From 1b75a67239b83cf40be4c7d1433cda690aabd3e5 Mon Sep 17 00:00:00 2001 From: Tomiwa Date: Sat, 27 Jul 2024 14:21:24 +0100 Subject: [PATCH 001/439] prometheus-agent-documentation Signed-off-by: Tomiwa --- docs/feature_flags.md | 2 +- docs/images/prometheus_agent.png | Bin 0 -> 154246 bytes docs/prometheus_agent.md | 44 +++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 docs/images/prometheus_agent.png create mode 100644 docs/prometheus_agent.md diff --git a/docs/feature_flags.md b/docs/feature_flags.md index 24d70647fd..73fb6a25ba 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -73,7 +73,7 @@ this feature flag will be ignored. `--enable-feature=agent` -When enabled, Prometheus runs in agent mode. The agent mode is limited to +When enabled, Prometheus runs in [agent mode](prometheus_agent.md). The agent mode is limited to discovery, scrape and remote write. This is useful when you do not need to query the Prometheus data locally, but diff --git a/docs/images/prometheus_agent.png b/docs/images/prometheus_agent.png new file mode 100644 index 0000000000000000000000000000000000000000..01a9f00f3904468c17bce3e5b02078becc307c4c GIT binary patch literal 154246 zcmeFY^moYZ(ZTEd$=W!l!E=&a~OK|_`eH|Nh7VAfA z#S<>NK%iuB^ZeO8IouZy|M)yPW^nqKOhvqT3-22(-3zuA2Jtu(%1oS3M7P{=@w&sq zyqYO|j-=0;FlU2m1qItjm#c5YA>5xU`(LiAyKQKW*dc!upB@$I4O2*hkM-Z} z;awj%<^TO4-cKa@zxRLMkU03ihwtGI-1^`9RL`Uc|M#xr$N%@s|Lix8#gvD&RzsY{qeo*Wt!L%{SL|D;^y&8spYj zZA$*`(z6btK-@i3r!b(4QBs_ZX73dotl3la$j4-iiQ4$SbttYFR+0#ffAu(=fst|B zP1>k3=JQzJwliJZ=%}Uu6(gt9kyW5W^{5VWkl-guO%;WcCfWyg!yr;D%L1HOD*nFm zLcRUCtDwrW*I(am`uicrMY|kyJ$nxbEb!>gSWxBz+vtsg#`q2NwAXab1aDA70d*T@ zm8mKID?gr=vAEuR^X_!X88X0UqPm8%+BCD*rw%g@U+?T*2&XVH)zT>I+ubNkrRyNd!JBkbzRc`Wrjjp>PF&>>_9ZDAnDPO4~SrK7ju`)Ab@Gq`7qY72)Vup8cO=kDh zAH4{pu*O|D4j{g}N930%+DixKp zaY0QB3yb}|J@?G*xWxU7oo#9U_w}+ti78_~EaOYTOAYwF(T01C!|P5qHa2sPo){s1 z_SwdbbZeitn62Ey5*Acb9}SuNTbm+}nPJ|LRRVPV^0&zb-)Ql{i*w$s{h$8Vd&Pru zVrFLM!of(Kq@kt)d*q;9yU#t_KRe-nAKUyD!g%?qJaYtlIoZ6Vh_&qR=;+v9HLBTA zJkqhVAaC&S?meIhHyAy5t2uWX78w~?(|oZtK0dxEbKRbP+Lc#VAGKUzzszuRx;`Z6 zFLZcn5~`_t9IV?`9s!Y2pciS;mqZRs;LcJ=*5y<_mT(MT>m`t6!H2&W6&=)-r;A@s zqlCwk4=E6DTboqY*hQ34p|!M16eCBwu$8pjj}cRJfkZ`-JGfcHPAn#E6!xy4ThV^V z-iipdq8|g^9^_%iPm;qj8(5`7vmzj=da$MX;+jU~_!k{2eaXEg{ts=W@#tXE@h{ZW z_m1Yy3xm#4FI!Ix{JZVLk6X?S6<9QoV z5z=hvI8m^T2+j=eabjVRqRb{+NYgnQ31*RJF@ioL!hONS#L3AH>0o1L-yZvPyStpi zL`m`SbmbR1;drTp${(duIByk{j^g{;qxwu2Kj9A_AL3ix5hn~XU zDh5{GxVw@*RAiawi%bXv;)#hl?d(3LCpPga{d^{fKRv4E=(HtWy5TKViQt#1m1N7Q z(^GF<-HE=tCCOt<&N-NaqkVST#(`vPcgk4*=3%(K_Tb3K$lt#^dsjo7iC!bV-G1r? z1qHh~^WAk(ZjTdLrCs*#h2Zt36iU!t2(`2jB&r3H7+KnM- zH>`Hzro!K&iiT8<=?2lWsi^%Nju7FXWm&7k?O2-`#TUeQK@w=K+Dns-)A0@A@U4qS z)z~&(pJtriJ{f@qfzk!4Eu~nrHvx{ z$M{SH2fOV~8ww>tqQgyDz(Z$gwzwMkuA))?79I8UKf$13JR*~xj{LNz*;%9pD$qI; z-?zOHe-6nIsHh;d2n={JY8?|Rofo1O!sRjlWFvPr)1DiT2NEfS&b%kj@f&t3ig;gD z{+dKWiZ#Qqg({K4ZWPw8-Ao2uVHg=+9;Rq)dd_q=Z)?n}-P))|;hbbMWcY2Tx2~d7jI7M?z4QqmHudy~ri6 zly81U#BlSyamwzh0kX&M8f!$UTy9iJ7~L`J@`DVyGgEzJo2 z!Jt$O#q42uMdVn5?lV|~n;D!Z?6vj!^?wp^*k#*mO}YRNFtkuvd>sH1X}2l8hu z>U~OLOx4tKb&kjM(4^DS%=+MRmouW=39~x4_$RKhv5`>p;O{o;VC}S4)?z28jzjIi zY4{i=$7vX1A#vMBqc#_bMx}4CPB~h!Jyw*H8}m7AtSRm1Lrtu|H#9OcTT0p3(kg|B zo26eQPL;Oo?KyjU3)RJ>HU(w+CursSTEwVXW2ucw{^|L^`Ks#cQ&Ljuv7`23laD*L z%IIgY)SYz%!rr{LjxH|TG@_gC%C~pau%lZs1GA3od#{8KwpQ*oZs61UAJ0`+S6`j? zU!msCq)YDM;M6Jy5=nH`cjS&l{-(V&Y$j=N)@5IA2*H%?EB3l;L|`|1-CEe5%9Y+{ zZ7g>2F5+?Qyx<{a49pv8A3LEf{jsn;*AQm55%~_n_dO?7h?o0u-~CCAhgg-IqDnSa zdRb)xy+2lM(W*@U;zwTbe|Wy(e-@tG#P!#kyq}XnEf4u5P4>lDDOHORJG$KFT1A9LwLNcLs)Tzb#{E>h+6k6%7Wo#cY6qDNe&-6#tk!n z73(tcX|%!9OVmiCq{Lf)Wlroz8e?6WAQ5QcV9eL%KXE>yEiL2z4{Z5>;4cF~Ti?XnZd&v}Fy+&A~>F1D>! zEEt(k+Z-gUB42O|*(w*>YCj)n((adLWz9s_JTVU+FbMGd7R;SC+7M6V>@`EV&Dis5 zs_H)LgO$s+Z{lK>Q@0E9VskBQB!`z}3Rhx(J8n&Awx*A%)V!3-SraV$u0%QCuiTX& z-JWF5*ZWdM>aG)qwWMm8`jwjxpD9Th0~t)s^qcoR2|+$M$PnWKmIiYU{IS+a;UzGamNxs-uALQO?pqUMX*PZx-Gg z;btX{zSA}rOhBXhxD+mAK-t0Ty?HLf)c$N?1XW}8CTFzb`uMwd&>_fk_^gj=5J^#? z2rMH9ttICKA$UQ&w*OTI7H^!LI`~|_#991IO*GAE9zh`dEKGMjje^vDlJ0))z%Cwtr`Vo5fkXXL>7*XGAxESU<$qX; z_*&F2v@A_|$2=nvM~K7!jChF-WX8v>RHvE#`j$pTyn!EvL_(G0(ld(bZH&G385myi zzm97W;nZ>{JT{IaT?W^nA@B3VR9m+HteN;Hf#w76hFY};?)kC4u#6=Q+VWWSrIt#$ z>_F1;K&giB0G7NP69cK8)*m(y$Q~Sx5;h&Zg{6K|o3=EYL4}mYJ0tkP>fxMk?)i!P zH}??CwbFpaBtg3YI`#PRjBC4*_RH^-cQ@Q2gY*_1 zg%KjKwA|caE3QR8Dx*KhHqaL4Ju(wFtC6(0VLPnwCdrue3tC{LFDzy#P;c`p0Byz1a?ct3&%X13OqAL7uWN*pk z&wIk2PH=q^#oley5JeVk@{sk{94EYrre z7iB2Ab3w|!yNb)+$p)zUF_TaN_u8A->2lHM=cO`8%jDM8N%n&JQ5wMv43F*!9nL>c zR4T{swvzaZX_T|w=AlOLHBZQXKSL#`|L(RvJpD?sIjrcS>E5J1sc;L=*jWX%dq+mj z{A}HgDzm3b!)1B;DqU*akvof?&9gp#I0_xD^^m=8roEWDJk7+Z;2cf&Pf_-#eu*RV zmFh#r*KcioJfqpcxaZ z|9)w>O0|=BrkY|ajzp4;VOpU(Htvhe{hB#k#qQ^Ut3{z`Hv5Kq-qzY z={s7>Q~Hbkd-*z(AK$o5{4r*%^od%$1$B5;T++AR7`(SLTg>~NPq8+Vy1_k+LblDA<;&n=MbKVVw5LQsasH4{zh(91DKDMo$B>_h@B21Y2)^ zMH^|&(rtpPOmpaa4n1*c4$J4DIdR97Z6jk+$9_cwz8+_E<21*aG zs|j}=Lp|~d%Z8N%MPJ?()?rRgM3Zza_D~Q2&|_z%M|XA6E98&kmFyQLWNh)xI=Qt4 z{(=gMFKO^kw!dSnW)Bu>Z+li^`ETivv7SZ7yQkwNK4O3HDJyqBt|9c{$uxQ4uu{W! zrQWy140)9Qq!ZZEpg+E;(;IX?VdX~K2|;h+bwr2l-hhn0{UQ$vKZOP|sRx}lVr}?H zlwI>2bYZXxc7jIa@sA`9j$jmP@4E5bp*J{7=ylKZS0cz=D5G_s2uj%i5O7O_s0k>0 zXd{!Yt@A^W{6t4hYUy)UDE$~#1d$oI;sXlOD-(nZ$ zj!s0Y_bv6*QqVqA&`qvG6-IF>9_Xv>$3fk=IZF`GkIcwr#*6KJ9=g@x%JmUxK~Wjp zxgaxl`?B$t_E&- zUmJzzg>h<6O^zrwZqOd#;5eq%7E?m;I7^BKK4d3kn4prpe58P5uHo~_RLRf6QzdOE>%^FI9eN# zhEY{3<`-^$K6a}`JHhTqy6Y6_rf^THuGhNzBNJjID`UgC{X4B-Nt@MEWC1e4X;hEt z8{>uDp;_;fG+FVXb_M!sip<<)d8^IygdY1{!3F%WZPiQ1AJbAR2g^Op=-o)pGs3OC ze%T3FFC`M0xSEblX6Y-h!xe&{;bA5X7@k-Mcv>48-=>cR-L#~;k}!JWn7Qi`>*!B^?^ zeCv$AlbxE+J||=>@zLb;z)UWjje_rf)--~rt4md?g;yAyg%4M!*S89l=h7F%HvouD zSAQL$YSZoC`9#9%->RRcF6>cL8qHxyJszFAL70J|Dn2)=95OTg^B4jd+#qz_`znLT z4N|f8eG4TB+x3SfwXss_7?-bNzNxoD?ZWkSC7FjNQw{nV1o@YEsFvCj80*)t25O2f z&>?E@f)m=Ht*Lf9>tT%NTMG#Yp^FzjN#7dumlt~apaKuFg(FeNP`|%WFW9AR~-T%}cRj4ds^+haU z`Ogqgi&Ko%c1(+pnB?KkpK z^GbF?!V2H)qk!7nlqV#TqQm7|d)~0vZ9?}G$#m3@4y0Aq6r?PVV)dJeA3(PfORhmI zpi^=s;2I~6uF^L8)odBCaz``6ALm{ayRfme+lfK#I1s@j=P%0YKr?@}KYUs~>mgsn zu{W<^F!1^oo-=4NLJnrjg36sbTwxSB{OAi%<3hw~qti0hs5b=LEXKH`l zp!zj&Wa0$w?J^CRnihHbDIaT%Hbr0cIWx z5{0+_-MH>|>{OldP^c!Tj#)F3$ok06t#h@i>P?R{vnf2Prc{Oy(EljbVd|}@(kZ0X z04u(;b10FC+46q5kVFI^0wrv<>`cIcsN!O`V9M*oGjn$$Q-tyY9JDI7dwTP)b?K_Rg; zB|z7v&|R%)leds!*@hA0Bu7k-Hf)}kAJ@iSeh7)O1-&VdNRZdqY+0zPh%MgN?I5tv zg(rTqI_*Q4yp#g3_!d?l(z3AR=5C&m_Jr&rv8d-gVwaiVc4W;Kq5IcQ4;yMHkitsL z2@7Aq**BE7Ag$<{Ry+(<@YSIk@}Pz&az+yy`zmB?uz&F$KR}Km7+A?%Iib^TGoHcx?!@w9Ql^G)%hvLr5Am@IL^J1SomQhY#PGSAZoI zl3QGC;|taQ>E76ExxdgUHA}`TzNW#qH~5rVXMgf7iSbQYj7b)$An$X2`iv}1u6S*= zx=hLWqgh7?7uU`Vwh6tDnygd5W}6PHhhQ;Fo`b4mV(LD8$IHLQLsPM@c(@!q>(zH$ z_A}=$~yMF8mPntNJFh|2?rpC`DA82 z9L!aNX{!$C z7+wQyD9LAM<8RJrqzxe0*$KKC`=DGUZn85K4J&2GgC86E3w}QZJbh1LWG`*jErTIw zfu50wW$_?y)-8&Wu|c_$#bZJ8CD;r0`%cD{vmVk>gaQPzp7I1?g!vBpvQ*I~Y}7bU9U8SW{}QoGgwKdf@B;8;#O8Qx4@-H23jw|HC@hRlxMr-2 zBYrjG3vY=PqzO}#sky;wKPd`26*PLS z%FJYU%Q9XN%0=%Z9ulaw&Nu7mv^(G4eAdvsvASBDF5K9g;deCt`SWL{Oy9i(dt5+c zvUf!K%h~=@5BWa@-$E5$@vWFup|WRy!0eZ<64;_?n(Sg9)O!S(VnxUT5v4($-9s#? zZ|(L8ZVFDNi@N#hE7jUEFFUGypsl)n@mrMr(++L(;)DbTKoN6*ux|LyHe>kO!9H}3 zrmDUzya4crm_ba_GmDNXNv5qSV_so88J|du&IyCN4mH7xwoE$~rVs!qu(X#lrxBPk zmHV7UW`B^om$;W7m`ZG2Us8r?TAQgx91t;F8w9b8wuz*@sG~7>Y@lL15HJCJwRsW zXoQ=Gk#=VCMvZmL`d)9ON2jaL3JZNDLU%=hUX<|$8|pvfBWd+9!HcmwOPb~xQTAV( zx~-jpwfOh*86}Vuv_v_6n$RqDv`oaVaDP6$?&n-XQK2udaMt78FgsR;i`St<_5Dom zr!EZ$b!{Z%UJ%#)x0assVQXt^(7 zw{*ysWNED0jE$w(oFC~0r%ka9@cFUQe0u^L@6d50nJ18}|1%4aL9u~7;~2kV2Hi4h zVXG>88QgGqTAq+>12+o~ds9K}X07XKBuFRaQ0zmE0QETOD|@b*0&Ftcb-Ct@g*SjV`UoJw^CLFLNW7qFn20*ts*m>%&DDe;rICCn>C&sk`*kwihMmprc z?9P)Y)^_hrdP(2nEO@uUYFs;ME3IvPm$NAA({~Em9YA;reGyra42bn>0s5whsUMPwL3=ZYj<>S zfU5_TC@X$EM7X)F>vsaTiLrS+Lx|TqZhS;{7~QoFOvmuh&_^JHSRlMg%w@02z2wQN z>U8yv8v33VO|^PdIM(kP8ym|(2`YY+s*gy02{W$k#59;D4)1*Rt`cCqE2L!F+sD4V zzDqtDno~u|BEIHGpY;-g%mK+6uG}doPf| zIlPdLzApZ?@qymn+u^alC;T6VP0nt-VUfngyZfcay@H(Mz2`a-VZX%}@aj5L3L?6; zQjadN%~wp?)BC8btgOr;%iNL@Ot<0X-t5Kn>{dVmBGdnLbG+gn`UTEML!OKkNXLIE zU%_WHoh%hO9|&mOGTu7Oe~KK2Vt65gHq|x2;7pTfD3vH-2cDKfp8k6?vt?wRe8~lm zBGd(tdOT450PFztJ`qHrKHdH7%{uR4_}8ZUQ*}d8+RX(YXK#%tLKUV@i6cODHe=w22575S!q!R#F z#Ry1JxPW(Z6oTk-a$bA2#^saFpVb(d?jFvw;lszI-)ZVrr=;!)_mu{I9|E{Lwl4 zb5hKK5@!<}p^FLoB?fud(;FDfrDuc|uy_w+=aSmM3LB|`(U7mwz% z>6&t#v!S6ObxqBque3}!I5ktsIU$L=$-sHqQ7^X>rg#916G2|3h-HSuj_|Nux-JU0 zTWu5uYRW%$d^p#7-4Jtm=~~(7#Kx$hw&hRsSlzh?jdgS~XNMj630f_RHrmYNwzAS! zWP+meeA7nc?*OI*%7LJH78Z(J7iESK%i4ApiPU|_6nr!hq=kt57?6uGjBM7T z{ZrsSfJ!{*%1%pt7GGZI+fe#hg)z|_v}vh=*Bds4m4yWtEV-R|y+u>cNy59vof=Ae zfcC))1MurS28_M^uhWtq53yQuB7SiDw0L?h;kMVc?2ktw2cM+#VWrYAOjE(== ziF7I2MDDa%W&)EXQDgV3Ow}vkIrEkk{VY2JL_#7os?bN@B2p8t6JsII;_RyejV9h6 z3t9rUsBQ*!!jNb86Z?=$1O_PSf!RL%?(_*wV;w2^Isu@xDOfgJoY!wJ&a**+bww_j zl>yLW-fCv*RU*?-NGFuKC@CQUT-EeuhCV49lxO`>IRkpAcaP^+R zs}Y^BWplr-;dNq2M_13}*|*xIf!4&P1Nc2!Uq=h(d!0^Vdoj_b=H}ZPp#hnhNlBjg z9tyZIYNt)&VqzI_aV|DB=j9^jJFW49{Q;NzXS666SJwlV%(E%2=F^e2H7f}@Ft4Dj zto)2f;J5kZVg!VpojoWhX!c^;!{h#f)?V`-YjgS4mK@8yyLavV&W?=0;b*f~mkt-B z4vo)!?%@bLYr@b2IrmWu-@LNG*W`iQO;~%FSyS;f^qUh}JNe*fFjIZUG?d+aNCBnp71Om53;Nk-BfGTmVU z8W@Oj7{M-R0l2<0oe3F0C5^cBrkNWoX^;J6af$ zwZ^tr<>|Q#!vsX|;XTx9Lm~g@=4Z9XeJY9-@x;WqKvuIsr2F2UQd>}XOfN66r4^&K z&Sw;@z50-L75LptQ^3`mQ0R3gOn(;m8)?*7^`@>hRt=oKkw$Ms_&8qXtC#Ry<6S zwcXW3$;E^up{Q#owFi}}aKKZr&8Gjm#l^*e4Yu3ToybWd@sL&wb-lYx%ElI=#hXAt zTw3}JURYdwc`KQvdB7d^U&u(?T~16)95Eop z(vqZ`ViNYJP5G~lBB~VRbZsgKD%#TU0x#uo;MWTww8*bLw|#{OQfGUK2iQ`Tuj&Jq za0^4fAJPF03HqzUHLB)^X?{k#UE{4m<02)2i6hBH1SR_syVbWkF*Sn~?H>|TGL8RO zp|XDgWwA?ho7y6b2z16sh_mrWW-GWk$Pr*kKY+b`t&=urgnr?7MedAj22xORsZHp3 zwK=uR@hjp2k3iF!QB{py$wz;M+01^y=`oZ^VGXX+e1;mIaZZWSORjS^@4*Q4wmh-- z{T&Rzsxfgs9XGUhWDkY*3cgJ|RqcaYaqYLd4t40wFO+Y%B)ri+{iStJbE;H8lGJ(9(5I| zVniVeajA#G<$a>6l@1$4`Jlt4Z?7JOXqvFcb|c2m0ZhLTf)?jFdzO}##*dmK zcq_8oh7dbsQLsX#tavRxmL}ZT*tp)C;5IgRb@H zft;cu>&urDKGPW)872OQ1tumYIXOA%>gs{tu&5Q$UFnFjh0f^N;8nUyzkiF1O?Qp% z;2b}hKNEwKHeUO|?|wN?eUPah=wN=6+^WW%l=^Ii5vTwdj2$5>9T1WbvoTDa+p|~S z-;I1!bI8$<7i$SmTmkkNlJ; ztCG22vXN$>2W$+$EW3971w|EZg`o=hE-mF7$#xt1)~Xn=JJZ9;KzsBaG2#br&cB4E znH#GP%yKZ^O;6D@(6p!otjf4Du`OFuQ%v#xS?jcvje+0qt{w(mWJ1t!`lIa_w6=~H z$WwrGkPe&!u<3vb;^YMG9FoKl@T0}bW0e?SOtXqkin=lp+`dloL~H;?cXF1snFf?f zLJ2@HgjyzYn>v1-WSrNHv?^1~KO9V(^%%F6_Ef#YS;C}yHoXE83xLk#S{<1~Yi?JQeRcmFOMtN_U-7^ZvH-a|20%NI?zXO<6;eB{D;5au@4B za+okLdq=dFmH}6pWlN)j`5z-*(c)!9n3?+aPOLSIC^me^xG>t%?#DNS2!=pyW;5}S zHueab>qban(sa`6usyYZRgZ4uAiXdBJSI6gIelaOxRHX>qcEheukU(*39zTD@#$&m z`$0`$+wk%5_->lep7mIWRWwZOE_7bYELuV5A9Zzt^z(bvW# zBP)w(#2QHa9vvM`VAYyA9j%!;pozN{af5Kh2F019>Y8bfS-%iMT6EWR&~*9<_x%RD;YlaR#t%AZSj_P^#Yi!tgys3T>`Rb2m14`d$dz4 z^j0@;41(;vTkiw|e)n-Q1hOmuJrBkQh9a0^*u1R>t#An+9xMd@)m=&G{;ns{vtS?3 zLplS4x3~kqTf>x(YtOyw)wBnllB3sa3jyp(Ac}&xuLq%g?uCRj2h?@HL}tOZum&3Y z>L4Ua*Lsc!Zr2tsOVU>}TOoZUq2xLQ{8EdK>1APUg%pZ4h~A@=BwO-#I#xO$AB-`8 zO!X>Sr=>jZ$}>nIqiVQH~4={v! z%-gJ*TDw@j)-YW@cm#_zw;*5ro$_dAB#QxfoS^Iby|z zx)T9U$21)lIW%q*H@M#yTp<)Y{@~~5CqOmB5+U{c)9Vi_EzV$(|1f2~*Vp$$vIndw z8d}@h+FDzmj&+J1wcvBwIyiJ1mw8^0FhNPl$QqZ}9ZnXR#7|%~s34*}rhq*r+Vg++ z49_OeGh7}?$;n>WhS_vbCK(x*l+C-8z-QRq10KT}ruhnpkJ;trWkx2ZOt0l+kEZ=} z2cOAFcbURkmj3xX@(rFGSZIqw71V74<;r`JwSyA3mk#lR$U^60%jAMjbm{{8#6zP^4f zq;^^fYpSXuBO*36D4skCbC@e~@Y}d2w(}`*f*u5gf!b`r2G^4n`E-8eWfFBJD~|`- zNFfuFB*b1$3FBXZ=M+I|KiSB}!gjaeyh-sonGpn|O(2!rx*W~ri(echk)>b;u0aL*Ab&})X029V zO`5G+iXS+cyyCqDsa`tlMlcW$IM$@3|6>>d)vMZs_R^Fcy~*T1-o^f0rXWGOWPyjF zL)14q0Y*k^r3NKv0Gr&4!P!~v0Z5%o)zKZXL!oLwWp4ddk5r#071w#iT+FdH{!x8*yskjdeC~59nq@%uWyC9 zTM?)ZVj5pC22bTKpbAHUkuxhuv#&uH&NbxR(bRCKx+` zgWS_Vj~G;x-Jh-=_^0m!3f!p*h9zvJKX!T_;%3Kt>kt})tp;!tmky03J#J$X+YNi` zYwYpodVrd#V{Oqjx>I_&riP-%k@%`lfh{(u` z&VcjIOBX7IH9(V)SC_L_XE}f$1#$Y$Ezn*a&z|?rq83080z2Eixf67z=F7tpLtS0t zh^6J_4&$=0u&`+VgPeu{n~asBk`fCuGwGaxYfc3-^V|7C`yx}SG(+RIPVyGE3S+SSU2cif^O>?&xpoWlEMcp5A1u%=D^JKYaQp3Ei%BE^7Vp6(TF|`lgjWvU5oZz zqzARAhewylpd6wz3Eoh?T?tTPx!zF;WB@6h%pMIhuUOLk&$>OHZJ<*m$z0p3% zSW6!32zfcVP{b#6^Gqq%WlDTJ>BA5Yd&yh14D&ex|D#=R|45v8 zSdz!D>pe|AeCqt~x1Xj=6cz1GeV>lb9Waf!emi2P{rLN=M{?q*&`1c#1y^%tXN#9} z&*xJR{*Jk>wny88zP!8&)acuB%y8$zOy!4@cg#@vIwd8czn~=Hd!RzzyB*FYeo6Na z5aukY2xkR(v2F&Ur`j2R3uT=BBPH`dxWBAzF`;L^DyD4WGZkdYjt>n+-ul=z3&O4K zTabcn1*u39u9<4_L3*ZOPx%D(A?L%s&;frzh~*L~P#N*6B@5XGX#`YtiRAByj1M|e z&|WeQ(l>p8u z6@gO5I;cXJE^O6-{7X9Y9Z8AxD+Lf~5BP)$6zc^9-XUoOvlHHJg04)Q`6?jJtwiHS zJEFUKYRdNxm+QsAbx7y}sY2 zd+{`Kn1~EDsMrI5;@#Mb?d|RK?{nU}0w!Kw zC&7x7lasSM!Th*%Gy&(k9*rCN9ZA<@juW*bAuB7ZsMrmD;#F8Pi5AsVRegiY!p?qi z+$nxJ5bf`p*+?TKBqVk^5Pi8`a=E@?fUUn6o-NAGp0CKf*aW|{TjIOKdM#(jn~&Qq zGQ5@x3@_FxXLjyN{Js{-r|4!eP+*2vhlT+si2>tIY5dJ6z1Mf2fSt+2`&RUD_?qT{ zrW|k_5^xNrY^88dstnJnVq#*h&u_Q*hVVcu=5iY&c5yI-opO0AGJ0|Zp6TFk+TAd` zIu*+a3An-r%uY?6U&CElwBP2qMdmvNK!~rqvuqR<3n%9+vO2SI^Sr6N+<%j~%1)T{ zzMZ{&hr{F=1{3XJ{VWid&i?`(;0GqeuV1nR{IMz0NBhl(F#*#0$+K6M#%r;xlcs`g zV?dVnVglVHgM0E@R?K<64NpQ2NIoRcW$8jAlaqQ{TKrtsH=_GNG~TOOguef7)wsO= zW=I9h0B}9dTUtIZ2q>_e(l5%QPdKaQw^I%}t%_LS=&SCp4(u7Yn=kI5l{QsraBoLI zz`Aj=;uA?wm{4_|;Q0%HMMeQ2A`<;s6jXLC*haqS+DmW=_qX+R|1rwuy>bnwY@No(ASW^^#(Es zrYEX_&w0wHfK1EW^Wh-sz8-N})BvF#)!JNDNcx^PwNB!~EpMOji7ADCkc%m?y}T;~ zQvY;AYqO=b1nCxP;s^2$sb=PSPtLqMevF>N4!lDHRIjF7`l-eJeK9F$DMFykcj@kW zpuINBI&tZkplRRZFE^SmH^eW7#6^LVdF?PYoiA~U@BF+4xNLx$y1L5+<+xYl%(V`x ztMl4hljRg+(Rn+k|E?&;s~-SI?zQRQG2^u&JbMByI=mKjxw(uIp_ntUYenc5 zNivVyz%F;UH+dvcSXAV?rG)6=hWcAE&ikk+j$aUU%8tfm`(6gbiCzon+~ z)r$<1vDhf;=*_@D6&Y;?<#CnoW`26iK_OyxS)|N4S?obAlDfK@%-pvXm`NpW;o*%t zp%PK1@-T#L_fu?!%l64d*1TaxsRZp!uPW??tie+)?0b2Bg}xR4zLlY29zH(vn#mf6 z*~`s}{`;RxZO6-ti;ID~24G0*Uk+wl8@1DdkDFyvTxWiDd6CNRFjMcgu|3^DCM|E8 z`+{nLIdj)9EITPa-gWN}Q}gM1!AiPEM(Z2}71i*XEB};ZYq5TVee=apl?__AXcUl* zYemZO^xgaSD664dDV-tV?OJ-(4(F4-CE2a>-*TPjC;OQCjban9#+)xH9@tM*BJ1kL z%_{&a@f6|ceg)@I{bWdVJ7eXM1TGyeU7#b!C|6BLbO;kyM}GeEKxTM;{$TWV(Yu^| zBI?_6-*z`%&P?wIr{ZM~Yv=NPkkKXJ||u-3a`VV)ob4T$@pnW@@rSIL+8=I$ZoX zOPpwjiBgBHYb9~-iC|Yxyee@P9ZQ*-JrCss0=w(MQ8+{DCwn}O;sw-dgM^Zt5Qf%7?=bDyf z-$=20VDgmW=d|))(?_i9S>s)$5;t!tyya$EgO}@C^04SD*EZ;|<&%c5+z8cXdtS5E zd7-b?Yy3;Ur&IzZ>by^5sZ7R7Is7|fus##|Rv`iZxKQM$leMd+mGEoX3$^L{@yjnh z-GEob#2px~`6k)YOVYsCBu%Rn_Y);@!u~jk{FO&d|BhjRjY{MsvpCIfz%TuCUs)=C z@unw;yrF6wf4@%z9AywaSWlaqE5HBPDO-w3=2FM1+?>Uszb%6gULfz77|Ot_6!S-= zL`b+5kGOp7wie{O)S7*Z-zGmr>Hp0FZ27v2Wv0JnlA*ZwLG9Jny1CrPPAcW&@^TuE zhRyOxhvth6pKWAuaYjSOW!U_#=W`aGPb*#>+S$D3Egg}RHFlGCagqv@yauGn^&NlR znFKSt=ewOCJ6%I$c(3A)!;wPDB5_xXjAxYO%v%^(RlBbYAAmy(kAs z$+bh7S09{c=zp@fxOh#*uPHh%U1q<>)diMm{vpmsVE#N)U3^Y@zsRupa>fTEx>u#8 z??%=!w&r^3+XNh+B(>8Cn-X4r{%e)6vAtbAxdlECtHf%WOJ5x*xn)K-z@x*Y!usL) z>!U?_ZJ`wNLzgd7-I8gH03aq5L`0s`r;@}W#9rcM~r|7YNoZgGjbW(JeiwFtb$N3&l zbI3t#RI<5g8@1rRIacPjF&aRpCGEmS%%HEPrUovSF&72#qfb&(Z%EsvR_P!infOnr zl1X;MaOqr-_hqGNmnCrG2c;-(`l{_bAIN496g&YKG4~>$eaA#*Hx=<$$c68zw%9CR9gT3~zCC}4dyv0oU0-;-(>c2^!_~^zT zees3qu@#u@hEAapjihUKNKN~3!tgJ*&PI0P?dMpSdD9;$-TG47kgWJBd?fq}cLfG@ z!DrHA&9ejL2V1T4rnBYL_qf28mrx58p@##1AY$BFYlE6{<}tDUhp55+Gfc$ThcW9G87H&^Ei{*25>s<^Ij58 zyg083BL74-2z0nsu~OVpu%J(;dUU@a5Tyns(iX{grRlgxS6jMChxit8Cm!ba`dvFf zfhrQi5?@b##M{_)trk;Cb<_E+Uw+hoO_gsmCOwb)iG%k5r{g;>Sqd8uPY2#}b<@S) zQb!M7J^rS5^hPpPqk@!?p-`bw^ z`uYpvqz$@huZgfNKE8j4%vA&|0%P(>vOLFl?%vJ7Z;yJ6WBS>#uUSN25$$jeZ5&?5 zMxfX932hXn+Sjjvo}g&@PhG*n+&t?2NCeT-Cr^@n=H}bO4FfI?g)yl8tPq-;IOkv% zBGX3l@8o3aZ~N8FO*R%5oks7&^^pQmn*Nf2i;e3Ewym03|D^=2KA>#dI>H2`ji>M7 zd;kUC54JEUmej;VP!;AlKZ6MTdhGN|@K zh{Fm&A#XQe+#9B^V>OUI95>8f*tdTryGENSF-Qmbd&f`h#^a?%k~~kRYn>K=)=fTk zK07z$Z^<2z)X6k?ILb;N{mCxxaF3yT@$=%l0oM=v=bm0{a^q>?W%R z1_r=2waU8Jo5&n<_&tb*=V_#jOSarOn#t*xp_MSRX?@$ebc9-+SBVfe(nh;ocpnFc z_MO3ITz=wCpU7)G>OLVwj^QQ4 zy8SJ~DYv*El<+t9CWShpV`*75LQVfdJ_0#5c?&mKhC5RaH%?ufE!&l*rO80$eFd7I zZ@kj0yaZ9yNX>P0&_Fx(&;uIzkVL1?b>*GDcjFtlt4Lnurb$_zb`6zaIS$;^SKDf` z|2enu(#^%+FPg;QZsH0~p(Ba{>7Q6$ZDTa*AMV#l8;{(KA|Kv~AFLpfSb2EI<;V=eXW`;=MAL0V7cJ1 zoVLAiXoz?FSMOi%lsbYFs&8=#O6ME&;N8S~5c`NW#Ix_)i`sBO0}bz#grhGK&`;WG zl1!b$<5reD1w(Pnywq8y3h$tBp4FUB{$<_}rggCeUpZ6G02ajP8Y1%&q!90YwyJ=z z1;S@?R#Q_Gq%RK%2#g{?s0ZXTacZ(st_t(wt|u|~6)>9vFf)yW;^#YH!*YJjp2c|7 zy}$N<9U8W(ubp9=(ZXF%5zw?i$@%R3x(D)P{QLL$Y{wki$+Xzs*Y2d)6z`s0giS9m zE#1Iz-d3Wc#@uYgO6uhCFI( zef{&F?;lS$hq9GXjPx`#&OuB|n3{TRO3OePkns^vex6Jn5R&+^s$pSa0SbW(72|S2 znhg|BQ@hSGfdU2j1!+-brIHs4#0O+{z@OJ1zeCjHCE@`Ad8#1QgKRZ(2htb3u-l@y zuJK)$Pv;Wm^4Frt<-h~b@6D0SOzr0?s;Vc`y0+G{)%ieq065_uRLNH_f?$eUi6D$B zWwg4%yX19Bo1|9_@IR<>d@#kznI_Pz0snbYyF=(RmigH1^#!m=`BkaTz8m+)0`brL z{^q=X-eC|Z3D0}}$h~f$3b;qh;)cVJ2|L+aM@D>)mIt?}^RI+x z0UjP6ksP~)W2Drqdtv}nbJG*_5K$*z-EMTg5|5KC(I1u-_-TSkb}W0Ebj%+Bfzb8Z z83+>Z6X?;3f8keVgpq3ey(nFYR&PPb=NEj5DfF;KA1=U18m7w$VncqP2`V$Ir-ixv%y*o8leN}2 zoag&leT?~T)4>$Kd1uy- zWE-Vqw$Rk(>^AZ8rZ`4E&3WuPX@h9UxJ${~8-hA&xax?;--7M=H$Jh&t3ZyWQmv4F zXB`Eg6g!ku7B59Ot0Q$r#F`0*Q=3S5AFO}wlB1FPyvJYF)zoTSj~0Omd0AYTVQFjo zii~V%c(|gnvhCq!Z|es?K)3+SZfF^J^QbUX)z+T1e}e~qIR{vI&*A5nr}T)^guekzomzLJdex{3))tHj#qEsPei&2UJ{U9x^$#F|DZ74ArvIPE9 zmnJgEchJ-cKe(-ceZ2tMVee9zVqw!^76urmSG*Kcx38QOe(BbJbq zp-juaF&5UCRg^? z4OA>#r3IHcYh+Avy+)@Uzz!_lq_Gl&^{)K{&lkxuVRzh32y8q047XMA>&BVPuGpo6 zUj9V30UW}8dG{~q?GtO?nlAaR(_r_})%7xhcM%2BaJ`vUuDLhFyClTJgO7sn5HKQf zO6(*6Q!GqNNx2Sh<$A2bIcI#V!$}|+DwOpn7g=+7c6rPr13GK&7%4rF=AR_EnkbR8 z>4ir`zB~gbGvZ|gS7(<<9G3AbPargQ=OzW6hXWm_wWb6d}Y zyr?B8IVY&Vb_?}UruMucyEHdqKk=Nc)AaHb)5oeuGrX}z9I8m zUeWaebKY$>z--;jL`!|6jBhCUlNR;IeCJn(M|kK=0GOsT6t6_jH)dYlAh?ETYxaJ^ zzHWCRNWtEEjXO{4Gd?Re@}ThYoqe~qN@na@hJj{TEg1#;!uXvD;F!l3Zg0H|*y+t! z--nE++uk2U{J8hrRS?(Lsru570xk`ki?FK2?6;;lIM2422fUtwJxy2P`6?H{<=d5@SBKl%|8DP*{5n?b%JhDo{Ft+G6o5I- zHDCO-+RFSY;{We#Lh0%JYgK*_=3UK^47)g(1A0+2XmW=Av9v-Xon zmYeIh)}W*}b8nF|Ry1Yl4fcIhc!^hc&KN^JcT6z{q3<=uQ#k>EeDL&+6*QU!=NW$3 zurfofO?GWlajon`^JrcjhMDha&{&xO*cU6?bUU|I;5dA>++M6zK0*5p# zEUc}s>G9F~eM>CRpp^ELLO4WDEFIa2yg&yBSh{owzD*BoaBy``=F$<+tLDJDZaeJc zuev))YQsSC1KM8U^ti?;ZR2LD6`-=m+qqE{@9TDPc}J`N!=9%JpGwfw@k!(Xj5Duh z&=0CA)L<#cMq`G9$cMybe-Ii)JeR7fB}ipqA?WrdbDA94B0%q+@~I+xxO)4#<$9Q( zp0#E9>I9rvqm>9dp8dz9$I2Z`&QDig(9R~RBOwqtwiyhxMlTK47l!h4@?sSbjX9C|& z7w9CF>oID)4x~c|c;MUg|>3PY~$$nua{P@3&&*K_5HN!~Hif(ftp)0+oD8Q#bF*@6X&m^t}%9pjKv#815-3oK&w@uaW3uMaryC_T{+1 zPtp#;dO%M(c%o=A&d){0z}^%T5g9(cQz4$g`UPm<2cR6;wwzA`NEDHa)&{fhA(qNj zDZDK+VSHuAPK1Pn)M}p7l{9F`{(C_rk~ z9Mlan86~B+iB2I#uS}kx)}lx0kf~g}?NyTZU(`E`xW?RSU>HwCz4yz#R^4_IP{G5O z9IGZ5QOOzDobkp>{mLAg!`@SiNcdF0`6=1UMt4xnf9&omt)9vLnFtlcM6D37auQkQ zE9mQbE9&J<`ubnRr})(2b-oK1T!YuuW`H83Qyh97Y@A2kbBv@o?g#?A;Rm*1*SB9*TP0YcslpN{p*%2aFtWF;y!bV+^)y9eYR>m4O%itFsOK_@jzUd60G$;Y7MJw*c}%9 zvbZ?*k&AeVf3mu?-0FGZK>6+C--FwQy(0)DF+NG#p2ntD=;5UFk74Ux!7IBKR!csb zbv(3go};6if0vjd4c50VLA3m=JhTm$qZ&!Qf9Fyef;I^WQFPoZTW%j|!I~NsUUPwb0^z```c_W)4%OjG{);F2TnrdCdDfJ7{fp zB9te2XcK^X$KFV?m%KJ-9xdSt z`uX$k6}$MPq$eqz2PsZl zZ-`g{{?=vjc5g}1pE~Mcl`z5i{FTiYI7k)++)p{55+>_gqvM9z7`Al+`{s7M>tg9I zm%puSjq?i9@5`MBb&P}TPe8xazDIR2`gi>PDNRIT-|GKUwUXc zV&Sfi_Z{aAGW&8rX#g1NcPc^Wd}i24A3hp$U}3>g^IGJ>?L}srD#ZR~nXy)efHh-T}#9Jmg3J4!sZSh6o(iHUhHW8&-`)~-5 zsgk(Hd{g`o?-Yv=xx&c@F|0se`05K{XRi%f@AgDOD^sQ{71U?1c!?w(sn+;jQ06fV z=%H&Ui08bPqT)VveigR46qEY%cCfRrQEq3GQ`h8_DX z-FgDthpZ=h^Na`=jX8UkbIN1c6^H@TenW+oA54b&ZuTa={;jfpY4lmkb0k;E`ff1? z`i7r?Uj6$lzicnc;f>wxtIX@L1xwj#?IpMSbC~Vv!g0k$0G|QOx&Q3togUWK*}w3@+ARkiYltPY68woUq)w z*y>4srC@6)5t}zFb9OlQ)uSPIuQBYk+r8fcfamJDH5r$Sf3JGx=}Eq+uU``RYSF3icqHI?e{%lyOF_*J z``vW$>pD`>e;$ulr3M+FR<3Aj-s_#-b@HpTBdzooa`@m);hRCgR`FK!{>2lOx zT{!P?jTk_iKWePfbb~UKsCK+0?dY26C5}}d*mAPDN@?4j4h)oZ9dY6M$Gjt z;I9E3H*)&O|IuJbYt6BSycSpSsY)1-X=va}ejH;zv{*TlDmF!}&13i7368qDfkWoc z*n-(48y6VC<(H~N44oX}toNx0wkQdm{F^vxR?=`b=5#vJcU?Rk&8<9~A0*m)c@p65 z=fq91o}h|kN~MyeDnN3z$EAc38EN1+m5}6h`dN(D`yq=WbmoI|w{=^C+DMh#-ovap zhMI=Lps7Q+oAB;Bkq;I>r;4S(p>?j&Lwqb6r2TA?8%Ct^{=(VQ<7nyba=0qGq|cJY z7uDT*rgcKI;(K9%l&RU$eCz41^VY+9D>4C#HoJ(3$QQSfH?7O~aRE6@ij&VVF}aWu6Y(Ff{vI_ZdUrx5OZLwmmQmh* z)iFDQ_}W!TN=QvK)U0EEbzgzGF6xXa>n0>M)#N))tk|R)t59jnxT{)gY_ukrc0Y@6pB6@e2zoL z%)UG}1=WA9(`GsJ{S{ARRG?yMXF@W;dE~bJb^P;mX_97`V!gI%`NWQpu&}X1ce{==MDM7c5~;X})C&%Z=Q>KeU4@H>{_kvDL?+40;W6OwoE zZhh1Za;NCOAwCs9rnUDTinAXGQ-#SgK44P{>}r{bLwx3chiBfM6>51OdBQ=4bz0Y@ zT^G_)6Q`ARZ|@vO>e||PnwpvT`Q7J$D`z+Iy3pgVE6%o82+?yjbOI2GAey71w}h~A{KK=52+d&bGMsM zWQnCB0w0)4|S=mw9MW_-+h?)<>SS$x03Lde3iWP+TyYd3tQoz zvLr97gTBK{_&BvvX6BK{VU1GFi~81M2QM!g*PY#16XujkgX3B(ZyboBlb0_7lDffb z3IrmBW-cHs*YD>Vi4~hNX71#4fETa@);{c?#VmvHOGtZ84Rr3y>K;BbN%YXSw|}xZ z+bPLSHow>m!2fUk-NaEWW-g+9vBtkR^Yr@VFqD*7*8PtfIjb?kQfc7z_4p8+kSN>I zl^ln=$@p5}iZCd;1W$FieTA)e&k!;<2+){|NpIIGkG^jIRA6-Jy|iuU@Z}TC7=pgZ zOHyaN*f$Wm`?KEVPsysrl7cz(0|=tiiHRKtUq^hn3vOFR=Lz4NYU+OeBLxgtI5zyR}}^*jSvj(>0!K&jdMNtd(E8p zTXZLZ>{Z44pVc(N_?O&s5gss_#Vfy~8RJ>4q;-vTwKe>pl)(+s|6eLRPSt!~B-RobRxMBH8U^6Gzgu#e@i zI1fm2Y^LD#27!md@}}hw%oiTQ3tKY@yhj6`F1H{4bv&#apYkTMG+Ih}>)VSQu_ZW$ zesv#dxm@`A*0ozwE8Y8sm9X5g5xx1oB}Qg4I-Z+H_b}+7)TVpaF3TawiT>fwzW3T1EIwMt#Ur+RQntYw8 zj$T0}fKBrtqBvfojqqO0t?;ECX_zFiOO3zlvQ%!>oK_VkVCR%D6)HiUh_3HW#Wm_e zz5#|oPT`N_w-WXv>bjnH=HphMrXb1CBUz6&Ggt9{AixhgX--sxw;KVr8hb%tPoF4Z zA+LCUmXxXgodblDd>>T)_*6DL&1V^J&vwCWz8CpI^|wL1k25=Adm6TwFjN4BJM?kL zNZ4go`zGUK)js{0g@ep^Uk+$xxL1(5o%~-GfJtOENm2N+d3`_Pb<4^8(ql}_Ctb&B zo$CvEoTvz%BU{hg^Nix{&r6e0Ha8z{zqKT8b3-sv%e@r80jc%?nZHGa#ny)|Og7}} z-eg>++w~N(!BT2moaaeNTxFvnZ2KsmE8B6VOg2;N>v5`DLczk4@THUb&+oNgu(SM; zf*cCG-dXoQ97>D|IXu7AZTHCZo?$DTkrxlmgc z>dOS@lyxd={;W@X8y6{+71=Tr-CX~ZAb4R$cU-^Z;CL#b9oapO@r$a7amErS{o1#5 z`s6TT4fu|Tj**ue?tS9>SEKE9KSZaQT4Q~YZWKhN~STP&|NC~Ls?M96y1t!7( z^R3a0^=YOVz6wxH4ZbFqQ__pr{6kw!{{QLdb)y&pYq4(=-unxN864aC7x; z>fiCz&q4}*B8xF~pLd!~fZNj2>sEDEb{Zp$TNEGud{Ci@vRB`8PYD?PVbbNC z=<$*8pFVc)x`!!!iw?g@h81qCgffUGZyPRBk6SZl9*;lReQy4Ulx*PbI6b#_j@(FZ>YKeY$HJ(U z@go)i3EP*QA5mvNuPqKC=<2r~@Fdu!?(QM%ky~)$;$r;3SmX}`QHcsVjy8FgVimJO zSp#HcMl4z+Y_%PZ48lhhGByyb&|>7Mn-<5E?f}7$vYEZQr4K_%a=Lce{~de8UUU|I zS}I~V2oSznMcN@yg~D7BV`tG)*cGy!Ei**a4!^@VZG5cAiFZi1F=xMeZ(mkwezqvE zFh=;D9X>o1ED2Qy@8cD{QFDheCkKDQS?#~Y4ENGj%7(CihP=|p_*)qkb%H-!&)(G6 z0dr6{qBr(j)#l?dB7NqY%63*HWQ0h2kKAbeKHx0b)5Az+reIxW>i~Fu*V_5aGCb1c zJyh?{!mEVWJpt_rRH;_zt6X*yS-gUy2(4H#HXcEc&(F5cH+lV(w^G`KUCT!rDW#21 zur$bZ#gY(&F_hnKUwv1v3wCY%Rx)0Ux@@sxR_MR#&LAA*=4oeuJm3(9e-07n91Un9 zxenLME{l8zPRwcr9YMw_%w7BoB7LxvTXf_VU)Rg$&!7rM+8#B6=Lvz}bxLVWbwOq+6%Jb#ZpGm%;oUdL7no#I8Kb z<&UvqdyxrRJasz{cwo^zLVSYZ0&!Yc-x=vaJ(Q$91G|mc81uz!T|ITb{Fu;dN#;nn z@&Mf8&LCu$qhzv|Z%mrFED3;sWcE+~`~rf+my2q$IX=ojOzex4tIUlF zqB!gzL&PVG%&_4YmPH!?asmg@X)e^Bf$A+pC{)C|w5=lMjwpw!6@&07bPnN?W>LrS zqZQ2-pf1CS%b=*>ZS09e7Xt23}thH zHpC^3g&YWc3{5_fR~w1K&BAIWAG#>>&gx2WsYgHKFVM0x>R2pD!^49BAZ2CvJ0S|- z<;HIcGtla@6lbrA3GnoXQ1yy+XkZLYfA$}*RzuNZm|_76$J9T(Lkr59iS#__9sa(= zKx6@XgP>?#8iEmcl;UjE{Cm+H7)@rzAYRJVZNDL=vc_TxW}G#2P()xi$ujA5_8I~K zXOAo>_^W7%`SJp7xFOg#e=)^t6jJMhaS?<@BW|1A%v!ZS4DMjCkayq1X!!g&9TBxq zi7!?=Y!nL#2Y0g=Q^KzjF2LoE|54FuZ3IlG*eb}8T__^{XtI3I-JiV`zIJurSTY~N zog&PFJnKihOWd#St&Ja7>NEwd@4nOXeHrgB1yK#8p!I@<^f8(tgMpBnMm!^DXY9-$ z&@yu$DJ~calfHrvjyZSk+2=Qc3eDs%PQ*XbLKWz9zpNf1=X9a0fY-}Ih9!On@35w5 z&b_m%*-pbnhRH;R4A+Z#Z30d+QjkvxCk&haH1hxG7T*|wo)?v*3iv3-w%BR@1c4ri zxJ7Ith*#mDM}#54LkFM5kM^rj)(873?^tPds`bTmZza{3YuhXYsd51ueDrfCygtox zfPgSau&@?46yjgjoq{GL`l}0W z&@|Z?VDzm1U-5hw9)POW{y}stCQ0eY(DXsbHYbPLkA`69Tek_|##(AoNE5`Z{n?{z z{SyBfs7W6k(8ylV-NQul!{Lk-0(xjxU-|UEY413^_(Avx@>jPnhMjl{=gO$nCi3w$ zEP*1n2v0Ey;x!m_^AqCI-p$l%3@Mlj{ktF%;lbOI`C&ifP{zbD!Ra)TXPkQC)Bj2K8r{iuIS6jwFkG_FNR#8soOmZj z`KtdQYH_qzdtJS2M{f+&VCN6OaPKq3MCKQ1g3NLqD_4#?6Pf(%4n<5Kh7jdW#GS0@ z%SZ0)Dt$tjXorZR0K&~5Bkb32pI;eqR)=2ksVXl}1sZs>{fFHLD?jga7SYL6)n*nM z>$L`(&yzz0wMD)-M?F*p@48n^V6*ogCZ8BX9X8^y@}M}r#F#Kd_&<|t?Ll$hVgq(9 z?oQT=ZahiPdte$bGbI;2Bqnw5F!v=(wYZjGmK1{-m<>07`C!0mF+x~d1e#c~)~`ad z(iMyR!i2z{t~&>jLZ6QTSGOSbQOFxCslm`K_HNBBD-8R$SXavLQA_%7aEfNjXHo(r zd=}@7Ntj3vH#dn(EjOB$C1jMf&0kvQs|ow;7koATQ?ZKMUeVE_OP)p#4S&MkKL^<; zMXc;8IKD_X;c&zF&Gm(RQH|otsBRGg#lo&`OrPM~R?= z_<*g$ePCJ@tX>np<~~370s;v?20M;AiIMZ-!cd7BI!M4w00{=2I@}Gem(nUV+TaC> z=HD09=GygkJkTLSZMqMbht_Koj$J1L47=MB$~zV&#|(h#h3l=pQuas<@P$@26F}O( zi+g$hnfPV4E^|sC2KyEn#W3Wl!_6{PiQ|B7ZmwtXHfvZOdYY7d_F0Pk`HPhbI?-Zf z^LEv^}Brx=VuiC-ZgU>FjOF6jjxE!FCg$y;HgtdCz52)E!q_H`4IBI z!^zB0@O?)*LY)S~z!VQ_$?Lmb+Co6fI+4<~)p}g+-ul{v(K9>QQ1L)xT*tgH#LbR^ zAlAqH?n0ifH>5375dltoH1^ldnQ>9lM`GGt9+YRt=fx9XG5K?4`ofph3Cg=4VDb#=d#9=m9=EbYbsS4H_ zDC(i(=}wcF&PegA(P^$`&#rqOA9NN*&CTRa`}tqkTdJqFk(#VyRU%)y+BGvYGGF~B zB?wz?eIdaAOY8-m&d2JAo7-ekKU3x-rp<6VfqmL} z>Z)gZ(to@WymOFVP#j;&AmHKbM5Q|F^F`^8%LVEcN?F4tn7I#y%)gw(0e~h7TXB z@)@sAQY!}ALg#=5rw4ukNN@5`sU7~^{o3&}O6Z89F)swVU1VUhfJro=!RD{oB!H{> z7c)ABvdUN_Q>%xa&QG6)y&Y^c+NpGkk`)Wib(KPTpy8Oyb^7UB^7*Td``>W_%3<)D z_ym$ch|U}{q=@UxM&H}zj_yA%E{bw~UcAB>NyyiAn!?5-)cpxNelri38Ul&mp}N;^ zsC-xQfhs{Awi?P>kCxY_R`R~fjBqrI5c9&Wx!vsNb-H*vgbdx#7ITs~YxY@rF*mzR zAzjQ6$d#dUpMjhQxaTsuQz$<)v8GkX{%2mF|d)p9w zkaQ!7=%IEU^hSVyNNBhe^(BY24c%b1V&U!8n)JfM)u{BMhK2^7q)uX#m)`x-W)6PH zcK`esQz*`hAdp}G&l(XLk&Dfn*rAb)r6~pj78?!Mg$6);D zeX#>MALr+!di{J$lU+As+99x1AJfuhwr_Ok&prB;U&CYMZ)Jt+@ep+A2kitKuw>A$ zYeCv&41E;CpM8MHh+61?`2nzPVmd#m5Ww&q&_iZ&&uW8BoW+lY=z0?(pR}pOVZ!-* zeh_`cvLcYD}7BfLFmgc7zUJ}DjzoTNx=3rbLk}PM8G&_E^?`7=x z-A#VShOXt{lTNv^G91KkOeg;yWUb|#nX`obkb=xd(IrrKF?>r=YHO*=Ai3{~-k<$s z#1#F{2kU)q#1ez@@3;_8^7z_IE{RPTj3&s_#eu*s5r%XiB=YY_ukAG*ezGA?8W{=eA0I>IlfgnV%cZsR3_#jtjI;4EL@R55Ug4 zb3xuG5)nX|?C+tt68ZNO#~xiJl8@nn44j91fIR^Z2&8OLJ$AGqMNINg0#JR2fkq7- z{P0GQby(MbJl(iU8mUp6<(UG3h#=P)=MM=S*)bi)z>F3S3SHF{sw12oDeAMjh%Kfm=r1Y}LK7nO?a zf=PIYbTxnc8vOd}eB|2z%2b=3@CVTFf#EPdvGtl0LnsLu`f`U&R!LtXaFCxts|q79 z-2LxVvulwkB;fs=x{$F}u||N%L#rr#?`4Cn%=kRwWY(RV8>k*@HK+L{h(2raho<>W z06bu$GxdR{?OIf&6Q4@Co*?EY`#h*mls@dYs2mzd-`1$@{-(i_5TOnlao{S$GN=^b zcor9X4YtaNacKChASf~7HdV^1@Hk|`?=RODU5XvPMZ=VOfM6a3*B*kCxuLniXCp{K z)%Q}NYZohOCH}w6*pU5shz{mcgzaf~6hzL!j3ga_-ksH33F!hFpD08+kz+_?xD|*B zapD|@AU_^6^u%xkQ(N(p)|FCoG2+nC(Scyc+<}d``PG$RNlD4yX7|&o)~^wp)c#l~ z+0!fH{{=J9G&E2(i4vz$7>CWJig%oZWbU3anabQzD&feUfKVOv;e?d{I|fWWEGRss zHMPN}N5K7>IsW5SenFT?-R>m9F(#zl9?VYZBE*rQZpS zk41t2ZFzp8L{SMXKtucsF*U-aDN z$$!}2+ztih*^ymNI4KWz;H>~r4;V6P3guS`x)ron;yU_`JGjd)CM_K(V+gT}ekTMS zL4rgtV+}Ys`htfC_G9;&!=8u;wRDLN8eg!duKmo%_!Q>EnNnbI+RMO5wj9v|npKlO z8yzzodjNjRMUy>=LDy?;p6-Y8UKMwcVf*`2JM!%lJiUGb#>baHOj5B7)UNZv?UJB* zc3BI6o46Pv3^nF~O(VoH_Cti1PCj*na)iM(rlxn=8@Pv+IZdI1AN5U z$gJJ}ADI@Y(vn2<8u<`r(MrN#J*^)^CRjCDmTAa{cRMo(jAikmmV9vm80P~+p4Vrj zRb?+I$d$$6{RUPr`Tb;5>xmfb=a*2~13mnr4FXvhojg=`ge2&s{e980zsG(IaP!n= z`9y|y48<8au|R0Zs`E|`KT-7l6XjJiR>X>-MyH+I22R!&$a^xNUQWK;af4u9KuVsW z#0baXO6(s5%Jz?brtbWpnAwGzGwuQ*k#B5_%UT4fpZMwf^u?mIdLrde$7^LF5aSuD&G)DO-L|_tncU|cJV;P)9 z(pMqPK1yjVX+xVJgj#tOt%y*i&B7sf*Jb7`03+}iy+g+dAQ!2++a*1C<@w~BdpWP# z^J9^MS=IvM0tOm(lyaBsGJ;(U?rjZ$7Y+h`%9>$ifQL}<9t9K_7ki*izJ#MS?xN>C zLc##KM*a`&_h2Mq#mOUnN1V~Z(iq9}{Z4S^h*Jpd6zMWh&VeD{%+Mfr4KUgF@v&g! zrmkPmMg0L3x=ueZI#^u*TD7umR-DnFiGrTo^f8Qb7tTNB!}2@3i2%4kAZn|iz+nK@ zu|WI%AS%wWhl~L-{Z98diLql*jL!IyY$4qhOTDt*5b84^W$p0*d1G#_O)nOID)C?wS?G#1p>vIG#A#3gFL;j3WRWA}cPK$aexA$Fd$&v18A z1_zb23sE{q(y8k?9H9$Zt#Vet;cX(FMa`T3h2h)YG5Jio=d5zNbEntTEy9B>TZ=Tj zmtG^b#Bx|POX}!+iSYp{maOGmD2XmhJJkLXbTGaM3@tPmUBxELvX>7{F#{QK_G+3X ziUkpV5<{vkYADr|N~O+(^HJ znMw>t^w!n1hrz79cy_E}y3{Dfk~(#=E}X3DUu`CVjB-9&ZH4jXXT==4Y)Ns-PCS!# z&Ko%sE{jTtKRntE*^~TQ9ds$wnG%Lv9>AP-=h`uf?`YClbbLGrEiKr*T37k_Wrsp7 z6^jlNB8N!?lFY>N6c8aAl+b51wD13-6zj!`d{dP?Nqn=k1JVvo^Z4rVigr#_W|45u zz$3pasT0LQa4bEV#NIxI7UC&evd(DJ{w?=s36GS^Gddq#+q4`xIZ%inrsz+WVaKVc zf_|@s^UcaxGhgEqCm-gS`95W~-5^0{#VMwelMfU2##7ESzDvkcr4~5w^8UH)5d9HN zwS41c2|}r?fIFQv`!fi|*4j&}KcQ-Q;*9+Rw7-5`Z@#Z6z`#P_GX13#!}zik!SQ84 zwUJbjL=sF;3*$gPmID_hB;KmbH%EqDm5u|3qV})+{8N2gbPz5lF)STUC-dmHO8e<} zzm`g>`Pf*9KfY+Ju{sNW{@NS~M0Pxod(8A(SyE`^mL)X1mlehxH{OGGps@cFH&2{) zXsD3N=qE_bO(LMgMk*EHs#1&`EAV_C(up~bjp6VY(yP^h$l~A-!{QZ7>PF{=G!wwj zf{)g6L{jH*j#-!uhDxMq2-pkcb3wil{!RNc5rDuN12)eus)j}f@K_Mh?%m=mnu-c> z^Wp^gw)1h z9~&~k0E45o!Gpcg`5=NH2SE_~{^M5hRltSf&J}ZUvRiAuB0CR!Xn`}503~9zNJT}u zZAI?nd)ZAIRpNf$DKs)x$TR56L82Pj5P5UkRVF3yVt4$D^Ykd4biC;?#iMiuK9b;* zVYV#j8PyrqU~%#nN8$_}cv-<{1fyj85g?)&xPM7zOVoU*XjAy}R6Far?RZS0&pC>9 zob+&vy}jgMhL+E0WcMbDw)wVqOqmsrts~?&X`r}6yqfSCh%~ajyRsTIOMfYxQc&Ib z@c0DmF(^=eoDn%W$)28Az}|{htNdRU0M145XCTwq0Zkq!^rhS$PC%sQ_UZVI`kwZX^_;rRi;E zegsIgLaMA~&h-zeiaQz~Dr*_DGYfHDYAx?TVlI`~H!mQfF?o7aa2YmHQ6{RKeJIYs zBk-Ji-Lq>PM87v68zEQ-l;-7N>6>m7;NASc6$Cr{6f$RZXxJX4d*-);ZhVgA!3y{P zkGGIp;=y69j5$d(0=`w&xkXNW8Y?qS-@V+#7(2n^5-Wi!5gMOkK zIEc>{q8hsAAe>XfR1wG z6TjP2Lv&0tw7Knf%dU$-QL{VW5ckY=8pjZz)c9TtRymX{$oQ8!C*22g+jZSt)YR;p zwVg>8ak?OK-X&$kmW4c~P>d)n{TFyg>7fc)pH^dDBI&odUE9@K=bSVB|6N$E^j zun@B+ekUfOz{tRw#U;}I5xm}2ZNDkfhKHph%J^E9t?mwEs}jeFWT;CYmp>QR#f3zx zTO_Z`t7hXJ!1iZ9joBoG59K={ZJ&OhGI_gkBH!;TL|@B;%IYk-3@0mI8Dv#tJYEeJ z6GbW%gY8d^WX?oSo~ep`2w0_3Ve}T8_n8A=;bqYdhPFPVcu=$DG^^`CroO89 zH;J-CRGrjWBb_{&#|5UU#Ill7oi^ZRdr=unXHavnvj5C>Wbi`=I`eT0W3YE0dluwn zVvLpGXu5f4ZbAWX$#+MrWm5rl68Zw}TT*1^08PQWq&QY##lFrw+(KMceu$_Ozkg_) z!#DU9Xyw`6MI+1LfH5b_f0Y|<*L(_dd}C*;V3Q)Pl{Wn-DfOwl}|?Ouj1V zJ)t9BTL!vVJ=KkBYDPH;r_NI8&)GYG^`wvxwQhVEity`LCQ4vP&x~Y z;%V`yv{C_pF=&nG#$}BtdQU(@+pvIU4j6A>+T#Gel#PP=tEv!pDh-aoV6! z#ynHO`b_X+H{9?jFRq8ZpE##Hu9P2Lc)_Xc8;5Be&QC*&wfB~A3hz|1oRL)V8J%?g zj2lU4H_$K>Hj?=k2N;5j1k+{7;ae8G-r-YX(ld}$i-5gRKUtMfn7oeBpNtx91j=J; z3pN%)u`&lz?zTOs`RZjEp^*ymY}*1B)~>0LBt!7j{N z`b>>v@$Uy|%%Y*s&s-d*^1ZwfJ7kJbc+OnlMMX2Hc~ojCpGU`=9~qC3!VA$44PGM0 z8Z$#>^51hlp!vr9tJ`X($9xtWuaKh9OMiS2(VF8OeGWRwjs*3(&X1e$?P@tk)96$N zh|jjeciJFPT|Xv%eG1v6`@7@0=uX5hJ>39^!Q=H(`cJ`+(@Eij!PX52Mm)4)b&^^* zs4BYvyxk`@BpTuWTVZsR;_&dPQ<~>QsL=tAb!M|Q-A#)*?_S;Pr?>L=|r%zPL)dp ziSfbAX+S;RTG?g3o<{14?lIoP!lO-2ovo$<%K{DvV_XQ#6d(@|;XHLk<=XA-oV7Tx zgK?_a!Q<_0?hc2*$ZACbWCw8;6kEYRn^#w7fh9@?G9#9T*cMUcJNV|w1jL3;g|P|h zx{x}QIjuT@jq(m30ZW-1W{!Z+0a~<_I8(guka#bBSft{QRXnmd3gWMb2%_8OD`{4! zDm0x`=H97&j(SG;#nY-P0@>wo0*zVqy5?5Kx(duuw2UB%#Hov0NxpR054&`Mv%XFDwqrGM~w& zrp`_Th%VFWxQk-_If;A1b3%G9-K46eYa}IJh3Jw|x`|0jduo)uC(oxqcc5{^fki)~ zz!MN`4)FAkoc;D#--drkIDV5s-hZ1G+2Ku8cI5aX&J{hzqTbpy32pR+Q zjRjJ%G8$Osi?Z!n>}qz4*RESQ3!hN5q}lwyPhneDPL_Lp*+{dwa6YHOZ@sH(U$ zIzBNqj6DHE>lq|#WRO<_os2b)E>(8OmvnF{K9rodI5bbKN0q7Eu-+je0fCzG7al|$ zw2)=(i+R+Jd8VP3&~RnP2#uT(bQIrd;yydtB|T(FR*sYJl!QMcMQ`;6t#N~DDwkjU zfTWzDmgkFV5pe>kJvE5JX@rJxvTvju4(oxJT+6mgXrhY%ts3VWLI)NtHM&}!QKVeJ z2q2;rt>R8jvC!vniIS2#J`}-;E)_Wl4y44{NWP-%7!aRm=IvU0e|0ti>;0df07T_< z;RK*TLhO4gSj~_1VQ+{eqr%Ghx@WXv!TDL&5v4L=hBlI@rK?}UR<8)?TDUT4yn_?V3NM8m~ZkQ`&qdUxT>#(RLO`+FIijI8DJENAHX&0msyv;-Dc0!A&c5HfY7`!=U8v-i0WC z7?{=yU9_P%2h}OW$(X}F6+KTNu~TnWge#Gv`;7?D5Jn_EGJnAi zN-}b-`~VE1dX2Zbqrx(U)?gm+Wmik%_c(= z3%)*}*mOT+{rK@0b+Fnw_%JM3?}}#(>gpEg6fzLVy7t)k3vY60A-(ZZu2-D86RCFz zLA6N7B+H-jJgD4EGHSK5lT*krn>ZaZje>;{93-cgHF@p{)_y7?K=#QQouy2EUsR+$ zpoSKz6sDMFcmT-}Wye9G#6ke*oKFb{0Zvj30FFpaiht_yk+rAa=V+a3A)%BSG3sW# zMP+^>QQO;8zwlOPnLyoAsCC>pTp3sknQz;vdnRir!X~_nJE2qB3n8jgZkZ zt&D_eo-O;rjkMm4il)@VNKQW}^4uqJUU~j4j>au5v>ddYoCi&gs~a%)9edkDCe*GO z#CX3`BOJ@4d5ArwotaMDGfr@98*RWXeShk$LCnjqpI6+YdSaGN{kuAHhW5AnM^7*> zKgI&|MBlhZ4$<7C>l*WWgU248iK5%~W4n4MC$@LElm;KQkA?RAH=`B8B)5`)aGeZs zT6@_~`;I>gimzPk%-=cmcwxrxIp0*sYm}(@X6DLHqg@sZ*@;s$q^UzHvu^ll{p;7_QL^ajj z4$z7lCiEqjZMh{hRU$aFohBgiqu-k?hWqT@S6SQ6u+}(BuZYQZ66>ca8tgHWfR5}g(*id`QPeJPmZ+#4=ltGUN%2e8o8}llZR0F77K&n+ z_6VlHU@Aw26Rg|V*w|7}57P=$Z>9N1d2ux4ZC6m3PyD_be7-yB_b+`)rQ^*EQyXUL zG$}NiQORE1bsFLt52%z5)m5*JUr#)g^=`damz7jY7N>oLtVhHzaj$uyv z$*2lFZz_vf&yQ0y&8h-ifrfJZV+WW8YLRLLnYM3xR_~cy`Nmq5U@&kQ=)w62epRX1 z`VNPpBk6`&Mn}?Rg@iPOqLNpbOrPYf{Ofw+fa24(Pcur@G64cK0dHnrPsy=` zr56>Q%rdGQbcvgAD!5|E(XmKdAwzhUDHN*F|k<9|hdPIcfwx*D}zr>DEx-HS#tXeeIw@y*U ztzzPfI`b35Eb-8fBr-J0csArJ3Mqtn7A-3*q(_Bl$PF9Tyyp2aM)6ka;Wi8x<=^Oo5=is9A8=Nr`CZe}OA+P7B0(uX}U$Bt*zAM3Z}Fowys z*lE^jVD=szp7VqzD5zQ1jn^C~+#J9jKN*z@|gtqr%R{!5I+mmMBqyHN6v zmj3hS4+h@8DlYa?;wv>vj3hpON`EvBp6BM~W@dh@tBbB5T<5H;%555&S@M>B)ZiOP z=QzNWb};(M21Z}~D(@_Kb|Kb+TX|)9&*o_17Rk*O%X^#uY`#tJQu;Vm{QikIu_ zF-WufqSEG&K!IByKYp}WQNO!V=0vI9-qF$cc;8HG#kPl0OieR_n)@k8J;NLIR2%Q(g+F0PALh&LvaB@A?Oc#N)GJvZEzHR_f5J&% z(f4mH^+f59SU&Q-!OKS&IPU(=_gb~q=I9zF!&2caPEq~G)P|TzHPzLPX1~IelZ=s| zAj-5B#fNFjT7Iwh$vj+d;~ybA#7z7E1GSvI)d+r=VVg=4Q~Dc@QGbX8PWBiSbbDj= z=jE>@S4{5GwS`%aJGXrOYG!I$H}hH}W%y*eaW)2GQw${aI=)(7_Ka@~?wNHIxc8~u z;cBu&Th~&vp-Z>_{SMp6x2@ReoDz|<{WD9en^`$jt0fwKVTdoLvtTaSRm>^s zzvj>->zy9*?%lib-s0HU*z&%KH$9IZKi;;}7(+>CC8T%k=q|KB;p3xB6Ui5S?_R9r zX_l_^!a_F;7|5i5C#`H()B}UyD+3?qMh(r)D=;1k!`I*ViwFy|va(WgoyTm_*!%ZQ zBpFvQ_--RtL)5PYzJC4sk=zAphf0oyNWZhn z{gO@`47OA1JB0UzbK}h(zzMA#;2bqrjfE9J*AM4p$&`oI^;37}^6+)xVsc~RYw7+3 zDSe;AM~=|#-)WrP0cRB5W5=jT$?b~k60G0%M^pg;8Wb!GU9ghfh4U`!u0esrYyb2^yp!OfY3dIgM(fR6MG)%`kS9UcMgN> zWaZ?%VhAg>zVhOV3T6Fu{p=3BzTsVC9U{&azf@x%Kjuj|)|tIZZfhx6rm3YRA||%! zbEU>%4LlkCqW*U@5ESC?4lTWr^3l-{ESuf4^xAJ}l*+BNaH#cEEPcsG_Nj>wW=d=@ zXy3@oF%SX^M`^J;acIud$HKxQS-`}|s1R=~nZ?1>)Kpzv9ryL~@|>{~)Q_tkIIt4T z?L_msnw5Qv={ksylZM|XaoACw@{K5**iFB zG`(3*xTPAH?_!c;9CKu_sk{3k##%ZQ^(1!h_RRP+RfVyFm@~G}aJ^+`d~nmy&Ye5y znV3#xcOVxDMIHA2P~~&``<=?H%uGeMVdfy_LzO(g87#9mB{e-Y&d#lv?|DP(PJ^*5 zt0x96S}Ed=P`PvNb$V=(JGn1*s;RbFSS%0L7iOg{=V`ox-ndj)_>LEKV!$AkF;xfe znVN zO)z?NndM@D#ZdB$kfyu4I|dRa&-S0PATOUiFM7h%bad~%5iJXgZ=u_Ek{^Hi^t_&Q zGmo_X^&JutqZdUVMMYhyX57n=dDQaW@ph5nUo+9yN{xnCQtLDH;^X4uDavQBMjd<| z`eBek)34 zmT+c<%q!_POS|ApVJ>LX5dHJIj~{6~$BrI5R{0~C+p|I%)_wTwPuM)RIq<_+ZdE0Q zAuoLMl9rZ^NX<;)`|M0fWDVO)j>y?O3_(V%MGQm4SV6SH5g_FCsw=HIs7qb^k_udw z_m-8Hw@A=0v(s*}s~&Vok*G>;-gyGch$&6my_!#`=k<C34zZ{|_dKY3ECO+NZ?q-^WV9{q?#V))`d-7W>EuNa_1cC2jo zAI{3w>YBFWzGRPd*%WUpPbN}`y-dOB>@;(2^eY>lTgI$eS2c4q;#%w&%07OLosNbx z5@z>MnJ;A6@7Ih?K9N;=nWiJjef?-O-TdhRE&l4K86itzx$gPU^7{9H$}>B~NFoZ~n_0 z!Ys6;xV80Opk2YBzkmPI-8?(qQ$)`$ z^;q^ifI9jAaxh0Q8ELj{6p0z5_sK~%vfY<`SIz{!Wd@F^pGbZx0(LI9<_TF#>`ATT zXKsgmSF>ham94$QDTisnmHS+OtI@2fY_DM8kDPdO#q7d`p_GFmLPA0Tad8$5aLt5u zYZ4|M6C=f8*E4H2=5X=inW`H;1_mt;cb{XEwot-AwnVval@8@)WnXeFt1f@9eDDBf znLpjjb)t>LSus8Jt?EJc2mzXEBTsR0aT+e~ErCll>TR=XHL>)=E@Tp|{cfdxu7FN5 zFb7$J{_yWyr)5*Zfd$AB&lK3|9Xb^9*2-hH<a#bmn(A zOqGR2L@;+zod&ChVW5~VG|0!78r%22U(Miti{0aWuA=J96_V1Pr>$?L+9|}tO|%L* z==dYq!Pd=lOG8Pj9#j2rTfwMD`R0RlBY*ywy*yj<>7E!KDI61ZkuQmX(HJeO-J^M{ zhml+1_1u>;wguM#UGaF2l&;H$Uhuyfa+ExpPs6eyC5ms>Q~3+;KQwkWGc$9sKadIk z_LB|=!prQ;<%%M}RoepGokRS(JpJoatj*HJU;a5n@uk0ok57t5Lp6sFa}@vebQI$C z(hbcgUNY`mr#T^Rq%y z)_;{6pq6u+z(#EHZetntgNF{)Vxm~ST}vu3&LVKgcbJ}GvKTurug}ESeG#=YnA42t zBv4?XD_}xMNDgY**a%N^4Ie)jY>fk9dgcs9Oy62a#G%!4|8wCSjrG;5S3NzK)5{*k z#>A{GHTfEgvtlYNtmGs;BOV_Uqw~kF!2B25029oeC8k4TGWErx9%7=|uTS@w-!T`# zUSk6j^DvM0dvo&?ye8IRZL|J6{<5=s$I+VtG((LK?y_5B32cr0bm_+3SM-@|enAz7 z`Lx)jmKGK_*5*24@3sa9P7v(}n=Fw3;ll+-MJ6UD{?Vw-mYN1UEUstqGN7%pKK*6j z?`(k&oajAIG0y#;767wn=mf8y-lXs=*(7Go@s8|Svvh4GjfN62H(pj%R#ujjcwoCIKC`oPevWp7oRJK}_Nc?3AMXovF-a^|j^@*T{Rx$9 zYG$%JB-AdhfH_B9#D)|nFsOGC&fi)FurA3 zceO6&^)uK;wv5dYkET}OVMV@U=gwZtC;#0T-`3h{;W?5na`XXiwpoN(AwFE$p8(8> zD*E7_&z6*3Kgdz)2m;-2)URNfu&>h8^)tlluvjOe)Qy>9ZLFbSQo{IHl;58k8sH&v z?!%o=pFbZvdUQL@&(Z8$U~*t*qzX2zD|PpG;194Y54l|OgF4t6HcH#$GIC3PhNARf zE+Y96-?2I6AZ(ACV4b-Y4#3(2S3AuYK5mcj#!tEaZYagP%DRC53S(R6@d5H7Cd zRfl2FrDIRw2izeo)b6u$$8dI{fx6oH9p{lZ1VDN5L+RkMZK3*a%{-CW5vq zmX>%Oh`%aIv1-AVaSc}=?uqIFAa7r%ofG=sgUWL;)+ILcvuxdZ z#?(|ha96M2lTD$iz*s5IrGqQH7H+@?1_iYb5&%qiJ=df(wY5zp*{Y7Qa&wn^FHIv( z$LnU~<>ftm_^_~0e&gV=TRq%PC4y2?P8isY8e-?JU2{v`U|n)Mb4`mJe!RQ6SUY?c z3Yg)uy2pdHJ-f=UO}3;RI+bI3^MCxU>vQ8VM9>Y4-Uk{Ytn$qAto%)LCep_+9Dfia ziV0yfjw%z6qE&J4h0l*~7bOx#)#U#jFHL+!Hg@*3*R(?-l9G4Mceg~Ic1<>BVtyCK z3NJVUxOn^cvU2L z3z(ryDy{q}a41?vP%H=Lm*>*!zdTBOTUa89h>BKD*X^XGrHzS+!4&g{NV@Z{hM7Jt zZm|?cL=$$qdbLG<4JiQ-FWA~HZu8$G2KU=W;7GcGhvf)XAI!!i7({UI<1^gQI+2a1;kLjjXA4(W+Ew+O=g+<2SY;P?V|{7pgrOmL z&TniKj%;7$zl=_5eX0w@J~)prxMvW+L7M5 zxupt&p(Q1sUpRTZg0%XaRY0GEyvn`*uSateUF#N_fPTee5;3X1Yq_^Re9*2EAou++ z?tV1ga09x*U)L>|gC-`N?%#+O)`3icpZ~H#{3e?@LK$h)K6pLMb1OIZ&e-}0-UIGr zMGxzNVAsRMLxC`dAMEA|_j;Rr%aT3U?5xbgAJk1VQnB=#;{S+YOz=42BH{ew9qPyZ zf`WpXlV@QHhq{7Ma1~KT^myW({1B_lVf3V00(k^Bz(jdrycZ=luoM0ql|xpa|M5Mk zVqWquJGjwR)MxWeGg=0k^>eb1bWx}i+;Ql7O#PrTJ(%-q8JAL?7?4V^YBUm$I?NF~ zl64?iefcD{(a&PrLn6!?z7$iw=jn4hQCKw}i_9bw)Ra3|#o|93o_K^(Hul!-+m95l z{48}HTK3(I`+j`_w{+q z(dJR|aE^>P@GF_DK=MF41wei5!{y~=cny8Alechf+;dR*fh7qJ0ABt6n*e1VL`E_j zy*pG!5JJe8Gt<+}(FUhacSmhM+ezh}adWqr7e-BkbCQ4GRwZl_xzYVWn?2KPSL+3_PRGa13-cT?F9o{;J zi!9}(rAvUXk0E+!! zDh5VtYinDm?fN|}82VEthL44XCBL9Rx}To^K0mIRqCUP6`vkxGjl##wtc{gI6Uo^3 z>UPPsFhn@^)U~J3p51qto}%ygCV5)|y5B@OB&d4ER|m8UZjL_Hiw~~Ywahd<&7CzC z0h zu4NBwle9kj6jSPS(pxLJR=g|Mk3ZeTMo+R*Q#ls)GB}1 zmU;a_1+`>oYVY`i`a$(3lq(Unu8UJYa2qdiWnnMqJ6IR2Esy2C^zZ#>yLg0&J>ruM zF(({W($h0Dsz{SF%QK1zpVy@ip3X6LzU_FG+Kt$v|5d6x3InfL;v0h%j5%@@C56TJ zag-oyz%c|YXd1h5r`6%7$N^%rF!S>AqJE6ubCGTS6$kUjALN$18Egw%T3ho`6#@`; z5g(I;V<8sO{ae~?3vdq=3xXbMqSE#JtSp_h793j3aOX-gA$cjqWI8UpX2=Yl%>YFmaLFd!UF;KtG!Kt(_6>C-HA>}kPMiksr07-MV( zCPPc>=8^O`cm?O)n9IfwZ8LxV>Xkxs#K<*d8nEn+FHMBDZ%18Fi8M6Bw^tr`d&Wa) zZ=YM+^)ny^*9WD2QNpfo5B&+6$kO|74KP4&6CYnmmVf$sC{a)z<6C_pDs@&dzk+!EOofrQOcxflR zpWXVJH5j0qCO{^>5t!9|e*#bKr$LuTx}vf5CnZigIzHD=`GK1yBgx9jehBA9=8;88T$&le#e(*l zAI?vnK1~}3x?PzlM@x@}#LUmnzjbQ?{t?uVL=~0fyM7%O^ms31q{hZZlw)^096<2G zy$Mwu{&#)1FP|hrw@F`dP#r zIe+ZfF*7qX2~e5CMrS+C*}p$)_9slZFvO>d=XP=zK0^Zs)`8KB7cE(+NrK2)%=J!N^G<9Efv zw{n1TG7>P_`cMsAd<6P_z*@WvUSG>5?kp)!sZ~`1s#fS_1 z!3%5N@|pkouPFF(f)hT2x?Nc3S;r{0#nNy~+9|UuT1_eQN=|ll_4UY>uN^uhZfmln zcB9J+Ua{fIS3qFn=O$!=kMS$ON*+s6QMPY=0wn0UzPbdadEF!)-wl?7=) zIIVYECb79tws760mhNYwA=ChOjtR5qfXq01RtOUv6el-sot;) zBcr4H-G-r<;5y*$i01pjgVpbD(i2?5qhOTV`Xi|PF$E;Y*y~aFfd)Yxmc9lGbj=Qn zas3jsrcR0)ac;qrG&c*x))TG!PjZBplU$P#oU0!1Yjac}d{D1kxq@OAzE+(RSQ?b? zyMff7QZjRM(~}n00F;m~(xTC`h6jLxkkC&MHir~gNu;<8gC2Kyog%*w4Uwore4jpk zJbd`@Vm1G|4CcckGo<+NsowbNR7^o49I3%{r*zt_?7ItgY=Mk~07~EY2X-CqrDk#$ zJ`3wig8=~(v+u@giIIesHPJ(a+zl+y>Nqj@z2nQ5N;qVEoE%7xwUCgO-pj%8>C-2; z>DoCtfq30uBu!mlsX-$^&Je5*Y%s(Wyg8VWQ*bu-22BMrsXUaoH@ua7%Fs~OXW7kB z*4>&w6r4q<*b&`WZhto_RV^_iLOc*7nu7az!U2vD=P+@8XG|Ti_cNns^?R*Yqbk_V z&CS`_vM$-E85mg_GZ|*JkIe<=-F|b2BRIEO%icb#<}A8yU#Wsz?$-~3^s+4&?=HB2 z-L*cw``iaf^D;LWZX$ZbgIk>*tgl-988$lVaPQOo=;#1SqjnbaC?dvnCbs0fA9NAS zeFpqDi1EtmC$WRTNE53F_sF!gG%9ruyv&wWGu9fIJ<#o0!RnylSz+~-d_%<&t>q4(MOpEK-9a>pV`GFs`{>n^B(Bw?Y+9c4`wLeJqGTA zBqXd!NTh=9Rd@k$G6cVW|9)d(V?D{Yc;~4XX0JV7`o1BN9_Moea$awi&oRy>5UX(H zL^C|g;nj&2H*-V`-jig#ww9Ll`3uPcNj88&!F1}(4Ysa`2U8bKjT^eU4LF_DB#)KxVzgjpK!+yozbXN$3V~et zUd%VwCS{l>SB<~wMRhliUWmcdHp0nf*fq~eSHCJ%N= zORKsK>)YFpVB3$4ja^az2NwsTYlpJ@-o0=#80vo;xH)9Iipn~u(nNs7{L2nlQexvY z#;>(jzrVdT7J%%+XP(&-JK0AL4i1lxJw;UbOyz$q8sKRAp>VX;wGJJEX#JC&{5;*s z`g})3T$JX}8ky#NMza{goliD_&bNcP*}~4FYk?PV`QpW+y1Hhs9J{ep0Q;{kCPVmd zbTv0S6WANOKDNV+{q|;+q}V{DylTwd=SWFW2I`t3#*Kp z@`H`6I?ZIsK={IU-y~}eoYVFb$Mq8S&tV#jX4F-j{#hM!)b^-D=V2ZWqb+I5FWQ=# zt3#?A>ZclPWbbnCh5s&g`m0{@Gg4+H&M2^f|Gt%9%Qchw{_0Y^YX9A;10*YmP2gn2 z=qW&YgJ>+O`;gi54WS4MD21~TSdj%}2umc8p}P?s-QAyNUTdduHbx|}Q$PjDpaNiz zF47`eDJBmZ>*`Dk?F`V60a@Q34$1pASPM5N8E~PEFNF(Qk;`bCk6va1+xF zY*C(CVquV_6ykGW&<6YBY0rvIn5p7Rsi0zsj%EQw*|KG~)7KJkhA8;wjrh?`BpjX4 z7ZFzeH8{A-ylfU54P-~Af1eY+^<=SApD8qk9UbZV;u*&hJP}2QhK3^EUA`+G$`>If zB61!l4dn)i7lZ7M%^?G5_$zxaPF*VQdj(KMZ_>`gjLA5{EJ;nBNBcPjTMyncfnpbg zI+pe=50mX-jz=%~NF;f^DkLquzBROw5Gerp8L0{TGp~%|Rj@PEq$w{EGoQH*exF&W z30lzB06mBH&dSd)X}`(aYR16`|W5p{PqW5@#pJ2R|o0G$c-viX2!rBqudks^}cfD z@A}%xUF~yDP7~q`_315kM6()|BT|-I!THnkFsWW$ooRw`wGs^>;~Y?o%h(q_c5>pc zhxU?>24181)vF0;22j)kb9sPV!gIeyE#LCzMKGu~lnaP!wj3Ern^{@4^d2j2 z@nSS1ef)9uzH0ut;2^-B=O!;9n_}Iz4XjIKYUb+d>dqG>8sK9liCzm~c!;!*I$#RM zp9lsp8U+hDobuU57LW({1ByAJjYR_cP80#a{?iC9wJH=d#7R>KeaXKHDiUZSZbHW3m^$Ktn>3;_B*(JhR(p zc@~%s>km2xwOV76D*q!6-AA(L(G?nI8~vvzK9W8fqwAlV35*0_j1^wTLAetahQoku zkon>T3bs|mQ=(U;WT7W|;>;QDAdIs^>xNJuPzR>u=er=Oqh?8zv3^GYwnSxaPR`So zmQ_~a76eJ$C31Dfaz|a zMZ&Ef4F(!(7nkXir_A>tjIBSGR%a&cFA2+S^e72BCE-|wMN(|+UYzMlGZ}CvACo~9 z11ooo%@N`%{RpbSC0bN)oEkI%a6ZGw^0fyN)L*co=tCHrn0Vj~!iFZ(nJ4HvPUgE8Z%_y>~S_&=^7i7*Y*({u<@{WvT=Of`-hI=2|I8R+^)mZ-9>I zkW;Qz+s$=Xx^nTS2oJ0dIoI1eJ1@gv`**`5XgC=7i8xQQLt1(Rg&!JmH*Tzh@x0I& zcXf5~FXGiNM9(MUqoF%re5|iOfGQHeMFyS3v=($>Ir#Z=_0c)|C)z6R9d|GHnz$$Y zX?WN!G_?}p14n&+e!e<{sosV#2Zqp>(7}L@MWYxjOUOS)2Z_R==mwg;kfRIS0%?Q} z3glUkB!BxxU|KMB;Elg@bi9=0W@mreId)J|dgJG?{cp9lSrBFfDusjs++}iivX9Dl zyiJfrdhw0!&{7Z%@8a9~lD9zbmUhFvamKs3KER5D`heLT>fp=4|J|CFRceZGT(D}8 zX2pFHPKn3IhHxkVaOWrb;u900;J==@u;dM}lb=6L0IMR04q#Ldw9%2~L`6h6L9f0| z?i|A*-bQ!R0wVQXCWZ(!cMHLH`4SmnGtaGCw>~yDR=^#SUD5=_|9RPNgv2typ=e?B z@U;v;eK?_i_QJgUSi*jhsKa(~m4gq?5G&Zyf<8Z5^zcU~41)|$Bqr^1wZ#EcYLYg| zvqH-W%o_TgIAKgZU5*Oam*;T4fl(Uc_CTmWD{bg*M-vDEVk4>IK*YtLU-_t2#SE{)F@NOkm>j;9ux_8RUD4Y0UI zw&gUXy4n|J0YNX>d)GHS9Gs-Y)3?#oFRe#4Me52nQS#b;emq2QN8kw7fu%7Ai%>9O zA0y{~ZEkM9QR$$m`5txsz`#I!d^{TLh;T?k`p@2F8l)-md)9Ud6X2 zJakW=2I(3Y5P&#~oCJGsw4~NiToA+<2RJJQv=u=-;>Qq-@#`7*lrVrRE0;f?oSZ!I z`ftw5moH;cAhp}r%p?f=f_C+7#bSpyN$Gd`C`0j)20&A#C4e; ziqHr7h@_cV8A>M zQtQf3&0y;u%UuQCe*_N(GjPg2S!1IjS4SVh(2ym30Kr~aomrA?r;?JAu<(0Fg#r`F zv8UIsUvG@xn~OWHJ9lPhWZ0rkdX=&%$CyxSY}u~uws5MwPhW~Qwtn1-7dJVgqW+?| z_|nQ#akEb?4Yc)M11Ck5=eJ%tiYb9FO!@l1#q?sa*9ny3OJ@-6Z{(@W(BcqSLx;3C! z<|wMeQhP;yQ)128BGq-gfn4))1xd+2|Iu~|91Lbsoz>MSd{rP_MV9ap8iZ8@3SW4- z-m4uz`Bv1}+4WZn3k?PZG8NP_AV;LYEbrUb4L~1$h@t&?icdd9?G==%#@XYt5-S=v z`S#OLr+DZc-%Q(8<0mCYi!bk{mSc=x{t8zg&{>du^!)ha_Y0~rE9?v=P_fpW9dRpL zIlrU8TNC?uni>Hs)fWh9UfqA6uB2`$tCXb9>hlPFY&|B6`ox&W?5F(%*as zK6x#@As_L9P?6&FllxnNWdKz>KRvyieJP@xCwxHkUY1Y#amY1)IHMGUv}E>Lih?+# z$Zj~gxZ?QgtqeoiDaV``z~g7s6^;t!1M`ox(_UWOOXn#l{_iQj2AbwrymCBkYx|pM<%8HOPx`#sge^=fx3kxa zmDJ@AmP}t96{jOAi)6jPw6t>?|31gV$JLP{aFGN71_c^2CUlu9D6>&|pl%05PFM=U z>qpxNls+~vf^Q?lsUIT+!I$?bPahMa`3ZGWv5916k%SBMKTval%h3{NO}#BgQ?_2L zO=hB?-~Xm=c`0)+uf@)$pj%wV+Q+fVxd{_fZl_>J~A_nb#NRv@Ag>%Y%_*Ol_g;JoW`0{sTl|5W~c6P&v zS7>VcczfeA&HhKHjd3>y=pHF;kAOfw3}m^X3keB1H2LYs8KfjY9VFJUo~Yz0vTB*k ziksSL)*KmY9V%J|fz*r=l9HO8Q@6&8D&}Mp8*R5z`cLyVi(-ZCtC!Cqh(`(@)TgH0 zA8LAzbC@m<>MD8p*~CpI_pSZkk=X@_vr{mZJbPAWB{Q|;-CL1oA_-)U`y8GXI91)$ zdtgdla~7y}VWrhBF1Wc`t=a1$?naY_t#PbCz>W4-zJK!)%n7RKOQ;CZU>>Rb<_OTd z50Doe6QWs}{njx#jg};B7 zvd^>c;bIIQ*qGc{t3umhX=&-UW~vj{=y!zpfr*LA^Yv^Ozml2vUhc{RbQRyT=jyv4 zTJgCzYYq9##&TPqIYp0}(h!mH>sLpg4tFZnBM|oV zrD?a&{h2Uhg4elmHZKp)bFUCS5f<7W-j^nhdfoy$`GBHeXmJN`A#6K019hp{w-Z%D zR(pBnO^O1VLDw?SBqk=p6m@u7C()+`G#gO~LUn?)Fi}4E4OG`is|Y0gdP(0^-I^m9 z{nqsanC}3!W_K*Ftn_+Gnu`huSb=?HsP^#k+Nk&)3!NwOn_^PawJTRfZtgIEGy>@^ zO;_}yoWkYHlOV3y+1c;kcCIS!{Xo;BFPhWx)D{#JoI7`Js42;}iWdi|V(Juanb~B5Gof$4so`(Bwfj6) zR;D>pxZCI2;>rUco2e;hH;WFRGwM4^cu##J=WzQ%Nznyz6`z%#Mv6Bz&}9BOGSV%& zQ(c@jI4DTzqZqun|d zFEH`y+9{8OoX21{(YNC^*K6?lt=-Z9OH|V{$1*|t`3rH zVV`UQT;5TrPd0kmIXFDa%tYzr-i=}Hq~<=3U_qVyY_9&82M_p^JY;ZP20fOMHNCn0 zOm({C&0(f;Bx^Y@@^WteVxRXix+(f4<>hBCnxs-SWJIUO%v}#nb>=Dll2{0pF9r>y z8zN?;rlej4bLDkLkZH;<`3trg$%v9H?~!I* z5Qk8{dE0-a>M718ZLD4}Hf9^W;BRVlOH&90 z7Se+CL*s13$S-FjZ&~Du>9l$>M!aJjet7%aB1C%$O4nJ+xsTXw-C{Sx5K$|~b@xkH z^*nu7pNUM_wY%>)yZVNy-HQ(g?V1(liUtDHNO`lq_l19tgCQ}p4sqbNv9{M=@E-_a z+!{@5w9Bz=S4iXX6KU2m@!9OUt&2p{NJtnFBMb`-rO+>&exdx7-u{JA9h zt}M4H$;pzMF1$3shN%9LeC&~8_^Fd8msVEJoZzsF!&DyB-jZzbld^s1<+)nufZgbB z2K>7#nWBGdeK@&bCH3s`3mz=fvcNNM;wvGs$0Js7J0d@YqjG?tK{ zTc@1mj)Wqe)L+#szZbC2i|NTwX2&QMJsUISOQZ%qsqM$xcFPm<7 zeg4mB1f{MnLf<{0_@Z-+iu^4E4Cvgf%u2K7^O1%dHH8ce4Wo;TuK{T#B_(MJjreQq z^NGS8x)!_FLr3NH%1cUkRJk>y(@dmUC+ONm?_r(aehm#TEhzyy_nNm5t4m$Ln9eY}^*od=Fj;0set>$!)YfYxsF~ zHhS3Tbt9?=Tbr4gp|t3k-IgYf31Lyf_T%MFyeEMzaFq_tpy}x&y1ERTH)~Usp?g4F zjsuL!>Tq~z!o$P!XJP^=8~H{c=7_<*7ybSH?d|QjV2pcpSS`LpjaFWPF)~s)E}o)+ zj}IUhZfV)iIK&5Xl!%==bqZbWCJIwmS785%433>~lyA?z+`sN>ReK{H`56#QSObMXNp#Z_K;-q`9NUAD?br%pXhPv=$fF3HS1oZE@sW7EUk1E0i_ z00*DHc%cFZ1z=9(SKL((9{{q#;xSY!O@G`@{I=}u6@(WovUWI6cap4;ARg5V00}ic zemA%ps7jRhCZU9Txsbdpv3Kv@9OE;FhOO%;H$)FuEcKcnJsPGdw1o~A6F)CM(HDPz z<#ka}M>qwA`HKZx<-_T%;6v7p(zr1gZt^7_5yYJ5U|kGuyF4!_aAm*Y0+AlU^zbQ( zv(llEocR6QeWW!Vcc#yA%tJSHtLAJWmjpG`o6q+$pa-=u`Mv5w6CaX+^AC+pn?i^x ze7~DG>MvQ@X^^ETZSUN=MO>Q!;x`!n0knuQ1|TI+y?ttI#03d0{@$DA=dLA`>X4I$ zP5)~>z-Sjg_&h7?imPk9(sk$nr}M0<|0ow$f`^CL33q_zr6E^>$SaZGP#RaqaU1`u z=&WDBG$fa)O=~)njLN7*aTV3ik5ujmou-=VJOs^`aOdM?E#H|+@6!1AGwg{3*>kQo z+z0dr)Vz2LlKb^eNyHGOGALShHZ}#%S-V2fXWE|!U0P__v<$#kcB9XQq^07sjQa47 z_Bq@SzHvk4+1hDHCC;2Fhb$A=IBj1CHYOmgMndauhJ!nbJS)CADqQQBn5AY5qk@cO zScf2E(HDW^IA<`Ytj$O0!GbsQjw1X0`J<#(N4mQ8rHOY_01Y-R#gaIp&ad(DU95DF zY|gdbul@m4Tx5D_vc>;JyO)?*P*70U`Enu;Re;D66Bd4MR@z!!op$l~qQuUfxTt^> zbmF-39@_>Ewy>+9iP0tuO@%18t*s4b-XNOl+5T{epmQ9?xQc}30pGcIZ!z8~;xlR) zun@orB_)as)nFu_N!@nXFQ6EH=ILarsb%H}SFp9b9wH+*Lok{FFLs@4sV0)WG8Pbh zCEa+1OZXQ-rU7BT3gMk91_qu#ZEss}I=7SQ7e#_b3I}jXL~Z$vwX!`q)HnG6)O~8H zJ^9)_@j%VFVC+#F^F?$H?u}!m(>ix9Eh)+J@L?J)17HO76?RHX^Q(A!96ef(egG&F z^yP5jMt?{y@U34@&LF=2uU~igl=tHnJ$59H9HlyOWg9|O*+&c=9GzoxqW5lRjCg(9 zdoZDO8y(g18~9Wl4B2d7)Pv~&i%Uya$DQVdKfa<$4gs6efE&X|)Eyl~{P&-YD+g&< zuMlQ|hzM(rsb4KyySi?zr*|JK9CB6!(s`oVe0la8Eju1WZSP2>s58n#0AY0Jddse6 zqo~AxKp>5dKWb%H$nb~*30;l69>$3z66P6*F8EHt)+p{=O1;5HAm$dDN##C!cJ`$S zFg9MQqXPG!Z1Nd3^imS92|z(9!lGkgk%;ndY!1ZZk>kfv5)W@`+%v9BNm>@mF+OTw zv@%KopmAycw4W5jK>BuF4~(-1XO_m3_!YUhxzV_w4amz`Xc%;{k9(D~PFnW=cHjrZ zk$&dMN6&giq~=a9EF|r}I*P!67J1k}Np9{W_$YKETZo3x$HNsC8|Z1Tf8SW+PilhV z0Vf&b0B|e)9EY|f8^$z%s{;LlqQ}?Qw>P5ZikU})ppg;&9$`^ynOi=tvNL0 z%RT3wX{YJv>bhOKHo3kwp{+kg=Jz7K!vWX#q5FYESPq>T2yCxj3gdTHRSYjqQuiBW zTqxjD_SHFfkb+LR%OV!YsGyr6C1iaQpaftjN3f1?G`dTX!uukRUQ!iE(I*fhQYact zKNO*PhRW@U;f`Qyq%D*&kQNc`dlwfL8cGnE#EpF3z1VBeMiV&35FG?l2s7T|V4ldPmWI+uQ3T=eku-N0Wquk_vrA_tfcz2f1qrB-H7QgSu` zl+F3`XgDezIB@1^+fAfBDk}BCk9_B3A3-V_F9z9gSPfTR%BEmz{Vx>Z1E8@K;xD~4 z>G3sBJ|6gAc^`RL7=p?VG7J-b$}~8`C_DAy6>`>&0$Tnz2t-(ig)$1XI1f+5(99Jc zetElQD9*lo{fhRZ>i4p%JNNI8s~sqM^~%`D2>L)w17o-JzJl+{5qwp~Ur12!2b5H0 zIIJjQbg?RKh4=2A*gGLXN{#@W1I=v%I0Xq#NHwR5_ae47E(al{2{h)^W*SMP8+QL5 zrSBVmD(+x)OzMOK}!)%DBxDEgZy0EwI89fu4MdzIbX{+NGW0Sgm2>HRs{RRO(?H-0KE?x-RY1NZ#f}eGz5=7-^NnIeE4fsZ#3U{(EWBq=Ow&;DWm_`OI|T(ZC>w|y zjkqvNPf5CyyRFAC>R{N{r=5VeD3LdF517W-xXkb>irk^J5t~mo+24oZ1nZYTx6ikj7+` z4Vm%shr8Te75RMBE&~x{7yW*35PPZ(LlFp-En+-c%5YTQj)7LTxF$Bct14LnQPgErWSQ|s#aKLIj-U(sSft{PeI3?cG2bjcB_2< z7jFz=&%2E{l{>C5s55(3@cFpJZ)>oXqhn51^Q)@z#U0x)5ebxyHmInu@RWzr?44~|2H@nr47G?MXbpJS*x1BlcvA3RkJs~z z>S~soc%|O>p~(cfPKJ>L+8Sh6J4wR}xGS25;M^3q=OT6&XC9CX(Cp)YPA7 z3K0r&bi}b72y>&)C_LlQf`AeuEW_T?(()$6FHX2O)}d+Q{*iDBZtR+AvmJ@JC9)Vdw0l%{d)z@VRXdaP$Z9~Hw9WNFP&%?c2r#_6d51k)-sGEV_ z#|w*!WfU6b`7MaUD64dFHv~mY!ec4BmnP^Hy~1C&dK-XdHF_90{x%mbT>IovMu+z; z`boLmh&1i4q#E$F!*JR>@Gp8pti1s__`r=BgS^YBd_fZJcc zlzyK0F**4Udh27b6@tmd$U4vs87Hg(cxXXi2h<>=z=)!#pn#U_M^Re@Oi{{Q?%v+e zP)?&zUo_@xXl#62X0aK3bHQ_@l!6lUgz^EBGql!RPM0nS5uSnUq&582JK6+UZf48F^)@1lk@!8dEDwa`T!}Z%7Ow1ROI~z zee(zZ3mE#d?+mSQk_$X5SUqT2);2aWA@|{pRlq<@N8JJ&IgCKq>Yn&xSmBru8n8dW zu*_%SyBTZ2!43u#;?wbispqp|w7ifiSEc3C&XV=W1yoaJmW-Sj46#g1&>&KDd_zBvYX1b1KGbdqCk5nA zWH;2n(YHV}#!6n$L!Ji4_aCd1=Z%d~(qvqQAHbfa0BU_LxgO>BH>4_mwzuDwo{@y( z@Wa|+;8(DX5cP8~WkMGQ)(Ik#+S*#II3V1#HNWv|RJa2QeqSq{hvbId!0Zh!6tk=} zA_%a9@TL2JGey}03OhYr0csuV&_gx@rIM6_hK!6%#Ik1Km#G-F7qF@O*Z&@qmk-2X zX`t{;-O6elngEOjoXC4j=z_4k7Hjce0rNimtwfnWqEv7=v4~pA!X)ZiONYEAiByP$ zpz?m?CdLY)E9LIp8{`Vt{`fHsRan`e!eO#k-KhzM<`GyEpuob`_pBX*zRl|SwegEG zv$B9q08-!w<}`OJyQ#s89Ut_du?w^ceIS;hIXkVdzMzT;7ymNri-v6c}y6zSNs3|2AOH*yyN>6#ITe zWj8b+hY!4%%QMz62=B@e9fyYo{Dbz%D*4T6{Ec}2u5TZGvA+$;*yEuENb8U)V5)Tg zyp>hZTtDP&>U6jM>gP_|PSlR#vdM7-* z2Ivzc9{`WJ8SX$Of~UXL*F!C0wiQ328OI=k$pC|4f)*bD6yi#Sfbmwu z`LQ}5qB{-$UyE2NaCtZaf!h(k9}GoEc7P4(riuu&h1~(LH`tGVF*2fbYVpmiZ6S<{ z_;tflVO?f)H$D9~4l^&=43tGjbf9D^h_1N&q3R1zfkMHO+S+4cVqbyCp;fanH)mxk zCkx6{=Rt@7wT5BPjLtQzF09`z%x^}<#=l#Gn4n6mKXKJmz*dDc|B`4RdyKitG6VeD=SA24er)p`km;$N*g~X~Vb*k{LK_Ak;B) zed!&r+wlzI%pyi10ip%!w+WEw@87>z3aTzH!uMZ2N8_kzPep3PJvJM<^!yNf-xV%B+$4(s4_bo5Nb=U-r-fItt2=@Q4VPwRoVUk!`+gd&}i3 z%=LyU2c2$hNmt<~(FdnaSS(FL@5COBNvpNF8CVKB{C)fO-Mo2oGGM6;gw>dv^s}?$ zqoa!`0jsNS>D8|;Jhig4Y_+XGBZ7-c*P)&!J;?~RpXg1){7%effp}*~%F4{_2X+;# z>Nczwq}YLx1yE{QrP?Lk24-O6nz60M6pP*ppkI95DgF`JXc3|P5 z{(ufS=;K{YG3v;Oh*5OK=sjUmYqr+X)rFE-T}vw*4JoKR;<<9~-e-KjGczGJtS5({ z?wEko4DZvz!UAf1Jc(1(T3mEvO+odzHC#9eX;F{|g{$Wx#^oY}5^$C`1Py>ar6f78C$j_*wxuIPoB97twb>*x-b#lP6OF3w{09 zLJ^ib`sHsEje*y(ONsjmdUHh1xkpg{s+>6B{X$FxPFUo(Y;JB6Q@22kajy(KS9y7O zo`Rj7ofuF9QVRYBSTq}h=66ur!~4`RiwjF2iw}$~VDWHw&vqW2$RjBBDdiA(EP#>3 zw5uJDAIcXajk&-GtKRqOXxY$0<=Mk6Vqad}awlf8h=_3ibkV0Alej)io(QI~&Lm zLY7XOm{4P1VOx!OMq&5hxssKv)IuPg=*iEt>wdsuG5(EQA3!#b&OeW7s)l6v^=q`R zlSD`pB5bIWA(;om@POkcjA^JY+VNO-C@i2MqNAhnJ4_meOrh8IqvD~Ih>ME@xY0bU zi{9=l>U&=JD|CRwl7$sT-2zs4yO1ps3**4`s~{PO!bTVYFv6`9p`sKq8>|+9=XAk@ z>~wsDKM0AM@q`^{^Evm5nDPrZwsk|f%ttjfKYQ`MUu9{umEeE3i?(~3mBH_W9vpa1 zCp&iLPMd9Kz`Ub7$G`uLCZGr{>wWheT-zj+Mbnt6B|qM@JSoBqpWD5pBuoKk1PY<& zBlzGqf#(5K5cwqN>2N5NSyTgeUqj9k z$Xm5Dv{#i5w1_cB?qy_@1f92JjeN>KfB37|3i>)mdRdPUU2 z{LfteuOK7_sXzrrkkY~EoK^@S6~ruRYBxa{r|PNZh(QU-&AlVkjV4LxDzWkLKzAkWHAZaS!vWDe6n1jZWGUO+;z(Dgs zHW338iJDLm=7+eCn?0Zg%`3WHpUP?0JPZY2o4M4sij`$ zD>FO9v_T3{$TI!np21H_UX^Gf}U4z{gofOe+ zg@O%0_Cy|xWFpM2{}XaI%~8a#OA|xG(ALsZk`a^0@F#Hgx+t1LAeP*NnhgNfg9Et@ zzcDIe4T5Gd5jIsw% z(Sdjo*ILWJXmPWR{t@qK9|TMf`Sf%6@X=o3{4>G~3KMD7r4J`!IiXx~`-g@QmIgp` zM}i6n!Tsj2pQ!VE`|%^}_L zw^t8S(Za$pG(Rx(5t*TmSzL-S+~ffW6YFxyvlfF;E3nT&t9o?OczEaX1_8E$Izr#G z4E6&_2#Mz&Z?3OLMMPjAhb7}3v5)%Lc^9A-Dw%$&L|b#S4f;Oxr^dx5vM~U$mF^o5 z1LeUFfjWk5ZxFCc8&I7J zGJW?`fDIY@4X^@epGKURpgQa;%)u|C9h!iT?OB6i6`a~#_Si(HU0s(t@A48$4aB}VbvY;q(&>Gu=|dQ2L0({D z1K$sEPP!q`jy;?TKf&AmcNEcthC!zfT!Bmoz;gmNMIP1VADon!+)rCuPXVKY0{Zst z+j!KsbW|uHzri7dC8~b*5|}s;LWWEkCbWXrvsecA%UpaHhk|r%`xg$5toOFKP+-sr z#ksQTUX4KF4TE_5j_Vo&$s>QiehcnM1U8FD3Y%5~Qra*iE%`mz4?J${iGwcUL6A#Y zy2dW|%C;g!k3UqkSquu*j_nh%5&ovUQ{r zvahIq)9IujZ|uOzBt|`3SU?a+_{svk6J`Sb^9;d(X90X`@j(#`Y$o)}oh8ECckdA4 z=j!6ZnP0Mh-W1Pi0wHEZI0}-7O$$e5dqrzjW_tR;)Hho3%e~7pzuQN=AM;R#bj)r6Dr!i3_>dS-2><^cG^IZ%jMp!P zx6R-$fbf=R0Y$3@l1SS3sIMAbD<(-aZ1Ts1y^kdFX%nR!kD(yN>&j=ZcNeRqD~DAq zk$i0c(sFIT_Kx{a@cW!S%dTOFVI^#ybp9Z;1@oDL5yFT)?(D>-1}ydL%dJow*8F3^ z{XcrnBij8v4>kriqYfOOK!c3^`#_1LWccS%=tCu=bu%+E z{(A3+rk0q_f)faF68NAv(9pADD`2Z##H`|YnH}LYqRFoK22R+Wj$#kKq4i%Hl_677 zd^|}G#enXEH!hvZ=BfV{gy;Sy}BD z{&Zn=I#`tOjPqAMwj+!Rke^#cafJgTey(Vsf`AEi9%4+_!w;ut0Ns)=rrWI={)vUO z95##A_V&Mxe?-7$2{UOxiSB4`pH2Q1$1os$cEtd1B-oB6?WBcUvbMhdsFaj)eDdMM z&a0U3In#q^>ntFX`JW~zCQ^yS8d&nj<>id~_pjpz1_n+A(ozrxb^_u0M?^W&qz_>% z1Y2YFif?|$Kn37IA{vI$04xBXj96QNqF{@Fzs$)d^`isui5?RD7XWK$mpEu{^J$~j zRoVT{e}9pglI&e$Vg56Ja-WrpYxcjdI7_iZr6{L^k!|=k=izJtz((j?QtL0CfART>z{!4FMZ2lxnp0?yso=NE^?c zx3E}RT&x46f;xwG?T$Lh-W|<1QT9;~(CI;FgS3LUHrpKM(H|9kvGYXL0;D6`MbD0( z4G0W;3%=Q%>sb(|TgR__Y#}G9s#lfg`<$e5wKyWw&B5UTsc`r5QXAa0bS=kH7vu9R z^aEMSTy&IVc!qc(5b!^-1di}WTe&Jn1_GaQ0t{MyVl|)%l>*RhR#tQ1LIVR+U%p&g zrQbpL+XU(gCzaf>V?=K%`~dEss0igppt#0_6JO{Iw@+v54l6+^DHATb-uuTVAwYwG zB>IF&ZLd8pB+C2j`newa&|Cz$wK57eJ!vU$VRZ+Tx076~zPyLSJb|eck0?NWw`$G+ zc1hY*L)`nYZ=dT)8)esczB{wBXP@`Y4M;^xNDm+<>qKlgIA%q%!M}^%a+s?wkI4c} zrc_)PXgs=3cyV6Zci9{Rd^{ib3PJh48T>+*$K~Srhz?_f35LTQ8xWDNKM#8VIo>pF+YE2-=)}8(%iUd?@;SZ;DL)D`;fluQd42J>U!bCgKZF zW^F52f0=>pxH8{(h89Z_#z5#D=={hD!JKxJmZTgV-0yb~21sQ&^kIF&7{bQRzPP%& zy0St;ae$~0!E!=>KqRJ!pbRwG>u%&XZD3(SPtuco1gLKHraQjB_3M1v#8^~lb+)c* zc|pg(8Jza?PnVpcWTFhv1Hd6L=Py6;At=Wd!%YJ2y|)*Gf8~GF88s6Ww}MQnWBA?z z$YW-iG<-OMf~l&vBhaS{GLhVPfs!?0w}a3^RLDa^8Ida(@r3wC8|l&g;@|&O2mbZ8 zwfl1-{fBlBm#1Q>c4YTsQ;x+~z6X{+J4G1h9FI+fR=iLQi#8m9S z%~OKQH+rlF#F5U7=Z5wbm(N`uKZUUrIWd}at!V4_($Ni+eZWG%%lGz*gm18MzjwQ=I2h$?|_nn4xz4$jI&cy*{1|Cgg~bFA9)GrZ{q4&AMi$yu9z>J zu>kNTWL~fTk3s*1Ru_qdki^eTY8ehAmlTn$%d@kL`}Y%<4X>o++BH)?!y8?w4|w&v zk$6)ukuQ#%gn%8^V050usC}Ry@RV5`rTDeu0a58rSSNjucgl^Lc$UItqV;0M^P-GA z)Le3M>@DSZOMm`c4UDDqPR5Z*1jL9D9N=w0N4y>y!ckOf-xUvq|N8ZFkT}5)3}=?F zkIzXht#|nBIt8k$h(Cio39b#~V*LRDO1zkWm`n2Ztp)n3jYWVDUm(W)_>qZ)WfQ;g zd2Y^v_8{cD*AUO8pQS*Qrk*;5V9b6@g#Q4|A}#|>B1TAmoP6LO-k^s<#CBRCRS2t; z(w8X7k`o`rl3c=u^?j3f_*!75P9YuJ4{bU; z!x*!P_h(tv<4z6}BE7G2;}MeelkVOX2SH2}lYz|>PFh5_0NM{il~~O6m^}gMVvsxZ zALWIo1R}KPjmseQAABFgEDBU4kSOQQox`rcX^q}r4{T;x**K0nj83zh89y&{LYT^Wr?c^wt z3Lh=X9blZs=aK`BXz1OcssJ7Ur{!Lr9^ykMkZL+U;ap}6proR@VUazqE}R62AwoSD zEdX>2hz-(IS0~cVk*RuxunWRThiR}cQy!KH{1k@H|G_HQW=It9$Gi?ghG@NqF}4Y@ z?I2Ey5F-vi1CmjIdl<_AX%a6D!W*l;^x+4L5_wzyfZzNBrHD@t8y*q2#)OCi4iA)g z{v#Jx6vKh%VJJ|!2-@l@CcCJpgs>a6wfzy&d}>_`jm)uQXNrc9G&Tn(wY!fGF`dpq z;tUk@mZHx+KSkgYpcRGd1N-zcWDWTfu&Ho@ybly1cnh?LOkd}VM*4Y}jRrx zlB%8sG<$GJvbl>+j&2s++=-~pT))gS@2l?8$^9d4;Q(qEx50gCwAsJ2BU_*efF=47 z4i^NLsM2mDkmd5FOB-LmQc_Zu_wid}Mw0mrG?6InLuOO~G2qk(kqPL?P^*Cn;OGMX za1R<#e2IX<(0AFfR^fICn3`mf%+E>$wx_1R8`WTR1VsorlG6R-$TY{~fe{fPOCf02 zXV0W1BwkNWPWo#77ug%!FW~EY# z*vJ5Bm`#-a&9$9^euJNnM{a0Vq8LICu+ecmNPRajy1vx=&pw>_{V)AemES9YRkm5D zr%|)AE)Nlt2>s#vU+?aPwmU~KMVRy9!p_Jx)mTdH_z{Q7fb>wA*h>?agSvc=vOT48 z^!oc-Qsd09&daW6UjM@J_5Z1zNZ%mT?M?IY0745_ICK=C-Lkz{2HbFcdzo2ezw3tv z6Xb4XW$6zg+;;+e03b;GyOC{}JV$_p{=9k(T{A=k)zwEMq7HciJ5m#tWbkQdUzuir_0V+KW&$L ztrlU!svZAe<+1cv73nS^2i}#BG4is1kFviWej&zM;9|Q)_My-jYozx3JLUxXgqmDw z=e)Zkf*uu5FNuYYlpIcEV+=3<7u~?Rhg>Gc<|54nFOu#7DZ3nx_~;X_SiMZ7|B%v?6N>Ov2DTwtrTJaQb`P-w~9Y`tnP%6xe9W^X@IHwg=k z&p4Hh6}Dd4IM4 zap^ss0*sWWkWG<3_XR&<9@UmxYNu;ibN4QyY@*sci=r4Ax>(mmnh!8?q_m%{TP8ok zlQp~0!$Yb2>10(pi;%muaKtecdcO2a#loL{l6j0Zm|v8!kvUtU<4^ysb^OE6dnFb% z?_WfR3ph!dro{@S*W?V6McSYJI5wrlAwb!yDpw1kn6pK3y8l=+RcU?lOp5-=Nud^R zmvb*=*Qu?H^(ou(BPrxwO9Yf1`^Pg>!l#%&R%6f6QQXa-Yj@bqd{@{%`iAn9sKOGX zd)H;hM{RrSm5b9lDcO1iDBEayqo1|EB4V`5rzw!-U=+PO4d#BX> zewy2{RqES`G~NjOImK|gU3H)CZZQgaHOx$DU%xuAhM7sHmt3wOc_}Umk|WIK!G&XL zLp2HeKzKZTZ{oo?w_C_`_sZf)=R|K^#?Q+!^0i2SfM8)rs^DTSbwJpvHMco4wWLYn z)VaUcFJ>0@ML*kX`sUhA9yaDKF>1V>O5rH$=@?NbFWDGL^8I1cWaqzXW@|ofKBmAN z8=HR8+*MNO-qZ=jw>_(|FUVq^)>5Kj)aAjaFh+@-PT9{7I6h>7G+2<6$>eS zniu`PebBhDNzK8CxlJvX_nN3*F6`5t=g$ISy|f`nk<+&eNLmBVe6$Hfq8oq?;;i_E zgtYk+psHga6US{th=PDthdD)83^Kl6kduKQZ|U!;=`FP$x6*Ym2u$a1gw}>G;Klv{uNv=LJHcM9;xWKAzwc&N6q-wqo-ZCldi5V^68%zwM~=KR>x+jBZ^!vTS!$bhx>VHE6H4?6p*`1Cu^P9<+mN5 z>dc_e5xZN1H7qs4OQ<19QuglpSMl*)NntW$zvI5I^_4Gez%KYwEsC<)bF)1-38yBHi8jf*O*%O{*O>Nw}7i?Sfp1JXyw(R-*bhPqk4*LWyV>s|Ct!IJU`VHU(M^gikU-m1X&VlPL=7XvTHp|dzP&K z%~3($j#XnYtlQ{IK@@rH?E1K8+5-{i=vuY! zNG6UnLu?heLS)k6GEGM!5F~ndBcNfh?yE2Kv8U?&r&IrkP$;zV2xlI7ywf`hPS{k5 zoh@SV%OSkifqW+S_;`EY(qkUj?IFu?mTMRF9xo3M?Ok6-lKG@uoeJ?$e(|UI8NxtTIyRDq`WqG|c`-2jr$XpuUW9=DFS#Q$E zh-RpKyWYCrq_1#a^1?595oay|xgB{z#x_+275DZtwaxp4=@vQp@44c%OP+66i-%X2 zk6r(n%e`9qKM#0NP}}uZUp^&WeTGws^+vke-c!%^-pV`F)7SHQ)K<7pfFtv8pJ&FJ zrc&nCkCTPZLb2fAnx3t6$>?yb)phpYBkF7Gza`F}C%zg|TY5@1P$)uPOfI@uY#;A2 zSp((CIu+U4@53ikZjxTGF*mwiQW+dMIA&w_uGw&YjjeISuZVKZvxAbE|2xlEM)bkt zJ2rP-a;u3kI~&S4yPa>-#)Y5${}VrUSaDLZ)QRwsRnSl!>4+6Fq__bF|;3|c?#r`e5^ zY|I%3~lJj3roDIkzyzPd1PMCZ2e zriIDdiA|+0g!R8VZ`D)$%RyMJY^`r{_4nbDYm=gGxQH|IcHHf^OEwoSw9ld)cA%LtKS!;)ju`=^LU069Dw- z>l>$WP!JhCGqaKAW)%oBp8s8Xjuf)L(+B=e&na!z9^XnhzV-F?aez7FZ{Hq;B0yRi zu$*Ik zwXL}PXOiD(bqWHaD}T>x9tXd@j!OcdK>cgArMEls3cV_20TZ`9$1-{SWGkD$(YOX* z=zRL2=Ke#fW;w<3y!6S)qghYxveWcv7j&8nzA_)IC-N7o1JTsZ!aqNNJZ)J=~5?U5^njf|Y_FVE&Sra8C3F_#HfvwNp z$fQjmpFoiReu#0q?neE7fE~!+{LN#AEfQ47%pHF6$xQ7`zn5?+lghSVtR7+C6Bv{48h5jGbk`R z@}!h+LzGeLH%q_D4VEJx)nx!}BqzyYc9axq@H2U)U9PdQR>+Rs3a{ zLfIW7pFYO?Dq^>o$+2e>Z2Bpylzb zSe~`sqQo&xoV-pn|I<638Qgb!lksLi#^I04Pc3h~`EG4v^G$2*s}Aj#FJGV^IPmXh z$Ki!(#gN$=fLFI$JT_OXygt8-=e*V<)*>>orJy>GY@Zg^>l+WxZxgRNjCQ+9dhK(y zGQ?cC0H+qiT?1S!kQmJcwS7>({rgV{4SUqL`e(Jz{G8qsJvLi&02iCSMY}chWe>lc z;Z~s|Gd11wFDsydVy8z{n69N$O-23QU?D}0Z36i1fmcE{R zv{kNI3vmKUBKDv0ZDn$*M6vx13UWPJ>Lf-&5q3QpPsz}OkR6yjKTO9HZ~mrNqivz- zWbHp09vO=k-27Pd=-P?B+>dW%a&X5me(#AUymctg~JWurtGlE$|hCJN^9=S7FM-BFwet3Fc;(7>t zJj}5Yv~D%T@L;%wDxXSK7I}z-!`uPuzltF<`rO&_g>q7yZ=!!q8jEq zs-GLt4kyaL^d?w(bI4PX1-dQPf3P|5>&RzC-%Iby`sKE6F=ZG>B}k?+n6%ez^hp=;w8sycR{NQg>Vb{?6XjW>VsILdc{LTPZ~*3(_<<0t*Uihh0P`n_8#~ca|H=4E6k~`@x_9w7gOW=WU#Zhh*lM`~v#@fz%`a6fJ-){7o^63z` z5=T69=zp!;TTGp?V~BdOgMo5LFkV%eYuWm=;G@qxxx=AC1^;C9yAnAZ`^d?B-j}Vq zt=Zgdzp*+jGQ>=h_`NtaSUi(++K!>MThh|Jb)or@=)CK0>v=)Gs6^RyY1&No_R0j2@zh9{Px_G9*`>Dpi+`Cocu=q;d-w| zJBozVYJ8nSk48HV7$s3E!aI*(eHymgfr28cTCds`Y6;eQwBjvOyM%8wJgF`;1g9gy zX(f{0rI`jbKeRgFv>^QjNCn5z?UrxZkx%^o!O8pQqrma4E`w{|Rak7leNf|~BdRiS z=fQd0V274@)HANa@AS_r^$-6hq*&2Lbz7dziL)RfbUP)le5jN@-rOP3;vA_W&$3rj z$K_|M)TzA=mFfDi@lVd$y4lv9n6vsdL1Uej$??rY+tM$wss13-I}cTM!hF*nImL@V zuV{Nbi_3B)`}2LuD#+{b$C=wT56m5=9(}fA&~b@#lB_ZdRh8$7R~YRJy+xI}^H)E4 zCfgg@&CpVw_|swKFZ5$RPU{DC92&{$=wiz*UVNFq@N7|Qv?B~0Jv|Kd%FnpSDCroQs9Cl6)BGQkxH%=0N-if?Npy7Z#W7GE zNO({uF#nQ?Wxs(AN0^&2+ZUcB$JW>E%Hcv!#sY^ZIYTXfP5w+7@(|{yNNTRfqA|Hn z|6cV|lf^Qha`!%WvmB2agSiX>H9;#UGd4o{PvBxHdI9@pJ0|8G3gibnH9GcDC~-fI ziq#%Cdhb(ny`^`9N#I0=@xTa+&ng2$IQM;bRJ>+RvGxZqI}Wbht-wbh&;{}yNdGy~ z{HSD2lv&~PjPb1o2{aZf&rfzQX{+ocyfz#2oIIIRlg4!VzTA}xuZF3t(0)P+&(M4EAjizesJ#-q z^~P`L(<~;aMTsY7Y&<&ONUUPd@BEPoLWg%Saj!4=*7Q|noouu!ua<|jzRufwbMqODp`i&36l<$p zYF|{fqs<4c+2uN0_TywJ`y`Z+u{E+_@7-|PDvR+DJ3g^PzVVvqH?}R+%T%mka`Npl zqy(;yOluxpSVRK1Q%b~4RdS{U6}n}|C}NKOxZ-|c$NiMT-!h+W$P4o6xiqwDbnFOj zdR7=vm|D0)Oi758`er5>!A_Kqy6D2tH>DsqO+k;TMCzyPc2^?%W7uv4iSplIGQIJL zLjIAvP-@}z+tVMYeH*(33L}&HUVI}c&t)uTBru(x&EO%lnh4X54a%4m6biTph~`C% zT^W*9s(hxnn{x1*Qekm^>y4o&I|$xZRw+2~1r-?Pm}b{;UDH#2hw{usS-Os{-%56q zn@v{lh>M2y@V)h~50GSKcY@llaRLT^*OzTCXXxi9_ttGmIYMpK8wnzg$XJyE}aWB5VoqMPJd&T^7rUJ*Db7{XVqUXQS( z7J3eTQ@t!SQ&A^-GP68h9Sn2%pvq#2gZ?vK`v zW(i$>`CGs^lP&L&iqW}PuZV@=?4}uhhiCaUyXXxqJeDk$D0j>99uRPp5FR>HL@#Sy z7xt28tH$2WD=9KuhGw(ooq9%^1IN$AW_dm1f>Eyv&%>JPwH-U&4RXg{l+7bht8HE) zp})(uE7aG=M#n0R{(H}@QdO@9&0klA9;u91ceNeFN~Yv{=byx=Uc92f!?T-{qW&>A zr|uq>V-}PIr=>`oru~Px3!Z7wous9FBY9Z*9D`?s^4r9mK%BzES5qUybL|w{(HoT1 zt&S_twVS?TidRL=;^v+FJ=KKT7Au|64D-l-w<1A`8=l z2a!zJ(4fH2PbBmgy!B6>RGg0>9&LQ+6fyT$E{r+!vhcEt(%4t$oKgIOrR2`g*0V#oD_x&Z7lnKK?QKG~dOkY)|gf z?;mLBSsaty&G~KcTX=`{JPtd{_|y{X6UP_j&xy*#Bo1enQ@&ADDbl<(W@L2s;o=p>GwmR>~@6>*W<+w;_afa^PdvEpThqpV<(N_>E~6;%$~a1h6iV4#ViQ?zbZ3NW@L2; z6u9nA&|+#Ux!@(EVv%}!oROd>Sn_hA7T>eP1TE#=*Js73Yv*83= z$_FwU4|;LggSr+Tyn`3WXwZwtSv^?&(z#uY>BT&71=mbkC+kzs%kQ)F&WJOnj#^lX z)Z(J#&M8;jyi~m6kzuN(!%_H*PyQGNhstONCGlt`dCUwBUms0RCY?R9Hu%kY{`;x= zrs~6){vojte6&0l*c_Y6n?0zxkA;Dp;GKJ4e~<37Ijw6u2}WV^^S$B2?(_nV!gk}M zB^I5J={$A&THZG3dIzxji7<_ZMv)K(%;r8#EXf{ID>=41EQT)r=v$Syy%$Vh55zLUF}wO6cXn#O+j-iK*hDP(wxnQY=Dzb$RqSr}|Mwsw5G zYWXD2iY(kNLeDW1v)ryohKQ1%HGG%aP8Qw%neSa(XS#Vwm~q?EJH&h*7ZV&rDHplLPUNc%+9qA%KCb-qwvQHgfh zuU#0!;!aik;E2DtSZZvQHD~;^?Rx-UC?UAwd0y1U{D*Vg^^}gx51!gB|I{i~Xb~tN z4X%76_8G<5V{QrC`*w8LiA?l$x|F*8}wOJCI??-wF(Ek5KmI?6L-A}XQZDfIf@ zT(CH^!if;UkN0luz=N);;~L45{6O(;Fh^mupiJ~@7n^F%e!3$6LKV7{N7Ku;Cxm?V zxO-^Z`W#|m-*I96^LlxmlqsRoQJEY=JyUGul z;`JMmkfiwpn5@)mET7D8-9cGAl((;B4b0#Fv;f_HkImnNtaZWee@8SZ4UmSCx;KzNYljLsJQKpm~iQlz&h9icVigc3K_1kMV($&R86YZZ5X{g9iN zJ8)lbmvEf2^4sB1?MrrS`E=ZmsRbOICJrvLe^ljZEx9l&tDLEraoD6p&Mt$4eaCSj z4>7f(p%zPfYW-Jl6oyJ1BO>*=4$^oHypSC;K2aZ)r#7VAOvy2jYFH>7Rbf1IYpP-_ zDAQ1*r2d75I+`RDJr>3zW>P8poM)|1b1n?aoc6fJCM1X|lXm6KyM2Ap)fkA2Zb#1& z`h}Q1b)p*gJCuH){E0&mMPz;4*7KB0K*sJ^5-;O%k5FB$B$xh)xBIF8ohzR$aqhOp z2~=@eR8FPz@-(lVit*ml&+DXm1PXTV^i}5@Cr);fNvOc89!X6V-60*pbYB&ON=TxT zDlP|IGAwmw^wjB{NvaUNCrvBQ7~PxudNOLTrfL z&7*0FeKcob$|aj``)&v+PUTBwq@XqS@DF54paC1u$ZxZMcv+_3rm3( z5l;;^FMUnjNvzDi`_RZKX$U@0&fP;8E;w_kkCa2e{xh1HpukN1^_w;?Gi#IJO`ym zyBp->PjoN!W^3BHrPq;1w*+xNPW+gJRt3WRiw zK<=lK5K;;O0j6H%kv@KdkT^}l7pnE<@Mo@@by7ucW#6faZT)W^6Hs-`T*kDfi+>8= zenoY3I9WLFK^i7@fHVue;ZAORpEiQ&=F=o59_mq3hWb`~r?$SKzen8E!DQMt0kzs8 z|K?7b)^7ea?;XKa3Yr2GIFJDC#GV{(h(uoyx@>E7j)Ky8Nw>NGs7C;&e~(B)e~jVV zZ1XGqOf;3Bj@8b|_4g*Ti*m;ppB`(@a=m&Y%lu7=Kg(XnR|~zmetZO0&1?P-tR}}9 zdKNL95`fS(Po(SJEhy|D;%#k*Hz(Vj-*bF6UE<7w&}k<@q4#wQk|>V<8jJ zG_+cGNbkf;j1>I7q7lgWB(!9Af~<79ftE3g-&e?z?@V4HC-XnH*i))f?j{i>GHXi-?ksAGW z$E-4!H3w(I5=%K2O`97B?dl6$_F;$+I9$?3y7fad8rs?0EzN6x+S|j178GR5|HfAS zSA_)qVu#(c=6tN4|2xH5V@F1le@~$HtKfKbPJa|z;0H|DaTc~@G#~HM1i3~tncii% z-PDvX+V{PS≫DKdohE`Y9Iv=-~TkS-&4jVK6}(8tsr_u`47yYJ)XUKEEL*@?^fT znx!8TPMo(2?JI?+PBlNstsxH<&VQhTE3+aElP}D#Q}zpTecBVQg}wrvr8OQ>oNa+Fr905sJpHg9UeD%=khx*u@_m znXa@nIvXjg@A0D~%Q>gqsPo8a$95c%gM<9`Ea`26YMYgPrWQTMS3TRehcv86y-!$(f{CAF5hkHqF4o{^)Q&>t8D_h9{wvp{8k$X#l!j3In31qagh1 z50mg?xg%3i!d7H2u0i8_9@nJxD>R2{RwVGEF`rK>+tOB&ASbMk_f6pm*DzI-?EKb% z0lez@4xS_`&JTjZ=xA)y_Bo!1Wn{%>C8U=i9s3(0s}r z&G{W9s~F34T2kX4#Rq_;MrRM19bKxo=w7G0pXf1Zpt5=C!5U+G7d=aYa>T& zCHh4S{J%gUxHPOkS9h+n`fgos{w4~Q2;0-)T&asp3xBE({=?4AqSOUng@~V?)i$b{ zK|fakjk>y=bnizlp4_<9*o4;@;vA-E^mUS5UZrjGmISJD)~M|gZe3*N3r#sz7#{v& zEx*;~r_6%-fqoILy?YAvZGK+)^w*+y;GpIqeLYg^?g1-1M=QBCGJPG2B;lK1>fDT7 zuDzD>w*IbNzQpd{Fzk0HK{cvn+j?%V7`4Tc@m22(j~ao^N!th!>2s>;CSmeXqz8&! z+J#a-R}2-~$Q@Jf-lye!ct(7legCTIoR#g#LtTgL1T;Ng`JA#Z{wA&@a~H(W1;6p9 zFQPWrMyrUei>!D(C-p4LwoYaKZOxGTnYFlD>R;R2Tf{yXR09zhR{FQ~Y83&#^Jhgw zuAS_CL~$$5<0|WNolf`Ctp+2oQ>pC4k;q;|s7%EuT*2@3rfB5eJ^A@*`}Yg?bBZeH zlw7Pk(D$GpjXTR;-EJ2{t&~My=R=p3jv0489{6na>mJKoGMFcP{gY<|3Z(Uv3|R_) zChCbwg~V(+?>i)0ZPIyYXAA`apeN@78E531q8sd^uQ|lR)b|N_xKH2x^?6adW3Q5s zRmo+^m_Obl&SW$-^L|D%%G@?l!WanTj&diyiN`@CTl??cwemyK=VtO|Z;1O`GqOsV zNqDX8zn_IIuRq%OZFZcgpqb3RracZdq=1-y@bgg;vis#e-X^=e0Ej6vQEr-`8TVo z?7`hdpXTc=zQ5Sd7pk+tVnE8TU<}xnyZ%(ku}Jmws*2JwQ7%LKFz$~@;w7DGUj)%R zAL|L#0KrSssdyBVl)2YEgKh$$?Y-t)hSAWhs@V>)LfJsmSBb;xt=lmS{^jHEK2$wZ zZ0X>a*iv6ecy)9Xxa+v27Dq;E( z{aGm2xFy0hIB_IC;>3k{&wu;JtDWW7WcB;4g$ssw$(rw$#xdMp5U?dSDuQcYX)Mhf zUZqH~u+GBWQBfhjbfGY`jW-f$Nb6q4^C!o18XLC}g@Cl4U%$_!5J4FsMX{5nr$5&Y zxK}Jjv2kNB+&!#^ewNz?r5Dt8>1F1KhinDF#KNu{l)8uquDjN6lvPmKAqK65w+G$7yBmFP*gZ3)}U~HfvJIihpqIqS{G#vs@-vN_TQn6M@e``@z){x>kro$t3C_3pOq2gr=56HJ(7Q~NA~dTov}&i z^Jk*WuVfkY}pI5#-q$6lOmBzN@DrAKJ;^0Px?bw?U?O9DB{xtLBr z%_CjpoQs!iEs@N2EeRwkn#0TiXek^rkT++F7E_zqwrn->jyb(xI>Y(p_1omu?*ZJ3 z9kSokx7aZxS+#Z)i5(N%Jm;u0F!GCu<^O0p4{)yc_y2!HA^Q;7vyhohMr4zfy=6xB z-XePwl2vB*%pMupBV=SIdy~C?x9|V@yUum4b55sAc)wq-`+nZf`}rXGQs3n~e-9+i zO9x?A;*X;A!C$TX2-L!lfiMiiW_ipI=hgqsX&U>&tPprI%%$z2Z*tjai5}1kXd@|= z-SCK|s`}1t3IF6#_b1nY2|>@Wi-D`5RItM;46Ub=yADAuEK4Jp+}&+I+CDN|Rl5~s z(nU3Ys;{gs;S!OvB~GM>W;>a6DEWhW8KcwdPyg?`p0}+|fe>6%NY8Z)FkM=DB!dd#+n z_t0(D-Q>n-$o6jZzu-&5&woq!uNNr>gdJ9S8LXuDTVgN(Y`i`7)mX&ziQy=Gd>+{7 z0P=O2owok!1J*p=Ht$|5bkT zeEx>z4X$y-n@?uko!Xk3mmZpWID{h0)|^Bln=Phr_=MO&&;X9_X*<}IZgvW9pBczs zJkMA3-oVc|6_=OqztBzjk=5gA!r5b=TC$l+WrzO8OWt3Avsd-*$5#5Bg%58{GZoXE zK_>t$)~%@a-k5ums_VNCNVAaUsW7hmvT7agi>c_J@_qlJ9=zZEPC@Sa%AS@E{t{JUP1?DQ;FoK zEV0v&Pq{Nam2T7;Z@Bw1HI7kWDuxB6-zH?RFx4cUgq2F?Yjb&KgmYF5yUK8PS00=$ z0z$7*E<>cV%l`9IHm|AQz#}iteeu5e+vAfNIGn3F743{-gM6dm{*I1}rzM~W9Ib$f z=TyCvQ`O(YQE3d*uK|Nmq2E^fqwaY-a!)!ZD9Ym(>bzSW2)Qr(^rcf(a&JiW4*Ib! zzC+l+o`L|sbk{}4r%L$#tldi^f#XHYp1YhECVHk-qP)*`3Mwk_>zCuiW&t>T?Rp)RX@qcD0& z$PRhm0KFTLW7FCQfKsNkwVkwtpObTRYzz|NDY|P!KAD)BLb?cse;MSq2?$uGd4U@) zV;P20*V8Rm)AMI(^X^dl+~#qGlMWLS?vk2KubwyivV0xkr+>wG9}88K;9u`poxQ*u zed1VH!po4=u|P)C+JFzdF`2!ia~Cl9C%!8t{f<1(1ZKV)TVpOYc4ibbG-+h_LdD>HZ*Z6{-`znE$E6 z58oj}m(KYQych0Y)7C$&SuAJbf34KeOO3Uh>emyBSM(Q}cz4gbSkYeDHYeg~ZiG`o zAqE_^xFI910pSr~BTEj#K*&=x7FsjQ3@AFpmeiJMlzy9RaQqVb!sltHj#ti0eDSPT8kI7MZoDeFAm(fVVQ5w7WyxH>m~dh1XcP<2 zvIE}m)X3YilY2dEMDW6zd+xD{QK@rt>cu;e{K6DoL0bAYd$S04jo!rkgYrL~^A~fl zRSm1=zSmi|y_Kk2y03IjFj#nCHPjh`-QIa!WXS--9dhJ=sa;S2jrw^1ZR_ss^DjQb z>tcGo1P~}~M9r{W{TkHbmi3P#0g{YFg0EYRS$RUqQNhaAT+WO$Lw(ofwXOKW&AH{< z173}4iuBzg;`zLG)~grtYJo*!)Ivj()9R1UVJJ;N(kjtez&SX&;=d$sJzz|B`aaIm zrL54lbm5XCw>zv7dQz3)?_G~l5-doW;7|w2*`ex73_Fxr^V-OIt67UOHjH+H&R8I_ zfCkT4wmGfu>=W11q}Jm7F1AXpkeA!SWdV$s`S0ewaP+IBg-D8 zRk@NNOQJi`H4mxe*5^1`Ks={EKZ;b-IWSF4eM7bdeO$z!a_Gh4Ia}F(-A$@)hDmpP z=9AP<#e@6_`##jA^325&f;`YKw>HS5%SJ};?=#8^f8jLFEq?q^fH8l#Nv_u-FM0f( z8$~}_L~BFGoXG~Zyl|pt2`4`Tw*kCbllxz0Jp1@&;z`)jwZ+KJmgiDBhrblBKV@YW z(@9Odu!*~L7@k8xdm~pFR+*v*jop;yo+doy&oi?Is_;x(@j>;mce1eZ>?Av6$M*X7 z-P%yTpru=?Ym>AOjHO-nzOi0hQ!28N(EMyHM9rhz`qw*t@C2#zqtU|( zX25L#w77FJoUPdn7S{gflXVlkA;$U(aTx$MNA`ndOV@8Ea3VSO@OkeE?&l51%=HmDrZZjuneZrdJE*VSdBlf0WsT z=SG1bFwaZO#;a4qC$|?`G!tB~l(Q+z8=AcfcCenib(_^-Ev9#dX*;=CNiLpQ}>S zr)0v)k$5*6D!j#Rwv*YmeE!#J;lw%ltOMOmyzjSQ>HID(YCYjrbBT4a?119mCCuFJ ze?Nt;zSEBxbo3A-zAxe5O2zs~8*ET(PJN=EJbJ7!$dzzspn)#h#R2arl&^;J#5+H) zJYPG9zgcYsBgQu?olTVkP-#gK5#L42b_p@apl>|wQ1$j?fzw^A?QLXI48v22#NerK z#oXm9%tUy_e4frX`fe%(nN5m|g=Vhp;QxY5#}9FFDnP_eNogoAk4;UTnV!D5bxK1N zEJP>GUmw>%l7KH3(qHgtQ@Xgi-e&Bz++4KqqE@rOk>~Tshj<;S#db_n(2j$X6V{+1 z6ekjvlwmOv{tmHXx8CrU=-f9I%*Y6_5L_Cxw&}Ugo9Bar3RJk@boCs zTn=`0SK}Z;FdzW=;LBOhd)$M02%=0Z{zKcR(Q>ZV`H>+kamR39_<+Jg2<3SBPbf=G zC)(+|MI~vAy}=fTr|2qI?sUKXdSM3iDj^1gk_-PncnJACzYTDgR|SDOM3cXRKG4wr zgMRS?Gn|a4Jn!KI(KA6ghO1imDV7ZN#xJyt51oNhPp(}5H{2eaNeoEF`t6G0-=(rS z>h(6Wxj};HvFIyh>4zXL7=gH8S$1$=Vdw#QapyiY_>hnECd)Jrj9+j>u~<`-^o=G;LaXw0O1S9t(b7p9`%QCTZ`%EH*aGd&#!=d{fXyx zbSuDF5aApiML3!`Qlz9~kLp>vSVW_uorU?Wkm-Or$!B6F7^b+*bgl;N{Et9^RSt}D z$=?hI-H$pC4FQfef!#ZzR6oY}81)WVpU|f1%KK}2Mqz}*p!T0dUiCNDv9vB?_wOG! zlFi;B3-f?vho&Xr7c=7;9v+go!S>M+Kdos_i#Rtx;g$WfBqb4RVOXVIf=>2)rGR595JlOcibgr^!q$qk503ZLSO}W8 z0@b~m{SkNClx9UdEwrxH)wmAVFZfuXgF~cCOoplL zWAIegu-xjW@YdMQ)FL4Y3IKy;$ZpY0KtXXm<2d9Ee+Z9h0QkiFo+bI7t@>V@AAHGI z0)#ZOADs;Byn7!1HlMG4|CA8(9`{Sy_m}?SlHv1C;>un>A888KHxl_1iG@7tLe^PisHsjvQ7OSAKp zjidL~LG+m;zeGRdgZo>CYcS`n2(&K`-8e)e!h?l@C+#O(yZ_l&p=kx0A<3(Kh*hvQ z9veUAQt08z64KiHh=ERfw(RB9XkSLURUej(tF!p11LN12?bhz+pLTHutzC3$aMyX- zf)RZrx$EWbiw%;`j*BXz#(4`-HbmevNj`gR%1IxnH1Yk~;^eCt#(YxB9n{|M*6;6v z^>y9ivqx@#H!oN`884C<28zd^*a6FHfl!_T7Pul{KuBHK?KO65ci>fa`=IJc`0(Sj zkIHWon~TT*y&u2Z5i(JPsfk%I3!_c@tink$N`^eFP88B4)6IhPd@77;{Q!8?|IZ7c zb7F00-_v<4kP+7-A&xrji`gGT!kR9?{I>Y+Zc}TZ1k@18bBRu7s`keb(lP+F^=|rH zwihVJQoZh@qWvQ5$$VJVN2SNX;N~PyBMrWQD;|wn=Kx<L_yn+}j;6Io2@0zP&_Ln~1rd-}BC3%OOe@6hk*Af$ry^}!9$jVf~53RznR zkHn;*Kz}uNIB{i6SV}@i zFW08?!GGFyYv%digkqNkjT$ZTa|Lr42=ApV10)yE;B1VJmO*h`O$b z0$=T@%X#gNY>qMJ+Bog)YChgLc=l-Zazx~Mv}MUWThHTPeiu$;0R!WIs?;r-X|70Y za99fn6$~ndJ&4TyZbKr8Fa|Typkm0k)%RO8k28L*E4$Rj7 zi%!**a_pCqE^Lw2-mdxvV?=&_)!r*E!sr5S#gMMtH+SL#vNMs)2_Je+!@x1?gKEv* ze$Upa67jZ*5NCF+3Ui5=;CQ3l)HzL3Oo45&{em)F7dv%EWLr~%!k!ytWi{kmZpi!o z#b+Eucvy&yzg2KQuze+4Bk{bF67e{1t8;cxRh?!~=SuH1D(N5XK@#SFQDkWrL;*IU zDCFt`%D;S;~0^VCChvF_*1U@VZP&H zcERs&)hF}ULn5}l1E;!Hrk6i(?z`uJ*xvA=wL#4O0b?n+k6_G; zQ0n&AZ&P7l2ikyYuehkl>mX(Rt(Db9fXFFw;r;-W&=Z|bS{2!-;TdtUvC3*{N47V| zwk!QVfEy?&+3DNZ|5sz-S~nn*0b*Vu!NArQ$g#?@-oo>+wsG?g$Lo)^9D66^JVVS6 z%xi(YMAijC?eMnT9{i9%uzQ&Z1D9y8C0fd}ZYiI$HaeMlJ6QL?Ao4kTL{CqDo0{Xg zo1V!~UjosxU{3@82*4XO)YWrya++RAmb-m~q%%lx^Ms77(UB1fCl-(;s8_IMl?im@ z$!=#~-@XZhID_*Kh<64i6NC)YHkB)VRRl>SQV1%|>tPoNI*gF-6Cm zjWM5EVyyos$Q|u9NSSmdgnyRgf2x|ErdQVydaS$Js{{#b{sKqXqMZkYLf_NNhSmm= zdkCFrPFP9#XpAQ`5qQZl>aGlHBAp&e7Pa67jSig{7wp{=VttAwl}#e@X5K(+CS8%? zhK(&5e6==U7Mc5K$0pm*lJP1bBLfG)?6xs$(XdBt>?#K%H(9IQP3ZHKB?Z16Eyq+2 zfp7;KV;pW$-7F87QgHD^C)UAU_J_t4E4yrjfFvS7uN-zT^g8^zoE)T&#G_X(6)NkdB9V8^ihN(~*YvTyprShKfvOX<^s$wA-lh#MGFwhk8Zaa%p&_|1PU z;@?Vzz7_a5Plg+|LvX!~wUXCRbv~-j_6AC(I_W|$WNkh4=NEH(OJ0Pt)wF(oy;gVs zgu0NE$H!--3@@$UJf8G?(ariF(-7h>EQuU4;d^IyP$!N*WSUs}GG9|)L?Fnv zl<<+>V5csq@_tSM;mSO?ga5dd*9S(+D6Mw#cS1Nx;%Y|~nHvtzHkcg+atILD|JZK+ z!Ad2~&`lR(Iuz{?)_iJk`NIJAH!*yV&3%spC@(*Q^e!cG4jl8H7A7Y4&g;ve8(pZz-qsjhHgK*DHqlRyl{0>m^-JU`551B=fPG(axy|{nta2jcE zfO`LQyr$XtXhUr@Tr`fZGLV{%E*qvpz^P1=18la))w*KKP0n>N*rR;m1$1hdg=-5E zCE>%S=AsLf|C5%RE7;{O^;%|1Vl(jh$*p~P9}@@1)!7-YC@rYkvivd`E@htYG|7Y& z^nXBwZ~2Y~zWEZqXI7oYFZG#!cBq5hv^h|cbAcUIQs8Ld!E4(mabuaM-LbgS=1?J0 zdSA9+yUsKnXH7iQ9lg0~n7>K7*)xjzZ7Y&D;2?Z;Jjy=t! z{Gi--(O+JS%lTC47qc^sh7BoeJ+Qfuz4lDs^eZ`(Am3IEe-x5$v9gxx>al#htl89= zE`8Iw=s@}`BjZPf+W zT8(p%bwr&AWIQ|B#5KO7h|i9owZ36b* zga%9~utBFNYh2%{5G5@WSxs;{a=Zx$Eh*DY%5|5T*8t8}w+MP23vP%Iyb=_jw~ zfp1P%H5JT|Jg>mlShso)?9bz=+|D#Ga-gsz`qu|;J(|q=`p=Z8X71PDN~oJ%f7R4S zj1x|6kVgw$2#$XJyY&>?MMjMiT-!YAhucSM@5K`e__!~cev%{p+!7p}H`Uo6UP2XA z#L#Ole2@%&yo^<(H^R1+zx#tB+3_|57xKpS0rQiW?oj|)KVULXE4+t_>bn;}>9u7{ zSy)tbspb5qna1v>-8bC<@~9!?3NpC}yHhEML}c;Her;Ym})_%V#AK9@k5Q9Ds~(Zln0GM$*C zY#m!IPa6C7IpqEqe5$4)y5vIv#&`i4GY-zYjCo4bvvUX6+0~P)0#W|mmVd7YXHK|z z9nDsncQ((xBqbp?|MdD|w~r=?gQG=hDF`FkR}2DOwY9Z@67$Qk+;Z%RjD^yQZvaCW z*Go2LWXd0CRBoy=m zLh~Q!(#4M|)uo*;E<4cFT(Mp3yNpM1HE?2$iH(t`tvc{442gKHM(yeQB%~xwn3*sS z_79`)Yu;SGYE*gUxb*WKJ`wiy2`ORLHI5GNy*wi6hSvy$2#>g0hFBwVXvCcm)jfRL zt><~t{J1Sj%*@#SM%8`KH<|W3$OHwn%H)UBUedHv>0fKA{)g}@5b9G*P6*soYQaVL zS){nCe-L9R=w=F%&SB?m=hC=e%`b6H@{3LUV_GgL>zXI(pu(t7M(UQuFtn7D4Ce_g zzW1)Li4lGs<(r)Ujj9(vT1f*Y0tAAE&=i-NTueW3e^e&6&gYG0f71MR4~Z@?u-b!} z>t6KGpS;f0gf1f@R}S-pY?Pkpl&`rCAxbgpeN>oPDNK{iE`t0-;j(uGhmJ{px3X%k z%FwOSr2DkZa7~Zh*%UNPT?(o%?-n<*jCjM^^?G!#784=rxBB&*!^SB_*8_{>Arb;N z&Bc5f{kvn>@`U26x65Z0>CHX}zbEYS+oWg+T(i7~#_JI}@v60Y(k27~RfoDY^0Ugh6ayOt|b|2jpSH6UF zz#YgRTv#wdz^?W+U>TYr^`uE;Gl5GKhC5&(k0_}FtM~OC@;4+F%dtuyRIo?ZYV2* za}V$Y7w6}nQc^rUJc5FPARrs!tM{71Au1D~8$A-C2j++%BOo?0V(mwVWUkrqS$w+A z%VXluxC3dBgyG9mR;5i|e2-}lHpK~26W_cwit5)xFXWqoc<&eHIp$Ed`<0`E z8>6^kv08n2ylhl&cwM-R7WTjTGK}VDeEO7bCTeKm_Bn%XboP!Crg{TfZhGopZl0m< z!_9M`&m~)p2iCbCpPaNnG--zSzkPqx$4|bywhv76(rI65ep=@UiMrb{OB$^Zgce7- z<)EGcmwQMy*|3eV07)P}#@0$`m-+MTxDqDgmqI4`=p_H$Mc`UH!gJBelc!sU?sD+o zHcx$iDksws0yerk*e3_|NHTCH1%qmRd5#tvn%kc&iT8acd zyCtT)(Kp|;laKG!e@mBUd)|0So`)mKps-;WB0nMA`KAJn6zt!!DTpE>Gd<2SyEq$(}W+p+nrcJ22gD zgU)9?kwPMGW*rh`lAmQcn;iKT;p%}U7O zyroCwrq}=uBl9aHdCPgIpol_D<&5)$9-!5r^AQZogj~pp&7({P2!Gn0EYB({``O<9 zLY*=`_DsTYnMap^kT5qp8(53Xy;~am+QmGO(E;GR%1TEIi?7ASFBOC?L0lUg7~qC{ zQ63&1US3knd%^8r#w@+PynyF!Tfd(B1ADL6Kl3Z(xJD z)z(uVTRgVmPP|1Wb#9hclA$6VrX>r_|NEKLC3~W$QANj*ZR&A7`+{CEb=JC4JHTlw zA(?S{cS!M$-}cBsym46y5k>s{hT74A2XAZA`=7MY;v@Y0zP~y$-7-vi{lnvzh-x-t z{Wr7Px}EUWWSk5iV+V*VU_Rw_m-(Gi{e#&sxt&&-^}dLPb+LQm-@y4~}^U-n+RbweQS{k2`!+y_3J0U7X&{5c8fB2~pJ?w!N#g zR4N+KW>r`qe6wbYq`e&WsBWxN0Y%hss;7!~@0D>#Fb+Q1gx9}G&9k1;bOk@DN7b91shRaUh$7{2i$1AHTwnN(V&TBkIpA4QdXaF2lQ7Ob zUv1^O@W~3Fru}zM`C+#bGvB{0KvGUl3Ncl*8)<)Y;3OevRgB{36wut1%w~S4rk&d) zs#$gi^M#O?9t}4x3V+&Q#`z-Gqesht3Gms-&%0%v2S75bpj#OzY_~7h4lzZSJpxfU zb{?%b5BJF}WCoZEcRvSNzt~y!Qcv!!@Ng$#iQME|)B_0B||2>!9#KmGgD+_v$2qHFG)>ic|4+oTYWaEWqZ6Ibt> z>9%)lQRD@~H%1rZI`M6)6#5@*AIg1w(0SfzY>=q6=C=LwQ`spDQK(BEH+SIKpl>BLp^Ir_;4hf9X&H_-&Z> z>agPuVQh$RAz7%=!zXr!c9fH~B%wp+m&@k!r>9Edmgi0Dg^OND2)_)H&1b%U15ZTG z{^&mP&qN7x!o=Rq_+*Oja4i1g8wsYO@V|`Ck{uDDGS^@OcHR-in(^;8>GPOF>Rv}{ zD-ZgZ$%7`E1;=Ne;gbAUag>|FWBy3eZR5jm&x}$M8g5?$;;F!EM9+*shKtN|ys!sK11f%{hM&G}v>f5WS+F$IN}P`AC}?93He zd4xtRfhv63VZw2>FS1j@cR0*skXH5T5H%PZ6*aiAs!mfkMQVj$lb#uw*0{(tF8QJI z+vW#Y_lb%sog4$Z%T2VAzyyeO0ys5*ppuYNq%|rj2YxRbv%F$d!MXW5xVF801+T)K zLmvy!ScHIfWN`g23AE*X2iJgxhG&S+6%{HYR+l?`Ky38?45Xhvojb||w=tWQzjd{> z&-GT{Hq81!)-(j2LXfw#x|WItTmz792slYMpv+0Wtt}B>yhe!KNfG3Yz~z zlvi$N|FV!(m(iI8&f+@!oW2YH%-8h5BK!~iw(|d4@Aw>d@4b90N9(ccXHh&&-A)s& z_N9*ZNrT)hjMPE`G(6oRC1o8$Z>WPKOjhpGJSBaO3qUx@EO#fTp^eLwowF^q# z!6!0z-tM#R%|`6V@!+79Skv!o7l~_be^=b7MeNPS;=;RHTGH7p0ef;fvK^M9sThJg z>u7S9e};qgJx**__^nKLW2BVUe6Lr1<9@YCFaOpMzC6sr7%QA8`rUB$Nwd?zqT~AT zneXP>8pr_;t-rEIN4Z^`UC`(3FFw{bK*ysaO&-xb-We`!kyUEDi}!B5N5(pL7-C~% zcd!K>E0uda5aD2`004u;SQ`GDFkKgxzdreefd%}L5|8kb5`-_N@~0{_oj77+*0aB! zTyF-lMP=0IyPX|hPzvklL#wuG_I@7^k6zez-Sh7^-hq|!UuA##{U2EJx;pr59ClG+ zYptl;G_gF?XgoWly*XO=g6rFYO6g;=olE+JiRnaSuPbV>}Gsi92Hs|Sw7q{r!0_H4Hq9FiV^UZlDgcXb%R?i9JsLFCzMDCFrmQPbW5AgUZz89 zgQRO+EiE`V`zmfPKU<#z(siYGi+u4tbV5u#JnAG%P`WikkAiv+64-&6okW>%4NT|A z=xAF@%eVq5_ya^!iU{-Zok9uRe72I(a9O_&KH>IkgGSzHkAVt^Abhwe5RD!D z3dg~0-%@;|4h;ubFF;SPuCD$OWwBqk6>`?(8*DT=G-L}XV~4o~Utl%P0gV+fKY&I4 z2k#%73m*ppVFx$|A4Gv0qjG!f(Hx9a=+~J5zlAV5WJJ`@M_IX ziaN)*IfbrTu?oXqcz=*nw`$9j<9K!B)Xp9H<=xneMLkDsGqZ|`$#cuV1!;FIgkSFZ z$@Rt9#QcRHK9Wova=;6~@gBh5LDfmd8yj?N^nJRgSNU0#TrTPCPG84JG~RtVdf=wQ zN|?+L_PlHW?iwU;&Xp+nDiW_2By5@b^R z+oSj{^kZCd-X8x_$2c5sjtZlaC_Ik_YSPcIb_G|uZZ1xL6EzF!n{F|ST&_>hD^@n* z{zv@sMK|!ih26Fj#s^tlMSSN|#ezrIH8sE5TJULWtuCw|osPo%^>h)P`kit6V;#3A zgb%S1hQamjeFWJqqAxO$P@W30$jf07CcF>)fsUaLU&W+#jCycaZmVT+E&Hz74iZAM zxH>~@+vt`R?z+qRmkjXFKUsm^0yZd1pvfFjEISiwLhC>MH7ROu)I9x(1Ajp zBgl896Dp$ z=Ai?46XU3Ie&rf*k>us&0Spib?q{~2wm^eA(Ba_CasM&C= z2Wy1n;@)1FerXa9T{!u#zf_F00D~Y0S`inQ6MX(~@?ka#klb!zZ$N{;J;{L{_=HH9 zSpcp>DdodhDGEjw7QlLHY!9$H9#N?Y{Uo>>aYWO`cpys^cpvfom}I@)QB!O@N5xn4 zb4u^oQ~cyQT@%?qWQG6V3!s%-NlYV6!-#5sP=Sp#81+&2*ORnCz0{1>%vbM!zbVKk z!w7w>@{ejOj`Nq7PL{J9!VqPP6cf>*7+J385$j-xhaQ3zhzR*NB4H#Ls}&L`iE%(! zpYtgtW6?gY9c{H6!ONpL%ZYY`S@6rUSl#p5ix{^f;%nKnoilIe%I3UTi;^b3e*4vW zRAl4;@hI=V%K^-=zpnaZtfZK?WF!RSh}aLvRA}( zChB{(GPH%M~mf!iU zCc%&Al)Ik0zJ`nEo6nlhn8ge281zzYgb|3rU+vYuK9dyVQ}d^GPb zTTunud2e(vBuRuK0kfetT;W-9sn|*`63TSyFU5U|IT#r9#M1*33kVsmb-`li$o`)} zSA^H(UeE%_fS@-(avFgI)P7HScoIK-5)cr0GJ6LXR=5Sz0elud3e}4Re6PJAUHI8<6 z5T*I~=-Al8zSn{%qQI_yPu2aF2hxcf>g&hH#$Y}SgXD>c38(~14O;X!ut6%-YRKoly;&R!aPwhNUj$Oh-PhlkcqV@&Lssr|C0P&cyx5Y1p`0x(2oX1O3~ z+S#!`er#4ApiKsjC_tMYw_&iM`R_H%U)hwmpC$a+*zlOPuGt2}EKIm>yUo|Zj^7Tx zBjZ423;Th;-&S{bOKVw6t@O5zjUDCTlKzK+6tut-K%teEKUqb-e~66O%eqp9@uK7veo|j`D`oyQwa@(`4Zup`#!)FB-PT`BD2=9G_?$yD-+9k}b^&+A?BlVdkqrrY#P_zvZ|%^|Q&93~HS!oK_hgRBY!*(D zY|fQhZIeHp*HQWJHL<^RAQhTG+D`puNlHz>m-C^fiuCs@9KqU^+~QdwoUy3rfx&+H zy3>!H{jRe<)8ubrUlv;*VWUmBFFm4{RB&QSxKC4w+lKCAiAl;;p!|`LnndFc!s3&# z>&8+jt$~}KMz(B^sVticS3Cv2`@|IvOzM#knY3+2>7-lh_8M_|_?%XWE87E+X=84x z1NIm$o-m+M*X7c3l~nyImQ;#eWSw8cAhPQ(v3qv)u)t>_HhA8BS14+6HMMHs2ga~_ z|A{M$Q*~_zT}khr_C}u7y!&Q~@5Mz@-rz$IR0?#tY2suw=6GlYH1i)H$lhv@DUOLr zVayVQwn5yiX;|E)O^04ua={E?85Gz5M%xVvKFAUTQK3B<>~T9CzS>w{m~%K8Do8mI zKDu0ZWuCSY8x+Ml`r)qs!q$7uq`Pv#Z9LA?AS(I^+TZHg2HUzL_2!{9o?%K{yfF(BO3svay@bJyS0&BYC9MniP2L#sW|jq3 zzgpADK7PKSuV@RPA&W07M1}jY)-xP);JE7PEX&4wc?43*z;RZ~J0yfojm=AW`OB@! zAz3-OmyhGy7jPw%zbL`e_ja!FEb7@#j1w_T6uh@4pRP0htV!Sl<`yU!@By_uAhPDw z?7UXfTnE#IE})ZIU0Jy{zDwp$1%!uYxT^{Z3)P;e0e1yZu}_jitiDY_L=Y&P0HD4F zY>x1H!c_z_3)_b6Y8U}R6$!jlRDfnyPum1gNfSuGJ7?S#1p!noKoT0drTnBRfNs+a z7(+S!Kqw%$W*bh1r6{jPGp@zsvQ5IMP34#HF^H25fHK*whQo0~7AYAIDlDKXr=~19 zlC^boN(X-hv@b9)F?p_i2?l7+_2sGvy!dP`;w}H#k%56WJS2nGcGw=9Nf}TtF`jlcj6lU!-%?vfa5Q!qSQ#kmB#Asdl;ovHw0QT z)Bjgl{)eG{q0z8rL8eY+K}bZ%W_#1AtkKlP2KzCUfA*8;aK10y+SFQ{cMy~Mv}&Ft zBkrv``YsK#X30iV8>j2`9ep^_GJ(>j6@@f?6WiY^6P{7}3ZKXahUu?aSXmLL`6;~} zB5$yX+WSwODpa>br)DQNl5|dJ56o4|QL_!>vlZlD(nN8Q_h+)jx|p~zc&2r2QPDVF%Y+zLqQD z#s0KH-_@k6SH0Vuwe-mf0BEb3+~V?nyFdF=2YW^_jmbB1)J@E6ekbQBAtA(j7aI4l z6n{VXk;3__#Ol>+9g+>o%e8Jt`d~#rtUCTkO%JKFQEu zwUbV^8Xt4`Q=Q7d1Se+Zbec%Tf~H%_Fy*%-mCC*FNKbd4pO@zvDUzP zcEIcaaLlr@vM_mQ_8oI4A|!O-PudrnnVs$H?R`v7513~7Es*XXBq1&iGTKW;#YMhr zy&H^mWti->qEg$xU0%!HtsAYk3q*eSsP*h-i<*q1~Bt)4f+@Z;JZt zf%kyxtVg`-T$rNp4aEQApbJtiEjBrJS=ZS7{!e)Lk@n;LF&(dgL13=4@Xp1Qwg(=5 zoUWcxb=p)A|FfRB!KESih(B4yBJ<8A@{FFMx~jRE$=mypa0eYFTu%B`SkP{eQ}ydu z7$b}DhuxEW|2^Hu$J_vx`T-*rx;Tl8Q)A^iN0<_8T_Cu)p2?!A=CC+WBTIP)vTrL= zEZr@|W@EIn9C4KKSD&3b}$s%FSMSjTkZ@__ZS}PK{X|$Zwl1JJ0oLke1+V8;s>|ueh_|&ju()~JLkT-7R%O7 zUoV7^V>Mt~CO<8wF8J#<7xzdpX@=^r$)^ldH6KhDdwLEG45!w|qcQB0j2<2FcP9$G z1E9$%a&+|d+MX2=V{eC{GK(JZR=%H+AS?0_4idty-TQ3|o(e-xRnGG-4^WJ3zA7&9 z*<>sJQ#?!7`er!>ROag>B3Q40L%iiyMr2b+1lrr5>HFb*=yz~%qIdP~iT+&H0_xZG zx>6it=Z!NR2M31+frd@{4*=U8d0fF!4_M9$2BdBPU0q(nKYx&2h6otYSYQS)mYWSJ z`bRKL#zUcACHJ`xd^@OR;dx*B25`DFjmtgT4>wkB`p@-dP!J1ztoIu$&k00$b9Bq` z_V?~0>}b7JB^dwWi<=QRWFf!Jz&paB{jY*r1xYh#eFRw)?ZnP|FKSwA_dD7urpUPB ziUI9596y<(3TkQZ?Na+&?j7vaEk-@7Hl+^@m@ShG=*p$unY)~%Qu8Dnc9&$;y2@bP zx$tS|Q}!WK&+jREfSV6jiz9|QN6o_%oZoO=BI5{tujYkeX*kcEviD(((0e3HN6X#cA-KAlKUDQ6`FBmO}A87}NNw_*pjR4t&X1GtweY{^{_Pu&Fm} zdK}rQ$RBifaH1NJm-mA~VS3{Ar-H(tFFqMZ3+cEwzkMZpc;!0dbCt+VdMw;X}R4;9_gc}Ga};0InVUd8DxG==w4;zduMOno9D#z-}p zv*eO_S8*aW_`5yqA+wzoOk;9k#lrbcajYouj@3PS} zh>~4pn-R z;D8ejr~MF4G(HrJQH$)DoI#c@pI<4LVPj!Jld;&Ydc-KbD7#WbP(+y9gfy}d@j@8#EmE%gf~Mz4J2$H;qrPJYh+@KP?i%@byFT?YngE zr1GeQ-(6NETh=1tpKv(d{ri`-Utt())rgFKG_maoi<)N;DLGwI@sIXJw#bQjQC^MD zt;%jH*zYsnx-uDml?=!XSrQCUEo-vwP?`{GaAAmZygBbofm-nFs>UJU_A>nUH?ys+ zJ+u~c^2bZm@7LgnJh4i046?LVAxw{v;MM;_FZJN=X4%g*IVNKs`xmn`U1HjLcDnVp zI^^yAE?b`BX)WjZU|xeg0GLhu*ex;nSR*CvR{PkBzsP=J;IYl;SJ~ttwhV-Xs1JkI z-$Jb)E&CsHe68v0m-hC`F+|z5YFc*WMPo>!pGnz9tewSeo)Ws`)A+6bdi9bhv?|BV z={~M_{kM`yIW31jXWidYlmk_=)8sWOEmDx#!_EU&7*uh9oc*d;-=mKeAu6xPWc?MK zBYW?e(Sq`(u)Z>&+zqx+aWE|2>`h=gn;Y*k?`$dJB%5o*Iv*6P+jK(uVkAJ|FrJ1i zSlZ1Z+IV?)nru|N56cCgKdIiKHny3{abi<($Ct^PSRqbKj=Q+DjpKgV9v_0rXR80$ z6|Gw9mk0qC8WFz&*&436y~XQK;6Qp|T(Z~L9vz;ML5U(NVkoh&y2|_X=|284Ay6_) zo|nuNpe8e+)SwM`|X1({D6}O&8W~cO_)g@{_pW3 z5>HfS){1mH4|5^C^`i{LpII65vVpl~dzY#rd~c7BQNic+`+{ev4eqhYfn_QKDZy*G z;bkej$y#WI-#{cD)y8OWKR0EaZ&|25Dh$VII?Qf;yPzxN)!6jvvMCY8LozThKfV%A z(csPVFL@6(Eorji;V0IqIdtS2D|JxUg<&I=Mmpc@*-=C}QnbvYznn;z7zdnh zlfWj>rp?PtL!th4^4ewCMgq@dTrjNG(||RsP8K>tIHdpye`A?~M?m1T(w`g}iY_A3 zlEKddN~naWtPlt*$P?Z9!G$CS1_t@7uWf7;N+(VZ59#RX*<6%F@cE+5&CJfv&Sdn$ z;l#wgExsop9~-uTnP6-`CS>Y^cga-Oa6_b-_y9#T)ELT94%W1gAr#z>Xq-_NZkm^| zU`AAz8cSd9d}*@c@Mg4IF}m_94L2+-(@*MAXtjGm-JN|LT9w4-r;l*3;U`};7QY^- zX+ndDVSo0+i3`1QRMP^JkdSF|~VLK zFQ~#HVvcuOln9~f=@*;)q4$v@kAzL^30AT)T0{Lod{$NSe}*W@pQ39$F%&$Piq^l7 zmb@yOdGB#nE@Zu>M1*^nS&-rgqxrA^MrWNEA-)XB)gQKP%_FeHwS_4X=w5cgi}a55 zdnH1&EX2*{^#_|`>XF2S*wB%?*Y};I%KA#Ac3u%!6Qp6NZfnIZ68x43X@~~1R{ddH zn_$t{g8h1#cHbe1MB&|V86@_8r-ulCC~j!BKbXK#|5lM{)($E9CGCHeIUQKM0?*aR zq^`ue5APsE;YBJgeeUHS`a!UV>pl6}-@E~YzhNXs(U`ItJ#}PMzZQD@edwQgR4eZ% zQ77;|eU07j)}=I5l%`Twe<9f=H>!MOD*n3qvGwmrT&KlIUOqN5(|Vl`5~O8ie6bEn z+~#hTGUbOP%nYE;9#%a~yT{BNCEXvSq-tVOa87`Gm!tHdNnHq8JLJ zz}u>F<*I3f=wJA%!k^jlz|a!>XfN;{w*QQI_y((#0>m6vR$(xhynOjmUw=+y?uXTQ z5kJh&Z#|ViU*KF3oYcbYnQ#xY{bqbYY&^QDKsjF(oV_r@Mj(1tUzgLKpTXk`5&m<4 z9e#~ULwk_5-$S2q%Y_D8NPd3)4c9I^WNr%y4cN~NQKrCEjNVr<1>fVr1mEYa$lb+@C{N$!`I9dP&jNUo zZ0bW`>>BzT7jAx(P3AGT*qXurz#yq8V&oHK*+?ZoF7AQ!cezXF@%qaB=dZ+NDgKfJ zU!0T`tuj8W&H1Y$mHCEg*FA|K$;N=6e;D43_F)|a zVgT0&g{%JQJ(Y%--%Qk)tMpRqx%Wd<xB3DD~&H! z(J;KN;Kao9Zzis^p7Oa!97w_TJ*_FXul(KWc8coe<^}`F!Giyz>%HTt?*ISsm)Rk4 z$ezcsMYhcBnN2p?dvA(lkL(c{Wsj^fqG^wejEI!1j7Wr1>GFM?_wV-k^XIzVuErno zdY$Lw^axjk<9a{`$8-;yo(hOr}YzXtaUjj1G`B8PXi<@!c9eD_2NTwBYZv+ z1bMcNy&_RPd?PmOgfvhLc6FL-%1+eX9)flZ)z7b67eGxxq=vDL>K!9o{Cwvm@o}sT zeT=y@CAMi$?mL)W1B433QwpEl!F>`>ZR+-MEgZdO3Tm}4kb-ZkxVgD`cz8HEu5E4A z_Snt0*vE3JaB*-Pj;?M^Rlv#^D1kp<6M^jS{uK!mvi6tlVl7V@Zn%rZuWXpMj21 zx#wHN5F<*;nF@aOS0=wSTun~shx!&FOU=#M_S)dlpbYhuyLwSs1YQ8tb(KPWSjh7Q zd$iBG@H4Irtxb28vDWViFrxJ;1#?Jwp~r_dIPF5V#if>L;VGXArpo|hQ`HmsN+&SC zu~i+gB&y{?TaH(@Y}Bq+QtrmVh<=lCA%G(?+R?s*hqe%9FNH+2F;y}O^)i}?26jpL zmVKQl!&_1PYP4iI%y%2ZaysUqXh^3zw>>NTNRW7e*j4_(>X( zF$MXSS=BV7%^GsIRG*JqM3VNBU_Ik+ypPUEycrzX7uj6!$;O7K;k;Fcmw%$v=0HVK zD5lVYw@P~cvizs8EX^C-G}e>f(%K&5>5ug~Oe2K;jw-m*RRzhvauAIxb=h$ZCMJTJfV9pZ1Z^H#ROjk~5dccO) z)5D{~YvG4}D&y(De=y!40!Yq1x#3}Ju5RH|-*#5ony+^-C`0Po=7{meW)GOpiHI0h z8Cf)%#`a9k&bH4}j+{oI)v}xy2GUpSm*Ffz#ZE#>8nInNmt!zv#PMPV~b^$t}9B4-vCBl4t0~N?eABx9!%8 z1O5l0l)B1%>c4W*(rQ$0Kx&1ThWqF&+1qK71XP%I7D`R!xM@-~(eC#pO&$~2ggHn; z4mToy)|Jh)4Iux=ljaDfqq(GB{#xb6T2}IDKfUa{ASr&O0p$Thi#7qWJfs_BEFHJz zq>g9D;QC3re|+amM?<Q)3mJqXZX|;661_@ zPukI1-EUPR+GxzgyHWCE#3+8Kx;(1q9{Fixtl?{PRA=G?$<*9fK#7ZzUZ#mPuP&CT zVUIV;G86d1qI$pFV*)j2@nl)1e@4G$V)~DEk7zv4rZyMV>6p}vGlIeoV?7{oxM_J3 zo(f^_D-)qqf``;w{dB4YHFE=1Lzl1DLch;<3WSFAEL|0@G<{_3SV1G0mo#CVWrol9 z4nce#=G05WP(nNW+~B<8729s?#iV{_l-lnR!w&{Tm;#;Adksc_H>&^1T$-Dz!9}1{ zdH)iW!J?khUoR5ERZ%GpMD7Qqkr8Ut8~WMdZ|sO2 z!j`@Y^J3KDFPyXVxmBP!qVeJsK>%Z^AWf=DdGJGnptrCi!lyP-b67bRTUB;aCsO}Y zpwi@0K&p`UF^~HE3Tt!@0G4*QheDo;dLO$f(q|Q)qa=;`xZW8?eei=v^xpZ9OV`b~ zMa2tk2%mRq2^DOM3fn{g%tCWG#;#!7%|YgV%e;vk_XX5xm1J50{Ei00>nmwUQ{|>9 zp4DFbpv39#UF=&t`kwM^J2MJCdT5fU!?2MI(UcHH<5tgHW%BKBS`LH*S3ksG<>L$b zNG|jpY-nE4G6nMR_ zxyfbCC9l(r$ zs@9ZDnFVnCgBQujd;vi>&~}U>s)m)!Nl)2azpn6TQ2+LXaD*VCJA2dATc($?%lnfw>mA6EC@cL zzCSN+-spTzMm0e>$o908WM80ij*7EuKai#JK<=zBZwv-2qN&4Y)hiA#QvpOh;0nPb zH&yic*e?_b3G8|C5KVbHpwp+aWQ{fB8BznjDrE0t_-cRhK>C;WI)Jx;oi8}21Dd`h zKR=;7fT?6mh$N*vtn_RuvC94aMZ)%OREKo;V|0vzNF5~BEAPhJ=2l4x%{9f5Xeifp z<)b(4(k{`K^rl^CktY_AAxYKCG75`v&TdkmP|X^%;qpH2daTsZ%+ex2OO-|(7f8UO zb4_BENwt!$+oRF0FsSs57@M2&!l8RID3Sp!1skf>D6O{zOnc5Bm@ridIyZ>x#Jz5x zb8OIvt-uXdFjWG?P*(D6C{`-BOvqf)X3xuqPI1vP>9FO z=v8j0RHpbqP58}e%p%8EhaBojM({}|_#Z@zTQ=ReM)X$v(u zzbNu@%h1+-XgwVxENmq`Ajuwdy?d-vQ#xh~U?seCXPNuZ(b7`Fw^c-8WoTiIofQDn zpihzUbs{0PiDO;s{UbqNxAPgXf@~Suv6z@R?(10KKacRMAP9nExx!H;*jeUdc_aBG9=$zkmq#p2Z4;>O_hF>U6Co4(mKN=Z zH-ufn%SH$lysND|{}qGwoIJ>n|H7Lq8%rsWn^LqwEL?6~&`AgzsIuI+!@sEi8bgOu~n(r`TVf1CKsa z1=#7+bzVklrAong&G=yh$kR}pt~cbGNaFffrt4@XE;vq>)U#mUz93S(kDoeus{f>4SWT#eoL11uZfJq7R z6r8Ez2{3gioBAIK$uu_wUw@@D1|4L|ILTeI}nMPe8}5B`rrYx^G!>x97D~z zjhZp6uDziaBi;K4ypccUVeS3s(IcD}{iD_{6?OHEH=A!7BOrzWB`@<4YZIE9R1J8k z4h~qP+es9N1A-x{3g`Z(#sgfv2P<@av(Sxl=a8rwF1$M8`I5lo9h|7x6>>#8x1A*52hx1_H`V{ZFm>zcU%#jiI!rriZ|?9o;ZEm?Fsw$fxvC$GR5xK>SUg*U zzM_wPsc7`@A>(COO{AQ9qD$x&Kk*xp8&b2k)23)%{9Dp)l zDD&U9laGEv*MspdWN>5$a$V9LG^f~kk@W7Cd;G$k%TW&0-KgEY3&kNFt;w2J+|n68 zj2+X@YruUL^=42+`&4GIUBOdQ{P>~;$|xm>ic@B*Bo;7fM$oV<|5+?7+h`sl`1Om> znAK%1H5WSAA(hW4F)>LQvr*9}4>>DJk%X1{yco`)xRl?NCm>gRo#zfO@B%p_NkeLj z9?;|QrkoHu;Mqfr_$C1zm3;^!niSR7^_YEl0O)ns>r8l4K)i|VpZuvjAh;0Z-u3Bq z!Z=(o8-!qn39O=IWV~NoOoj*-jT6WAxU}NVm^AZ{#FZf5#=8JV-l`5uS z2`&&K@Y+R~4E%@`1~zonH}0M1P$p{%Vkb#5uqy-lG%N!a7u7eKls~?Vj}Nb|Ff1s# zlj|WTN{JNAnWZpagIh~R7x$aN^2R_K`Es%}j*F<+G9g+r4lV2S7L@&2PmC|nVaPaU z?y;A?ir(BBOa(S|;Zw$g3r?1v(CYz^F+=A?+uH}MFg3#k3ED*Bgu|J&wlZxe2AC=w z99?wb1||SnkBWgPsfY)O-u9|q{CWUgKj(d>U8--UNhXTC2^uwugnZ`Qh3V=OUdk$t zEz7*yRg@!LVL?xv1V;*Wp`?svr{|gMMlH#~@ZDj>A2!!<~$O{X;as4!y zoAxl4V4t2WriIeYcq zLl_h;iK=Zhss4+>8*PyngeUt7b;!E5P^RRZNJ#`Xl!lRtL^aFj*|2_ER`|fZUCA*O zV3`jGCYd3$j2r@Ez&MRw!w%u2l;v&}l$yR1ii29%K&Q@xKkv=&-@hOH5&SeW!;w;6 zne8H31cqat77aEK%u?`S3UWz{qlVQ9LAn9c0G(&W)cm)furYg7wy#93me@B1=Fu=R z#+79xx-+mJdy-rMXOrU4xcKuEft^bCl8ojyv(1_zKogi-GqSP@$?WQAa45)=Q}aV) zH*$ySS;2X5b((8i$$m-s>D-f88z}EtOUN+h*_;5Hdw7=dQY<5*UgqpfI(1tDE6hGmLcAx$)zeYkr{LPSv&g3*C-J z8($xuyEiqnVMMZRt-@txtg60UcM??hp5uw-{0FM0nWVq*nUo~MqGKi-sT)damgFmR zw?dLj*K%j*miGYJ`D%QbS6W$()8MiDE!R}x$qUyRF({KDs_J0M#J>xFNMAk8**{-h zo_&(;(ovUyy8DnytW5iSWj)%FW!1dakQP)Bo`ve{g2bLD`LV8q&-QR4yoW7su?*`G9T6!IO**4hCY=}Az$0G z4YEu(q3bK<3(0+8Qu&s>Z!PoF2Cwv+PzIWrjVvWfgniCnpncA7nPD3>LHfTNBM z;pd-So!f!Gc|i*=1pu4HXp}f`Z=Vj#8__Irf;oMFMo9yeY3j2*Oxt} zqP8(rK z3?I|9Hi9@X%o*^J|6Gv~k`cLxk~8+#-tyq6$QfDIbZVd1|4ug+8#KuLkq3=nj)Fhv ze;;~@zm;3KYDe*TiELJHSW-Q1a9PH8O0Ij(EPzF@b0RfGpYB{lSG8DY5l=~RntL%G z0_SScJk857+7u0kY*^UWqqo?4t6yhKjpwCQdbXpo{&R=HZw0N7)=XHtg};mqbS6?D zip(qyDH%x8l$MxZsBhWpBC)xPcq?(C4=0z#BxaL3M388#bG1`c4`Y47Do1BxGge+> zS^^o8A*H~-V>V%#vhyNycC~hR^SQ25Whog-fyv&cP)aneD_kkt!82tGX4 zP>p@ySyyB1nq%*eC{DcVZ!gB<)E%hS!aHOU{AqkH=^;wST_!s7kzg=2OZjSzmS~)m z=|C^wR*Zp@9#>}h+hvfL>O&{uaz=Y%; z2x=kGfR0A&fIoEk@@1T00$^5r$J{Qk2hC8N$L(yXMs}3)S)H1amY0_UJuj0 zo&kdKndqzMy3&Cn{Uj3xTt~sm+1#<%k!azi`EJP)cKqJDPBP7@)nSa4*~G85wI#f^ zDB@~=e7Pa`NK_(`*f@Rk#_9w$8)aey%?uqHZv{b`w(nN#qZwF1c7X1gCCh+@NXb3; zth9H}A^Q+A9$lz!4_bx8yYN-{Rr?)OYE}7B(k8Q7V*FCbKwG$z9U<;0gcM&zNVKKB zS~g9aEfN&&hrGd_PNr0K zpA4ljXZWguh>DY;L_qfi9-?y{&$G5%%$MU-nK?eOzsGi9jLXQ7#DQ7ofU-wUwPuX- z1%c`z=Dm4**Gh11~cYdX=D zN|s6>H#B$euv%SI>i%6#Xj%nas;f1(zW2T)VAl$CO@T|L_Q4B2cI30sP;SY?)#=(K z!u}`LD)DYCL@3HvJ#qS(!jt1BrNkkZO0JsK-(6m0WU2Wv{LoEpP!6;270kY9pv2{5 zAi{?zm%$jv1RohmA{>Epvl~p!+8c>a%@D}mxPJHe=i1p|@KVq;8B*rPl2`BL_W;8$ zd0|H8u#wolmTsA_UqqB&>jxLSYZFzlv-9+0Nr-9)GRq{UnyftWG=0*&RI1>lt}vpY?r@FG=7~tCrQ8j7GABc#Jth3cT~V*m z5b*g;hV30aQcm6%3ohlkqvW|Y!msYpD01nYLkA4Xls&zegl#&+ z#$fx)u69&wGHN3}|;vp#P(_4}Sv^9paqpNH!O=zcM90qxpiHk*%tRa<@+Q0dliBR(J?Hw80@E zww*nlynaoW5ybpu#4$&P+!N5=IV|mc;$>tsU9Ofo$iozafpN7bh!6s39=67Hj~>aI z-IxblZ-YfEC=Ng=`W>uUVq);O6@iK6>*gl){pj!CzljM6P5;Lp}U?r;Kl~G$knX;PNSAYE*2=aaoL-rFXjwYp{q#7^bTUlh> zerI8nK~gq4oXgy29h5S*h&YB)vk~DpYd+w(4lgKfiM**wFK8ZW)?S6Xo=wg>m;8io zYby}UJZrgL)0IG~7C}x^cMicX4a{ZTyuU+T_;f|ax8oHvnotpiQB~)}F49|>B}fq? ziYLSbG3)m|2uFn}>6Hl8@R3aKyYj|k?N{t4WJJY@7rbxz@}R>$ekvkMyMrb`{B$MK z%T*x%(&Vl4N)1bx@QZF_ulQ$Ea=$m}GbyP)7ONeGVA@bdrKk!2%&4%j4dy76fHONQ z*(vO!1kJ?&~oekD>c4{0ad|*SEKssTr ze?ka9N5?Z%;Gv1rM54BY)FN*i+OELigSAf8imWBPh-f7Qp~;GFsPOV8`RtADa~OuE z{h{#Wn$$}u>l_C0b=LYNOvRjeh9B}mwZ)$c1sfz1vY!qzLZWhbRFzUP{kg0c6C!9B z4*e)@Akh^^Qmf`)#<)5|p)BjDPHMcwqvEjDzYTP(t#8YWkFxrRuUo?HX5Mw?|50Y7 zyW?tiS;jwTK5Esof4(lxy~$ogN;l)K^_r?72thG1Q7}sxSXl5~7_1zNdA|4Tnl?3f z)!BW16&biW0T`Tr142>$`^!B`OSVPhF2eK^FO2|vvj~dKAk%>Z*dHDq;uOqxcX$1Z zybY;kWo2*wclhAkxpTk)0VEHw5Z7S?xTeC9z6Nev0|T2jk^g-^`>(mZefIr(vuDzB za&o>)FRni_Wswf3tgrU~h&r$^v#${$c5h4N;TrMr@&fn~jHiKz0!kn_jxp%?&)tpyI%0Zf`W#I2ENau+S-E5ic#y7hYv9$du7j`KQAwL zfNg#`$Fn`R_7AXS0tj$ZQxoKsYUM|>KJzUrGx`c8TZ%rcI06^YT!1fj6;yrzWPA~{ zaK_qKWN55WG2KrZfIBkzUae+qeUYr!fNeE=Y@j4XTe+d6k&i0zPo>E%|Jn=*6ZScB z{}@!Rj6eR;JImteqvAOK!3dfwy*K6MIe2kUHW-(}o#d|`C!*iumBFTy&(yV| zUbi|tl6p5JS)$ZH5IARO7y*|}V%E;mmEWFmZ)2pC&!wKUu9Cq>{~Q9RrB~8)M$1A~ zlkC0Gc`0IJ_pCm>!D{4=>&_d55IrQ^c)mJ#rRZ&vV?}485JMKCMfI2*Lh$$P?oTgA z;5y95|93fQ&7i8x!RVnf5wg5U_R~1r$&b1ub7YRXRbn7pdSHQ=*{1Pl&KY~(YHhy` zBGObvyU+;N%#KCpi+foO3r3Yt!`M(Ovl2ZddY#DHlOV5{Yc?4{`d{e{)%}B`T9SJK z1BDnARm?-%a2I{(YCTM>gWt$0y^15tLtFRqCBSV0w9zsUL9)%kEyLUc+I{A6v zDFY@BuH8Tt`yE%0YriKd0R%rrHfk_089sVD%tv;;eQsl;PuDdlcY7v1kqOibjz5(Q!rPJW{5U3A_hGggHt zYw7{O27w~r&*Nih=5!FpVd|S8|3Rq?0&PN|t9v~L6ZL220gh_{v`KGoZy+?QqYmU= zhaYa7oSeM=@3@Gu8C1gnes&ubF*1 z?i&?rUoh_S99WI%Yzn2ofCe@+{PXbB+#lZn(&M!}4DK>vP#_l42nCz3?`J@h7fT1? zgb_he02_znr3Bu-0JkAf2@SXa=SFLFmwMswR@O0>K)_0AXbiVIxJBb<=dd@F`Bi zNi{Pg=d&z)Y5wG;sw3o`I^lO2e-kos1tuEW6;(=mDVoBhu85ZQDrH}ujSZ~a{^@ogI9sd@+*)8}EW5I+}Y4H&5{6=KqLJ7^n?Y zW>Y33ic9Yx*DCI~8c-%HQWw_aS63&IrB)cbLY`dpQ!oxu-uljcE2egfvslBP6?qdxph81KK~^0~y|aUi+YOfO+UPi%OQwWG zL?veO2Y+^&i*%|%luDGF8_0i8K`di^BnOtmH==$!gFz8!GU7-On%5rk=oP!UyOU5* z+y;7bMMVY3D$)sBg67ow`FVOY7s-`Qr?e3V#inbTGGoRtc zYXfQ;(0I(w4p<+i%ozcOmjvX#!^?J87zmZ(5)vjJDBNh`$t`T7Cre0Vi<)-xu;}p+t%)8)>UMOz8J& za$2X%ZZ13ny^(Ls#1w2q2x7QQHVcuWM1)Qu1b<=Z#FSv;>m3TIFk_DQ9}q24Xr;Ot zp3*3yvIfT9QR7qc6|1yV48h25voqiF)wtr2YbK2ZTKVzQ*g6^A&dwS6zWHH3Cz00K zcr?56X1vbdsr_hC`9!y!xa!Ni`^rU1yE}1&de4a{gO6`HO0=XYr&OORCzVl_wHfS+ zI`B7QyEE3&yI)s#Mo+$C(|lDUNv~|Z_BLH3@H&ONXECydRxj@0RgHvsMd%h@sr`LxbMP7$%Q6C z*>0>byktjl+GuUC2uAsY}?$X@&&CvEG|~n0=3aL{7W9xi#RI~`UDFBAew~{AO^<9z-OUiVp;?# z&`+Nl4S4rnpGA@N-M@byK_G3)>dtJI`U3U7PZ&Yy?w$Q{23T1I=v%>h{Dlb$(BdOx z3HG129WGJWJO7v1-Q5ksi4__dKaY;Utb~l5T)U*$$3aeB{)+d9pukH4qQ-Qo13y4` z05sih{}|zc?8~X87o;jGnnT5jkMulk&BZCyfj+8Oi40c8OS8&ZBO%&Xcz4frRQp@- zml%w)Hq-|$vX)_lnablKMRtl(j8X!3(?qMeDS1?r{QC&HXFeNR#R%g+HE<|YanjcypN?+EDt6--BCc4v@cA}pF1J2 zK4idA?hjOcn*4o-^Hh01pWJPidPx2EPi{q{xQ7Y9YqG|BpGsg+{`v|g zHbNg7RSgJDt?}%+sMP#BZ_qOUxU`DVmyL3q;MQ%r%Dm`u|7J;lC(W%LM2c^k|Ib|# zB$`Dko#YS%q;8H(tMb_3A(0+kY2#7g}jZ2d?jpdr2P33CZfd(P3?h=uOD%W*@|+ zAKPo2>Ffx8dJa0Mf>s@;;iKmYCa%#lsB>Ph4#K`Ycv*t)c#c%UPXfiE0${gzZD9($ zc3R%AXyx+R7*(b9@GNIM!c#Zapn!Nr*U49G>~YMJKD88q0kNd#Z&=0;sy}>wh0{Ig z8JK3RNzt&}tUXFeBZ#k6kGol;qJWljNM?-$nDb-RNYM)lBV zEN}55QuM8gWIS?_>_hCOL!Vv%;|Q(x(c+vJ?Q%n`W_NM0tA3OhVdxa)RV9Nm>V&`k zW}!~)iey8+m(RZVY!gNoYLm+%)d8959=BD0;FXhzNPVj2GFX*>F3; zJ2a+t^u*sxQFu-MG2b*Wal;=e!teO~VqEyu{^-$pMwSF^iAw60#}^R{`7wLxK=DV6 zH}Ns|;qt?LEJinWBMQSz-AJ3|h{Wyd$ewJV@D=G9sJ_ju_Y>mXaURXLxwPHmB?(85 zinxwv0K|6oR#yXHupqnF3BnI7uh0H{=S%tV^CvL^TGwFUL4d$Q3}Qd%NPuAc*?yve z7pOs$gf1<#*sB0DWAnjnu-R#cDaGfr|9$~j&xjl!(j4!=?)G+_QC-B%n?*%MRWVoN zGC>p(QqK!$Z9uKpd!kqx-AiY0NI09%N$)6a z+*HhV0tW|e?KfpAi8|cIAnK*3mu~IG!y#q}!cM3Ez$!#Gw8^sZ)t{Fv@)2$A?Z-{e z_p(J@oSnFf9~D?)IQ`1KGreV;g#MYgUGUhDYu4-xJzbf7WVX4vnM&}lGl!|$f6EUs zL?&dI5zQ%dpJu2Ks%-wf5+R|ACHr{4u)hA39AZHn8glXPbK%>!d;6IBNKovmSjJtv zSEX_H6x64bZ(dZ-j0;87r=|n5`nZomp;j*sM-Go-@rBIQk+trZ1ekw~KEpCdRi;ZQ zf-LzA7Jn4JLa`mJ=)v`QHsR(UrFc^1BsWF_afQkFn}3w6NO^OU9P#4z7pv^X#xL^c zMpsfQcQ0J1Yb);~{}ic8ra>$A`F;GZ>BlQ5nJ#;@vXBA>Tk9{gi0$!-?=bv$1QcedhzAqbP)6!b z~oj!d0Yw!Vuo)h;`~*-1FuX=p72E+0D%K5>%C| zh~eP>6wRxtT`%l>^ml%~z_L;OE{j2muYyieo#4Pf<%fd|2xdXXi0}@ETFnpwmLrHi z&cnNx&(2)NJ!tC*wBo^5aPA;xS+n0Yp zmip7{+h^Ha*YNT2F>%?fQ2T&BG%kX}#7)TNnMsvVBM7>{#t(Y`MVJnNeQG#=82oPC6;Rb63ztFAkNMt9R;O22j2@)I4|VkOr>Tj_(XU@O`aJ8$ zbTj%^wzjtq4i9s#5j2s2x;A(E+IxPjvL{b|Kzju&sV7GmvG9}sY$;GA%ugTxJlpyM+3m8_$gCcjKE_HiWfJkCsgf*z`eo93~71mv;oVRhiBRQj|cw%EPLw|7Muj}b|3yZR+D)d`DGseOQ z>a$&vy;SrkM~L@b}|oxj8vm+bdtHLW{B;w85 z*&k=RHPZi*K$eN*h`moKK?%Y@n!S| z&_iHu)&$2Ng1q!oMhEEc;fzOulF9eZwREtW3M}%}G|p;h5QxD53E;Avb4oc**cDiL zELU$8U{HyD?0K0`w%i?ir_A4(*$U^64Mw*hZtLUYLqI@491ADR2)76aF;s9@y8$f+ zXcV0zj`}WAJQiQPIUIbvFZcHFwEa^J0ERYhPFzM)o%pIHHdzoa+u4*7buZ} z1u3T%HVzdNwowSWZ5s^xHV$uamRtZfz7ck26Z)y$t|ta|v0xN-3Ih|+NrW|*Cy4Q{ z^d*9K{3z(_03AH?^k4|CFLP!Ay<;l*?ZG(B{==VZQQrrK&pkBD)QN$)t4H5V+PN@oB z8C&EL+{Fa@a$fEFM3CqQ4+5MNdU)N``1m(?t(#JN;FEa9Qv>uW9cj+y#1US3$N^o? zpO2<&RB{Om3j^_fxhED@hhXdnGo`4gsEaJ`(z;jtsA*{GOzMx0j*`=zzurMJGBV2e zEh~b`70~-s6G0R_Fwif3%b+98Q%D(KpUAat`nC?y+nC(@5~VF0XYt}6C3ON$zF8r%a^M6JMwaJ zoI`8iYiOk!LC?w>1%omXGFa0rb_rHHHxcGLKdP?>JSj*db-^>$I|$RmEr-3Sv#(j9 zRotkaFq$gNiOu1D#{9LXkH4uYF4r!5UH_wBo9U%lU$DV3=jhu~jh}R}Acu`b zyIYnSOo|Kst1x{u>(Or83MS^;yvOYf|uq@@Kdo;X2!_u%$_zd=7ma%x3K1nl$R zvx2eZFIc(3d(t)nwQdN3@&NgU68GhGS|}thPk<`cJKJaQ;j7|n#E+R&oyCz(BuI^m zvoOp4NBrIp-#^8VqNakDFNYNqd4#;yoRfch$e6|rCk~74UGW1S2#|z=+SEdT7RY&} zt`_;u7Nwh@X{u%9k)t&O^+fi_Ba?r6hSa#Cd;5G0Mv9rdbd!dZ=hhzPwPDPxM*p8fud`UtWyQSzv(0Y|ge9XxBw)GY54o(bsx->XrUyB0rYkC8v@M!P_QA=Rdqumme zWz*NfZU59XHeO(-W@OZ!5mz%bx=f&E$UJ07{bgmP%z&3S<+W#_8zlKM@pTTrvq22S|`pwXA&Fmn^6v4q4P@S582R)O-8yzfY5zI9TQBf*z zu>*%77--~Ar{aP%v(X$u+`gy;-TxlQj-Hm4wG=$FKYbFMwHC1M+&A6&`_O>bWvn18 zC#MyT?>W{m%xd|!ey7WJK|w*Vg{GvWg!;4u9*7=%en6^4l%5Q#<|fo`KPfq=HOg!N8R`&*V;+u>w}E^!WT&zDqJ!em*tz9$ydd{+^g5@ zG^4ytu}2%h^*7Ytf6Ajhr+zb?^NKJThlE7&mLF8Duxf@eUvfr&v+?agzH>`I1sg{1!?Qwp3rJtyFfiO&=N6apxUj3RN|U^8<80n}FsAe7Ob%wSj_md3iawN7dJVg~9VCpdC|aF8y~@ zqY$5%s2_CKIK!gVrM;u$&&i2S5C#~33YjBC#|DOmS>QbN-X?Tpy3dJNnI6asxw-GA zrltV%U}4eaG?HyqN&FTrDhQDMe}&SMBR?+K*U-{RP8sTmfT@mtvD5)9c!kVc6z5o+ zd>GK^uqlz=Hs-)wZ>xH_bl!YY+lOz$?jGr1^p5R_dA;yPfpUynlYe_ZygoOvc7IVg z?LCDoeV=g1^GG8t5Iu}nhR8Aa+-6uu4URa>Kj5pXh^((bha0t`7Muvr2v}iIKkV$3 z>bf(fb_f0D8I--b8`ec5*L(sw2pEUFB(F$FTp%OU8J==(D(>!9d};gO-n}bQQs#z+ z@MKY`Izl2if8f7=Kc8KuTYY1#KzrGm9FZG{uW2-s&+)>{3F)Hjl6o=W}F>NRiM=7DLQ%xSbpuQ zcRqmtVeXQg6p!Q!tOY0VjH`dPU{=V4bOSo6`Mv&wtMU^6`w|v5n79R$@Gfb3q0WLJ zKuH@DfuBD5bI( zoYRK9xvZpT-226XPQ2^K+8>;4S19d>jNbVy>+usagsPt7Tm62WQ$dr+)jq|uMXJGs zpD*RU*L19QRj+UZoFcGMvfX~UO@czEt z?8_Y?8f0}@RyLfwU+uiDv~dofos5J8#1mn2&%(k2PMe`0{m<#nTx7WiDvL;*!OxlO zuHVv^WQtqhhJVoGSW?m$Gt;-yIN|)EGALV;TJG~5UiyHzZAyV`?g<9z&+ljYqML z^u+9K<^qs_-d8jvuStQclj)<^_Y!JNA3<&P;^j*l8=K~EP0<`QNTI^HDJ~W`4rgIz zZh)3aK%k+q@#VXMH@ML(R5ekK9H3~I_4|&FZIMtb(2J_lP)=^#Ojn45ExN z+X|hi=p)@b{Sx^dB4mC3#m-PUG&|jlL%%&N=e4zUq9arioq-_EqhESAIT?gg{GkN% z_wS;*01zA)dn=*WV^Xaymgrr9)^zl!9H*j!Q%qi106gba?+Z7uHtkFvVD z%QV*gU(S?^kqarv=nMQH8s(777Up`WUk4{bZnd>{xAGH8CD)<^LUiv}+BAubFET z&x_UHuVfBgDMWP7vFlN?8WNSHgzAT|o$lN~LTvb#h6V>4OfP9=ll574t9#s6P-Zgzmg}tRx!(H3T91cFWE2qM8|-JTsie z+dKRre^gm9P_nMWLVNo-qGF17*7_sZ_<=*p#|t3D3l&K#uIaaI+lr^NEvS`hz7&Ij z;tyyzhZFy}6E=SXmc;90r$6Li?W!NW7XUksBNu{59R(nN2NSP+5vC`@tqGwS-2HM) z@bXV{DJayFgHJP6w^&IPw#UIGBLi0A%tne&cVdfS!74dH7ns0M1MR)GQ2+1i|fQa_2b50t78Y6co;y zZ~xnY4(>nj0(OA1{vt~jyygor7!+>u>yhOF3bcPb!9nNxb!Tx#M+b+APs=9aI#JR0^jbk~^6o+lZ#uqkOq3;AR=%;;4;u(5aqfH2 zWM`$XT#Sv(p*|SRBe~@FEA=S>IH2Ra5QhBev?&Be!OQSzTN`-lNC^owH#OB@uuH9?0Z0CpPD)%9WYpAEEN1Lh^ax?3<-aC)>^^1=~zI_<= z>VUcrwD+a3SS%M8qy)gj>Y2QC0W`!7@GJuF1t_0>f{hm&sCWUE1W3^-Ri zN*bEr8G+$Kooa~MkmIggc?vBNsS3K^Y1eNBI#d{%GBe8olJ@tj&gai-Dk}+zi6tc@ z7}5qIg&tNW0_b;QLJyj$BB|C1kM=!~XWwalt*4^{7~6s^KOe#v3>g{OvQ*U64f{v4 z5&O$ZakyS9Xw9+6(p(^)&Ni&9tLwMW?cbo5_ZioA2R2elXFYz>$g|63wa{Ul)H-9B z`flb5y7aO2&6WN1FAdw`{qAoRY_d9yC`0M?7z6s0b1Dwk3xoaM;pSVmvjInoj7=+kTrs#%|33wyuahuL z&fw$G)zgat^%mTg5T?|-Fdz(u@l)DerHwxzXABxc;7I|{XJ1BwPFQQg!1p7Z)*mS! zW-(pKy3b-#Az&{r+bwj2l2X#s7i4Bmg5&Sh6xhNC19ZTeH_)T~Ly>vJC*$zjtY7zaT;-zx*KXk z0s|#)f5ms*a@m*#dO?Cc^N>Gt=-$6?-@e`ce@wjzG}dkRJ+4wDQwW(usLaU}i9*JZ zr@@ptgv@g!Q^qo9C}d7ZW-3YuNytzXl7vth%KYD--tTX%zxA$XJ#SCA@B4FI=j^lh zKIg2DbHX4y`sK?L@Rh9`zV>iiPwtnXq@seL-xn&V zt-q}b$i&uN_cwYpL^?5aeODMz#xF>|pjj92JPpdNJBgJQ*_<0^IMzoZ__X!(3iWXSM<7RlE=-S)cRi9|d}gAPsj}LXIS<5F zdHDjWveY|+d9*VFrCpt!|HL+&$#C_6#Ms*9TTfrdlohWm#}ykHK7aj@p<6>A*I{n2 z=E7L_aDL64`k8lbuLZ4spVI$2TNB?E9DIlzCz)66{)Hk^0A5X8W3xyf0y|6Ym zzxUoy#!X%|5fKq=ZwPBb(jP;nkd(~RD?Ae?H!sj5dR4Ca<%bX6K8BYZHxv5l%y&1`S(lO6RxDP5;Pd46w&cW7Jr+3g5^N04>ez z|3m?kmGf{;JclKp5M|R(xzh6)Bj=3sd$GC#)@msU3HU4##?^Cs+S;a6lx4Rex+x%+ zBavXcX?wAb$AO3kaD4uhS0-OtTqL|=cxj%W+y{bRN&bgbYnykl4iP2G2&U{96nmgv zjICwFvofgdkFV-+u%A(T^#RZ7zmys00b14BZ;dfvtE0V3@lTM%8E@IPGY z*&L`>c$)lDasPsf5<(6r9Z*B);KeQv1+Sl@@H)uj6UJPAJg_znJuE#msI;@U{{f*% z77H3LUHSk%2UE(y2Tje)_?rCD(uH=$mQ0t^_PNx5>4+jN3LOx|MDTar@DZcLOdpPK z?M4?ED4KH_bBoW20F)5`x){$1znz8IH63Qg}jtx z1^g&7!Z&ri?4lqzV1wfR-rfb4oxaMW(!$3Ip1ZKVR*rymXMpm;7An-uq_L`x5*<3K6Yb?MxHYEl_+d3P ztE@oRx|;P;d9QE2_=@6TOUAsXx0=3q58t>M8L9o?-@XN0#&c$7V<7Ui@5vHF>*3bI zyz9BY-M9S{?%d%rehZZ#1N<+TJzoruR*hl%0C5w{GOk}a27sd#ENV+meb0@XT!Z#i zuD9u+h?Zw;JgTkrp6M^fb?Nx|GC;pIL0LED zHpas7wxeSPGD-%vOyC!<5%L$IgF*ozx(td{_R_ut9SRX7owp6-_p4X0P=hQkxxt9u#x}}Byz^<_I3)? zhf{7NehZ!WjjYItv9WtmDq?jEhTu<5dVU@pa;yeE!;4K+@59yY?%sC5@=_AU8};VT zEUjv&@h7V?Gz0X;x4l|efBpIvS|HGf^tXgJH<7okD4pZ-ePElwK#VHdA37#I*B&-Z zegM8wC=hrHAjJFo`@6dVIltjk`RAB8t&5|nlKpGAn)$$K+E?!K+0MdChs-Z6HwHpE z8^hB|coY*LifPkWArKK6X;|wr5iN6Q_f4*gNr{BkmWFx~tcwEN?tt$G4gMQAHgAi5 z{`vCxXJf3R^VqyO0=f7N;CkG5YUvnwG}1PG54+Cp^?vnX z@VuIvbf9z^w_P|&jc-4$bd1;^`9vXoJnxqd@8^6?|2q_^)Kh8+$Ot%#rSQMns<_C3VIc3}7vi7j=3#4v_HpIPB&36) zhKJOYlu(gi(?{9*L~zdmC?5H?b&nrE&dGU&;D-ItpbbEi$;ikYJJtsTA)Afg0s;qn zre}?&0?ok2t-((qCo8-9ptD=#cC$H}bzXONCThF7xh-O~%fg>OoR<}nbI2kdl*~{J zql6e8<@#H7bRdzl!(MV8D_wwO#8Wh}I~TwdG#-jH{Dh7UjW~|q3kzwfsaS~tekUht zha#;LTSH0t;`};;809%b!?jMSk(|6dliWqj9CQaTk8FP8H1^W9W8-3y5E|zl9JH|L z>Q>G2^}L*KsV@CNZ#;WHd|1J5D1LB^eHp`C)Wq4DD*PgjM#7pKg zat5r6g_xyk`PNJ6>!XO*$`7YlMWg==jzup= zoIRjzuKgTb6(k3>Ew?lN^Ja5Pp?S84cDk)|=rXWVW8qMo6KqA6mIUy4hT~84EF$8; z3-5wa0dP7sjbYm7OD6_$y-TGOAA~wl}UR%O_bytBfNAc)9m^ z{Sxh30IUAA@vC;Vcs#|03j<&wF1JMgMiRw(Npon%s6z1wAS)ufLRtLtL`?5+A{K?i z;j|8clgOpu>_=PS7a-OgV_fsG7xbiA12Z`1SKiz$gE$x(x(ksKqyQof<=Wzm2s#lnxZUw80kB<-DK-B)DboBIS^}4#H;OogT@IFzJdR_QK6CIa* z;K{+*>*q+NUe$^GceU@lL*Jb5m5<)}y>I95r9AJQ5;x8(uJTB$tE_Y{`glI%;h92x z2HMk-JGbrDco~=Y=+Ea7PSvCP8*kB4ep%QE0?-}%rbgZ--z7gO)7OXl44`-}PZNTj z+WGU0w_AH>mXNM3#nCz{DIxK9Wd%MkHC{UGIRX)6mhO)RG^`b~1^RcXHR6j07C=8- zJg*GYLaeIDmIMl7aNnaMk4u@oP>|N?EWi+fiU7$6jR80br&UxyWew4(Q8pzcCCv?0 zW98WtLJ}6ks6Mm0$)$2AMf{cL-u5Z{fXg*biiZ=i`WBILVj|>AL!oMpZR&#pv;cki zVjg?u{lI`~Tn+$m_-X8Of`RA{3>na_yZ3I6PzCGJh)!=CzN>%u9)2?p2>LPXu3r5! z_lcE>Y2eykt%|qGSTJfNT#5mHGc)BP16^ZIEq{XdkZRQv^yycZ+9Hen#th1ks{{9$ z0mcy9vS&8b&z zjSjpB=HdMJQK2{D8G_+?)NLb5xv;z(1Au?~_GM_l@W_OD351{ijmy(@S8SZH1EB5# zv9*Xh3jFm{ZfjCb%Ij(f3CKA1p3tg;~p-|DUZ>~(D=p~9{9&YX$ zqdCzC9UX=x5GRfwE7V|3yWDUBO<(!>lW>|~9rR_Og?!DFo7O3xQr`!GX|jkaYmaDh&*>@hj&!Wtv+h1UL zOp@Rw*f!iY*H?f~9zJ~7eMa8Y!pXT4k`ywevWm**FYcj%JLcx*h%?zSL0Fssq5p?i z@B95$6Di%&>{oUbre>GT8JNMGHS9XnJ9p0E&y$OfjvrXMVRrW<*IlV=_lvTb-L1Wjslm;Jh83;w(SVWM1~8%Loo@Fa`k7$ABzhWIdCsr=MUb zoScle=HX#&Wd-*Lu(L1F@S#J8z|38P{V$`d+MpPf4QSjWLP9_SSO~0o%vr~S=lJn@ zRPm1~htQ(eK`I6MYOhOqS7S@XUqw8&(s1D$8 zL031B@%J#?gD1I|eWtgmX=!a>2m|VnN(6qQ_ZN+62S=OatTvfc+76wl4hQb9Y4j$T6b`1?1`U$!j~*a^2u zl>_SJE?_E%pxp<#LzG@jE}&M*e()gl`b0L|xZoHs2r$sdFTTTb!}hFel?LEZ0@h}$ z5r0vYy^mJy*yk#puB#;vJ+Ax%%Kd9w{Hzpgt=+F<{WI2=sSr=tXlc>B5|mexN%OT^ zLtfg82{kedHh#Dd0A)}F)N|?nYh{RRHZwCr;ryLSG=U6-@h}htR^`4k;J{x|%6goi z&kqpB$M=8E3_xBSUjS`TG|S#(P)zg>BiZ5a0(*vS$VPSR(%=~nq2hq_$+aY!2X5z) z$ukt7zQ7AEwSI`(_;wum%7YA$s}TrLU5gmiyc-<62u@FA$?3?RlpDGsp>kLZ^UB@H zHZ`eZf*2};0vgi87-7`TW`_c?LY&}KL+zz|8$B*7ybe9C*>iN(3PB5U?yXynPn}4v zI1dJ@J*~-o`n6ARg!w|)abC)%#d_JvnVEXPeNNTwZFB5^#&9V-9}>(QgV&gF>ckUb(!KL8O4auVp=zsL)^Fd=mi zXACtXs!qJVYWU3kLE|-<{OEn&6@(RN#`Tio;x4#dAVO)$NnwYqT3Ur~N%-<}dkL2R z4)0T!&$Kywn$AwD)?fG)XxfD>qq@2pHRgFPTFtwnFT#0b!O?+SeE0eD{kS;X zh?|T%T0}J={4!9WQqn(+fDijPST5W%grV!#pEa9{u|zLxd|g~xLP!uE(M7r5bt`p914r*Cc!S9z1-5bg?6 zy7V zA3ZD+%2>RpaC`471U$!aDlHX(g$SQ9H}@CoBc2o?N@Vlgsd)hMXkZj!pa3fLmNPLl z90Ho_FEM6iVE8&Y$9aPZ8FV5`+xFyu z9Y|*Ik=^z15xKCmF8W3d4Sp<|ISZEWVrEn~cO!p(3%0MW{2M@=A58NJfs9tfOU#=-U4DnXKcRo))t- z(rs2`xxeV<$M_#{T~Dv--w?X%q}c1$P|6R@g|NCDJ-Pt;s_@Yx^b0XYhVZJDU4%JX z%yZ62pW73hhpw(JkR^~KMaWC@vQgA_eL8 zvWsl=^heEJlImx;yo-}!P|oUmq`FkQW7NQ2(lA$V1B@TC+O|+L0SDv@bXp|G$HyaW zrXJ+udyX0ur1C1ke?XSZ0-VFRlJa>LD)uH(n_1=|1*Y4rY2sECG2=v;Pjo#M>dT@M zeee6Iu&~x?Y24R>YxM;slOn@XKR*hm;Hq^oU;^?1M zZ>KF)KAzb-Ma_A_57!Q5DLflc7-&fY&CdxiAnf}*^9OuEdk3;YeLZX6Pk5oa-n=1v zr`J1=MXMkuhMv%K=ZaQwog0?P?v}P{XIM+sLAJogZo#z9sy=qrLNZ(;>aZqJP0)h= z{+qEps#s%sRf1*b_HBpfq|wlX+UN{#9A30>y$7aX5zB~|`#g}~@kzrdldCrsvWJN{ zEk;I<`s|P`2xyRC7~5vqWD-^hE&z4R*I!;pu((!u)d*O{sbD=JFly-Po0@Vd(pFhU zY)HX^JA=gxkFnS^HI*gk1=_2gfhvauPC!_AF7Kt&y6|rpW;;8Px9BKBfaYKs94fzZ z*4+F{=AK8eL=hx1!k$ZQddCE8z7yZReTE1JdVH`_dl;7rirqU~?|)l8JsKhs&lgpe zm&Z4oXDJjYWf8a(;q3F^;P-Fe@_MI`7S-Zl*`E;o_5;J2T1TdIX0|OdadK|K-wjoM za$*8K$*9t$P?Ps>pj)Ws^X}laa0o7dVMm_i4<9fg`d6 zeXvY`$B#H=mR45Je!kp8s(2>12NcMS8ziVav!Ys6^4*r1gtjpUpq#~5g@KTP`eS%# z)>jr3U}lGpIyNzJ3Gz|1q-ql=*5_+;XbmS8-NI_<4vYNOO2*bD!1qp*Ce@JDH`cHY zRLxqJf51s5*54A(fT;Zs9a7_tz1#T>;1Vbj3!S4?c#=EZWk2EgAOL10Bv>BVoS&Sm z2Ca-NkMwq5#x*Y|#}cRn3YOs$%QjHeDtc8P>?#@e?IZXo^RmBqHLzlx+UL-n&&{%A zDw&3G%1alXkpUPVM|18Xz%;RBfUs|^)o zp=LNIsNE~Bg`1eX*PRCp!HFOoy+}dW#*M3d?buz9lhH{JHBWMf6*Kk>cXV{1LO;zNe(M$- zkmRt@o#D0V?1m)`HY`~EWhEruFGtyDb0B|03x=ule3p(EIQsh9T6P70Kj2vWV5u~p zM#n{AbuF#RXl_J*)xtZ0WVT2~V9S6?LX^hPHG;YlUZM@~xE9P+E)(h!ZymcIC|8Z& zYX0KCMScd<05EYkIoC}tqGtfkNCUwnAcmeUJr}l1&ea>JmI1HCD?i+c^>`Z>m;1xNq@0e7A&P0Y9# z^g~dVqEREtlUImxeK$Oov&SzmD1(0>f^m8iCq4ZF4s?g*69hRlY`bD)f!CLCe#eKa zI}9{0WOoAqtZf(x6Wcua;z34WN0A0EU9FJ`*h2;e3E(1(u9CPsiY`y&nh8Gph7R*G z5IXp>ew%&?VlEIHu__%^pKej#5T7Z=Byy3comSL`GRWz%ob|u)J_g0h;3+!aycuo? zP>@Z5qv`v>FHX`ZHB#h#J#=*Ll$MsFT0Xz#A31!uXUffHvGph4=;_m^Ev>B|gztkT4ys_4yD-3*mR2V! zaorQ!wQ5FPY72NQDh)tgVN^;wIk8ca^(Zh5K=2+8H{!DvG16{6c_#2lF zAj5e9nd7+NH}V>opUu@{m-G6Gq8Ui|J|>f0_l@IKi<4lc+(ADK<^ooQ)?fc)rV&~8 z-fM-`>;M-RAj450EM8tq=2IN+dNCCKh4Rib@;4NnYKOs_;4ft%ZBV>a$`Tl6Ba?V3x zNwY8?bil(ok9B~^T0e3AyYPxAOYF>OuD6nrl2XpI?u_ExQj@hmCUrkkOE$(n!l}Ky zLOW~amvYgAB-4f3y|wEVmFWdCIYR=}U-!@MW&)Cwy(cU!?WIYj`sCi8mjz@_WZY_c z+_8gykUCKe!iI#8c6;C`p%7yG5!9zO*kqSjCH&Ms%8z>tLgd2p=_t|1g_v$N49^}yqA?uuc)st~*> z&?rA4#bT4lGa$x~oW@aWx4Fnq~AW#L+X}X(| zvZ8OvjdJ25K0LtBM&C~BqtGv@TYt$sL_q@@9C4l)x|5Lbl`HqA9*vD7Im;RsUHCV2 z9s%bi?B5%H8czua6D|N-8=I3BnjpdLFLYDwYQkB=peeu&*!!UH2{Lwom%~UpfS%N} zG@!rUe`0e0_!=-8ckeP&HtCpz^0poDkOFOQf{~0UmjZY44cbfIZk?H$n)>nMRf&+h zecEi#;NV}B-G>s@;if?CL1d!)$N?;Clpo&(?O&}YZxXF|lkaf%zET$Ezb`K=fX_sR za`p5KfEx+Su43HR(uPeqt>A-4n8z3}oUtDlUYwMXu3+3vkl<)Rf~*3ED#z{C@n0oI zwGVUzz+17huz(1jtD6zlA3gx(AXb$FQ|!F>D{Nkd1i3IfE$`kniT*IBF(nW<$UhmF zIDuQD@}Fzj)fpo;X{SuY2k1x-A5z?tj^p*dvV5h&VD<`h8-jqA^P<(FW+Ssn?YFBW zRjjG46{GC!?_X{|QhMt&x8Zc)g91Q(9A+mB7xT3Lysham7!nZ>W`9uFA}WqAJ{-`# zprhl2&lD}UswygVvW%nL7viZK_N!N?vTDEM6XJJ!;?(Y{?-}H?6>gwP>tFHcIb0MMT5CI0|CBGLKkjz;^A76w)8qgPCc%%=DiZFU2 zAtQqrrG_EFbD<45%SYndqEsw=ee3(|a*5WG_x=6A3EzPG)N@_XhV)$@Lz+Oe;cTJ~ zgC~FhY`CC1(mF97jYr)Tpo^&05vjq`VVoLWcyQBL=JRxfRk)TqCcghRM9c&ZhDqyN zSB64@GA;7^ACNJg^Ph3cFscC7l8gDhZ4)lo;jVvpA4LdwJG)9=HR`el2MlNvIwq{jSe3I+uuU`kqA7o{jA+q9hcD#KHTbqLax|gzYGu}K(O^|nZMfqnl90T^h z*_NgA9m`B@%rxBmi_RBKxX7gDG>Jxa+d%nb4o&>0eQTj%r#;~iKmig9siNS;$EKL* zXp>?oA!sh}f^x$+_@|c~YPIJ)>0?9S4;i>?ZAnObR*8{?c?%**K&VjY&^D8w+@>l@;pGFrCkP=LU#<@xxC)i<&u<3>C-Y0jU*~;!YL=ADc~xiH>``avsr*Fgm{Zf+NC*r+Jv-|FV;R0| zZ53HvYM-Kb;35w-gRIbX@AxFw(#c*tUYtWDeDku4eDVM3>pulOZC6QHFY@xf0c0Yj zvr0J<4aW!v+WA*)ZQ(@g>bl)Ec{WA-Algb{S312U((QFNH4p3tF=PP$(@K{X6BWJF zF`>th+h+$_4-D=T@PNmD05reSP=f96%i}6R;^4*0ch53Gz5gG*B^;a0eO(09Bv)dVanV8yXjfz%O6KWRZ$Z6K{i;wUjK;%!+WZb*= z04;e8n+!KCIhw+OMQ}He;89S5`2pbwD>J`VY6WCXcpNZ3s<3YwbRz-*R8HtTFwoP` z&>*7XsUt9Bz6}!-6QbLRmxH<-0qdWXfEieD)e;tD^f`fqW5f6eG#`D`6dicDKsfqbzX9iRK-Q>DZdhz{}k zPy(nNyzPvPjG&;)sOsU{KyrW%k>5MDvU)}Oe~lQ``IK!=>P*o zH)RNL5qhxUD#HT>fuALUvgZ#p1`HhoNI>@*n2R>kO8k5=hOVfh6$~H(swQg^ zFM9Mo0t+J*^U;#~Z>^2AzIv7EybTpB!XOlug6||5UHodlLnrsaTlg;P6pDpI2K6s` zd&_HUWzhe@9UB}Hk`;Rw0TP5)N5@rx2|9o=V0%<|c=l)S$+|Co*S)8mf4&GNdu-#n zb?YQ&P64YQk|M}?9Cf(m!L)+8AUw(214;gCPf*V#z$@&TymVLy@nNuskhbh)(j zkzQda^P6`g%C{OC0-$W;Os?JzSZ}fv$NM{sni9_!03UsU$s(6ArtY|pce1dt59Ts} z>dl&F#)QZ|GETq7$pskaZjIOE=U;(OQm=3w#~URKxUC|{*+-9Vtgq|rocR0K3v44` zGOFbsiF<$=IN&>X?hFYI&irx8&+qB%meu!`@42=A`82WvbCMws<=~7X ztYk`Aw8aYRP`*Bmd57T;`NK+sD@#mW!RW59?2XLIhWm@!TqfKuPf5BqiG3_EBfH|I z>HXGkYV!W9hI5>j*RF*J-+t;1JPGP?O4_4hpS8-9?YWKyb-n`!;QTgYZqU$WsI$R* z+SQW6(o&2g;EpXQEv2D~fLIHFT73R0@sFZ`neGDP3Xi;I^D={C8!w?I(SqZ!kA9n) zQqyI4??W;kWKt>uL0DM0YBtN(AipO)}_G@qNm9@3(&-ykt1=mG%1sE_b1_$N=_xA09rtqBt zocWy?>5t;Nr54Wzq8E&}@pC^c(#)nglCWP*Qxj*`#`?Kl;rl5!ETyYD$X@~r`&X#b zOTT`3<5NT`a!@!6ey{Aq58!tkJ^UcJg^vphzx_p#jd&&}hZhxMB+RsL5-y~Zg<-h< z5*Ik1{Y4>qicPu8x*S{@HxCbfk}=``_+b05?HUw0{`{R7{1|{OPRl@$<(6G&o2~MaoCM{T}PtREfg!^4_j45mYpom{^=WH9igx%m(3S zg7Ph)750zKMyEmM6f4}Q$(nq))~93`@R4Q7#UlIwcCI8N~80qioqA8A%vuKGz z4t#&&*fDo#!?**elrGhKIG|#K&%&+}Kn?Av1vjT?gZ6)^f8BoqA~M)yLbL+22iF^) zR;n8q9;CY`AAUfYU;FzPqRgl?2L{B!c*npHY<3Z@k{vX1wD@;AiSN>&L2wBrBo=>; zH6e)7^J#iv80$QW7705GG=At6N*q3ni?5X`p(e-(J3-ll2it`N_0FGH(S4713IrP# z>%$yLFARYVUGS~v{@D9r-G?8X>ZlIz>=&Sm!?Fq0A^U7JG)kO8yh-8Y$P+i_G%@lq zI9LvBJq$j-K}P>UYO9mw&Ihpv)#;9UE`Y_U85?3I9}0@zd+DB7cqS*ej^g>^6U*?^ z+<{MbFh|<+u&Y<6h_GjC*?H*HsV*P=rqSx-DcGt;-656AT7Dy=E|mRF)ZX!&6LXnh zZ~zJ45Z!sOqk&W1*fiT8s_4B<|3gO--%0+nHYLYy`tR+eXmMuK;^8OXUc7-IAFM=7 zRX)u<=2FWYdxkp}9n0RnzS+Br~raC+T!h6i~;Bqp~6V8qXZWM?+E35iYLA zp&?&r+UU{$6|oOQ0_eWCot?^sWQ>tkhOBgE%z4?_r*O`jzhp6FJ>E-BN}a^XNKQ)h zAIo`*Pq*(S1s8;`K#skv_I;K&xnL_@?fmQH>T3AhPv#?#Kz%pe{f$k_?@O(sKQOid zNdU&pQhg8 zlqvi)@%=Q`0`uHyV6)+qK>?lC`3;E13BwBdd3Tj+_*ZPUI>yHy8tK{PhYu z$I#c)^;smt|F0 zeLloV{As(L#=6I+!62kuT5!wjbMV3S_s0Yqc9+ELgAerSv88a}oD0PURvA_&n`muP zB~vs(iS+i;_f60l!MW3Ia8**EqjCHfK3Y4`r-$jFtE2cu@YAC)^~0@6gC*ywcX`qF z(S;Q>%HZpi{q6A@p(#-%_1q`f**l`+8_dQ**dR{BWkSt%2-pf6e1--F!c;{LNr4)$lry|WfDi)!A@G+M>OK!FkfC5!?#aG(IeM-IXQ9?JQlYMMSPxFCMGJ;i9WJ1=ZRi6B8nNa$I#nIq{8p|Yh@ir%ER4#7Cj}Hx^oGL{`maU@#%L) zl&!zY&eu@iDC#|8ph0Ohq{S4Fx8^e#r(d0}kdo%Fd;JfIw##6grOCRBNsoHc-(Afu zj=6^CLJr{VN=f};lzaD%7$Q|dy^Tav&*u3)mQLrUmKcp4loRAqJO8u8D2P226m_3U z6EX`a=)zxN{1BrYo54Jfuy6@bZ*g(-0|B&WKjINhLvw=KdX0{+SjJKN%vl;`=lRmx8LJ&P|wbkvv&=( z3u)vdES-XZ z87)5gRBz`n9&&TGuk4tmpw@$_mrY}*m6cI_fPp|_XjuMub$oQ_Dl@8k6f

Z|@!U zp0PsP>&~4J@-C2VPY;9=Ql7gMmE~7oI47EGyVYe(y61|sOpn_7d-wG3-Z`z&SGel$ zbR<6`CAnd?u(+6xS$gnWu|+ng%DO)+%65vuCDPw!Cdhp5MbsfsC(^25AHoq$-iqAfMV7d;Bs@iujTEzl2vPyRCvA)ZVHu&IR zO@v?oxfU!^!`i2L_`SYOXBv%TE}T4Od>E{M=qrn2;AMC8m1w@G#klg}`4hYhPFusC zZLQ8XqZj04%zFP8UX?LBO8!FYQT8__(IHGUBVp!zSAc6X_5}b1_41_(o4q+4j;OD7 z;ZHLKlK?^x-~^py%tSxsqeqm+=XX{gPaqoefKIV|!5?~&5qs>_r>oONsMCFkAyGhC zzsgJx*BBdX$TGKk-n9#OATeO~;XzK2&Ks(XwzGyEyE|KIt1oRbsmS9+V7MSN`w~3c zeQ@6NP8t3F-gpRE4`z_ah!eD=in{tB%&uRT&*5mHl1$U;t*EbGgKG92*ml~+{)HAF4+v zT7O={RA`bY8ViXfl9p{JG8EvnMa>9t3}i+eFBrVm(U(f5UIGz4c9o#~p+~8ytGl>M z3!|`CAbeZ#d|E_P2trO)5UJ6qT~F3~C&&E4Z>s7%4RxISPI0Fqa)!TqKTSCHBU3;9 zYmLOau~R|cjM$0aOQK>IrCoyp1AU^n;X^$=ztKvDavW7vCzP2!#ch+Lzq|lR(DYeW zw8J?Xh0^BcpN6w%&Jcnq%KHQJB#y@t-0mc^v^R+Nt-n*48l`_5ZOnVdxas%V#-BsJ zy3T(miRauxCYcs3vu+*seg|_b98s7;;_l`qN*Ny;J2ve^;uHgk7jua=fItyOSkD^s zfN`ueQ0Cqnxs!9#qOULKR7#fLWB@<=ne=9}@a)=Oo(u8w7Aq}v66#lO>S6!`Nux9g zy>Mee8ipGM@1{rl6xxl9+4qpVEklJKlE4#66Gv4iQ zyZ+jaT*qg18C=eB9m}`J4-PkJ7%8qs75#oQ{^K$weteC^i{L#@N6@7WBQQyNc&Bx? zn8mmzVkHi%iVDoOEsTaeW5WY^si!vLsgZ_y4x0>r%1M5cs2D>H!USJYG@@hpG#UfY z$_?N1v_RuAu=o=#u9sP_Qhz+iei8AIT75EqxGUQDjL)um2`suJA-TcIoJ~*6t%8~X z^r^wYjz4q~OK`I}B9*@OQQslM&2-#e*iiP;ovIktV_{A-PPPfz)O<$7H;?Z~brSog zM*~;{I4sCm|FwG?{y3RPCZ0l5PCJ`GdppO4xDLxaIkaBkp8hi>_-D-Dh~V8!_-l=S zVKsJQ<8&<~E4xSue-R0ZXVO)*Fr;Ap3;0GSHvU2Bkx14PLCJMV^peq|FYcVg-ys>c z#zdZB61#u5EBj57TfyI!i`VbyRhlxR^GDl-Jd2Z`J2|5&iJ+WnYGvOUQvKgeY@!-) z40|6$4_hYDIFb~Udz$!p;U#?Rhokn5Fc;AnX~@z_xADo{tm$?|MERgNB^^0AebRWx z|9%_shQ|tC2<@?Y3jP!$^{~X1RPhdUOy0<5mc_|T)>DeKq%l}=Q{_;BevyyXKQNSr z<93+%*{VJ2)RO5#_V1R#_TX}}QBUT@h(3+Zk~qpt38j~}&424a^(IR7spM*x|ISp& zSJ@g!ExCS;b`WV^Vv#=a<4k4%TWa2{)as|FOH{<9G4cJkCo8lMsl}lbc69U{&UGI+ zsTQ_OVbLr+9x^{0N6prAB;o(+p1VZ|-3%{@&t+sAhKe&lSHlg{>volo(IyUwPv}*$(g^OikI2 zgtH7|j9=49%dsD6-^9J|Ycta-#1<88CMZ+(`Ol1trqZQ1J0Nanz{-u_dT zghBbk=ZHaClNH74ZY}sqG~|s+uQ$|({;iQZ-7A3(fsqf_Mg)I+R2ZlFGw2T%WAV0q zK&d95FD9~CiGMhAKlK?%FQ_a09JG{k2ueK{e?|X#xlO$<3eC*%(ve!Lr+@Pae#?Jk zw$fTJ`|p&K$O>yN1h#8ORE~xIr^i)_x$n;npHX(|4(cU zcdMA%`|aQ9hZH9t|9NA>Y)S!B_x|SvaKu_FR1C_+=V|2Q{Lgm&D$OjZA4eb53G_=q zsI;tkA-YREQ`K|$WUd}+j4a-_jjeSiltGS^Pp3wOTdCfKnv_od^0Jp)^9idANUaJK z-LbHJ&~(1bzrp|9{C_ZOjg7sSaB=WOtps8-tf`D$r%<2cgx_C`mUA}GBXo?nvJA{@ zT4D(+(`Uk_=LLl#R39qx7O+RW3&h z3vA6Tx1-i|s5kdGlC$0!MN}pB9Zhdz6*LTF8PcQKF=9L5OGaIKFg4(nb-9BOJg6C6 zsxt?&<%07@&C;bmjVh`H-V$|-30#H^4>dC3BV2*WULp!+GE?kx(*i`GB2j5( zHXeI`fQ_OQ4Ba@-5=cg1PD~1-riSsQB!X0{wZ^P!H;HJ0;iYY60-V7`h)JVTBqdFh@FW-fUrTQ?ReJ2MWs@-K)a=4(LTuR}Bs<-bADH*&ZQR=Af-Av=V>jZ5p4ZP(zjGk>Hk>2ZgL1K~G>HP`X{4p~#zOXWIq%5FZ<;m)r4)xWBKx4z!Mn#fk z4NfSy($DE=u4M3pj7v9(-a#ysW}o|^O9Pr0oEkjh?q6=+^9)vuqb7a(h0bx?@auZ7 z!mH9fYSOCT@vb!>y+9`Z3sL*4>E45e7A=m&4-~hK|7ZwFxhivf=&hm@+hUQ!(1!S4 zYJcjdGT%AYHg^0mMO7ATB#bdQb4<)0_q3mn%g1<{%>NB~ZnIqacq6N1z;tw^Sy5w% zBQ>u(_b3^Slnu#1z09k6M~@+oP=~EAG^$QsKGX>gr-Cwf{|L)qcr{2lh-6fC3v7Bg+|HQuPaYEQ>=NCgoYx za3BdX?rC*?e~Gw6brhux`rK3XT=rx5{S!y;&djRr z(vqhfwr$8CsSAFyl+5Qw8?=$p=-;%75(4GY{>A3gu-@yzO-Ul^W6wFK4b-Bmua5x^ zFcfF8$W1l!$mI^-tz9ROq6Cm44!~UyT^X2F2G7S(MzGmHNd~2lhgH4jXcy~!IV!(f zic+ih^p}QA+3w#knGdF9b?WhW=1~6Ojd_`+_SrX6;3AGxs%HN==aN(rfG_QC?`q|)O~LI+_(XJ%4R<#Nx5r>x;%t+ zg@!V{V=b4i)D9evd0~3;La4*pfGFlZ{+QLV?joFMP}!pYdG zOXA(QuD`4;i~M`&>fpBB%+yQQRn!UICbb2EFFfV>}>;F-;K-;kTi)#Hh)R@ zFKLb4!=9V`rN_4Aw=&uIJ7gY-WS?YS$x985`bdR&4bWCE`8-QY!<>x_&_ctvX9x1x z)iuaEcYS7S3Ej9lIDe3RkuTH??R|$6KT?Z-j}3bw|9e*qt3`G2RXv*jH6vi$>cQJ{ zM&Hlw%*J4!w>vZVg^!-;34HazFG)I-)l*8r>R5+Vf8T+dhV7d<^oUfRBl81-ONEsQV9a*u)0V}Doa9CsT86W}taw)VtC9Ujul;y;)?IqZ zL?SLGog#l~%+5{xg2oz`){Gl5cOBmE&~MyDBF*Hwu*c>&{7UG6pl4(I@o2|q|6Llg zcjf|tr`O+CroXdKpB$T}{g=^N4xEvXVgPSJ616Li_ugh~TzouSdr+p+eFuk5ZRMOg zs{KSwS3qi7*nir<6(GFhSC80(57)Bxfbxd&{ zy5m-p& zd2wy~wU^xZ#Yq8jp)Wf25O3G%NDbeTsRYY$Sy>E8BC;d=v6=Ds58RBA&DQ0%sVe(8 z8#||x-E^u)dBfQ6yo$cue%7S(ceJy+AMpc7JQ)iReL#JXh_VpFI?1c~k)7i!$Dfw{ zK7OjtU;Wd~c%v%b=;WH48BR1)^kdV1|9#)wY2x9mz28A_LT{Dgco{H^uOhK5RnTD~ zG~$}(u`mIx<8Qw><-~ip6?xGo-7(23i6N>%lF_$23&UccWH^mYUql(yYV*hm#5Jao zVY=GIN57WFZpTpD#%`-}mw~T5V3zykK<7wubY{g=e07c-;tH zNyJy9C>}YRA}wXpNp@+H8?IT+$L_6t(1ZO#Utasv4@>n4b1Hx=-jZ> zG)SLpOM85mIC&?6Mt$pLmHz#xpPYAmoVF+U+&F-@$**L^cW?8B`9CGm^|rT$809HL z=IhmkWsMh~Hx2tmlrcL6(tsKC|0t_%!}xaT{~e@Eg@GK7SLV)`#DDsgSAoQ5Y%+pL z==>4lSW0)>y!WYjCMI#}l8tfxR8LpSlY8I#>7vBxw~I-6>{YOk=jYusN&bI#frL37 zhf+Mneq(hB_8DPsxBOF|c6{drE3 zSf#TrNnc6#OZ)f6qArxE$-^B7qO|U7*1fl_t*uxwz(o!Uj)eKbnYiG@V`8D!0!RB? zKj{C7&sslnwbdKY=ed-WIUn(o+$ea7!kC1_agWZ7S&q(LX@R4=Nt{ZKUw^stl-T*B zeb?JAUVrB_cckb6BmbqE#x&OZO=mB6#cDOz&XrK_{r9bu`-r$fp}CzzCFp~_bxCdc z2}_%ZtT*SCQNEn8nDFc`(O_=l4mNl6q87O>uPT{1s(3(+_|_Xkl=#;6{v$WUaWD9l zvN_V%&{YZLd2+D0{M0#0t0xs1Qujw3o<7c)u8oQnR2pqUkHPD~##pVU15^KgyOZzD z_f2O$IWzHYBdHEfOJl_hpL-q{LmQ7lB=K$O?kDrJilh4MZI2vHAF_HVf4a~>G){~B zsmxd}@nM?v?!KR2bc*!f&uSvW=~w!XgKG`9w4@ui4`ksa$GhD`qeF_J2Ia)@PIpxS zEmoDsyPr$(+shIkq2I0-^tGb0EjhVv6gOwIYgTGSqYjuctM^9eGWHy0=8-9h*+u;H z(S{mpV|2B`0|i^p1QF^{tx|)*$BHZq=o#2y;^r=BvA0PS6C~WVwcnz(5*-RfP5Wn9 zSXl1Ekd1B(_}QYrN{?>@g}e~hAa4U+B@TJU8{5^UaYd^R-|0bop*| zM%E8iB0V%cYL{(F>C#H}Zz(9-e0xP8zVJlp{2QkJIgOH-f1DN$c9f5eo!!SO2Sc<~ zKxop~?u2%)toEi^*eOaNQ=zv3K8BJQD?vC^WpVp}Z+d%rfH%T+FzK@&g6Ud70MZ@& zERbWGPQ3kVd@0^c;E08OT3Q4?w>Qx{-7eFpJSKmBizlu9>_}tyUA3RjjM%^amyp-# zCYB*KQ`_xI9yr)HW~9vOBXmKXxhS0H-su{$%zJovvyUW=S5w#%-u{ksbqN0VtTqr` zGxZ9E{}Hi2h);8+MnuR>Blsh$-IP z(Gd_)9zHxAep;M*SFtJcd6ZfgFAC6wn_$ZVZ1SyrV^Ouq7mCj_VTP@w@MHU>FK5zcv~7oM-P0x;-$gPX zl9=As`pb;Q+Xfx)B_+aO+72CB!N8TIxHxfe+K_NC-7%{3$Qv*k0k9xp_|(dz!j|`s zk{&4u2IYy#$t}D;EJCS)Kge6|GgScKDM;^^FJU+S(BF@)8+XvHYb!Fvq@OPOdPCZe z%^)nu>$pVT-qDF>KP}F|rn4_|_7O-^1wR;+TwPLn< zNzVAf{YsejSbpiG%3JQ>ptF{l?~55%EHvLCs$8@$-aED`{O55fgwjn z!Az>d2Bn!)&!kmvvHD#4Y!-Y?aBA=C-{G&Eu26cP9AOz)Id$%%hsMl)rBRKaa>&aS z+JWWX!B@zSa|JSl{5&^zMMH0YP&>ECv19jhDra$ZiZ6&KBucuCD{=N5Pj{o?x^D4P z?5OX*>+30wM42G8B>IB~O^=xUSW#EEitg@za}yva{|wmr6M(@doS0mLKA?a{#lfn* z75`>!!dA1f+JX@gu`1c#=-DuxuEjX7tySmVAce%2s4_E1oq_a zB^{fadpqU?`5*j0`OSAW60rGq(bD)DN)r==EL!Ivz+-r)^bzOLVOwDCe--xwLouHa z99_*NBgwuEOM$KS9^LS629?K7F|7O&hIep1y$6=!sF_1`?P`(dK#yDn*)W!)$^}Hs z)DT54Mjw6K^bz0H3(sEO2T zoU^13-zz;#GWCUmRcYJYTYXs6oD*}B>3I{X&dZ{@ezOXHWjGsK*xbPPtUq$B?qQr8 zh@@3wi55iT3nt!Hh%=zyT3YK4+N9>$uEHvbA#4W~Y0(5>FL_)_$`j5xaAL4s!?zCD zlULOAR4l#?>rg$3EgF~ooCPLGV3vg2B)j^vkA@&)1H@uB>MLmY!ZWwI@vOXul-k5Z zIC&-^jJXco+URQ}hJxXSQ>S)I7^CJ&PKHfi2lK4@`>iVtW?3;w75DN)#_i2>Xj_I0 z@_Z2$)D5Y^FxXu&9RSVaSPvj$#XzYptHUty%E}GWFO2RAJhD2op8^G0Tr#j?6H9awbtBKu9-hl zF?s0{DxN07jyQhB_J|ebGwIf7`Z%Gx>(WT~rwqkHU#dx3RO-l6q22NKZvNosh?2O* zJv_IB?%j%KdszL!O{_~l;1#!sm7q$xP@^}8V1u;DkL^{wAqu|^%mpjwsM6-z9!=kn zn%e(ncPB@M2T$&E^M@-hE>?~_EHHbt%V@MI(vbW0;OpOMqd#|xe`}$yvk15n)@2&b zv*x}Dd)=_puuc^_&&31-$Jpdp?nBpJnx_k8Rw#}$ZO!-S|7#CXZJ^cDoaAVkL z&EC1dmW7f1B7R-eXTHAPbxAof>kSn3Q_a=iIzlR{cJRxYYjc)nEi|Btw+RJ zBHi>7Y~KAa8wf23;j#hNvrSA9Xk?xE3#DYJXgH4L^{UU$oyYYm1|O!OZv=LNAFIKqKuQ&I&7zEhSE4T|Ocpn@47G`!=*~FUe*8cXW zOJtXf;(MxC-_D9BI5f-5!Ga^LYE!pZUgjdzA+@}#kc<+sF2UqQo*=W<(WDbC=C5S7 zr#Ck-BM8nM8NsX&e*4hMXE#T^Da1Lx(j2it}RmF?U5 z-W298QHGAcG0ifUu(B2I-})W%-z7+LR}Plu`J@p*qo2)OP4D8D@2s2HqJuA$?Rqr- zUwQB0&Sl^Kk1L5Nqez4ZWs{JM$|{nXohVUOHX%`Fp`}npqR0pd**lRfD=RXSP-a4Y zk5l*kJ3gP|`~45T@8dqM`?`#`_j#VL*YoupkHINEYecgr5ISZ7@((fRDf=Fz$zz;j zsRak}&pOWEGaP&J^@r)HgC+rOfup9_kuZ8FHMk_7o_J4pjQAagNMBUMOL6-2ioOy_&P8$ z2m1&N5U85ieMZ1yq^E}#4v3b$or4fKzjOEQ6f$49m!t#{QsxBzy0sOkDgsORP&o&D zo4uvQc{|nG2%H>JO&G}!3%RF2cFQ6Kj#e;HI#1J+ca}XZ|a37^PwL`pX8*T#{FVHXiYsacD&thLWh9Em7bEyzfykY{UREM~KP1(7mHcna(eZ|uTTFYlnCsK` zkub=eeWcjd7najkmzy@`ly2uCSk|3q{I=N8W9`Pd^uAAGjw;SoWL1LVjD3z|Iw#zM zE(Uft&8`gZp{u_wBAFDow6EktmEp6w29aN0P1*D&XLH%gl}es#9@%-9>|N+uSR>aR z8B>>jFKh1JoW1cp^#uz%?Js(r`?YKparpSQKYJC!eiU2@w`PrtJMhguIjXv-dQG*X z_+a9v;4$gNb2WAMyssY3d-iLaelGI~ig1+y+?`;+@o@%VfGQ8Ft#`)$hqC9I(c{c(C`gw>+iTcg4zIVP|e(C z^i3Ke-3292xSPF}61bf-ky=2?;csCWl~i9>C$KxjC_Q7bA%TZTesGB37r{n^o8Alr zysBe8o`F&?%JCAg3 zf3-`=5tkmaTTluz46T-rLqM<7H8S!nQ@inO9EfcKSd99n>+*=YAhcyY<{PU@tut#? ziyNOkvObpG8TC%LWq|9YjLWyz@Zq-Z*4LVB`8v&;o*0&^6F>W0$A{_8iPqN!dbTBn zLCs|^9p-ihA8sh~5#ALa^TCeb#K+A0@0b~8jaFWkzilo~OLnQ`T_kPrqII2^ z-KDY{C&flIbHdCXQ#Pq~rvxO@B>A`rACT z^0#?@E?b3JUh?Xt_!He(FP->SzOKLPnX7Tjm0*i%ufIwXJb^oImoBMk(^sc582Oa` zOr5CJUyRX_*P_fj8Ie7C^~k+sqj|kkJF*6(4s@SpFcx1p!JA9pNb$id>$O3=;QC)J zT9x8+I!7y72Xrh~>3c|QK6sz|Aa^?D;p@b)>`5N7dPd_g#b-OF6|Wzoq&iQE2BJcf zc&h>!_0SRfQTdfrT_>wcq2Gu?B3ck&yyRp{K%ekG{(pYj(p+P zSg0sTNy({W#BKDT-eKRDpM|hYi1uB%Vj~s>Ff1&8fS8f6ZvTc@3{a*)u5|}$_#;@? zTJ(U?Rf)iA0r25d_t~bf7NNX|EElZNJ%LZb6PFn=p|6GvAmP6NfjX(!d$fc&IaX`~ zB#Yo*;8X#=vG>{^*8(k_2@`{eM4r>$NvU0v1aTKz1uRb=V53L1Dxs?n4puQ1CfE^C zNuvy#ez>hb51R0V1nK8xVqznXMG(|RU!8|&HBOdOp?e-iMHO1M0{NVP*|ORbk&ERw zH-YKq>VDZ9Lu1r>*tdVzR_gO6p6~<%I|~Uq48ItHegW16Rizb%>=nz@|6&1_2>&<$ z{ogqbe~9`L0#H8eUcyfk9)s8neDuMDCMt;FpC%Uw696b23_kaqF}&sfAT%@-+z~UdP*gYgf{Kd;iqClUKJ(SMRS%g~dwxrBDqji;ydd3e=_!cPQN6 zsW4i&3tPBn7VSC_ygBaLJ*&RJZMVm$gu0!Z)8vQyhQH7^Uh|heqBzI8U&UXK?xaeV z_32+V>*Qp6eO5x(7MTKygr^+y#MQH`gGaftO@`9_X|rCpIC;N#f3C0aOzxGqA!l{o z=%owME?@2AcFjNa*G@K~(aH6W*im?Gvl8!~U@aWTUeia9&B)N%zMH3ymy8w8f4{Xl z`O=3^cjd1#t#E(g`z4{B$i@>M$RqB{aZbcM@)QkyJza1_+C5Fm>zyid%R7wIu5=%W zoyzlmWf>8|q)MN&wrG|0`pF{ouzaBdU9+rWeH12F!%jse=4 zQb%YY5{1;rGRh9iA2%5G94H~$u(1fEsQw=~sXS4g;y+OOs!Wp%ppOdOmQpNQ)ER^r zj8znygzab4p}kOP!;ywvNXVo@k!^bw^uGzMM9>2WjyJKU0Z8sL^@$zMCw>NM^kCN6 z4V0F!$U+6l15HuLI$;TQBghyV7)WB%0r5wG{}3UZRKWbvP7&bePwOd?n@+OJ?j}gN zFyj?E_pZTQw7<|5e1<^?^^kLn{8OU*g-q1s`cy7l1lwwAuHgOS;2H?GlYg=pAbK17 z`1l+zJ0ACtAp1)?njts=pA3B!{1b)s$L3~;nldP5Q5X0%UItgC59byR>EWT&r=jV5 z1^nbSj`#LX`Le@w#T(AU)L%kywu0G)-ccx);k`>E$LPQ|w9A!BHgU?NAnPH(JC0KP{Gfvix+)Z`swykL8=u$xDECEyNmf~Lko${mk;i*4>+q@XCKXRk-DZ*E z=^i}Te&~{J`k&UGn}wz)aM~#^ihjJDI?vhq0&%AzG^@2`_wg;_|Rngm!7J=b5`L|Dh9DVhBm8$NIiJeAqs9x_vp8HJHag;2|?(sOo&D?gFF{%-X>9A~Zow*}S?a@gY_%Vaknvk`qt5PiUJ!>O^HKs;^; zrXIZqsR}&M1CLU`Sw~z)>4Pk>=;VuK~*7ewuD0gHaqfcG#t?`s2h)*IdtGa(vajsw95&S zkffv}7uN`^86i+Mfz*Hq9^h5I22XvXM|tlm_Wl&V3}j0iqr|a|_llY;3kJrda|iq9fT(c~qe2tbiO% ztXY_Vhtm$t%n;pbv|HMpZnvQyjx%`j>(@i(wGV;=icJBOkYgl?N@5DcZfHXOfP;|$ zR3p#8N1Z_#pdq`UgB%oksZvOSuP@qqP$&=p|5(>`gy+WI68ywIz$dF#J0{#35(+cLbNBj$U!d2E@HzMMr$MQg2U2OAmzV?1;~xEh zNyPf!H9*=4onSEQZH{MeZJxs;>%}dHC*jD`Q?bcIVvm|+on@87%2oCJ1m~VFw_Bqk zGSE|`;Gm33(6v=Q=-RMHH*M3xHdgbT&bYi z>$;kJI}$s2OXLfZ85Ofl1o+KKrIU>o6Cc~nrKS;5&kZ73*Z zcW;qn_XjUm*q?SoEGZZ6+<_MZsacu#?JI)}7m?g0B%s~`IWfOi#1jWH_ z1=SZ}l7oZ_{r%8Ahs;3c5pN9DYz1Qu!~y=`p$8la=>E+V_;^@AcJ2{~K&H9Yx(*IM zp_#&JzWd$))w6LZy4f(F`I_&k@l0kSf3pQp!hmif%C9WV&`?l-BQl2L&Za=pMn&Z# z+>CrM2ZG_mJanQYg4z|mkBnYq)b+8>>xFp_;>~j|8(8lL~4Ebw;75J5Y(qy;;V22 zVvhCNkg6S4q2LHp=DO3Tf1g7oOtDBu&s=T5Ugf++y^Mg-X^Le|7mDfRX_1p1o+MLR zORrP+7iu}X1|9NEdH!9B&ShA__EA$rn@vMaO+F^D!WY`R9XhKlS}%f z)HL0<9NIL+RrJ)Z{Y?C#cK!;jJiXl)<8~_FukR}RgF64DNo$z-UJ8&ta;IN$kN6Z* z!`7TvB~>`wY#*BG*g6QL+nAN)RC&+w3^T@~2&J4+Y8BYKr)ul9oPJ4THLK9LXUVvQgO4v2sUSq+ z;KZdUFJFlOXb=I#sr&G-^_aVCWE7>Bb^|Ueo{K*`psWX)4TNZkRfh%v!%D&ib3Gc0 zFOgiFoWqa|`RY-Mhc4564#42*Sp*RKB$k`rheK+LQNftS$G!ifqY z5SC+J2B5J|b_z;K89kGRfEZpQpd(`a|I-OWt@9B>xGA^mj*eL}!k6$*b_$@f@!45{ zokseFuEZRnjEjU5RACP^S6TI4p-hRw&=i>`s=k4)Uan?Mg5gOtBCwmBKFPJ^bXbpi zn6MplXkH>ri*cm9UOfP;w>4wEi)IN3FD}Nn-ol3o+&4z-iTd!M$p zSza$XVT4H<-UJCUYHl8wk6;3^9r@q=`;nJ0tjgOEGCy_irIi$v)kH*o;`%UzLaKrO z6bNp`scHw>qL3TBHf+O~(XPJ`H76u@A}l~h#S2rezmQ*o^oSS69QMV(K0vU7wVBij z!sMtBAe@2mz^PMRW=0h&T;pe(rWQ`nY1OFMb+WWRNlUgE*rM`yOOa(mwu$Pyml^N+ zrQEL3zVC10&|{S~>KHTZ;jA?b;PTgeZPY!)B(L0ILX#6zRyZY+CDuh`e&*S7huZtU zecc`BWB4|@uc-8Ur-x{DG6e^H6>qxQQGbI*NLmR`5@aba*+w$*9U4Cs|dZ{4)sc>Q4AEljvk4C=54^k#BezNppRivS8 zI@|HR*`6cdL*{q$7)rHE8B=epCvRy|O5~p%t!v%tFm+0E`k3OIL&pT4KHt;B|5#hZ z|1$F5ki-J?X+p`5S+p&+v29YM5BSvURbkQkq+Y4?W(F;`Sj%%-*>-)8%pj>e+R&4j z`-H8qN`6+GreZ%nmOlRi;ooiI6XEQJr5JMeZa5b&j4v^CB!Ve6&ek2wE@KT$^0!aE zV3M#kfJ+9-9&nDOpcjne%upZ#!)$a5pn)IVcpz>7Ug1aqF(}A*$AzRGFYtn4!6_gl z^aWd&=yTjIeqY*3M`H*)8KpPX)*>%^)U46w0)-^J5)WVd z@!1hgAZ+t>%$>v`?N-ze<@v4K_pk?>LF@~r35eE&JGxU*#ny*EYQ(lh3{+}D#SVO@x>nl*EC3FqV=hK<_sZ{ z2X6K;Yz&X?`luM}q^iIj5In@O(=pqSyc7G$$mlzgOPF;-c;n=kJrE3{ zqXQ8Fj=>Y7qvB9$0GI8fvo3M|Bhy}lsKLU*`5enuf=lTja{PiGCfd-LgTR*lZ@(uXLUqGQhE3j96eAR@;cP%BN$NMqw?*?O=OIoal z7frZnxhsSs%ndVth`!2rHy#uA%BXm+)RyXG=}^5~rf+>M@y3m9+?uwUp(|(GnBpSd zMW1@zu|i4E^@_zbp!hhxl|2 zgvloD3UY|%t4c55oz00ZuVgptUeq{Tk!AaQ*XER!BF11dKQqJicRS{LGsg-8mvEt%GDijml+2hz`(8t7*+`Dt<{x{MPQMlK)GJb>@+FuNx_Zqd9c1_X$p$6siA*fT7t zo~YfCqZ0)EU9fS`81d}3Se(vYEJVOhoC1Pl2T{CUwVg~-~h*KYhw zZtoP)`6qfsxJ+pEVVtBJ^X?tCI7T6$lgtd5LUsy0NE~e7B$ur|jf1WQn*XDomZ%}m zoN2(#MQDJG8B!2#73c=J ze6RS~+#mdX1suP22anP-AMfGUiC@T+67)Tsp}b4Gv5P9vgQ_rNyKX+2O3$4xtnLX?L~1oowtkTaUMfjd-Duq3Ru9t^Adi!UH(~H@vRz5yJ8& z$vstIzWH!SwbS2Kk9)%?MA%k4Vr>xo5dn&^)z50kiran`fo|oBEYYY^RXxiWz7&&M zZY!Z0_EIm6RbZ0+$Xx2XsF5Q*zdUKNTAm?pY&aP8A<}(G(-(%vqHzHfC))rn6^BB-|KI6QSWFA#~VK>&zF2dM>C~=^snibpCRK_n)!ExXbn5xjLsL{yYw^Lj< zbC@y)($_$$jp#~DF9pvwPzls`=6I9qaA{ncm`~y6CsG207zhf9;Ysxf7ivANrQ$ZU zD7#_e<`iI{cZMe*Y31DXFMIwzX5H?y54P+zeQ>$pXZGls{ig1t(~Cv%3SwP@hSVSK?#)B&LEnj1 zo=8N|y1{XI1nG_0i}MIW@o@!^)C2g@6poH4lw+h^0e<_lvI1kbMHCWP&^y{4&+A2# zQL(X!>)Kc8ve&;|3g0g~#w_OYpd%?WxN2BQtt7&QUu0&O1kR|ZFP>%Hx=h95T$GfXpV z4Y*VMzW2w`l(ECd4colw=8w$S>9^3Qo0Il9PnnMFCvPiES}5Cqox8^nXJFKa2c#tJ zK{EQvtfl9;DlQil_PP(A?Y%p2n@8j>XU>f=$A``93qOkApSx;U)0mQWHS!u|U`$aj zgP0OO)4JRDMb7+MMoVG;jg>dJwA>WhKBtL(awuh2)f)<#zfp>_hP)I@b-MvqwF;K$+9YM!dX{73mByY))rCABTP;UMr3~_idB$I?{!H;7Ti?myuWi-;%sn7PnO5~} zy8gB3v--W+8Y;!3)1^C=T2EV?Xzy>SkbWI+zk$Cm7jK2D)Lt}pId{qGcy9Vh!h(bc zkWh>wkwAjK+EKk}JZN~>>?bazAY$n>;PJ`vQ=I7_pLOK&G6@F2~={p@)aKh@skmW&@obmG_tba16{$%>o z%Jb@|6={iG`h3izNvxRO`soE*b-x7fNK)a7tQcj^ zhl#|1pi|2?L#)e8xV~5)jW%#|D$w*Snr)sAq!uiw9q*VjHS;>ByUGF2@EURBtc^#N zDnGAj`fcIoH4c?en8zK)kR8c7lsMpk5l1{Zd_up{8|W7jD$N)Ys!;$b#)OX!-`*1^ zPQ+@sgWrR>JT%scL@=#GD)Z&fmWS76b>^1o$?hD9WL@cWad=q#cr1>GOex#t#h7vQ zmW+N1glvwN#Tr{UUMlouv@1vzSO!`BMJ6DZ;>o7pWaS@yQb z?X*oA_^ZM5@qo|k6cvj2$59zkq>Mj+w_v(Jcm%MgsPU(Z`w72Q(eraSc(!GLcDj## zdv^lIDG@+QKBd>wyZ)%B6d(J?>wD#>Hh&?@0Q8JTku#|L25vb0#R-oER@8sbRfx@r zUg<8=VZvGih=?@(Omwh_UT{q))7h&vLC7izWC21%7*{Ty2AZrLO{s9!xX*HQ7D-~% znIx=G3_GMHB;E~m1Ah#KT?igRtDC644`O_Skv%#Hkekk|5`+&a`oC$&4c1nc){EAM z%4K!SDg$cIWlm?A^sYqq-J%URt(qdkrFv(1-e_(j_g?dAq=ylD;z8sWf@s>S@ z{Y6*`Wr!zo)43pNA)#v$Zq=0{XP_L3K9y=6$^Cc;z@eqL14aUvzM>ofT#6=Az)%#! zFl;{E(Cuhch-0b2Bv6~aBUy_x!UDCJ3|e)-Nd;elIQHL>M;ZXhOG5@95($pRFyKKm zl3)l0Ka$LWnFfHgaUvHhaS{|^w?f1coim63F}R|#&oUqu2=L$NBJhife&S`=Op`sk z3F=@p2^Z8fvP{14MleLqoaD7s*`H8-LLl(Fh9WOT;&UD{trnl+B<}rm4-fjPeI09* znEJ4_S`?|jz3PWo+|?cuzrndjHgxHOc!&)~V5s?CzZQVJc-Ur103RdE%8u8+%@Cq! zr&Mbr+WUwVXvPtmd)QZ)0z0B(nx>Zyhb#XF*kTP);*&(5a=fVoJyr~DZ8A$Do;=CN z)cD7b<24-vNNr*7W`IN`S?dKP0!?*vfZ-TI8wfLhwxdojKPe%QL%ps9Wi-ap2&O?6 z-2~_+T;HS{K(Hy*Qk264pavNhM&jxzl zSPQ$(OfvW@kDeNDKX15R#RO`oPT??{?wa)B<6;_RlShB2ogDW5a-Y=xq$at(s{AI`5mM znl{V#^XRZ1{_yPD7biQvg&|=t$3o38?q8kco*@0<7Py~4lyw>!ys$&YT^WL*^t*RM zNfMet5%#dt0+>RA`7RhLS}5A!kb`Og(553^za5~;qN45OG>bb#!2y*E%sc_zLK7VK z2OKn3DVIGGwohcPj-WgP9T0{N9ExAkk<|J>zzG+92BxASkVT#ifod7l;!ARJTrj!9 zZ!U$hq0xtd)C%?y%ssJc*mh_h`6-er*~n$;@X^bOEbh_i6tSEuyM&w`6aq4x=el;# z;=)>jK^c$Q>KQp7KW!_k`_DZYS%?dKE~-e=#1XvTDJZB98sB&tU1(uot1n=Z1~=xg zmxxPfC9DBl0hSEJg)i<=37hvUF(CwSF-1KGFo}Hp6AanW9zjb09qqrc3;TFg-s4Dg zBN$U}F;PR~{YNtTD@jE;%5bkfywU zU*qbS6k!j7G-Kq%OX_S@=IvXH`eB2DyGtx8^v44)VmOl4zlm*U0#DDBn>>Y*R;Vdg zm)(=f{z#j{noOM`W;(9YuVg-`y=c|x0k8Efvp&eE-qlJa3{L>=wpt<*UFff7(l}~MMY-` zgdR4C6(Jdkkp?EyO-Kr0JG_kpF$20q*Fi{tCMVpTiH;0*{X@zd<`f**4Fv8CScA#@ z&*(q&q3;A+R>Tfyuum^#fF1S=7SV($7oj)3(4i0fCu~M%fz-h#bH(l;a9ug1$Es*( zj(+9yIn{nOD6N+|vw54~ZdJH9Rhu!v&_ex{^GT+S%)y$h)zA5P7ct5le`bTm8RTPu zNFA+@#2ass1K!lOE2AF=88+tB*w{%5g51VSQHjT}ClJkVBG&TRDVqiL9EDi@&JXeQF@Y0}QkV4+Db; z-4q?*644MNJ{Kz`;V*~p-DMW%gKRT(&@OpMmT_`&G6ha_WfC2_YBMV=Xdvye=z)AN zb`2^#PsZ%qH(7C~6=w8+3#uR8;9i#BK<)wXEdXH3t_t_olN7aF%&0c8!3m{Mq9{vX z^6`RR9Ly7N9A^ofb1exqVivi8hCgZD`UKo3&_5wk<{VNP8F)0U8xtaB$irqD{#(Vo zS*_$IbCBAee=~6KH3jM4dx~20N)MB)yz+=2EFXxqNdVy#DU7j`)vHJ6$b8u^l#`K2 zK9mWg>QC8tFS?r<#Z9+%Zzhq9WT)__^~&1#MNSf|EehY8#MkMX=U!Iwy&=9%;$w;w z@I)AwA`Qu)v0G8Bq`2pxWJc3m_{IloC*R%WLVX78?xyOwV&^hl2=_oVyw%dnJOb~8 ze5H&^k!C^nSR=o7o`=s8bveKZ z7euhOqLLUWw_n&qGXAD_O5OAbc|6iVaMJv7iRfWj)bD;a4&^(byU%NH9Ut;wD`7)D z&AfkOt&liu>r$ivM2nw4gci@oSexXa69V(kpX>&tG$b*Uts8G9BPw#@c_|#!xwa9% zUGT$Ax>T3=D}${!zf~L+Y2<>WL;|7lMB;AF?&(1NY#b5`oSR5u5;+pcp#>(!9P2Jm zeEVk4iTObmr}>)JDsOhc;-Mm}(9z7k8wbC5ijXwh+_Zh(AAlSN){>|V?7 z3x9An0icB2i`UPaL>Yh7?!n$|xAAEVnG?FIINu}_Q!tWM)`H89YO(4dW|Z#MqDo9xZy_;7VgWUPZSeR@?-pS@+~ZoiJb1meicP)V zWi1wUi^{^DG0SJiA86n6$b<(0(>MW1Y2iPq{(qt510Mf7&-Yg=_>N zjt-<1l<(Vg$sBe=G6tUIU*t({*2J*LN@jI!EQgzE`LAw;+7s5`F7}YQ6vdKO-6(f-hX|?3?ms~qnZ7EiFWrNS}{s8qVW0iJx1V+q~xu zCnwS7IV3Iy78?tXjVmW&QuQ8N3q*%vl&2R4`wFn>#y!3mUfHk{6Eu$6J4wz;FPr|=PBg6bA|MHEGt$t~qvdTW(!3jD_9d~p4)V{#fCP6Qa zDexgz%4#r8FO?i^-8oAd1rKXan6NC$gii(DXgh;DCdVb!Rc zDUlFPmGx`2q*u}cUtvY(dZ3Y=z&cl-^NWfLnrR&Z^-{TJ%7P(KB7seiE?>BFrb+G| zAX2wcP=L@(?UrU#9DF=|$FjlEzz6Gw&r)?l^+(o&kwu@W7I?9mJj*!xirXJqIP83X z5VjbQCIB}i_WKJ_gpGasn&(zI-(W}MwOYcCKoemBrJD@XGEAcBDV3(O{ess8e9liO z<*vW}Z902OVf5L0`sX((i@B>5k*Sqn?L4y7yUN=?w^Mv-*!@z+4RmMV$@0VPx|($4 zarJ9nl`DmXe!aXtjgBnMbE8`F1*xY#W+IhYHsjF=2>c_fTNDkBX0;jJmU!#>vIBqv zhj9fi*9(`bzXB0JB04>yE8XH;iP!bz5G*J;+7UJ}hU4o*bKbpFfJA$db?2fy5|HoIuJ`1NU4f*Gksx?ZoXgdDB$`Q$pwC?Wf{ z&Oi_Yc4an+KpBOhw`57?eSd#R&s=T#P$9#M#W)sVN38Q8Ne8G&VM$p6kKUiD2iN!O z^l48W$3`42Pt@L`c~wZ0m($^m#&Q5(3F~Oe+1#?(jgMQxhb#Wo@&v*T2^%9Gwxw8MAw6+%KR?SMsQok$VZMuWw-@s4 zCYje<4T*0J{u+8~bpTRJa^QcUN|R{@rR!q&zv%mb2y}y4<>*e$We9EP8OYo--Yxrk zc^ULhD>3GOLK(#Su#rUW$VZWW3JNOdr=$^($&D8cagL%+D*FEEKVSJ-ej#Hc6~KT0 z`?MS1c)p29Pu&He!YX2*xzUd|Ch>vzpv-QhLl}3sKkJ%s$q?E&eq~H}V5@r`!QLHc z7EX>WY0N|fF5g&ufC=K`0bhYKM2Po4f0aarL@0WagPYp|ogJ8hWbpluZ-lVHpbEOt zTuXm9u97(a8T6nWwDG#R)gMvz-)rMe->ktb3CZez?}^(UeS$W~w&g|ZlgE$W&v})3 zuenIYriE;r3is)VwB$Y+k}aDqp=aYoE)%~0-j9t7vj$x9Pu$<)G}Vp2ai23%i3SS+ zwHGSv|Me;VogJvwHqMU!d?4b^E73qo8e@=uFE)=Scx?Y(|7&3o>jh7fxWThj`jHz~ z$|n4qlpbG&LG+)+DgH8t|1Jx8m9UZTqPV4gXaQ1iu+$V3vJ>z5)7{Z<9(9MVfdPDx ze!#goY-2s+TjivKSW6o-)9PT15{tk=`uF-tk}u%NBOoAiR3cq?iBTjaHe4`&Ynn2QzGanMPo8A;}gR)Yl@S4n?bqFS%0GIO6}&6H-6Qh2QUC(AXEGK*0e_0oIkSP3XlZ;I|A8UO-UJ4D~yG zdCvhcF)<)*OEa{mIH;!+sr6H;$Uod6bDpmGbcZb4KmfmnlAGo$2Vy~)%Y=5W_-@Sf z04F}(d-vopkZqagi}b9n!)FidW?*pBOg5hNbGd>8o-LCLfR)rXH1wfU0O6l?ZT3JW zz*qrl;%C)1Fvv%b3GhN)*}XN6=q-Qm$~2v#;qc$iEM^ISR=C7v3k%@omm8iQ&OQE; zS|5WW6yQ~WgQdRX2IvYcTK%c1E*D%giqY#R@cFNT!m@_q8@dBFKFWG}ozSOz5*DVt zaB|usJT(Z%0e*A`=vse+9uIi?nP97MadV?O&ggl!^jyyGOt^mO92=XQyuqS`4TK10Mn-*C=_-+MHi1a{amjlZpQQTuw29p)3vyu5Poe9eC9cYP z{e9#Ix1BtyFNxH0#UO0T+~{fbN6ARM+Mdp=<2^_+a_@`PTf8hVu~_ zqMA1H-=G{UKiNkNJ4&C^H|1*YfeEI*aA8uJEm!SBdbnW;=>h*=!~ literal 0 HcmV?d00001 diff --git a/docs/prometheus_agent.md b/docs/prometheus_agent.md new file mode 100644 index 0000000000..f0c2858e0a --- /dev/null +++ b/docs/prometheus_agent.md @@ -0,0 +1,44 @@ +--- +# todo: internal +--- + +## Prometheus Agent +The Prometheus Agent is an operational mode built into the Prometheus binary with the same scraping APIs, semantics, configuration, and discovery mechanism; this agent mode disables some of Prometheus' usual features(TSDB, alerting, and rule evaluations) and optimizes the binary for scraping and remote writing to remote locations. +The Prometheus Remote Write protocol forwards(streams) all or a subset of metrics collected by Prometheus to a remote location; you can configure Prometheus to forward some metrics (if you want, with all metadata and exemplars!) to one or more locations that support the Remote Write API. +With Remote Write, Prometheus still uses a pull model to gather metrics from applications, which gives us an understanding of those different failure modes. After that, we batch samples and series and export, replicate (push) data to the Remote Write endpoints, limiting the number of monitoring unknowns that the central point has. +Streaming data from such a scraper enables Global View use cases by allowing you to store metrics data in a centralized location. This enables the separation of concerns, which is useful when different teams manage applications than the observability or monitoring pipelines. +The Agent mode optimizes Prometheus for the remote write use case. It disables querying, alerting, and local storage and replaces it with a customized TSDB WAL. Everything else stays the same: scraping logic, service discovery, and related configuration. It can be used as a drop-in replacement for Prometheus if you want to just forward your data to a remote Prometheus server or any other Remote-Write-compliant project. + +In essence, it looks like this: +![Prometheus Agent Remote Write](./images/prometheus_agent.png) + +### Benefits of The Prometheus Agent +- First of all, efficiency; The customized Agent TSDB WAL removes the data immediately after successful writes. If it cannot reach the remote endpoint, it persists the data temporarily on the disk until the remote endpoint is back online. This is currently limited to a two-hour buffer only, similar to non-agent Prometheus. This means that we don't need to build chunks of data in memory. We don't need to maintain a full index for querying purposes. Essentially the Agent mode uses a fraction of the resources that a normal Prometheus server would use in a similar situation. +- Secondly, the benefit of the Agent mode is that it enables easier [horizontal scalability for ingestion](https://prometheus.io/blog/2021/11/16/agent/#the-dream-auto-scalable-metric-ingestion). + +### How to Use Agent Mode in Detail +If you show the help output of Prometheus (--help flag), you should see more or less the following: + +``` +usage: prometheus [] + +The Prometheus monitoring server + +Flags: + -h, --help Show context-sensitive help (also try --help-long and --help-man). + (... other flags) + --storage.tsdb.path="data/" + Base path for metrics storage. Use with server mode only. + --storage.agent.path="data-agent/" + Base path for metrics storage. Use with agent mode only. + (... other flags) + --enable-feature= ... Comma separated feature names to enable. Valid options: agent, exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-at-modifier, promql-negative-offset, remote-write-receiver, + extra-scrape-metrics, new-service-discovery-manager. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details. +``` + +Since the Agent mode is behind a feature flag, use the --enable-feature=agent flag to run Prometheus in the Agent mode. The rest of the flags are either for both server and Agent or only for a specific mode. You can see which flag is for which mode by checking the last sentence of a flag's help string. "Use with server mode only" means it's only for server mode. If you don't see any mention like this, it means the flag is shared. + +The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. + +### Hands-on Prometheus Agent Example: Katacoda Tutorial +To try the hands-on experience of Prometheus Agent capabilities, we recommend the [Thanos Katacoda tutorial of Prometheus Agent](https://katacoda.com/thanos/courses/thanos/4-receiver-agent), which explains how easy it is to run Prometheus Agent. \ No newline at end of file From b6e06aca9ff071e4efdec16c9f97c4e23ce10627 Mon Sep 17 00:00:00 2001 From: Tomiwa Date: Sat, 27 Jul 2024 14:25:03 +0100 Subject: [PATCH 002/439] prometheus-agent-documentation Signed-off-by: Tomiwa --- docs/prometheus_agent.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/prometheus_agent.md b/docs/prometheus_agent.md index f0c2858e0a..7f72299fdd 100644 --- a/docs/prometheus_agent.md +++ b/docs/prometheus_agent.md @@ -4,9 +4,12 @@ ## Prometheus Agent The Prometheus Agent is an operational mode built into the Prometheus binary with the same scraping APIs, semantics, configuration, and discovery mechanism; this agent mode disables some of Prometheus' usual features(TSDB, alerting, and rule evaluations) and optimizes the binary for scraping and remote writing to remote locations. + The Prometheus Remote Write protocol forwards(streams) all or a subset of metrics collected by Prometheus to a remote location; you can configure Prometheus to forward some metrics (if you want, with all metadata and exemplars!) to one or more locations that support the Remote Write API. + With Remote Write, Prometheus still uses a pull model to gather metrics from applications, which gives us an understanding of those different failure modes. After that, we batch samples and series and export, replicate (push) data to the Remote Write endpoints, limiting the number of monitoring unknowns that the central point has. Streaming data from such a scraper enables Global View use cases by allowing you to store metrics data in a centralized location. This enables the separation of concerns, which is useful when different teams manage applications than the observability or monitoring pipelines. + The Agent mode optimizes Prometheus for the remote write use case. It disables querying, alerting, and local storage and replaces it with a customized TSDB WAL. Everything else stays the same: scraping logic, service discovery, and related configuration. It can be used as a drop-in replacement for Prometheus if you want to just forward your data to a remote Prometheus server or any other Remote-Write-compliant project. In essence, it looks like this: From 3c8d76a55f6c4b13b5a8d7b32ae84e3e293f22f7 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 11 Jul 2025 12:03:12 +0100 Subject: [PATCH 003/439] Fix up: feature flag and Katakoda have gone Signed-off-by: Bryan Boreham --- docs/feature_flags.md | 2 +- docs/prometheus_agent.md | 5 +---- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/feature_flags.md b/docs/feature_flags.md index 73fb6a25ba..24d70647fd 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -73,7 +73,7 @@ this feature flag will be ignored. `--enable-feature=agent` -When enabled, Prometheus runs in [agent mode](prometheus_agent.md). The agent mode is limited to +When enabled, Prometheus runs in agent mode. The agent mode is limited to discovery, scrape and remote write. This is useful when you do not need to query the Prometheus data locally, but diff --git a/docs/prometheus_agent.md b/docs/prometheus_agent.md index 7f72299fdd..b92699f8de 100644 --- a/docs/prometheus_agent.md +++ b/docs/prometheus_agent.md @@ -41,7 +41,4 @@ Flags: Since the Agent mode is behind a feature flag, use the --enable-feature=agent flag to run Prometheus in the Agent mode. The rest of the flags are either for both server and Agent or only for a specific mode. You can see which flag is for which mode by checking the last sentence of a flag's help string. "Use with server mode only" means it's only for server mode. If you don't see any mention like this, it means the flag is shared. -The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. - -### Hands-on Prometheus Agent Example: Katacoda Tutorial -To try the hands-on experience of Prometheus Agent capabilities, we recommend the [Thanos Katacoda tutorial of Prometheus Agent](https://katacoda.com/thanos/courses/thanos/4-receiver-agent), which explains how easy it is to run Prometheus Agent. \ No newline at end of file +The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. \ No newline at end of file From 065c0ca6640ef67e3701ffe65c01c1923e4ee1c1 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 11 Jul 2025 12:05:21 +0100 Subject: [PATCH 004/439] Update Agent docs for new --agent flag Signed-off-by: Bryan Boreham --- docs/command-line/prometheus.md | 2 +- docs/prometheus_agent.md | 5 ++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/command-line/prometheus.md b/docs/command-line/prometheus.md index e90a7574ba..1d337843bd 100644 --- a/docs/command-line/prometheus.md +++ b/docs/command-line/prometheus.md @@ -59,7 +59,7 @@ The Prometheus monitoring server | --query.max-concurrency | Maximum number of queries executed concurrently. Use with server mode only. | `20` | | --query.max-samples | Maximum number of samples a single query can load into memory. Note that queries will fail if they try to load more samples than this into memory, so this also limits the number of samples a query can return. Use with server mode only. | `50000000` | | --enable-feature ... | Comma separated feature names to enable. Valid options: exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-per-step-stats, promql-experimental-functions, extra-scrape-metrics, auto-gomaxprocs, native-histograms, created-timestamp-zero-ingestion, concurrent-rule-eval, delayed-compaction, old-ui, otlp-deltatocumulative, promql-duration-expr, use-uncached-io. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details. | | -| --agent | Run Prometheus in 'Agent mode'. | | +| --agent | Run Prometheus in '[Agent mode](prometheus_agent.md)'. | | | --log.level | Only log messages with the given severity or above. One of: [debug, info, warn, error] | `info` | | --log.format | Output format of log messages. One of: [logfmt, json] | `logfmt` | diff --git a/docs/prometheus_agent.md b/docs/prometheus_agent.md index b92699f8de..9e2d922b10 100644 --- a/docs/prometheus_agent.md +++ b/docs/prometheus_agent.md @@ -35,10 +35,9 @@ Flags: --storage.agent.path="data-agent/" Base path for metrics storage. Use with agent mode only. (... other flags) - --enable-feature= ... Comma separated feature names to enable. Valid options: agent, exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-at-modifier, promql-negative-offset, remote-write-receiver, - extra-scrape-metrics, new-service-discovery-manager. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details. + --[no-]agent Run Prometheus in 'Agent mode'. ``` -Since the Agent mode is behind a feature flag, use the --enable-feature=agent flag to run Prometheus in the Agent mode. The rest of the flags are either for both server and Agent or only for a specific mode. You can see which flag is for which mode by checking the last sentence of a flag's help string. "Use with server mode only" means it's only for server mode. If you don't see any mention like this, it means the flag is shared. +Use the `--agent` flag to run Prometheus in the Agent mode. The rest of the flags are either for both server and Agent or only for a specific mode. You can see which flag is for which mode by checking the last sentence of a flag's help string. "Use with server mode only" means it's only for server mode. If you don't see any mention like this, it means the flag is shared. The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. \ No newline at end of file From ad5aadacddf7423c399bdada008ce214ef10fec0 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 11 Jul 2025 14:17:44 +0100 Subject: [PATCH 005/439] make cli-documentation Signed-off-by: Bryan Boreham --- docs/command-line/prometheus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/command-line/prometheus.md b/docs/command-line/prometheus.md index 1d337843bd..e90a7574ba 100644 --- a/docs/command-line/prometheus.md +++ b/docs/command-line/prometheus.md @@ -59,7 +59,7 @@ The Prometheus monitoring server | --query.max-concurrency | Maximum number of queries executed concurrently. Use with server mode only. | `20` | | --query.max-samples | Maximum number of samples a single query can load into memory. Note that queries will fail if they try to load more samples than this into memory, so this also limits the number of samples a query can return. Use with server mode only. | `50000000` | | --enable-feature ... | Comma separated feature names to enable. Valid options: exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-per-step-stats, promql-experimental-functions, extra-scrape-metrics, auto-gomaxprocs, native-histograms, created-timestamp-zero-ingestion, concurrent-rule-eval, delayed-compaction, old-ui, otlp-deltatocumulative, promql-duration-expr, use-uncached-io. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details. | | -| --agent | Run Prometheus in '[Agent mode](prometheus_agent.md)'. | | +| --agent | Run Prometheus in 'Agent mode'. | | | --log.level | Only log messages with the given severity or above. One of: [debug, info, warn, error] | `info` | | --log.format | Output format of log messages. One of: [logfmt, json] | `logfmt` | From 54847efd17b2d98b91d93b30dcfd5eb80a0c451c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Antoine=20Beaupr=C3=A9?= Date: Wed, 19 Jun 2024 14:07:09 -0400 Subject: [PATCH 006/439] explain how to ignore WAL files and cleanup resulting grafs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We don't want do backup WAL files, so we should show how to actually ignore those files. Also explain what happens every 2 hours a little more clearly. Move things around so the paragraphs flow more easily. Followup for #14297. Signed-off-by: Antoine Beaupré Co-authored-by: Bryan Boreham --- docs/storage.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/storage.md b/docs/storage.md index f472cce140..76b5f3da8f 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -59,12 +59,16 @@ A Prometheus server's data directory looks something like this: Note that a limitation of local storage is that it is not clustered or replicated. Thus, it is not arbitrarily scalable or durable in the face of drive or node outages and should be managed like any other single node -database. +database. With proper architecture, it is possible to retain years of +data in local storage. [Snapshots](querying/api.md#snapshot) are recommended for backups. Backups made without snapshots run the risk of losing data that was recorded since -the last WAL sync, which typically happens every two hours. With proper -architecture, it is possible to retain years of data in local storage. +the last TSDB block was created, which typically happens every two hours, +covering the last three hours of samples. Excluding the WAL files (the +`chunks_head/`, `wal/`, and `wbl/` directories in `storage.tsdb.path`) +on backup or restore will ensure a coherent backup, in any case, at the +cost of losing the time range covered by the WAL files. Alternatively, external storage may be used via the [remote read/write APIs](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage). From 91ef9a50b98e39225580ad49523524919e0a983f Mon Sep 17 00:00:00 2001 From: machine424 Date: Tue, 2 Sep 2025 12:26:03 +0200 Subject: [PATCH 007/439] chore(prombench): re-roder checks on nodes to avoid deadlocks start and cancel and in restart the start job requires the node pools to be gone before creating them, and the cleanup and restart jobs require the pools to be running to delete them, When the initial start is partial (only one pool was created), no command can move forward... the preconditions should be relaxed, for more robustness. Signed-off-by: machine424 --- .github/workflows/prombench.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/prombench.yml b/.github/workflows/prombench.yml index 65d1d71917..5eb9becc0a 100644 --- a/.github/workflows/prombench.yml +++ b/.github/workflows/prombench.yml @@ -38,8 +38,8 @@ jobs: uses: docker://prominfra/prombench:master with: args: >- - until make all_nodes_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done; make deploy; + until make all_nodes_running; do echo "waiting for nodepools to be created"; sleep 10; done; - name: Update status to failure if: failure() run: >- @@ -73,8 +73,8 @@ jobs: uses: docker://prominfra/prombench:master with: args: >- - until make all_nodes_running; do echo "waiting for nodepools to be created"; sleep 10; done; make clean; + until make all_nodes_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done; - name: Update status to failure if: failure() run: >- @@ -108,10 +108,10 @@ jobs: uses: docker://prominfra/prombench:master with: args: >- - until make all_nodes_running; do echo "waiting for nodepools to be created"; sleep 10; done; make clean; until make all_nodes_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done; make deploy; + until make all_nodes_running; do echo "waiting for nodepools to be created"; sleep 10; done; - name: Update status to failure if: failure() run: >- From 8dea1f04a5afc28f0e0634aade3bf977ee4f9adf Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Tue, 2 Sep 2025 14:51:19 +0100 Subject: [PATCH 008/439] Scrape tests: Better series references `collectResultAppender` on its own will now use the labels hash instead of a random number. This avoids the situation where a series could be added twice under different references. When there is an underlying appender, use the reference it generates. Signed-off-by: Bryan Boreham --- scrape/helpers_test.go | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/scrape/helpers_test.go b/scrape/helpers_test.go index abc2011bef..3fc4cdca4a 100644 --- a/scrape/helpers_test.go +++ b/scrape/helpers_test.go @@ -19,7 +19,6 @@ import ( "encoding/binary" "fmt" "math" - "math/rand" "strings" "sync" "testing" @@ -148,10 +147,11 @@ func (a *collectResultAppender) Append(ref storage.SeriesRef, lset labels.Labels f: v, }) - if ref == 0 { - ref = storage.SeriesRef(rand.Uint64()) - } if a.next == nil { + if ref == 0 { + // Use labels hash as a stand-in for unique series reference, to avoid having to track all series. + ref = storage.SeriesRef(lset.Hash()) + } return ref, nil } @@ -195,10 +195,10 @@ func (a *collectResultAppender) UpdateMetadata(ref storage.SeriesRef, l labels.L a.mtx.Lock() defer a.mtx.Unlock() a.pendingMetadata = append(a.pendingMetadata, metadataEntry{metric: l, m: m}) - if ref == 0 { - ref = storage.SeriesRef(rand.Uint64()) - } if a.next == nil { + if ref == 0 { + ref = storage.SeriesRef(l.Hash()) + } return ref, nil } From 5915a013b788856e14596749ab5579e04486c306 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Tue, 2 Sep 2025 15:17:24 +0100 Subject: [PATCH 009/439] Scraping: detect staleness via unique reference Instead of the labels hash, which could collide between two different series, use the SeriesRef which is unique. Signed-off-by: Bryan Boreham --- scrape/scrape.go | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/scrape/scrape.go b/scrape/scrape.go index d3315a1aff..5c611c1f74 100644 --- a/scrape/scrape.go +++ b/scrape/scrape.go @@ -980,11 +980,10 @@ type scrapeCache struct { // be a pointer so we can update it. droppedSeries map[string]*uint64 - // seriesCur and seriesPrev store the labels of series that were seen - // in the current and previous scrape. + // Series that were seen in the current and previous scrape, for staleness detection. // We hold two maps and swap them out to save allocations. - seriesCur map[uint64]*cacheEntry - seriesPrev map[uint64]*cacheEntry + seriesCur map[storage.SeriesRef]*cacheEntry + seriesPrev map[storage.SeriesRef]*cacheEntry // TODO(bwplotka): Consider moving Metadata API to use WAL instead of scrape loop to // avoid locking (using metadata API can block scraping). @@ -1011,8 +1010,8 @@ func newScrapeCache(metrics *scrapeMetrics) *scrapeCache { return &scrapeCache{ series: map[string]*cacheEntry{}, droppedSeries: map[string]*uint64{}, - seriesCur: map[uint64]*cacheEntry{}, - seriesPrev: map[uint64]*cacheEntry{}, + seriesCur: map[storage.SeriesRef]*cacheEntry{}, + seriesPrev: map[storage.SeriesRef]*cacheEntry{}, metadata: map[string]*metaEntry{}, metrics: metrics, } @@ -1103,13 +1102,13 @@ func (c *scrapeCache) getDropped(met []byte) bool { return ok } -func (c *scrapeCache) trackStaleness(hash uint64, ce *cacheEntry) { - c.seriesCur[hash] = ce +func (c *scrapeCache) trackStaleness(ref storage.SeriesRef, ce *cacheEntry) { + c.seriesCur[ref] = ce } func (c *scrapeCache) forEachStale(f func(storage.SeriesRef, labels.Labels) bool) { - for h, ce := range c.seriesPrev { - if _, ok := c.seriesCur[h]; !ok { + for ref, ce := range c.seriesPrev { + if _, ok := c.seriesCur[ref]; !ok { if !f(ce.ref, ce.lset) { break } @@ -1808,7 +1807,7 @@ loop: if err == nil { if (parsedTimestamp == nil || sl.trackTimestampsStaleness) && ce != nil { - sl.cache.trackStaleness(ce.hash, ce) + sl.cache.trackStaleness(ce.ref, ce) } } @@ -1829,7 +1828,7 @@ loop: if ce != nil && (parsedTimestamp == nil || sl.trackTimestampsStaleness) { // Bypass staleness logic if there is an explicit timestamp. // But make sure we only do this if we have a cache entry (ce) for our series. - sl.cache.trackStaleness(hash, ce) + sl.cache.trackStaleness(ref, ce) } if sampleAdded && sampleLimitErr == nil && bucketLimitErr == nil { seriesAdded++ From 8563ed03e03b89c6e9a9c3a988381e65705a409d Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Tue, 2 Sep 2025 15:18:14 +0100 Subject: [PATCH 010/439] Scraping: use clear builtin function This was added in Go 1.21, and is neater than a loop deleting all elements. Also move the comment noting why we do this, because it could be read as saying this is the only reason we have two maps. Signed-off-by: Bryan Boreham --- scrape/scrape.go | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/scrape/scrape.go b/scrape/scrape.go index 5c611c1f74..5b658dc5aa 100644 --- a/scrape/scrape.go +++ b/scrape/scrape.go @@ -981,7 +981,6 @@ type scrapeCache struct { droppedSeries map[string]*uint64 // Series that were seen in the current and previous scrape, for staleness detection. - // We hold two maps and swap them out to save allocations. seriesCur map[storage.SeriesRef]*cacheEntry seriesPrev map[storage.SeriesRef]*cacheEntry @@ -1059,13 +1058,9 @@ func (c *scrapeCache) iterDone(flushCache bool) { c.metaMtx.Unlock() } - // Swap current and previous series. + // Swap current and previous series then clear the new current, to save allocations. c.seriesPrev, c.seriesCur = c.seriesCur, c.seriesPrev - - // We have to delete every single key in the map. - for k := range c.seriesCur { - delete(c.seriesCur, k) - } + clear(c.seriesCur) c.iter++ } From a2adccadd2da91bcaa983e46829fa5c58d754825 Mon Sep 17 00:00:00 2001 From: Charles Korn Date: Tue, 23 Sep 2025 12:57:36 +1000 Subject: [PATCH 011/439] Improve assertion failure message Signed-off-by: Charles Korn --- model/labels/labels_test.go | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/model/labels/labels_test.go b/model/labels/labels_test.go index 80194c5068..e3a7c47be8 100644 --- a/model/labels/labels_test.go +++ b/model/labels/labels_test.go @@ -67,17 +67,19 @@ func TestSizeOfLabels(t *testing.T) { require.Len(t, expectedSizeOfLabels, len(testCaseLabels)) for i, c := range expectedSizeOfLabels { // Declared in build-tag-specific files, e.g. labels_slicelabels_test.go. var total uint64 - testCaseLabels[i].Range(func(l Label) { + labels := testCaseLabels[i] + labels.Range(func(l Label) { total += SizeOfLabels(l.Name, l.Value, 1) }) - require.Equal(t, c, total) + require.Equalf(t, c, total, "unexpected size for test case %d: %v", i, labels) } } func TestByteSize(t *testing.T) { require.Len(t, expectedByteSize, len(testCaseLabels)) for i, c := range expectedByteSize { // Declared in build-tag-specific files, e.g. labels_slicelabels_test.go. - require.Equal(t, c, testCaseLabels[i].ByteSize()) + labels := testCaseLabels[i] + require.Equalf(t, c, labels.ByteSize(), "unexpected size for test case %d: %v", i, labels) } } From abf67c8641e1438da3a27269ff99009ad0809818 Mon Sep 17 00:00:00 2001 From: Devansh Sehgal Date: Wed, 8 Oct 2025 01:31:44 +0530 Subject: [PATCH 012/439] docs: document sigv4.use_fips_sts_endpoint in Signed-off-by: Devansh Sehgal --- docs/configuration/configuration.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md index b3ea571b80..330474d78e 100644 --- a/docs/configuration/configuration.md +++ b/docs/configuration/configuration.md @@ -2809,6 +2809,11 @@ sigv4: # AWS Role ARN, an alternative to using AWS API keys. [ role_arn: ] + # Defines the FIPS mode for the AWS STS endpoint. + # Requires Prometheus >= 2.54.0 + # Note: FIPS STS selection should be configured via use_fips_sts_endpoint rather than environment variables. (The problem report that motivated this: AWS_USE_FIPS_ENDPOINT no longer works.) + [ use_fips_sts_endpoint: | default = false ] + # HTTP client settings, including authentication methods (such as basic auth and # authorization), proxy configurations, TLS options, custom HTTP headers, etc. [ ] @@ -3011,6 +3016,11 @@ sigv4: # AWS Role ARN, an alternative to using AWS API keys. [ role_arn: ] + # Defines the FIPS mode for the AWS STS endpoint. + # Requires Prometheus >= 2.54.0 + # Note: FIPS STS selection should be configured via use_fips_sts_endpoint rather than environment variables. (The problem report that motivated this: AWS_USE_FIPS_ENDPOINT no longer works.) + [ use_fips_sts_endpoint: | default = false ] + # Optional AzureAD configuration. # Cannot be used at the same time as basic_auth, authorization, oauth2, sigv4 or google_iam. azuread: From e894a22b88dabaa461fc30c686fe3f368d80dfad Mon Sep 17 00:00:00 2001 From: Will Bollock Date: Wed, 3 Sep 2025 08:37:05 -0400 Subject: [PATCH 013/439] feat: add config label to refresh metrics Adds a `config` label (similar to `prometheus_sd_discovered_targets`) to refresh metrics to help identify the source of refresh issues or performance stats. In particular for HTTP SD, it can be common to have multiple disparate HTTP SD sources that should be identified and not lumped together. For example if one HTTP SD service has failures, that should be evident in its own time series seperate from other HTTP SD sources. `config` seemed more appropriate than `endpoint` as a general standard for `prometheus_sd` metrics. Docs were also updated for HTTP SD to point at the new refresh metrics rather than the older metrics. Signed-off-by: Will Bollock --- discovery/aws/ec2.go | 15 ++++++----- discovery/aws/lightsail.go | 14 +++++----- discovery/azure/azure.go | 15 ++++++----- discovery/azure/azure_test.go | 6 ++++- discovery/digitalocean/digitalocean.go | 10 +++---- discovery/digitalocean/digitalocean_test.go | 6 ++++- discovery/discovery.go | 5 +++- discovery/dns/dns.go | 15 ++++++----- discovery/dns/dns_test.go | 6 ++++- discovery/eureka/eureka.go | 10 +++---- discovery/eureka/eureka_test.go | 6 ++++- discovery/gce/gce.go | 10 +++---- discovery/hetzner/hetzner.go | 11 ++++---- discovery/http/http.go | 16 +++++------ discovery/http/http_test.go | 28 ++++++++++++++++--- discovery/ionos/ionos.go | 14 +++++----- discovery/linode/linode.go | 10 +++---- discovery/linode/linode_test.go | 6 ++++- discovery/manager.go | 1 + discovery/marathon/marathon.go | 10 +++---- discovery/marathon/marathon_test.go | 12 +++++++-- discovery/metrics_refresh.go | 12 ++++----- discovery/moby/docker.go | 16 +++++------ discovery/moby/docker_test.go | 12 +++++++-- discovery/moby/dockerswarm.go | 16 +++++------ discovery/moby/nodes_test.go | 6 ++++- discovery/moby/services_test.go | 12 +++++++-- discovery/moby/tasks_test.go | 6 ++++- discovery/nomad/nomad.go | 10 +++---- discovery/nomad/nomad_test.go | 12 +++++++-- discovery/openstack/openstack.go | 11 ++++---- discovery/ovhcloud/ovhcloud.go | 13 ++++----- discovery/puppetdb/puppetdb.go | 14 +++++----- discovery/puppetdb/puppetdb_test.go | 30 +++++++++++++++++---- discovery/refresh/refresh.go | 3 ++- discovery/refresh/refresh_test.go | 1 + discovery/scaleway/scaleway.go | 12 ++++----- discovery/stackit/stackit.go | 11 ++++---- discovery/triton/triton.go | 10 +++---- discovery/triton/triton_test.go | 6 ++++- discovery/uyuni/uyuni.go | 11 ++++---- discovery/uyuni/uyuni_test.go | 12 +++++++-- discovery/vultr/vultr.go | 10 +++---- discovery/vultr/vultr_test.go | 6 ++++- docs/http_sd.md | 5 ++-- 45 files changed, 307 insertions(+), 176 deletions(-) diff --git a/discovery/aws/ec2.go b/discovery/aws/ec2.go index 539cd84c4f..da02a27326 100644 --- a/discovery/aws/ec2.go +++ b/discovery/aws/ec2.go @@ -112,7 +112,7 @@ func (*EC2SDConfig) Name() string { return "ec2" } // NewDiscoverer returns a Discoverer for the EC2 Config. func (c *EC2SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewEC2Discovery(c, opts.Logger, opts.Metrics) + return NewEC2Discovery(c, opts) } // UnmarshalYAML implements the yaml.Unmarshaler interface for the EC2 Config. @@ -168,23 +168,24 @@ type EC2Discovery struct { } // NewEC2Discovery returns a new EC2Discovery which periodically refreshes its targets. -func NewEC2Discovery(conf *EC2SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*EC2Discovery, error) { - m, ok := metrics.(*ec2Metrics) +func NewEC2Discovery(conf *EC2SDConfig, opts discovery.DiscovererOptions) (*EC2Discovery, error) { + m, ok := opts.Metrics.(*ec2Metrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } d := &EC2Discovery{ - logger: logger, + logger: opts.Logger, cfg: conf, } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "ec2", + SetName: opts.SetName, Interval: time.Duration(d.cfg.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/aws/lightsail.go b/discovery/aws/lightsail.go index 5c356c8c45..99fe2dbddb 100644 --- a/discovery/aws/lightsail.go +++ b/discovery/aws/lightsail.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net" "strconv" "strings" @@ -94,7 +93,7 @@ func (*LightsailSDConfig) Name() string { return "lightsail" } // NewDiscoverer returns a Discoverer for the Lightsail Config. func (c *LightsailSDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewLightsailDiscovery(c, opts.Logger, opts.Metrics) + return NewLightsailDiscovery(c, opts) } // UnmarshalYAML implements the yaml.Unmarshaler interface for the Lightsail Config. @@ -131,14 +130,14 @@ type LightsailDiscovery struct { } // NewLightsailDiscovery returns a new LightsailDiscovery which periodically refreshes its targets. -func NewLightsailDiscovery(conf *LightsailSDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*LightsailDiscovery, error) { - m, ok := metrics.(*lightsailMetrics) +func NewLightsailDiscovery(conf *LightsailSDConfig, opts discovery.DiscovererOptions) (*LightsailDiscovery, error) { + m, ok := opts.Metrics.(*lightsailMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } d := &LightsailDiscovery{ @@ -146,8 +145,9 @@ func NewLightsailDiscovery(conf *LightsailSDConfig, logger *slog.Logger, metrics } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "lightsail", + SetName: opts.SetName, Interval: time.Duration(d.cfg.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/azure/azure.go b/discovery/azure/azure.go index bed4861787..3c38bbf3e6 100644 --- a/discovery/azure/azure.go +++ b/discovery/azure/azure.go @@ -127,7 +127,7 @@ func (*SDConfig) Name() string { return "azure" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } func validateAuthParam(param, name string) error { @@ -178,28 +178,29 @@ type Discovery struct { } // NewDiscovery returns a new AzureDiscovery which periodically refreshes its targets. -func NewDiscovery(cfg *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*azureMetrics) +func NewDiscovery(cfg *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*azureMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } l := cache.New(cache.AsLRU[string, *armnetwork.Interface](lru.WithCapacity(5000))) d := &Discovery{ cfg: cfg, port: cfg.Port, - logger: logger, + logger: opts.Logger, cache: l, metrics: m, } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "azure", + SetName: opts.SetName, Interval: time.Duration(cfg.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/azure/azure_test.go b/discovery/azure/azure_test.go index 69c815da2d..ee349bc556 100644 --- a/discovery/azure/azure_test.go +++ b/discovery/azure/azure_test.go @@ -659,7 +659,11 @@ func TestAzureRefresh(t *testing.T) { refreshMetrics := discovery.NewRefreshMetrics(reg) metrics := azureSDConfig.NewDiscovererMetrics(reg, refreshMetrics) - sd, err := NewDiscovery(azureSDConfig, nil, metrics) + sd, err := NewDiscovery(azureSDConfig, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "azure", + }) require.NoError(t, err) tg, err := sd.refreshAzureClient(context.Background(), azureClient) diff --git a/discovery/digitalocean/digitalocean.go b/discovery/digitalocean/digitalocean.go index 5c9795440d..d2fbee1d94 100644 --- a/discovery/digitalocean/digitalocean.go +++ b/discovery/digitalocean/digitalocean.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net" "net/http" "strconv" @@ -84,7 +83,7 @@ func (*SDConfig) Name() string { return "digitalocean" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -112,8 +111,8 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*digitaloceanMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*digitaloceanMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -140,8 +139,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "digitalocean", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/digitalocean/digitalocean_test.go b/discovery/digitalocean/digitalocean_test.go index a282225ac2..ca99e83b20 100644 --- a/discovery/digitalocean/digitalocean_test.go +++ b/discovery/digitalocean/digitalocean_test.go @@ -57,7 +57,11 @@ func TestDigitalOceanSDRefresh(t *testing.T) { defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "digitalocean", + }) require.NoError(t, err) endpoint, err := url.Parse(sdmock.Mock.Endpoint()) require.NoError(t, err) diff --git a/discovery/discovery.go b/discovery/discovery.go index 2157b820b9..70cd856bb2 100644 --- a/discovery/discovery.go +++ b/discovery/discovery.go @@ -54,6 +54,9 @@ type DiscovererOptions struct { // Extra HTTP client options to expose to Discoverers. This field may be // ignored; Discoverer implementations must opt-in to reading it. HTTPClientOptions []config.HTTPClientOption + + // SetName identifies this discoverer set. + SetName string } // RefreshMetrics are used by the "refresh" package. @@ -66,7 +69,7 @@ type RefreshMetrics struct { // RefreshMetricsInstantiator instantiates the metrics used by the "refresh" package. type RefreshMetricsInstantiator interface { - Instantiate(mech string) *RefreshMetrics + Instantiate(mech, setName string) *RefreshMetrics } // RefreshMetricsManager is an interface for registering, unregistering, and diff --git a/discovery/dns/dns.go b/discovery/dns/dns.go index 24af8f65d9..1e0a78698b 100644 --- a/discovery/dns/dns.go +++ b/discovery/dns/dns.go @@ -78,7 +78,7 @@ func (*SDConfig) Name() string { return "dns" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(*c, opts.Logger, opts.Metrics) + return NewDiscovery(*c, opts) } // UnmarshalYAML implements the yaml.Unmarshaler interface. @@ -118,14 +118,14 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*dnsMetrics) +func NewDiscovery(conf SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*dnsMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } qtype := dns.TypeSRV @@ -145,15 +145,16 @@ func NewDiscovery(conf SDConfig, logger *slog.Logger, metrics discovery.Discover names: conf.Names, qtype: qtype, port: conf.Port, - logger: logger, + logger: opts.Logger, lookupFn: lookupWithSearchPath, metrics: m, } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "dns", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/dns/dns_test.go b/discovery/dns/dns_test.go index eb37f1a98e..4a7170cc7d 100644 --- a/discovery/dns/dns_test.go +++ b/discovery/dns/dns_test.go @@ -259,7 +259,11 @@ func TestDNS(t *testing.T) { metrics := tc.config.NewDiscovererMetrics(reg, refreshMetrics) require.NoError(t, metrics.Register()) - sd, err := NewDiscovery(tc.config, nil, metrics) + sd, err := NewDiscovery(tc.config, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "dns", + }) require.NoError(t, err) sd.lookupFn = tc.lookup diff --git a/discovery/eureka/eureka.go b/discovery/eureka/eureka.go index 11e83359cf..6d726966bc 100644 --- a/discovery/eureka/eureka.go +++ b/discovery/eureka/eureka.go @@ -16,7 +16,6 @@ package eureka import ( "context" "errors" - "log/slog" "net" "net/http" "net/url" @@ -88,7 +87,7 @@ func (*SDConfig) Name() string { return "eureka" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -125,8 +124,8 @@ type Discovery struct { } // NewDiscovery creates a new Eureka discovery for the given role. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*eurekaMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*eurekaMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -142,8 +141,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "eureka", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/eureka/eureka_test.go b/discovery/eureka/eureka_test.go index 5ea9a6c74e..def6126e86 100644 --- a/discovery/eureka/eureka_test.go +++ b/discovery/eureka/eureka_test.go @@ -47,7 +47,11 @@ func testUpdateServices(respHandler http.HandlerFunc) ([]*targetgroup.Group, err defer metrics.Unregister() defer refreshMetrics.Unregister() - md, err := NewDiscovery(&conf, nil, metrics) + md, err := NewDiscovery(&conf, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "eureka", + }) if err != nil { return nil, err } diff --git a/discovery/gce/gce.go b/discovery/gce/gce.go index f5d20fb740..106028ff93 100644 --- a/discovery/gce/gce.go +++ b/discovery/gce/gce.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net/http" "strconv" "strings" @@ -94,7 +93,7 @@ func (*SDConfig) Name() string { return "gce" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(*c, opts.Logger, opts.Metrics) + return NewDiscovery(*c, opts) } // UnmarshalYAML implements the yaml.Unmarshaler interface. @@ -129,8 +128,8 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*gceMetrics) +func NewDiscovery(conf SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*gceMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -155,8 +154,9 @@ func NewDiscovery(conf SDConfig, logger *slog.Logger, metrics discovery.Discover d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "gce", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/hetzner/hetzner.go b/discovery/hetzner/hetzner.go index 5c5252d3d7..8e52d21e39 100644 --- a/discovery/hetzner/hetzner.go +++ b/discovery/hetzner/hetzner.go @@ -78,7 +78,7 @@ func (*SDConfig) Name() string { return "hetzner" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } type refresher interface { @@ -138,21 +138,22 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*refresh.Discovery, error) { - m, ok := metrics.(*hetznerMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*refresh.Discovery, error) { + m, ok := opts.Metrics.(*hetznerMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - r, err := newRefresher(conf, logger) + r, err := newRefresher(conf, opts.Logger) if err != nil { return nil, err } return refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "hetzner", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: r.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/http/http.go b/discovery/http/http.go index bbaf4038c8..d792bdacd7 100644 --- a/discovery/http/http.go +++ b/discovery/http/http.go @@ -19,7 +19,6 @@ import ( "errors" "fmt" "io" - "log/slog" "net/http" "net/url" "strconv" @@ -69,7 +68,7 @@ func (*SDConfig) Name() string { return "http" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.HTTPClientOptions, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -115,17 +114,17 @@ type Discovery struct { } // NewDiscovery returns a new HTTP discovery for the given config. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, clientOpts []config.HTTPClientOption, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*httpMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*httpMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } - client, err := config.NewClientFromConfig(conf.HTTPClientConfig, "http", clientOpts...) + client, err := config.NewClientFromConfig(conf.HTTPClientConfig, "http", opts.HTTPClientOptions...) if err != nil { return nil, err } @@ -140,8 +139,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, clientOpts []config.HTTPC d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "http", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.Refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/http/http_test.go b/discovery/http/http_test.go index 3af9e4e504..c553c21504 100644 --- a/discovery/http/http_test.go +++ b/discovery/http/http_test.go @@ -49,7 +49,12 @@ func TestHTTPValidRefresh(t *testing.T) { require.NoError(t, metrics.Register()) defer metrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), nil, metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + HTTPClientOptions: nil, + Metrics: metrics, + SetName: "http", + }) require.NoError(t, err) ctx := context.Background() @@ -94,7 +99,12 @@ func TestHTTPInvalidCode(t *testing.T) { require.NoError(t, metrics.Register()) defer metrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), nil, metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + HTTPClientOptions: nil, + Metrics: metrics, + SetName: "http", + }) require.NoError(t, err) ctx := context.Background() @@ -123,7 +133,12 @@ func TestHTTPInvalidFormat(t *testing.T) { require.NoError(t, metrics.Register()) defer metrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), nil, metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + HTTPClientOptions: nil, + Metrics: metrics, + SetName: "http", + }) require.NoError(t, err) ctx := context.Background() @@ -442,7 +457,12 @@ func TestSourceDisappeared(t *testing.T) { require.NoError(t, metrics.Register()) defer metrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), nil, metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + HTTPClientOptions: nil, + Metrics: metrics, + SetName: "http", + }) require.NoError(t, err) for _, test := range cases { ctx := context.Background() diff --git a/discovery/ionos/ionos.go b/discovery/ionos/ionos.go index 021986395b..c74013d109 100644 --- a/discovery/ionos/ionos.go +++ b/discovery/ionos/ionos.go @@ -15,7 +15,6 @@ package ionos import ( "errors" - "log/slog" "time" "github.com/prometheus/client_golang/prometheus" @@ -42,8 +41,8 @@ func init() { type Discovery struct{} // NewDiscovery returns a new refresh.Discovery for IONOS Cloud. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*refresh.Discovery, error) { - m, ok := metrics.(*ionosMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*refresh.Discovery, error) { + m, ok := opts.Metrics.(*ionosMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -52,15 +51,16 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove conf.ionosEndpoint = "https://api.ionos.com" } - d, err := newServerDiscovery(conf, logger) + d, err := newServerDiscovery(conf, opts.Logger) if err != nil { return nil, err } return refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "ionos", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, @@ -101,8 +101,8 @@ func (SDConfig) Name() string { } // NewDiscoverer returns a new discovery.Discoverer for IONOS Cloud. -func (c SDConfig) NewDiscoverer(options discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(&c, options.Logger, options.Metrics) +func (c SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { + return NewDiscovery(&c, opts) } // UnmarshalYAML implements the yaml.Unmarshaler interface. diff --git a/discovery/linode/linode.go b/discovery/linode/linode.go index fe61e122e4..2dc4d5f796 100644 --- a/discovery/linode/linode.go +++ b/discovery/linode/linode.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net" "net/http" "strconv" @@ -103,7 +102,7 @@ func (*SDConfig) Name() string { return "linode" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -138,8 +137,8 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*linodeMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*linodeMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -170,8 +169,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "linode", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/linode/linode_test.go b/discovery/linode/linode_test.go index 7bcaa05ba4..533bc0fb62 100644 --- a/discovery/linode/linode_test.go +++ b/discovery/linode/linode_test.go @@ -238,7 +238,11 @@ func TestLinodeSDRefresh(t *testing.T) { defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "linode", + }) require.NoError(t, err) endpoint, err := url.Parse(sdmock.Endpoint()) require.NoError(t, err) diff --git a/discovery/manager.go b/discovery/manager.go index 6688152da9..878bc5f6d4 100644 --- a/discovery/manager.go +++ b/discovery/manager.go @@ -479,6 +479,7 @@ func (m *Manager) registerProviders(cfgs Configs, setName string) int { Logger: m.logger.With("discovery", typ, "config", setName), HTTPClientOptions: m.httpOpts, Metrics: m.sdMetrics[typ], + SetName: setName, }) if err != nil { m.logger.Error("Cannot create service discovery", "err", err, "type", typ, "config", setName) diff --git a/discovery/marathon/marathon.go b/discovery/marathon/marathon.go index cae040ca98..438b8915df 100644 --- a/discovery/marathon/marathon.go +++ b/discovery/marathon/marathon.go @@ -19,7 +19,6 @@ import ( "errors" "fmt" "io" - "log/slog" "math/rand" "net" "net/http" @@ -91,7 +90,7 @@ func (*SDConfig) Name() string { return "marathon" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(*c, opts.Logger, opts.Metrics) + return NewDiscovery(*c, opts) } // SetDirectory joins any relative file paths with dir. @@ -140,8 +139,8 @@ type Discovery struct { } // NewDiscovery returns a new Marathon Discovery. -func NewDiscovery(conf SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*marathonMetrics) +func NewDiscovery(conf SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*marathonMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -168,8 +167,9 @@ func NewDiscovery(conf SDConfig, logger *slog.Logger, metrics discovery.Discover } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "marathon", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/marathon/marathon_test.go b/discovery/marathon/marathon_test.go index 588532d218..53f7d3a1f9 100644 --- a/discovery/marathon/marathon_test.go +++ b/discovery/marathon/marathon_test.go @@ -51,7 +51,11 @@ func testUpdateServices(client appListClient) ([]*targetgroup.Group, error) { defer metrics.Unregister() defer refreshMetrics.Unregister() - md, err := NewDiscovery(cfg, nil, metrics) + md, err := NewDiscovery(cfg, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "marathon", + }) if err != nil { return nil, err } @@ -132,7 +136,11 @@ func TestMarathonSDRemoveApp(t *testing.T) { defer metrics.Unregister() defer refreshMetrics.Unregister() - md, err := NewDiscovery(cfg, nil, metrics) + md, err := NewDiscovery(cfg, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "marathon", + }) require.NoError(t, err) md.appsClient = func(context.Context, *http.Client, string) (*appList, error) { diff --git a/discovery/metrics_refresh.go b/discovery/metrics_refresh.go index ef49e591a3..8a8bf221b8 100644 --- a/discovery/metrics_refresh.go +++ b/discovery/metrics_refresh.go @@ -36,14 +36,14 @@ func NewRefreshMetrics(reg prometheus.Registerer) RefreshMetricsManager { Name: "prometheus_sd_refresh_failures_total", Help: "Number of refresh failures for the given SD mechanism.", }, - []string{"mechanism"}), + []string{"mechanism", "config"}), durationVec: prometheus.NewSummaryVec( prometheus.SummaryOpts{ Name: "prometheus_sd_refresh_duration_seconds", Help: "The duration of a refresh in seconds for the given SD mechanism.", Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001}, }, - []string{"mechanism"}), + []string{"mechanism", "config"}), } // The reason we register metric vectors instead of metrics is so that @@ -56,11 +56,11 @@ func NewRefreshMetrics(reg prometheus.Registerer) RefreshMetricsManager { return m } -// Instantiate returns metrics out of metric vectors. -func (m *RefreshMetricsVecs) Instantiate(mech string) *RefreshMetrics { +// Instantiate returns metrics out of metric vectors for a given mechanism and config. +func (m *RefreshMetricsVecs) Instantiate(mech, config string) *RefreshMetrics { return &RefreshMetrics{ - Failures: m.failuresVec.WithLabelValues(mech), - Duration: m.durationVec.WithLabelValues(mech), + Failures: m.failuresVec.WithLabelValues(mech, config), + Duration: m.durationVec.WithLabelValues(mech, config), } } diff --git a/discovery/moby/docker.go b/discovery/moby/docker.go index cb5577a131..ec1187278b 100644 --- a/discovery/moby/docker.go +++ b/discovery/moby/docker.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net" "net/http" "net/url" @@ -94,7 +93,7 @@ func (*DockerSDConfig) Name() string { return "docker" } // NewDiscoverer returns a Discoverer for the Config. func (c *DockerSDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDockerDiscovery(c, opts.Logger, opts.Metrics) + return NewDockerDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -129,8 +128,8 @@ type DockerDiscovery struct { } // NewDockerDiscovery returns a new DockerDiscovery which periodically refreshes its targets. -func NewDockerDiscovery(conf *DockerSDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*DockerDiscovery, error) { - m, ok := metrics.(*dockerMetrics) +func NewDockerDiscovery(conf *DockerSDConfig, opts discovery.DiscovererOptions) (*DockerDiscovery, error) { + m, ok := opts.Metrics.(*dockerMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -146,7 +145,7 @@ func NewDockerDiscovery(conf *DockerSDConfig, logger *slog.Logger, metrics disco return nil, err } - opts := []client.Opt{ + clientOpts := []client.Opt{ client.WithHost(conf.Host), client.WithAPIVersionNegotiation(), } @@ -166,7 +165,7 @@ func NewDockerDiscovery(conf *DockerSDConfig, logger *slog.Logger, metrics disco if err != nil { return nil, err } - opts = append(opts, + clientOpts = append(clientOpts, client.WithHTTPClient(&http.Client{ Transport: rt, Timeout: time.Duration(conf.RefreshInterval), @@ -178,15 +177,16 @@ func NewDockerDiscovery(conf *DockerSDConfig, logger *slog.Logger, metrics disco ) } - d.client, err = client.NewClientWithOpts(opts...) + d.client, err = client.NewClientWithOpts(clientOpts...) if err != nil { return nil, fmt.Errorf("error setting up docker client: %w", err) } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "docker", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/moby/docker_test.go b/discovery/moby/docker_test.go index 430669c113..88c832db1b 100644 --- a/discovery/moby/docker_test.go +++ b/discovery/moby/docker_test.go @@ -48,7 +48,11 @@ host: %s defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDockerDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDockerDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "docker_swarm", + }) require.NoError(t, err) ctx := context.Background() @@ -226,7 +230,11 @@ host: %s require.NoError(t, metrics.Register()) defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDockerDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDockerDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "docker_swarm", + }) require.NoError(t, err) ctx := context.Background() diff --git a/discovery/moby/dockerswarm.go b/discovery/moby/dockerswarm.go index 44abb0ab25..2761e891b5 100644 --- a/discovery/moby/dockerswarm.go +++ b/discovery/moby/dockerswarm.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net/http" "net/url" "time" @@ -81,7 +80,7 @@ func (*DockerSwarmSDConfig) Name() string { return "dockerswarm" } // NewDiscoverer returns a Discoverer for the Config. func (c *DockerSwarmSDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -124,8 +123,8 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *DockerSwarmSDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*dockerswarmMetrics) +func NewDiscovery(conf *DockerSwarmSDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*dockerswarmMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -140,7 +139,7 @@ func NewDiscovery(conf *DockerSwarmSDConfig, logger *slog.Logger, metrics discov return nil, err } - opts := []client.Opt{ + clientOpts := []client.Opt{ client.WithHost(conf.Host), client.WithAPIVersionNegotiation(), } @@ -160,7 +159,7 @@ func NewDiscovery(conf *DockerSwarmSDConfig, logger *slog.Logger, metrics discov if err != nil { return nil, err } - opts = append(opts, + clientOpts = append(clientOpts, client.WithHTTPClient(&http.Client{ Transport: rt, Timeout: time.Duration(conf.RefreshInterval), @@ -172,15 +171,16 @@ func NewDiscovery(conf *DockerSwarmSDConfig, logger *slog.Logger, metrics discov ) } - d.client, err = client.NewClientWithOpts(opts...) + d.client, err = client.NewClientWithOpts(clientOpts...) if err != nil { return nil, fmt.Errorf("error setting up docker swarm client: %w", err) } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "dockerswarm", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/moby/nodes_test.go b/discovery/moby/nodes_test.go index 35676a3a8d..c65b9411ed 100644 --- a/discovery/moby/nodes_test.go +++ b/discovery/moby/nodes_test.go @@ -48,7 +48,11 @@ host: %s defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "docker_swarm", + }) require.NoError(t, err) ctx := context.Background() diff --git a/discovery/moby/services_test.go b/discovery/moby/services_test.go index af6ce842d1..95702ced9b 100644 --- a/discovery/moby/services_test.go +++ b/discovery/moby/services_test.go @@ -48,7 +48,11 @@ host: %s defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "moby", + }) require.NoError(t, err) ctx := context.Background() @@ -349,7 +353,11 @@ filters: defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "moby", + }) require.NoError(t, err) ctx := context.Background() diff --git a/discovery/moby/tasks_test.go b/discovery/moby/tasks_test.go index afb19abbee..3f38135096 100644 --- a/discovery/moby/tasks_test.go +++ b/discovery/moby/tasks_test.go @@ -48,7 +48,11 @@ host: %s defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "docker_swarm", + }) require.NoError(t, err) ctx := context.Background() diff --git a/discovery/nomad/nomad.go b/discovery/nomad/nomad.go index e204b740f7..f2971fb01b 100644 --- a/discovery/nomad/nomad.go +++ b/discovery/nomad/nomad.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net" "strconv" "strings" @@ -84,7 +83,7 @@ func (*SDConfig) Name() string { return "nomad" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -121,8 +120,8 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*nomadMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*nomadMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -157,8 +156,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "nomad", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/nomad/nomad_test.go b/discovery/nomad/nomad_test.go index a73b45785d..099a347cbf 100644 --- a/discovery/nomad/nomad_test.go +++ b/discovery/nomad/nomad_test.go @@ -150,7 +150,11 @@ func TestConfiguredService(t *testing.T) { require.NoError(t, metrics.Register()) defer metrics.Unregister() - _, err := NewDiscovery(conf, nil, metrics) + _, err := NewDiscovery(conf, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "nomad", + }) if tc.acceptedURL { require.NoError(t, err) } else { @@ -178,7 +182,11 @@ func TestNomadSDRefresh(t *testing.T) { defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "nomad", + }) require.NoError(t, err) tgs, err := d.refresh(context.Background()) diff --git a/discovery/openstack/openstack.go b/discovery/openstack/openstack.go index 7f23757297..61dff847cf 100644 --- a/discovery/openstack/openstack.go +++ b/discovery/openstack/openstack.go @@ -78,7 +78,7 @@ func (*SDConfig) Name() string { return "openstack" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -145,20 +145,21 @@ type refresher interface { } // NewDiscovery returns a new OpenStack Discoverer which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, l *slog.Logger, metrics discovery.DiscovererMetrics) (*refresh.Discovery, error) { - m, ok := metrics.(*openstackMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*refresh.Discovery, error) { + m, ok := opts.Metrics.(*openstackMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - r, err := newRefresher(conf, l) + r, err := newRefresher(conf, opts.Logger) if err != nil { return nil, err } return refresh.NewDiscovery( refresh.Options{ - Logger: l, + Logger: opts.Logger, Mech: "openstack", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: r.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/ovhcloud/ovhcloud.go b/discovery/ovhcloud/ovhcloud.go index 69c7cd6004..df150b8ce4 100644 --- a/discovery/ovhcloud/ovhcloud.go +++ b/discovery/ovhcloud/ovhcloud.go @@ -100,8 +100,8 @@ func createClient(config *SDConfig) (*ovh.Client, error) { } // NewDiscoverer returns a Discoverer for the Config. -func (c *SDConfig) NewDiscoverer(options discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, options.Logger, options.Metrics) +func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { + return NewDiscovery(c, opts) } func init() { @@ -148,21 +148,22 @@ func newRefresher(conf *SDConfig, logger *slog.Logger) (refresher, error) { } // NewDiscovery returns a new OVHcloud Discoverer which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*refresh.Discovery, error) { - m, ok := metrics.(*ovhcloudMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*refresh.Discovery, error) { + m, ok := opts.Metrics.(*ovhcloudMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - r, err := newRefresher(conf, logger) + r, err := newRefresher(conf, opts.Logger) if err != nil { return nil, err } return refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "ovhcloud", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: r.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/puppetdb/puppetdb.go b/discovery/puppetdb/puppetdb.go index a5163addb0..db5fc2e2fb 100644 --- a/discovery/puppetdb/puppetdb.go +++ b/discovery/puppetdb/puppetdb.go @@ -20,7 +20,6 @@ import ( "errors" "fmt" "io" - "log/slog" "net" "net/http" "net/url" @@ -93,7 +92,7 @@ func (*SDConfig) Name() string { return "puppetdb" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -140,14 +139,14 @@ type Discovery struct { } // NewDiscovery returns a new PuppetDB discovery for the given config. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*puppetdbMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*puppetdbMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } client, err := config.NewClientFromConfig(conf.HTTPClientConfig, "http") @@ -172,8 +171,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "puppetdb", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/puppetdb/puppetdb_test.go b/discovery/puppetdb/puppetdb_test.go index 57e198e131..a96310553b 100644 --- a/discovery/puppetdb/puppetdb_test.go +++ b/discovery/puppetdb/puppetdb_test.go @@ -70,7 +70,11 @@ func TestPuppetSlashInURL(t *testing.T) { metrics := cfg.NewDiscovererMetrics(reg, refreshMetrics) require.NoError(t, metrics.Register()) - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "puppetdb", + }) require.NoError(t, err) require.Equal(t, apiURL, d.url) @@ -94,7 +98,11 @@ func TestPuppetDBRefresh(t *testing.T) { metrics := cfg.NewDiscovererMetrics(reg, refreshMetrics) require.NoError(t, metrics.Register()) - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "puppetdb", + }) require.NoError(t, err) ctx := context.Background() @@ -142,7 +150,11 @@ func TestPuppetDBRefreshWithParameters(t *testing.T) { metrics := cfg.NewDiscovererMetrics(reg, refreshMetrics) require.NoError(t, metrics.Register()) - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "puppetdb", + }) require.NoError(t, err) ctx := context.Background() @@ -201,7 +213,11 @@ func TestPuppetDBInvalidCode(t *testing.T) { metrics := cfg.NewDiscovererMetrics(reg, refreshMetrics) require.NoError(t, metrics.Register()) - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "puppetdb", + }) require.NoError(t, err) ctx := context.Background() @@ -229,7 +245,11 @@ func TestPuppetDBInvalidFormat(t *testing.T) { metrics := cfg.NewDiscovererMetrics(reg, refreshMetrics) require.NoError(t, metrics.Register()) - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "puppetdb", + }) require.NoError(t, err) ctx := context.Background() diff --git a/discovery/refresh/refresh.go b/discovery/refresh/refresh.go index 31646c0e4c..e0bac2af5e 100644 --- a/discovery/refresh/refresh.go +++ b/discovery/refresh/refresh.go @@ -28,6 +28,7 @@ import ( type Options struct { Logger *slog.Logger Mech string + SetName string Interval time.Duration RefreshF func(ctx context.Context) ([]*targetgroup.Group, error) MetricsInstantiator discovery.RefreshMetricsInstantiator @@ -43,7 +44,7 @@ type Discovery struct { // NewDiscovery returns a Discoverer function that calls a refresh() function at every interval. func NewDiscovery(opts Options) *Discovery { - m := opts.MetricsInstantiator.Instantiate(opts.Mech) + m := opts.MetricsInstantiator.Instantiate(opts.Mech, opts.SetName) var logger *slog.Logger if opts.Logger == nil { diff --git a/discovery/refresh/refresh_test.go b/discovery/refresh/refresh_test.go index b241704b94..a5e0d99a45 100644 --- a/discovery/refresh/refresh_test.go +++ b/discovery/refresh/refresh_test.go @@ -76,6 +76,7 @@ func TestRefresh(t *testing.T) { Options{ Logger: nil, Mech: "test", + SetName: "test-refresh", Interval: interval, RefreshF: refresh, MetricsInstantiator: metrics, diff --git a/discovery/scaleway/scaleway.go b/discovery/scaleway/scaleway.go index d617e01905..16a9835848 100644 --- a/discovery/scaleway/scaleway.go +++ b/discovery/scaleway/scaleway.go @@ -17,7 +17,6 @@ import ( "context" "errors" "fmt" - "log/slog" "net/http" "os" "strings" @@ -167,8 +166,8 @@ func (c *SDConfig) UnmarshalYAML(unmarshal func(any) error) error { return c.HTTPClientConfig.Validate() } -func (c SDConfig) NewDiscoverer(options discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(&c, options.Logger, options.Metrics) +func (c SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { + return NewDiscovery(&c, opts) } // SetDirectory joins any relative file paths with dir. @@ -185,8 +184,8 @@ func init() { // the Discoverer interface. type Discovery struct{} -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*refresh.Discovery, error) { - m, ok := metrics.(*scalewayMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*refresh.Discovery, error) { + m, ok := opts.Metrics.(*scalewayMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -198,8 +197,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove return refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "scaleway", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: r.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/stackit/stackit.go b/discovery/stackit/stackit.go index 351526e016..1f9bd22469 100644 --- a/discovery/stackit/stackit.go +++ b/discovery/stackit/stackit.go @@ -87,7 +87,7 @@ func (*SDConfig) Name() string { return "stackit" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } type refresher interface { @@ -126,21 +126,22 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*refresh.Discovery, error) { - m, ok := metrics.(*stackitMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*refresh.Discovery, error) { + m, ok := opts.Metrics.(*stackitMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - r, err := newRefresher(conf, logger) + r, err := newRefresher(conf, opts.Logger) if err != nil { return nil, err } return refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "stackit", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: r.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/triton/triton.go b/discovery/triton/triton.go index 9300753015..209e1c4deb 100644 --- a/discovery/triton/triton.go +++ b/discovery/triton/triton.go @@ -19,7 +19,6 @@ import ( "errors" "fmt" "io" - "log/slog" "net/http" "net/url" "strings" @@ -82,7 +81,7 @@ func (*SDConfig) Name() string { return "triton" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return New(opts.Logger, c, opts.Metrics) + return New(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -146,8 +145,8 @@ type Discovery struct { } // New returns a new Discovery which periodically refreshes its targets. -func New(logger *slog.Logger, conf *SDConfig, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*tritonMetrics) +func New(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*tritonMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -173,8 +172,9 @@ func New(logger *slog.Logger, conf *SDConfig, metrics discovery.DiscovererMetric } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "triton", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/triton/triton_test.go b/discovery/triton/triton_test.go index b0dccbf898..731303677d 100644 --- a/discovery/triton/triton_test.go +++ b/discovery/triton/triton_test.go @@ -90,7 +90,11 @@ func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, er return nil, nil, err } - d, err := New(nil, &c, metrics) + d, err := New(&c, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "triton", + }) if err != nil { return nil, nil, err } diff --git a/discovery/uyuni/uyuni.go b/discovery/uyuni/uyuni.go index 6419d8d365..0320a0490d 100644 --- a/discovery/uyuni/uyuni.go +++ b/discovery/uyuni/uyuni.go @@ -124,7 +124,7 @@ func (*SDConfig) Name() string { return "uyuni" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -213,8 +213,8 @@ func getEndpointInfoForSystems( } // NewDiscovery returns a uyuni discovery for the given configuration. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*uyuniMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*uyuniMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -238,13 +238,14 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove entitlement: conf.Entitlement, separator: conf.Separator, interval: time.Duration(conf.RefreshInterval), - logger: logger, + logger: opts.Logger, } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "uyuni", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/uyuni/uyuni_test.go b/discovery/uyuni/uyuni_test.go index 46567587a8..4a73fa9ada 100644 --- a/discovery/uyuni/uyuni_test.go +++ b/discovery/uyuni/uyuni_test.go @@ -47,7 +47,11 @@ func testUpdateServices(respHandler http.HandlerFunc) ([]*targetgroup.Group, err defer metrics.Unregister() defer refreshMetrics.Unregister() - md, err := NewDiscovery(&conf, nil, metrics) + md, err := NewDiscovery(&conf, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "uyuni", + }) if err != nil { return nil, err } @@ -127,7 +131,11 @@ func TestUyuniSDSkipLogin(t *testing.T) { defer metrics.Unregister() defer refreshMetrics.Unregister() - md, err := NewDiscovery(&conf, nil, metrics) + md, err := NewDiscovery(&conf, discovery.DiscovererOptions{ + Logger: nil, + Metrics: metrics, + SetName: "uyuni", + }) if err != nil { t.Error(err) } diff --git a/discovery/vultr/vultr.go b/discovery/vultr/vultr.go index 79a7a0179f..27f3e11064 100644 --- a/discovery/vultr/vultr.go +++ b/discovery/vultr/vultr.go @@ -16,7 +16,6 @@ package vultr import ( "context" "errors" - "log/slog" "net" "net/http" "strconv" @@ -86,7 +85,7 @@ func (*SDConfig) Name() string { return "vultr" } // NewDiscoverer returns a Discoverer for the Config. func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewDiscovery(c, opts.Logger, opts.Metrics) + return NewDiscovery(c, opts) } // SetDirectory joins any relative file paths with dir. @@ -114,8 +113,8 @@ type Discovery struct { } // NewDiscovery returns a new Discovery which periodically refreshes its targets. -func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*Discovery, error) { - m, ok := metrics.(*vultrMetrics) +func NewDiscovery(conf *SDConfig, opts discovery.DiscovererOptions) (*Discovery, error) { + m, ok := opts.Metrics.(*vultrMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } @@ -138,8 +137,9 @@ func NewDiscovery(conf *SDConfig, logger *slog.Logger, metrics discovery.Discove d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "vultr", + SetName: opts.SetName, Interval: time.Duration(conf.RefreshInterval), RefreshF: d.refresh, MetricsInstantiator: m.refreshMetrics, diff --git a/discovery/vultr/vultr_test.go b/discovery/vultr/vultr_test.go index 00ef21e38c..8975cfb455 100644 --- a/discovery/vultr/vultr_test.go +++ b/discovery/vultr/vultr_test.go @@ -57,7 +57,11 @@ func TestVultrSDRefresh(t *testing.T) { defer metrics.Unregister() defer refreshMetrics.Unregister() - d, err := NewDiscovery(&cfg, promslog.NewNopLogger(), metrics) + d, err := NewDiscovery(&cfg, discovery.DiscovererOptions{ + Logger: promslog.NewNopLogger(), + Metrics: metrics, + SetName: "vultr", + }) require.NoError(t, err) endpoint, err := url.Parse(sdMock.Mock.Endpoint()) require.NoError(t, err) diff --git a/docs/http_sd.md b/docs/http_sd.md index 3bd6bada39..d329ce07af 100644 --- a/docs/http_sd.md +++ b/docs/http_sd.md @@ -39,8 +39,9 @@ an empty list `[]`. Target lists are unordered. Prometheus caches target lists. If an error occurs while fetching an updated targets list, Prometheus keeps using the current targets list. The targets list -is not saved across restart. The `prometheus_sd_http_failures_total` counter -metric tracks the number of refresh failures. +is not saved across restart. The `prometheus_sd_refresh_failures_total` counter +metric tracks the number of refresh failures and the `prometheus_sd_refresh_duration_seconds` +bucket can be used to track HTTP SD refresh attempts or performance. The whole list of targets must be returned on every scrape. There is no support for incremental updates. A Prometheus instance does not send its hostname and it From 2852c9c431dd0920d60641ffd023c4ab263aa0cb Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 17 Oct 2025 10:26:46 +0100 Subject: [PATCH 014/439] [REFACTOR] TSDB: Simplify series creation Refactor the code so that everything proceeds linearly. Also renamed `getOrSet` to `setUnlessAlreadySet` to emphasise that the caller is expecting it not to be set. Signed-off-by: Bryan Boreham --- tsdb/head.go | 48 ++++++++++++++--------------------------------- tsdb/head_test.go | 10 ++-------- 2 files changed, 16 insertions(+), 42 deletions(-) diff --git a/tsdb/head.go b/tsdb/head.go index 4e77314b02..a02b05f95d 100644 --- a/tsdb/head.go +++ b/tsdb/head.go @@ -1753,17 +1753,17 @@ func (h *Head) getOrCreate(hash uint64, lset labels.Labels, pendingCommit bool) } func (h *Head) getOrCreateWithID(id chunks.HeadSeriesRef, hash uint64, lset labels.Labels, pendingCommit bool) (*memSeries, bool, error) { - s, created, err := h.series.getOrSet(hash, lset, func() *memSeries { - shardHash := uint64(0) - if h.opts.EnableSharding { - shardHash = labels.StableHash(lset) - } - - return newMemSeries(lset, id, shardHash, h.opts.IsolationDisabled, pendingCommit) - }) - if err != nil { - return nil, false, err + if preCreationErr := h.series.seriesLifecycleCallback.PreCreation(lset); preCreationErr != nil { + return nil, false, preCreationErr } + + shardHash := uint64(0) + if h.opts.EnableSharding { + shardHash = labels.StableHash(lset) + } + optimisticallyCreatedSeries := newMemSeries(lset, id, shardHash, h.opts.IsolationDisabled, pendingCommit) + + s, created := h.series.setUnlessAlreadySet(hash, lset, optimisticallyCreatedSeries) if !created { return s, false, nil } @@ -2061,43 +2061,23 @@ func (s *stripeSeries) getByHash(hash uint64, lset labels.Labels) *memSeries { return series } -func (s *stripeSeries) getOrSet(hash uint64, lset labels.Labels, createSeries func() *memSeries) (*memSeries, bool, error) { - // PreCreation is called here to avoid calling it inside the lock. - // It is not necessary to call it just before creating a series, - // rather it gives a 'hint' whether to create a series or not. - preCreationErr := s.seriesLifecycleCallback.PreCreation(lset) - - // Create the series, unless the PreCreation() callback as failed. - // If failed, we'll not allow to create a new series anyway. - var series *memSeries - if preCreationErr == nil { - series = createSeries() - } - +func (s *stripeSeries) setUnlessAlreadySet(hash uint64, lset labels.Labels, series *memSeries) (*memSeries, bool) { i := hash & uint64(s.size-1) s.locks[i].Lock() - if prev := s.hashes[i].get(hash, lset); prev != nil { s.locks[i].Unlock() - return prev, false, nil - } - if preCreationErr == nil { - s.hashes[i].set(hash, series) + return prev, false } + s.hashes[i].set(hash, series) s.locks[i].Unlock() - if preCreationErr != nil { - // The callback prevented creation of series. - return nil, false, preCreationErr - } - i = uint64(series.ref) & uint64(s.size-1) s.locks[i].Lock() s.series[i][series.ref] = series s.locks[i].Unlock() - return series, true, nil + return series, true } func (s *stripeSeries) postCreation(lset labels.Labels) { diff --git a/tsdb/head_test.go b/tsdb/head_test.go index d32e632074..8565ac3bbe 100644 --- a/tsdb/head_test.go +++ b/tsdb/head_test.go @@ -6628,18 +6628,12 @@ func stripeSeriesWithCollidingSeries(t *testing.T) (*stripeSeries, *memSeries, * hash := lbls1.Hash() s := newStripeSeries(1, noopSeriesLifecycleCallback{}) - got, created, err := s.getOrSet(hash, lbls1, func() *memSeries { - return &ms1 - }) - require.NoError(t, err) + got, created := s.setUnlessAlreadySet(hash, lbls1, &ms1) require.True(t, created) require.Same(t, &ms1, got) // Add a conflicting series - got, created, err = s.getOrSet(hash, lbls2, func() *memSeries { - return &ms2 - }) - require.NoError(t, err) + got, created = s.setUnlessAlreadySet(hash, lbls2, &ms2) require.True(t, created) require.Same(t, &ms2, got) From 42b52ecc4be1392a54653a38698909eda03a9a40 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 17 Oct 2025 11:04:34 +0100 Subject: [PATCH 015/439] TSDB: Allocate series ID after seriesLifecycleCallback This callback is not used by Prometheus, but in downstream projects it is wasteful to allocate an ID only to abandon it. Remove lengthy commment which I feel is distracting from the flow. Signed-off-by: Bryan Boreham --- tsdb/head.go | 15 +++++++-------- tsdb/head_test.go | 2 +- tsdb/head_wal.go | 4 ++-- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/tsdb/head.go b/tsdb/head.go index a02b05f95d..2c71977b1a 100644 --- a/tsdb/head.go +++ b/tsdb/head.go @@ -1738,24 +1738,23 @@ func (*Head) String() string { } func (h *Head) getOrCreate(hash uint64, lset labels.Labels, pendingCommit bool) (*memSeries, bool, error) { - // Just using `getOrCreateWithID` below would be semantically sufficient, but we'd create - // a new series on every sample inserted via Add(), which causes allocations - // and makes our series IDs rather random and harder to compress in postings. s := h.series.getByHash(hash, lset) if s != nil { return s, false, nil } - // Optimistically assume that we are the first one to create the series. - id := chunks.HeadSeriesRef(h.lastSeriesID.Inc()) - - return h.getOrCreateWithID(id, hash, lset, pendingCommit) + return h.getOrCreateWithOptionalID(0, hash, lset, pendingCommit) } -func (h *Head) getOrCreateWithID(id chunks.HeadSeriesRef, hash uint64, lset labels.Labels, pendingCommit bool) (*memSeries, bool, error) { +// If id is zero, one will be allocated. +func (h *Head) getOrCreateWithOptionalID(id chunks.HeadSeriesRef, hash uint64, lset labels.Labels, pendingCommit bool) (*memSeries, bool, error) { if preCreationErr := h.series.seriesLifecycleCallback.PreCreation(lset); preCreationErr != nil { return nil, false, preCreationErr } + if id == 0 { + // Note this id is wasted in the case where a concurrent operation creates the same series first. + id = chunks.HeadSeriesRef(h.lastSeriesID.Inc()) + } shardHash := uint64(0) if h.opts.EnableSharding { diff --git a/tsdb/head_test.go b/tsdb/head_test.go index 8565ac3bbe..44daa7cddc 100644 --- a/tsdb/head_test.go +++ b/tsdb/head_test.go @@ -1152,7 +1152,7 @@ func TestHead_KeepSeriesInWALCheckpoint(t *testing.T) { { name: "keep series still in the head", prepare: func(t *testing.T, h *Head) { - _, _, err := h.getOrCreateWithID(chunks.HeadSeriesRef(existingRef), existingLbls.Hash(), existingLbls, false) + _, _, err := h.getOrCreateWithOptionalID(chunks.HeadSeriesRef(existingRef), existingLbls.Hash(), existingLbls, false) require.NoError(t, err) }, expected: true, diff --git a/tsdb/head_wal.go b/tsdb/head_wal.go index 3c5390cab4..cb7397e502 100644 --- a/tsdb/head_wal.go +++ b/tsdb/head_wal.go @@ -255,7 +255,7 @@ Outer: switch v := d.(type) { case []record.RefSeries: for _, walSeries := range v { - mSeries, created, err := h.getOrCreateWithID(walSeries.Ref, walSeries.Labels.Hash(), walSeries.Labels, false) + mSeries, created, err := h.getOrCreateWithOptionalID(walSeries.Ref, walSeries.Labels.Hash(), walSeries.Labels, false) if err != nil { seriesCreationErr = err break Outer @@ -1590,7 +1590,7 @@ func (h *Head) loadChunkSnapshot() (int, int, map[chunks.HeadSeriesRef]*memSerie localRefSeries := shardedRefSeries[idx] for csr := range rc { - series, _, err := h.getOrCreateWithID(csr.ref, csr.lset.Hash(), csr.lset, false) + series, _, err := h.getOrCreateWithOptionalID(csr.ref, csr.lset.Hash(), csr.lset, false) if err != nil { errChan <- err return From e1cb29bf8a7e435ed1fc7d88f46900017b4ee543 Mon Sep 17 00:00:00 2001 From: pipiland2612 Date: Fri, 31 Oct 2025 21:55:14 +0200 Subject: [PATCH 016/439] create common struct and function to DRY Signed-off-by: pipiland2612 --- storage/remote/queue_manager.go | 136 +++++++++++++++++++++----------- 1 file changed, 88 insertions(+), 48 deletions(-) diff --git a/storage/remote/queue_manager.go b/storage/remote/queue_manager.go index 25d3a94b6a..a5c215ec2e 100644 --- a/storage/remote/queue_manager.go +++ b/storage/remote/queue_manager.go @@ -30,6 +30,7 @@ import ( "github.com/prometheus/common/promslog" "go.opentelemetry.io/otel" "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/trace" semconv "go.opentelemetry.io/otel/semconv/v1.21.0" "go.uber.org/atomic" @@ -1737,6 +1738,20 @@ func (s *shards) sendSamplesWithBackoff(ctx context.Context, samples []prompb.Ti } reqSize := len(req) + sc := sendBatchContext{ + ctx: ctx, + sampleCount: sampleCount, + exemplarCount: exemplarCount, + histogramCount: histogramCount, + metadataCount: metadataCount, + reqSize: reqSize, + } + + metricsUpdater := batchMetricsUpdater{ + metrics: s.qm.metrics, + storeClient: s.qm.storeClient, + sentDuration: s.qm.metrics.sentBatchDuration, + } // Since we retry writes via attemptStore and sendWriteRequestWithBackoff we need // to track the total amount of accepted data across the various attempts. @@ -1772,33 +1787,14 @@ func (s *shards) sendSamplesWithBackoff(ctx context.Context, samples []prompb.Ti req = req2 } - ctx, span := otel.Tracer("").Start(ctx, "Remote Send Batch") + ctx, span := createBatchSpan(sc.ctx, sc, s.qm.storeClient.Name(), s.qm.storeClient.Endpoint(), try) defer span.End() - span.SetAttributes( - attribute.Int("request_size", reqSize), - attribute.Int("samples", sampleCount), - attribute.Int("try", try), - attribute.String("remote_name", s.qm.storeClient.Name()), - attribute.String("remote_url", s.qm.storeClient.Endpoint()), - ) - - if exemplarCount > 0 { - span.SetAttributes(attribute.Int("exemplars", exemplarCount)) - } - if histogramCount > 0 { - span.SetAttributes(attribute.Int("histograms", histogramCount)) - } - begin := time.Now() - s.qm.metrics.samplesTotal.Add(float64(sampleCount)) - s.qm.metrics.exemplarsTotal.Add(float64(exemplarCount)) - s.qm.metrics.histogramsTotal.Add(float64(histogramCount)) - s.qm.metrics.metadataTotal.Add(float64(metadataCount)) + metricsUpdater.recordBatchAttempt(sc, begin) // Technically for v1, we will likely have empty response stats, but for // newer Receivers this might be not, so used it in a best effort. rs, err := s.qm.client().Store(ctx, req, try) - s.qm.metrics.sentBatchDuration.Observe(time.Since(begin).Seconds()) // TODO(bwplotka): Revisit this once we have Receivers doing retriable partial error // so far we don't have those, so it's ok to potentially skew statistics. addStats(rs) @@ -1811,9 +1807,7 @@ func (s *shards) sendSamplesWithBackoff(ctx context.Context, samples []prompb.Ti } onRetry := func() { - s.qm.metrics.retriedSamplesTotal.Add(float64(sampleCount)) - s.qm.metrics.retriedExemplarsTotal.Add(float64(exemplarCount)) - s.qm.metrics.retriedHistogramsTotal.Add(float64(histogramCount)) + metricsUpdater.recordRetry(sc) } err = s.qm.sendWriteRequestWithBackoff(ctx, attemptStore, onRetry) @@ -1850,6 +1844,20 @@ func (s *shards) sendV2SamplesWithBackoff(ctx context.Context, samples []writev2 } reqSize := len(req) + sc := sendBatchContext{ + ctx: ctx, + sampleCount: sampleCount, + exemplarCount: exemplarCount, + histogramCount: histogramCount, + metadataCount: metadataCount, + reqSize: reqSize, + } + + metricsUpdater := batchMetricsUpdater{ + metrics: s.qm.metrics, + storeClient: s.qm.storeClient, + sentDuration: s.qm.metrics.sentBatchDuration, + } // Since we retry writes via attemptStore and sendWriteRequestWithBackoff we need // to track the total amount of accepted data across the various attempts. @@ -1885,31 +1893,12 @@ func (s *shards) sendV2SamplesWithBackoff(ctx context.Context, samples []writev2 req = req2 } - ctx, span := otel.Tracer("").Start(ctx, "Remote Send Batch") + ctx, span := createBatchSpan(sc.ctx, sc, s.qm.storeClient.Name(), s.qm.storeClient.Endpoint(), try) defer span.End() - span.SetAttributes( - attribute.Int("request_size", reqSize), - attribute.Int("samples", sampleCount), - attribute.Int("try", try), - attribute.String("remote_name", s.qm.storeClient.Name()), - attribute.String("remote_url", s.qm.storeClient.Endpoint()), - ) - - if exemplarCount > 0 { - span.SetAttributes(attribute.Int("exemplars", exemplarCount)) - } - if histogramCount > 0 { - span.SetAttributes(attribute.Int("histograms", histogramCount)) - } - begin := time.Now() - s.qm.metrics.samplesTotal.Add(float64(sampleCount)) - s.qm.metrics.exemplarsTotal.Add(float64(exemplarCount)) - s.qm.metrics.histogramsTotal.Add(float64(histogramCount)) - s.qm.metrics.metadataTotal.Add(float64(metadataCount)) + metricsUpdater.recordBatchAttempt(sc, begin) rs, err := s.qm.client().Store(ctx, req, try) - s.qm.metrics.sentBatchDuration.Observe(time.Since(begin).Seconds()) // TODO(bwplotka): Revisit this once we have Receivers doing retriable partial error // so far we don't have those, so it's ok to potentially skew statistics. addStats(rs) @@ -1933,9 +1922,7 @@ func (s *shards) sendV2SamplesWithBackoff(ctx context.Context, samples []writev2 } onRetry := func() { - s.qm.metrics.retriedSamplesTotal.Add(float64(sampleCount)) - s.qm.metrics.retriedExemplarsTotal.Add(float64(exemplarCount)) - s.qm.metrics.retriedHistogramsTotal.Add(float64(histogramCount)) + metricsUpdater.recordRetry(sc) } err = s.qm.sendWriteRequestWithBackoff(ctx, attemptStore, onRetry) @@ -2266,3 +2253,56 @@ func buildV2TimeSeries(timeSeries []writev2.TimeSeries, filter func(writev2.Time timeSeries = timeSeries[:keepIdx] return highest, lowest, timeSeries, droppedSamples, droppedExemplars, droppedHistograms } + +// sendBatchContext encapsulates the common parameters for sending batches. +// This reduces the number of function arguments (addresses "too many arguments" issue). +type sendBatchContext struct { + ctx context.Context + sampleCount int + exemplarCount int + histogramCount int + metadataCount int + reqSize int +} + +// batchMetricsUpdater encapsulates metrics update operations for batch sending. +type batchMetricsUpdater struct { + metrics *queueManagerMetrics + storeClient WriteClient + sentDuration prometheus.Observer +} + +// recordBatchAttempt records metrics for a batch send attempt. +func (b *batchMetricsUpdater) recordBatchAttempt(sc sendBatchContext, begin time.Time) { + b.metrics.samplesTotal.Add(float64(sc.sampleCount)) + b.metrics.exemplarsTotal.Add(float64(sc.exemplarCount)) + b.metrics.histogramsTotal.Add(float64(sc.histogramCount)) + b.metrics.metadataTotal.Add(float64(sc.metadataCount)) + b.sentDuration.Observe(time.Since(begin).Seconds()) +} + +// recordRetry records retry metrics for a batch. +func (b *batchMetricsUpdater) recordRetry(sc sendBatchContext) { + b.metrics.retriedSamplesTotal.Add(float64(sc.sampleCount)) + b.metrics.retriedExemplarsTotal.Add(float64(sc.exemplarCount)) + b.metrics.retriedHistogramsTotal.Add(float64(sc.histogramCount)) +} + +// createSpan creates and configures an OpenTelemetry span for batch sending. +func createBatchSpan(ctx context.Context, sc sendBatchContext, remoteName, remoteURL string, try int) (context.Context, trace.Span) { + ctx, span := otel.Tracer("").Start(ctx, "Remote Send Batch") + span.SetAttributes( + attribute.Int("request_size", sc.reqSize), + attribute.Int("samples", sc.sampleCount), + attribute.Int("try", try), + attribute.String("remote_name", remoteName), + attribute.String("remote_url", remoteURL), + ) + if sc.exemplarCount > 0 { + span.SetAttributes(attribute.Int("exemplars", sc.exemplarCount)) + } + if sc.histogramCount > 0 { + span.SetAttributes(attribute.Int("histograms", sc.histogramCount)) + } + return ctx, span +} From 9e6a626dae6d0477b875fdd94c8eaac37cd00797 Mon Sep 17 00:00:00 2001 From: pipiland2612 Date: Fri, 31 Oct 2025 22:17:45 +0200 Subject: [PATCH 017/439] create timeSeriesStats to reduce return variable Signed-off-by: pipiland2612 --- storage/remote/queue_manager.go | 156 +++++++++++++-------------- storage/remote/queue_manager_test.go | 10 +- 2 files changed, 81 insertions(+), 85 deletions(-) diff --git a/storage/remote/queue_manager.go b/storage/remote/queue_manager.go index a5c215ec2e..e3d6396a78 100644 --- a/storage/remote/queue_manager.go +++ b/storage/remote/queue_manager.go @@ -30,8 +30,8 @@ import ( "github.com/prometheus/common/promslog" "go.opentelemetry.io/otel" "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" semconv "go.opentelemetry.io/otel/semconv/v1.21.0" + "go.opentelemetry.io/otel/trace" "go.uber.org/atomic" "github.com/prometheus/prometheus/config" @@ -2088,48 +2088,27 @@ func setAtomicToNewer(value *atomic.Int64, newValue int64) (previous int64, upda } } -func buildTimeSeries(timeSeries []prompb.TimeSeries, filter func(prompb.TimeSeries) bool) (int64, int64, []prompb.TimeSeries, int, int, int) { - var highest int64 - var lowest int64 - var droppedSamples, droppedExemplars, droppedHistograms int - +func buildTimeSeries(timeSeries []prompb.TimeSeries, filter func(prompb.TimeSeries) bool) ([]prompb.TimeSeries, *timeSeriesStats) { + stats := newTimeSeriesStats() keepIdx := 0 - lowest = math.MaxInt64 + for i, ts := range timeSeries { if filter != nil && filter(ts) { - if len(ts.Samples) > 0 { - droppedSamples++ - } - if len(ts.Exemplars) > 0 { - droppedExemplars++ - } - if len(ts.Histograms) > 0 { - droppedHistograms++ - } + stats.recordDropped(len(ts.Samples) > 0, len(ts.Exemplars) > 0, len(ts.Histograms) > 0) continue } // At the moment we only ever append a TimeSeries with a single sample or exemplar in it. - if len(ts.Samples) > 0 && ts.Samples[0].Timestamp > highest { - highest = ts.Samples[0].Timestamp + if len(ts.Samples) > 0 { + stats.updateTimestamp(ts.Samples[0].Timestamp) } - if len(ts.Exemplars) > 0 && ts.Exemplars[0].Timestamp > highest { - highest = ts.Exemplars[0].Timestamp + if len(ts.Exemplars) > 0 { + stats.updateTimestamp(ts.Exemplars[0].Timestamp) } - if len(ts.Histograms) > 0 && ts.Histograms[0].Timestamp > highest { - highest = ts.Histograms[0].Timestamp + if len(ts.Histograms) > 0 { + stats.updateTimestamp(ts.Histograms[0].Timestamp) } - // Get lowest timestamp - if len(ts.Samples) > 0 && ts.Samples[0].Timestamp < lowest { - lowest = ts.Samples[0].Timestamp - } - if len(ts.Exemplars) > 0 && ts.Exemplars[0].Timestamp < lowest { - lowest = ts.Exemplars[0].Timestamp - } - if len(ts.Histograms) > 0 && ts.Histograms[0].Timestamp < lowest { - lowest = ts.Histograms[0].Timestamp - } if i != keepIdx { // We have to swap the kept timeseries with the one which should be dropped. // Copying any elements within timeSeries could cause data corruptions when reusing the slice in a next batch (shards.populateTimeSeries). @@ -2138,16 +2117,14 @@ func buildTimeSeries(timeSeries []prompb.TimeSeries, filter func(prompb.TimeSeri keepIdx++ } - timeSeries = timeSeries[:keepIdx] - return highest, lowest, timeSeries, droppedSamples, droppedExemplars, droppedHistograms + return timeSeries[:keepIdx], stats } func buildWriteRequest(logger *slog.Logger, timeSeries []prompb.TimeSeries, metadata []prompb.MetricMetadata, pBuf *proto.Buffer, filter func(prompb.TimeSeries) bool, buf compression.EncodeBuffer, compr compression.Type) (_ []byte, highest, lowest int64, _ error) { - highest, lowest, timeSeries, - droppedSamples, droppedExemplars, droppedHistograms := buildTimeSeries(timeSeries, filter) + timeSeries, stats := buildTimeSeries(timeSeries, filter) - if droppedSamples > 0 || droppedExemplars > 0 || droppedHistograms > 0 { - logger.Debug("dropped data due to their age", "droppedSamples", droppedSamples, "droppedExemplars", droppedExemplars, "droppedHistograms", droppedHistograms) + if stats.droppedSamples > 0 || stats.droppedExemplars > 0 || stats.droppedHistograms > 0 { + logger.Debug("dropped data due to their age", "droppedSamples", stats.droppedSamples, "droppedExemplars", stats.droppedExemplars, "droppedHistograms", stats.droppedHistograms) } req := &prompb.WriteRequest{ @@ -2161,21 +2138,21 @@ func buildWriteRequest(logger *slog.Logger, timeSeries []prompb.TimeSeries, meta pBuf.Reset() } if err := pBuf.Marshal(req); err != nil { - return nil, highest, lowest, err + return nil, stats.highest, stats.lowest, err } compressed, err := compression.Encode(compr, pBuf.Bytes(), buf) if err != nil { - return nil, highest, lowest, err + return nil, stats.highest, stats.lowest, err } - return compressed, highest, lowest, nil + return compressed, stats.highest, stats.lowest, nil } func buildV2WriteRequest(logger *slog.Logger, samples []writev2.TimeSeries, labels []string, pBuf *[]byte, filter func(writev2.TimeSeries) bool, buf compression.EncodeBuffer, compr compression.Type) (compressed []byte, highest, lowest int64, _ error) { - highest, lowest, timeSeries, droppedSamples, droppedExemplars, droppedHistograms := buildV2TimeSeries(samples, filter) + timeSeries, stats := buildV2TimeSeries(samples, filter) - if droppedSamples > 0 || droppedExemplars > 0 || droppedHistograms > 0 { - logger.Debug("dropped data due to their age", "droppedSamples", droppedSamples, "droppedExemplars", droppedExemplars, "droppedHistograms", droppedHistograms) + if stats.droppedSamples > 0 || stats.droppedExemplars > 0 || stats.droppedHistograms > 0 { + logger.Debug("dropped data due to their age", "droppedSamples", stats.droppedSamples, "droppedExemplars", stats.droppedExemplars, "droppedHistograms", stats.droppedHistograms) } req := &writev2.Request{ @@ -2189,59 +2166,38 @@ func buildV2WriteRequest(logger *slog.Logger, samples []writev2.TimeSeries, labe data, err := req.OptimizedMarshal(*pBuf) if err != nil { - return nil, highest, lowest, err + return nil, stats.highest, stats.lowest, err } *pBuf = data compressed, err = compression.Encode(compr, *pBuf, buf) if err != nil { - return nil, highest, lowest, err + return nil, stats.highest, stats.lowest, err } - return compressed, highest, lowest, nil + return compressed, stats.highest, stats.lowest, nil } -func buildV2TimeSeries(timeSeries []writev2.TimeSeries, filter func(writev2.TimeSeries) bool) (int64, int64, []writev2.TimeSeries, int, int, int) { - var highest int64 - var lowest int64 - var droppedSamples, droppedExemplars, droppedHistograms int - +func buildV2TimeSeries(timeSeries []writev2.TimeSeries, filter func(writev2.TimeSeries) bool) ([]writev2.TimeSeries, *timeSeriesStats) { + stats := newTimeSeriesStats() keepIdx := 0 - lowest = math.MaxInt64 + for i, ts := range timeSeries { if filter != nil && filter(ts) { - if len(ts.Samples) > 0 { - droppedSamples++ - } - if len(ts.Exemplars) > 0 { - droppedExemplars++ - } - if len(ts.Histograms) > 0 { - droppedHistograms++ - } + stats.recordDropped(len(ts.Samples) > 0, len(ts.Exemplars) > 0, len(ts.Histograms) > 0) continue } // At the moment we only ever append a TimeSeries with a single sample or exemplar in it. - if len(ts.Samples) > 0 && ts.Samples[0].Timestamp > highest { - highest = ts.Samples[0].Timestamp + if len(ts.Samples) > 0 { + stats.updateTimestamp(ts.Samples[0].Timestamp) } - if len(ts.Exemplars) > 0 && ts.Exemplars[0].Timestamp > highest { - highest = ts.Exemplars[0].Timestamp + if len(ts.Exemplars) > 0 { + stats.updateTimestamp(ts.Exemplars[0].Timestamp) } - if len(ts.Histograms) > 0 && ts.Histograms[0].Timestamp > highest { - highest = ts.Histograms[0].Timestamp + if len(ts.Histograms) > 0 { + stats.updateTimestamp(ts.Histograms[0].Timestamp) } - // Get the lowest timestamp. - if len(ts.Samples) > 0 && ts.Samples[0].Timestamp < lowest { - lowest = ts.Samples[0].Timestamp - } - if len(ts.Exemplars) > 0 && ts.Exemplars[0].Timestamp < lowest { - lowest = ts.Exemplars[0].Timestamp - } - if len(ts.Histograms) > 0 && ts.Histograms[0].Timestamp < lowest { - lowest = ts.Histograms[0].Timestamp - } if i != keepIdx { // We have to swap the kept timeseries with the one which should be dropped. // Copying any elements within timeSeries could cause data corruptions when reusing the slice in a next batch (shards.populateTimeSeries). @@ -2250,8 +2206,48 @@ func buildV2TimeSeries(timeSeries []writev2.TimeSeries, filter func(writev2.Time keepIdx++ } - timeSeries = timeSeries[:keepIdx] - return highest, lowest, timeSeries, droppedSamples, droppedExemplars, droppedHistograms + return timeSeries[:keepIdx], stats +} + +// timeSeriesStats tracks statistics during time series processing. +type timeSeriesStats struct { + highest int64 + lowest int64 + droppedSamples int + droppedExemplars int + droppedHistograms int +} + +// newTimeSeriesStats creates a new timeSeriesStats with lowest initialized to MaxInt64. +func newTimeSeriesStats() *timeSeriesStats { + return &timeSeriesStats{ + lowest: math.MaxInt64, + } +} + +// updateTimestamp updates highest and lowest timestamps if the given timestamp is valid. +func (s *timeSeriesStats) updateTimestamp(timestamp int64) { + if timestamp > 0 { + if timestamp > s.highest { + s.highest = timestamp + } + if timestamp < s.lowest { + s.lowest = timestamp + } + } +} + +// recordDropped increments the dropped counters based on what data exists. +func (s *timeSeriesStats) recordDropped(hasSamples, hasExemplars, hasHistograms bool) { + if hasSamples { + s.droppedSamples++ + } + if hasExemplars { + s.droppedExemplars++ + } + if hasHistograms { + s.droppedHistograms++ + } } // sendBatchContext encapsulates the common parameters for sending batches. diff --git a/storage/remote/queue_manager_test.go b/storage/remote/queue_manager_test.go index bb72b7f998..08f1a141c7 100644 --- a/storage/remote/queue_manager_test.go +++ b/storage/remote/queue_manager_test.go @@ -2323,12 +2323,12 @@ func TestBuildTimeSeries(t *testing.T) { // Run the test cases for _, tc := range testCases { t.Run(tc.name, func(t *testing.T) { - highest, lowest, result, droppedSamples, _, _ := buildTimeSeries(tc.ts, tc.filter) + result, stats := buildTimeSeries(tc.ts, tc.filter) require.NotNil(t, result) require.Len(t, result, tc.responseLen) - require.Equal(t, tc.highestTs, highest) - require.Equal(t, tc.lowestTs, lowest) - require.Equal(t, tc.droppedSamples, droppedSamples) + require.Equal(t, tc.highestTs, stats.highest) + require.Equal(t, tc.lowestTs, stats.lowest) + require.Equal(t, tc.droppedSamples, stats.droppedSamples) }) } } @@ -2339,7 +2339,7 @@ func BenchmarkBuildTimeSeries(b *testing.B) { filter := func(ts prompb.TimeSeries) bool { return filterTsLimit(99, ts) } for i := 0; i < b.N; i++ { samples := createProtoTimeseriesWithOld(numSamples, 100, extraLabels...) - _, _, result, _, _, _ := buildTimeSeries(samples, filter) + result, _ := buildTimeSeries(samples, filter) require.NotNil(b, result) } } From 704afd8529cf3df29faac33a0a6f5c868884e723 Mon Sep 17 00:00:00 2001 From: pipiland2612 Date: Fri, 31 Oct 2025 23:19:53 +0200 Subject: [PATCH 018/439] add timeSeriesAgeChecker to refactor filter code Signed-off-by: pipiland2612 --- storage/remote/queue_manager.go | 87 +++++++++++++++++++-------------- 1 file changed, 51 insertions(+), 36 deletions(-) diff --git a/storage/remote/queue_manager.go b/storage/remote/queue_manager.go index e3d6396a78..73a4896f19 100644 --- a/storage/remote/queue_manager.go +++ b/storage/remote/queue_manager.go @@ -645,63 +645,78 @@ func isSampleOld(baseTime time.Time, sampleAgeLimit time.Duration, ts int64) boo return sampleTs.Before(limitTs) } +// timeSeriesAgeChecker encapsulates the logic for checking if time series data is too old. +type timeSeriesAgeChecker struct { + metrics *queueManagerMetrics + baseTime time.Time + sampleAgeLimit time.Duration +} + +// checkAndRecordIfOld checks if a timestamp is too old and records the appropriate metric. +// Returns true if the data should be dropped. +func (c *timeSeriesAgeChecker) checkAndRecordIfOld(timestamp int64, dataType string) bool { + if c.sampleAgeLimit == 0 { + // If sampleAgeLimit is unset, then we never skip samples due to their age. + return false + } + + if !isSampleOld(c.baseTime, c.sampleAgeLimit, timestamp) { + return false + } + + // Record the drop in metrics. + switch dataType { + case "sample": + c.metrics.droppedSamplesTotal.WithLabelValues(reasonTooOld).Inc() + case "histogram": + c.metrics.droppedHistogramsTotal.WithLabelValues(reasonTooOld).Inc() + case "exemplar": + c.metrics.droppedExemplarsTotal.WithLabelValues(reasonTooOld).Inc() + } + return true +} + func isTimeSeriesOldFilter(metrics *queueManagerMetrics, baseTime time.Time, sampleAgeLimit time.Duration) func(ts prompb.TimeSeries) bool { + checker := &timeSeriesAgeChecker{ + metrics: metrics, + baseTime: baseTime, + sampleAgeLimit: sampleAgeLimit, + } + return func(ts prompb.TimeSeries) bool { - if sampleAgeLimit == 0 { - // If sampleAgeLimit is unset, then we never skip samples due to their age. - return false - } - switch { // Only the first element should be set in the series, therefore we only check the first element. + switch { case len(ts.Samples) > 0: - if isSampleOld(baseTime, sampleAgeLimit, ts.Samples[0].Timestamp) { - metrics.droppedSamplesTotal.WithLabelValues(reasonTooOld).Inc() - return true - } + return checker.checkAndRecordIfOld(ts.Samples[0].Timestamp, "sample") case len(ts.Histograms) > 0: - if isSampleOld(baseTime, sampleAgeLimit, ts.Histograms[0].Timestamp) { - metrics.droppedHistogramsTotal.WithLabelValues(reasonTooOld).Inc() - return true - } + return checker.checkAndRecordIfOld(ts.Histograms[0].Timestamp, "histogram") case len(ts.Exemplars) > 0: - if isSampleOld(baseTime, sampleAgeLimit, ts.Exemplars[0].Timestamp) { - metrics.droppedExemplarsTotal.WithLabelValues(reasonTooOld).Inc() - return true - } + return checker.checkAndRecordIfOld(ts.Exemplars[0].Timestamp, "exemplar") default: return false } - return false } } func isV2TimeSeriesOldFilter(metrics *queueManagerMetrics, baseTime time.Time, sampleAgeLimit time.Duration) func(ts writev2.TimeSeries) bool { + checker := &timeSeriesAgeChecker{ + metrics: metrics, + baseTime: baseTime, + sampleAgeLimit: sampleAgeLimit, + } + return func(ts writev2.TimeSeries) bool { - if sampleAgeLimit == 0 { - // If sampleAgeLimit is unset, then we never skip samples due to their age. - return false - } - switch { // Only the first element should be set in the series, therefore we only check the first element. + switch { case len(ts.Samples) > 0: - if isSampleOld(baseTime, sampleAgeLimit, ts.Samples[0].Timestamp) { - metrics.droppedSamplesTotal.WithLabelValues(reasonTooOld).Inc() - return true - } + return checker.checkAndRecordIfOld(ts.Samples[0].Timestamp, "sample") case len(ts.Histograms) > 0: - if isSampleOld(baseTime, sampleAgeLimit, ts.Histograms[0].Timestamp) { - metrics.droppedHistogramsTotal.WithLabelValues(reasonTooOld).Inc() - return true - } + return checker.checkAndRecordIfOld(ts.Histograms[0].Timestamp, "histogram") case len(ts.Exemplars) > 0: - if isSampleOld(baseTime, sampleAgeLimit, ts.Exemplars[0].Timestamp) { - metrics.droppedExemplarsTotal.WithLabelValues(reasonTooOld).Inc() - return true - } + return checker.checkAndRecordIfOld(ts.Exemplars[0].Timestamp, "exemplar") default: return false } - return false } } From e3e104816633d75c50b4f6d849db1bc6a11fac1a Mon Sep 17 00:00:00 2001 From: 3Juhwan <13selfesteem91@naver.com> Date: Tue, 4 Nov 2025 16:51:41 -0800 Subject: [PATCH 019/439] Isolate fix: Remove 5s sleep for 99% speedup. Discarded unwanted code. Signed-off-by: 3Juhwan <13selfesteem91@naver.com> Signed-off-by: Sammy Tran --- web/web_test.go | 54 ++++++++++++++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 21 deletions(-) diff --git a/web/web_test.go b/web/web_test.go index b07e26cfa8..aa63d09aef 100644 --- a/web/web_test.go +++ b/web/web_test.go @@ -118,12 +118,10 @@ func TestReadyAndHealthy(t *testing.T) { } }() - // Give some time for the web goroutine to run since we need the server - // to be up before starting tests. - time.Sleep(5 * time.Second) - baseURL := "http://localhost" + port + waitForServerReady(t, baseURL, 5*time.Second) + resp, err := http.Get(baseURL + "/-/healthy") require.NoError(t, err) require.Equal(t, http.StatusOK, resp.StatusCode) @@ -235,12 +233,10 @@ func TestRoutePrefix(t *testing.T) { } }() - // Give some time for the web goroutine to run since we need the server - // to be up before starting tests. - time.Sleep(5 * time.Second) - baseURL := "http://localhost" + port + waitForServerReady(t, baseURL+opts.RoutePrefix, 5*time.Second) + resp, err := http.Get(baseURL + opts.RoutePrefix + "/-/healthy") require.NoError(t, err) require.Equal(t, http.StatusOK, resp.StatusCode) @@ -426,9 +422,9 @@ func TestShutdownWithStaleConnection(t *testing.T) { close(closed) }() - // Give some time for the web goroutine to run since we need the server - // to be up before starting tests. - time.Sleep(5 * time.Second) + baseURL := "http://localhost" + port + + waitForServerReady(t, baseURL, 5*time.Second) // Open a socket, and don't use it. This connection should then be closed // after the ReadTimeout. @@ -477,12 +473,10 @@ func TestHandleMultipleQuitRequests(t *testing.T) { close(closed) }() - // Give some time for the web goroutine to run since we need the server - // to be up before starting tests. - time.Sleep(5 * time.Second) - baseURL := opts.ExternalURL.Scheme + "://" + opts.ExternalURL.Host + waitForServerReady(t, baseURL, 5*time.Second) + start := make(chan struct{}) var wg sync.WaitGroup for range 3 { @@ -555,11 +549,10 @@ func TestAgentAPIEndPoints(t *testing.T) { } }() - // Give some time for the web goroutine to run since we need the server - // to be up before starting tests. - time.Sleep(5 * time.Second) baseURL := "http://localhost" + port + "/api/v1" + waitForServerReady(t, "http://localhost"+port, 5*time.Second) + // Test for non-available endpoints in the Agent mode. for path, methods := range map[string][]string{ "/labels": {http.MethodGet, http.MethodPost}, @@ -688,9 +681,7 @@ func TestMultipleListenAddresses(t *testing.T) { } }() - // Give some time for the web goroutine to run since we need the server - // to be up before starting tests. - time.Sleep(5 * time.Second) + waitForServerReady(t, "http://localhost"+port1, 5*time.Second) // Set to ready. webHandler.SetReady(Ready) @@ -709,3 +700,24 @@ func TestMultipleListenAddresses(t *testing.T) { cleanupTestResponse(t, resp) } } + +// Give some time for the web goroutine to run since we need the server +// to be up before starting tests. +func waitForServerReady(t *testing.T, baseURL string, timeout time.Duration) { + t.Helper() + + interval := 100 * time.Millisecond + deadline := time.Now().Add(timeout) + + for time.Now().Before(deadline) { + resp, err := http.Get(baseURL + "/-/healthy") + if resp != nil { + cleanupTestResponse(t, resp) + } + if err == nil && resp.StatusCode == http.StatusOK { + return + } + time.Sleep(interval) + } + t.Fatalf("Server did not become ready within %v", timeout) +} From 554ea9ebfeffe0a42f58944923ac3bee9ddd812a Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Wed, 5 Nov 2025 11:57:24 +0100 Subject: [PATCH 020/439] promql: fix resets/changes to return empty for anchored selectors when samples outside range The funcResets and funcChanges functions now correctly return no result when all float samples are at or before the range start for anchored selectors, consistent with the behavior of rate/increase functions. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- promql/functions.go | 22 ++++++++++++++----- .../promqltest/testdata/extended_vectors.test | 2 -- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/promql/functions.go b/promql/functions.go index 2ff25c3acf..2602c25c1d 100644 --- a/promql/functions.go +++ b/promql/functions.go @@ -1709,15 +1709,19 @@ func funcHistogramQuantile(vectorVals []Vector, _ Matrix, args parser.Expression // pickFirstSampleIndex returns the index of the last sample before // or at the range start, or 0 if none exist before the range start. -// If the vector selector is not anchored, it always returns 0. -func pickFirstSampleIndex(floats []FPoint, args parser.Expressions, enh *EvalNodeHelper) int { +// If the vector selector is not anchored, it always returns 0, true. +// The second return value is false if there are no samples in range (for anchored selectors). +func pickFirstSampleIndex(floats []FPoint, args parser.Expressions, enh *EvalNodeHelper) (int, bool) { ms := args[0].(*parser.MatrixSelector) vs := ms.VectorSelector.(*parser.VectorSelector) if !vs.Anchored { - return 0 + return 0, true } rangeStart := enh.Ts - durationMilliseconds(ms.Range+vs.Offset) - return max(0, sort.Search(len(floats)-1, func(i int) bool { return floats[i].T > rangeStart })-1) + if len(floats) == 0 || floats[len(floats)-1].T <= rangeStart { + return 0, false + } + return max(0, sort.Search(len(floats)-1, func(i int) bool { return floats[i].T > rangeStart })-1), true } // === resets(Matrix parser.ValueTypeMatrix) (Vector, Annotations) === @@ -1730,7 +1734,10 @@ func funcResets(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *Eval } var prevSample, curSample Sample - firstSampleIndex := pickFirstSampleIndex(floats, args, enh) + firstSampleIndex, found := pickFirstSampleIndex(floats, args, enh) + if !found { + return enh.Out, nil + } for iFloat, iHistogram := firstSampleIndex, 0; iFloat < len(floats) || iHistogram < len(histograms); { switch { // Process a float sample if no histogram sample remains or its timestamp is earlier. @@ -1776,7 +1783,10 @@ func funcChanges(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *Eva } var prevSample, curSample Sample - firstSampleIndex := pickFirstSampleIndex(floats, args, enh) + firstSampleIndex, found := pickFirstSampleIndex(floats, args, enh) + if !found { + return enh.Out, nil + } for iFloat, iHistogram := firstSampleIndex, 0; iFloat < len(floats) || iHistogram < len(histograms); { switch { // Process a float sample if no histogram sample remains or its timestamp is earlier. diff --git a/promql/promqltest/testdata/extended_vectors.test b/promql/promqltest/testdata/extended_vectors.test index 8e116b1ac5..8f431dcfd3 100644 --- a/promql/promqltest/testdata/extended_vectors.test +++ b/promql/promqltest/testdata/extended_vectors.test @@ -319,7 +319,6 @@ eval instant at 2m changes(metric[1m] anchored) {id="2"} 1 eval instant at 3m changes(metric[1m] anchored) - {id="1"} 1 {id="2"} 1 eval instant at 8m changes(metric[1m] anchored) @@ -342,7 +341,6 @@ eval instant at 2m resets(metric[1m] anchored) {id="2"} 1 eval instant at 3m resets(metric[1m] anchored) - {id="1"} 1 {id="2"} 1 eval instant at 8m resets(metric[1m] anchored) From a2ba619612efe138533a6c2279397134fbdc143d Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Thu, 6 Nov 2025 20:06:56 +0100 Subject: [PATCH 021/439] ui: make update-npm-deps Signed-off-by: Jan Fajerski --- web/ui/mantine-ui/package.json | 56 +- web/ui/module/codemirror-promql/package.json | 12 +- web/ui/module/lezer-promql/package.json | 6 +- web/ui/package-lock.json | 754 ++++++++----------- web/ui/package.json | 8 +- 5 files changed, 372 insertions(+), 464 deletions(-) diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json index 90f23898e4..219d357f0d 100644 --- a/web/ui/mantine-ui/package.json +++ b/web/ui/mantine-ui/package.json @@ -12,63 +12,63 @@ "test": "vitest" }, "dependencies": { - "@codemirror/autocomplete": "^6.19.0", + "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", - "@codemirror/lint": "^6.8.5", + "@codemirror/lint": "^6.9.2", "@codemirror/state": "^6.5.2", - "@codemirror/view": "^6.38.4", + "@codemirror/view": "^6.38.6", "@floating-ui/dom": "^1.7.4", - "@lezer/common": "^1.2.3", - "@lezer/highlight": "^1.2.1", - "@mantine/code-highlight": "^8.3.5", - "@mantine/core": "^8.3.5", - "@mantine/dates": "^8.3.5", - "@mantine/hooks": "^8.3.5", - "@mantine/notifications": "^8.3.5", + "@lezer/common": "^1.3.0", + "@lezer/highlight": "^1.2.3", + "@mantine/code-highlight": "^8.3.6", + "@mantine/core": "^8.3.6", + "@mantine/dates": "^8.3.6", + "@mantine/hooks": "^8.3.6", + "@mantine/notifications": "^8.3.6", "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", "@prometheus-io/codemirror-promql": "0.307.3", - "@reduxjs/toolkit": "^2.9.0", + "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", - "@tanstack/react-query": "^5.90.2", - "@testing-library/jest-dom": "^6.9.0", + "@tanstack/react-query": "^5.90.7", + "@testing-library/jest-dom": "^6.9.1", "@testing-library/react": "^16.3.0", "@types/lodash": "^4.17.20", "@types/sanitize-html": "^2.16.0", - "@uiw/react-codemirror": "^4.25.2", + "@uiw/react-codemirror": "^4.25.3", "clsx": "^2.1.1", - "dayjs": "^1.11.18", + "dayjs": "^1.11.19", "highlight.js": "^11.11.1", "lodash": "^4.17.21", - "react": "^19.1.1", - "react-dom": "^19.1.1", + "react": "^19.2.0", + "react-dom": "^19.2.0", "react-infinite-scroll-component": "^6.1.0", "react-redux": "^9.2.0", - "react-router-dom": "^7.9.3", + "react-router-dom": "^7.9.5", "sanitize-html": "^2.17.0", "uplot": "^1.6.32", "uplot-react": "^1.2.4", "use-query-params": "^2.2.1" }, "devDependencies": { - "@eslint/compat": "^1.4.0", + "@eslint/compat": "^1.4.1", "@eslint/eslintrc": "^3.3.1", - "@eslint/js": "^9.36.0", - "@types/react": "^19.1.16", - "@types/react-dom": "^19.1.9", - "@typescript-eslint/eslint-plugin": "^8.45.0", - "@typescript-eslint/parser": "^8.45.0", + "@eslint/js": "^9.39.1", + "@types/react": "^19.2.2", + "@types/react-dom": "^19.2.2", + "@typescript-eslint/eslint-plugin": "^8.46.3", + "@typescript-eslint/parser": "^8.46.3", "@vitejs/plugin-react": "^4.7.0", - "eslint": "^9.36.0", + "eslint": "^9.39.1", "eslint-plugin-react-hooks": "^5.2.0", - "eslint-plugin-react-refresh": "^0.4.22", - "globals": "^16.4.0", + "eslint-plugin-react-refresh": "^0.4.24", + "globals": "^16.5.0", "jsdom": "^25.0.1", "postcss": "^8.5.6", "postcss-preset-mantine": "^1.18.0", "postcss-simple-vars": "^7.0.1", - "vite": "^6.3.6", + "vite": "^6.4.1", "vitest": "^3.2.4" } } diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json index f40dc65432..f850342728 100644 --- a/web/ui/module/codemirror-promql/package.json +++ b/web/ui/module/codemirror-promql/package.json @@ -33,14 +33,14 @@ "lru-cache": "^11.2.2" }, "devDependencies": { - "@codemirror/autocomplete": "^6.19.0", + "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", - "@codemirror/lint": "^6.8.5", + "@codemirror/lint": "^6.9.2", "@codemirror/state": "^6.5.2", - "@codemirror/view": "^6.38.4", - "@lezer/common": "^1.2.3", - "@lezer/highlight": "^1.2.1", - "@lezer/lr": "^1.4.2", + "@codemirror/view": "^6.38.6", + "@lezer/common": "^1.3.0", + "@lezer/highlight": "^1.2.3", + "@lezer/lr": "^1.4.3", "eslint-plugin-prettier": "^5.5.4", "isomorphic-fetch": "^3.0.0", "nock": "^14.0.10" diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json index 5e1ab771f1..05511c2b89 100644 --- a/web/ui/module/lezer-promql/package.json +++ b/web/ui/module/lezer-promql/package.json @@ -32,9 +32,9 @@ }, "devDependencies": { "@lezer/generator": "^1.8.0", - "@lezer/highlight": "^1.2.1", - "@lezer/lr": "^1.4.2", - "@rollup/plugin-node-resolve": "^16.0.1" + "@lezer/highlight": "^1.2.3", + "@lezer/lr": "^1.4.3", + "@rollup/plugin-node-resolve": "^16.0.3" }, "peerDependencies": { "@lezer/highlight": "^1.1.2", diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json index 144a5c76d5..2631802e53 100644 --- a/web/ui/package-lock.json +++ b/web/ui/package-lock.json @@ -13,178 +13,79 @@ ], "devDependencies": { "@types/jest": "^29.5.14", - "@typescript-eslint/eslint-plugin": "^8.45.0", - "@typescript-eslint/parser": "^8.45.0", + "@typescript-eslint/eslint-plugin": "^8.46.3", + "@typescript-eslint/parser": "^8.46.3", "eslint-config-prettier": "^10.1.8", "prettier": "^3.6.2", - "ts-jest": "^29.4.4", + "ts-jest": "^29.4.5", "typescript": "^5.9.3", - "vite": "^6.3.6" + "vite": "^6.4.1" } }, "mantine-ui": { "name": "@prometheus-io/mantine-ui", "version": "0.307.3", "dependencies": { - "@codemirror/autocomplete": "^6.19.0", + "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", - "@codemirror/lint": "^6.8.5", + "@codemirror/lint": "^6.9.2", "@codemirror/state": "^6.5.2", - "@codemirror/view": "^6.38.4", + "@codemirror/view": "^6.38.6", "@floating-ui/dom": "^1.7.4", - "@lezer/common": "^1.2.3", - "@lezer/highlight": "^1.2.1", - "@mantine/code-highlight": "^8.3.5", - "@mantine/core": "^8.3.5", - "@mantine/dates": "^8.3.5", - "@mantine/hooks": "^8.3.5", - "@mantine/notifications": "^8.3.5", + "@lezer/common": "^1.3.0", + "@lezer/highlight": "^1.2.3", + "@mantine/code-highlight": "^8.3.6", + "@mantine/core": "^8.3.6", + "@mantine/dates": "^8.3.6", + "@mantine/hooks": "^8.3.6", + "@mantine/notifications": "^8.3.6", "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", "@prometheus-io/codemirror-promql": "0.307.3", - "@reduxjs/toolkit": "^2.9.0", + "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", - "@tanstack/react-query": "^5.90.2", - "@testing-library/jest-dom": "^6.9.0", + "@tanstack/react-query": "^5.90.7", + "@testing-library/jest-dom": "^6.9.1", "@testing-library/react": "^16.3.0", "@types/lodash": "^4.17.20", "@types/sanitize-html": "^2.16.0", - "@uiw/react-codemirror": "^4.25.2", + "@uiw/react-codemirror": "^4.25.3", "clsx": "^2.1.1", - "dayjs": "^1.11.18", + "dayjs": "^1.11.19", "highlight.js": "^11.11.1", "lodash": "^4.17.21", - "react": "^19.1.1", - "react-dom": "^19.1.1", + "react": "^19.2.0", + "react-dom": "^19.2.0", "react-infinite-scroll-component": "^6.1.0", "react-redux": "^9.2.0", - "react-router-dom": "^7.9.3", + "react-router-dom": "^7.9.5", "sanitize-html": "^2.17.0", "uplot": "^1.6.32", "uplot-react": "^1.2.4", "use-query-params": "^2.2.1" }, "devDependencies": { - "@eslint/compat": "^1.4.0", + "@eslint/compat": "^1.4.1", "@eslint/eslintrc": "^3.3.1", - "@eslint/js": "^9.36.0", - "@types/react": "^19.1.16", - "@types/react-dom": "^19.1.9", - "@typescript-eslint/eslint-plugin": "^8.45.0", - "@typescript-eslint/parser": "^8.45.0", + "@eslint/js": "^9.39.1", + "@types/react": "^19.2.2", + "@types/react-dom": "^19.2.2", + "@typescript-eslint/eslint-plugin": "^8.46.3", + "@typescript-eslint/parser": "^8.46.3", "@vitejs/plugin-react": "^4.7.0", - "eslint": "^9.36.0", + "eslint": "^9.39.1", "eslint-plugin-react-hooks": "^5.2.0", - "eslint-plugin-react-refresh": "^0.4.22", - "globals": "^16.4.0", + "eslint-plugin-react-refresh": "^0.4.24", + "globals": "^16.5.0", "jsdom": "^25.0.1", "postcss": "^8.5.6", "postcss-preset-mantine": "^1.18.0", "postcss-simple-vars": "^7.0.1", - "vite": "^6.3.6", + "vite": "^6.4.1", "vitest": "^3.2.4" } }, - "mantine-ui/node_modules/@floating-ui/react": { - "version": "0.27.16", - "resolved": "https://registry.npmjs.org/@floating-ui/react/-/react-0.27.16.tgz", - "integrity": "sha512-9O8N4SeG2z++TSM8QA/KTeKFBVCNEz/AGS7gWPJf6KFRzmRWixFRnCnkPHRDwSVZW6QPDO6uT0P2SpWNKCc9/g==", - "license": "MIT", - "dependencies": { - "@floating-ui/react-dom": "^2.1.6", - "@floating-ui/utils": "^0.2.10", - "tabbable": "^6.0.0" - }, - "peerDependencies": { - "react": ">=17.0.0", - "react-dom": ">=17.0.0" - } - }, - "mantine-ui/node_modules/@mantine/code-highlight": { - "version": "8.3.5", - "resolved": "https://registry.npmjs.org/@mantine/code-highlight/-/code-highlight-8.3.5.tgz", - "integrity": "sha512-KZhYPilo6hUbJf/ls0e/lCjXKWI6/cnzkMkSRLAEVLD9HrhJKIBonwQtLRFgJbLmxKu9IE9KuJjq9lA5kUJCFQ==", - "license": "MIT", - "dependencies": { - "clsx": "^2.1.1" - }, - "peerDependencies": { - "@mantine/core": "8.3.5", - "@mantine/hooks": "8.3.5", - "react": "^18.x || ^19.x", - "react-dom": "^18.x || ^19.x" - } - }, - "mantine-ui/node_modules/@mantine/core": { - "version": "8.3.5", - "resolved": "https://registry.npmjs.org/@mantine/core/-/core-8.3.5.tgz", - "integrity": "sha512-PdVNLMgOS2vFhOujRi6/VC9ic8w3UDyKX7ftwDeJ7yQT8CiepUxfbWWYpVpnq23bdWh/7fIT2Pn1EY8r8GOk7g==", - "license": "MIT", - "dependencies": { - "@floating-ui/react": "^0.27.16", - "clsx": "^2.1.1", - "react-number-format": "^5.4.4", - "react-remove-scroll": "^2.7.1", - "react-textarea-autosize": "8.5.9", - "type-fest": "^4.41.0" - }, - "peerDependencies": { - "@mantine/hooks": "8.3.5", - "react": "^18.x || ^19.x", - "react-dom": "^18.x || ^19.x" - } - }, - "mantine-ui/node_modules/@mantine/dates": { - "version": "8.3.5", - "resolved": "https://registry.npmjs.org/@mantine/dates/-/dates-8.3.5.tgz", - "integrity": "sha512-LkIdC4eWPNQFv1BU1c52U3Z3RuA3yU1asvTgMEIQ/MdJsGK8GePwpgMH/jKQ8ba/AW9NfksdvtOJ6uIqPwjCkg==", - "license": "MIT", - "dependencies": { - "clsx": "^2.1.1" - }, - "peerDependencies": { - "@mantine/core": "8.3.5", - "@mantine/hooks": "8.3.5", - "dayjs": ">=1.0.0", - "react": "^18.x || ^19.x", - "react-dom": "^18.x || ^19.x" - } - }, - "mantine-ui/node_modules/@mantine/hooks": { - "version": "8.3.5", - "resolved": "https://registry.npmjs.org/@mantine/hooks/-/hooks-8.3.5.tgz", - "integrity": "sha512-0Wf08eWLKi3WkKlxnV1W5vfuN6wcvAV2VbhQlOy0R9nrWorGTtonQF6qqBE3PnJFYF1/ZE+HkYZQ/Dr7DmYSMQ==", - "license": "MIT", - "peerDependencies": { - "react": "^18.x || ^19.x" - } - }, - "mantine-ui/node_modules/@mantine/notifications": { - "version": "8.3.5", - "resolved": "https://registry.npmjs.org/@mantine/notifications/-/notifications-8.3.5.tgz", - "integrity": "sha512-8TvzrPxfdtOLGTalv7Ei1hy2F6KbR3P7/V73yw3AOKhrf1ydS89sqV2ShbsucHGJk9Pto0wjdTPd8Q7pm5MAYw==", - "license": "MIT", - "dependencies": { - "@mantine/store": "8.3.5", - "react-transition-group": "4.4.5" - }, - "peerDependencies": { - "@mantine/core": "8.3.5", - "@mantine/hooks": "8.3.5", - "react": "^18.x || ^19.x", - "react-dom": "^18.x || ^19.x" - } - }, - "mantine-ui/node_modules/@mantine/store": { - "version": "8.3.5", - "resolved": "https://registry.npmjs.org/@mantine/store/-/store-8.3.5.tgz", - "integrity": "sha512-qN4fFsDMy86IV9oh1gZlDTv41RAsO0grjx90FGyT5QCv7NTgcavwxB74GBkhp45W8xn+Ms/awKy+6NxnmLmW1w==", - "license": "MIT", - "peerDependencies": { - "react": "^18.x || ^19.x" - } - }, "module/codemirror-promql": { "name": "@prometheus-io/codemirror-promql", "version": "0.307.3", @@ -194,14 +95,14 @@ "lru-cache": "^11.2.2" }, "devDependencies": { - "@codemirror/autocomplete": "^6.19.0", + "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", - "@codemirror/lint": "^6.8.5", + "@codemirror/lint": "^6.9.2", "@codemirror/state": "^6.5.2", - "@codemirror/view": "^6.38.4", - "@lezer/common": "^1.2.3", - "@lezer/highlight": "^1.2.1", - "@lezer/lr": "^1.4.2", + "@codemirror/view": "^6.38.6", + "@lezer/common": "^1.3.0", + "@lezer/highlight": "^1.2.3", + "@lezer/lr": "^1.4.3", "eslint-plugin-prettier": "^5.5.4", "isomorphic-fetch": "^3.0.0", "nock": "^14.0.10" @@ -224,9 +125,9 @@ "license": "Apache-2.0", "devDependencies": { "@lezer/generator": "^1.8.0", - "@lezer/highlight": "^1.2.1", - "@lezer/lr": "^1.4.2", - "@rollup/plugin-node-resolve": "^16.0.1" + "@lezer/highlight": "^1.2.3", + "@lezer/lr": "^1.4.3", + "@rollup/plugin-node-resolve": "^16.0.3" }, "peerDependencies": { "@lezer/highlight": "^1.1.2", @@ -826,10 +727,9 @@ "peer": true }, "node_modules/@codemirror/autocomplete": { - "version": "6.19.0", - "resolved": "https://registry.npmjs.org/@codemirror/autocomplete/-/autocomplete-6.19.0.tgz", - "integrity": "sha512-61Hfv3cF07XvUxNeC3E7jhG8XNi1Yom1G0lRC936oLnlF+jrbrv8rc/J98XlYzcsAoTVupfsf5fLej1aI8kyIg==", - "license": "MIT", + "version": "6.19.1", + "resolved": "https://registry.npmjs.org/@codemirror/autocomplete/-/autocomplete-6.19.1.tgz", + "integrity": "sha512-q6NenYkEy2fn9+JyjIxMWcNjzTL/IhwqfzOut1/G3PrIFkrbl4AL7Wkse5tLrQUUyqGoAKU5+Pi5jnnXxH5HGw==", "dependencies": { "@codemirror/language": "^6.0.0", "@codemirror/state": "^6.0.0", @@ -864,10 +764,9 @@ } }, "node_modules/@codemirror/lint": { - "version": "6.8.5", - "resolved": "https://registry.npmjs.org/@codemirror/lint/-/lint-6.8.5.tgz", - "integrity": "sha512-s3n3KisH7dx3vsoeGMxsbRAgKe4O1vbrnKBClm99PU0fWxmxsx5rR2PfqQgIt+2MMJBHbiJ5rfIdLYfB9NNvsA==", - "license": "MIT", + "version": "6.9.2", + "resolved": "https://registry.npmjs.org/@codemirror/lint/-/lint-6.9.2.tgz", + "integrity": "sha512-sv3DylBiIyi+xKwRCJAAsBZZZWo82shJ/RTMymLabAdtbkV5cSKwWDeCgtUq3v8flTaXS2y1kKkICuRYtUswyQ==", "dependencies": { "@codemirror/state": "^6.0.0", "@codemirror/view": "^6.35.0", @@ -907,10 +806,9 @@ } }, "node_modules/@codemirror/view": { - "version": "6.38.4", - "resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.38.4.tgz", - "integrity": "sha512-hduz0suCcUSC/kM8Fq3A9iLwInJDl8fD1xLpTIk+5xkNm8z/FT7UsIa9sOXrkpChh+XXc18RzswE8QqELsVl+g==", - "license": "MIT", + "version": "6.38.6", + "resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.38.6.tgz", + "integrity": "sha512-qiS0z1bKs5WOvHIAC0Cybmv4AJSkAXgX5aD6Mqd2epSLlVJsQl8NG23jCVouIgkh4All/mrbdsf2UOLFnJw0tw==", "dependencies": { "@codemirror/state": "^6.5.0", "crelt": "^1.0.6", @@ -1386,13 +1284,12 @@ } }, "node_modules/@eslint/compat": { - "version": "1.4.0", - "resolved": "https://registry.npmjs.org/@eslint/compat/-/compat-1.4.0.tgz", - "integrity": "sha512-DEzm5dKeDBPm3r08Ixli/0cmxr8LkRdwxMRUIJBlSCpAwSrvFEJpVBzV+66JhDxiaqKxnRzCXhtiMiczF7Hglg==", + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/@eslint/compat/-/compat-1.4.1.tgz", + "integrity": "sha512-cfO82V9zxxGBxcQDr1lfaYB7wykTa0b00mGa36FrJl7iTFd0Z2cHfEYuxcBRP/iNijCsWsEkA+jzT8hGYmv33w==", "dev": true, - "license": "Apache-2.0", "dependencies": { - "@eslint/core": "^0.16.0" + "@eslint/core": "^0.17.0" }, "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" @@ -1407,13 +1304,12 @@ } }, "node_modules/@eslint/config-array": { - "version": "0.21.0", - "resolved": "https://registry.npmjs.org/@eslint/config-array/-/config-array-0.21.0.tgz", - "integrity": "sha512-ENIdc4iLu0d93HeYirvKmrzshzofPw6VkZRKQGe9Nv46ZnWUzcF1xV01dcvEg/1wXUR61OmmlSfyeyO7EvjLxQ==", + "version": "0.21.1", + "resolved": "https://registry.npmjs.org/@eslint/config-array/-/config-array-0.21.1.tgz", + "integrity": "sha512-aw1gNayWpdI/jSYVgzN5pL0cfzU02GT3NBpeT/DXbx1/1x7ZKxFPd9bwrzygx/qiwIQiJ1sw/zD8qY/kRvlGHA==", "dev": true, - "license": "Apache-2.0", "dependencies": { - "@eslint/object-schema": "^2.1.6", + "@eslint/object-schema": "^2.1.7", "debug": "^4.3.1", "minimatch": "^3.1.2" }, @@ -1422,21 +1318,22 @@ } }, "node_modules/@eslint/config-helpers": { - "version": "0.3.1", - "resolved": "https://registry.npmjs.org/@eslint/config-helpers/-/config-helpers-0.3.1.tgz", - "integrity": "sha512-xR93k9WhrDYpXHORXpxVL5oHj3Era7wo6k/Wd8/IsQNnZUTzkGS29lyn3nAT05v6ltUuTFVCCYDEGfy2Or/sPA==", + "version": "0.4.2", + "resolved": "https://registry.npmjs.org/@eslint/config-helpers/-/config-helpers-0.4.2.tgz", + "integrity": "sha512-gBrxN88gOIf3R7ja5K9slwNayVcZgK6SOUORm2uBzTeIEfeVaIhOpCtTox3P6R7o2jLFwLFTLnC7kU/RGcYEgw==", "dev": true, - "license": "Apache-2.0", + "dependencies": { + "@eslint/core": "^0.17.0" + }, "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" } }, "node_modules/@eslint/core": { - "version": "0.16.0", - "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.16.0.tgz", - "integrity": "sha512-nmC8/totwobIiFcGkDza3GIKfAw1+hLiYVrh3I1nIomQ8PEr5cxg34jnkmGawul/ep52wGRAcyeDCNtWKSOj4Q==", + "version": "0.17.0", + "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.17.0.tgz", + "integrity": "sha512-yL/sLrpmtDaFEiUj1osRP4TI2MDz1AddJL+jZ7KSqvBuliN4xqYY54IfdN8qD8Toa6g1iloph1fxQNkjOxrrpQ==", "dev": true, - "license": "Apache-2.0", "dependencies": { "@types/json-schema": "^7.0.15" }, @@ -1481,11 +1378,10 @@ } }, "node_modules/@eslint/js": { - "version": "9.36.0", - "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.36.0.tgz", - "integrity": "sha512-uhCbYtYynH30iZErszX78U+nR3pJU3RHGQ57NXy5QupD4SBVwDeU8TNBy+MjMngc1UyIW9noKqsRqfjQTBU2dw==", + "version": "9.39.1", + "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.39.1.tgz", + "integrity": "sha512-S26Stp4zCy88tH94QbBv3XCuzRQiZ9yXofEILmglYTh/Ug/a9/umqvgFtYBAo3Lp0nsI/5/qH1CCrbdK3AP1Tw==", "dev": true, - "license": "MIT", "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" }, @@ -1494,42 +1390,27 @@ } }, "node_modules/@eslint/object-schema": { - "version": "2.1.6", - "resolved": "https://registry.npmjs.org/@eslint/object-schema/-/object-schema-2.1.6.tgz", - "integrity": "sha512-RBMg5FRL0I0gs51M/guSAj5/e14VQ4tpZnQNWwuDT66P14I43ItmPfIZRhO9fUVIPOAQXU47atlywZ/czoqFPA==", + "version": "2.1.7", + "resolved": "https://registry.npmjs.org/@eslint/object-schema/-/object-schema-2.1.7.tgz", + "integrity": "sha512-VtAOaymWVfZcmZbp6E2mympDIHvyjXs/12LqWYjVw6qjrfF+VK+fyG33kChz3nnK+SU5/NeHOqrTEHS8sXO3OA==", "dev": true, - "license": "Apache-2.0", "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" } }, "node_modules/@eslint/plugin-kit": { - "version": "0.3.5", - "resolved": "https://registry.npmjs.org/@eslint/plugin-kit/-/plugin-kit-0.3.5.tgz", - "integrity": "sha512-Z5kJ+wU3oA7MMIqVR9tyZRtjYPr4OC004Q4Rw7pgOKUOKkJfZ3O24nz3WYfGRpMDNmcOi3TwQOmgm7B7Tpii0w==", + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/@eslint/plugin-kit/-/plugin-kit-0.4.1.tgz", + "integrity": "sha512-43/qtrDUokr7LJqoF2c3+RInu/t4zfrpYdoSDfYyhg52rwLV6TnOvdG4fXm7IkSB3wErkcmJS9iEhjVtOSEjjA==", "dev": true, - "license": "Apache-2.0", "dependencies": { - "@eslint/core": "^0.15.2", + "@eslint/core": "^0.17.0", "levn": "^0.4.1" }, "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" } }, - "node_modules/@eslint/plugin-kit/node_modules/@eslint/core": { - "version": "0.15.2", - "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.15.2.tgz", - "integrity": "sha512-78Md3/Rrxh83gCxoUc0EiciuOHsIITzLy53m3d9UyiW8y9Dj2D29FeETqyKA+BRK76tnTp6RXWb3pCay8Oyomg==", - "dev": true, - "license": "Apache-2.0", - "dependencies": { - "@types/json-schema": "^7.0.15" - }, - "engines": { - "node": "^18.18.0 || ^20.9.0 || >=21.1.0" - } - }, "node_modules/@floating-ui/core": { "version": "1.7.3", "resolved": "https://registry.npmjs.org/@floating-ui/core/-/core-1.7.3.tgz", @@ -1549,11 +1430,24 @@ "@floating-ui/utils": "^0.2.10" } }, + "node_modules/@floating-ui/react": { + "version": "0.27.16", + "resolved": "https://registry.npmjs.org/@floating-ui/react/-/react-0.27.16.tgz", + "integrity": "sha512-9O8N4SeG2z++TSM8QA/KTeKFBVCNEz/AGS7gWPJf6KFRzmRWixFRnCnkPHRDwSVZW6QPDO6uT0P2SpWNKCc9/g==", + "dependencies": { + "@floating-ui/react-dom": "^2.1.6", + "@floating-ui/utils": "^0.2.10", + "tabbable": "^6.0.0" + }, + "peerDependencies": { + "react": ">=17.0.0", + "react-dom": ">=17.0.0" + } + }, "node_modules/@floating-ui/react-dom": { "version": "2.1.6", "resolved": "https://registry.npmjs.org/@floating-ui/react-dom/-/react-dom-2.1.6.tgz", "integrity": "sha512-4JX6rEatQEvlmgU80wZyq9RT96HZJa88q8hp0pBd+LrczeDI4o6uA2M+uvxngVHo4Ihr8uibXxH6+70zhAFrVw==", - "license": "MIT", "dependencies": { "@floating-ui/dom": "^1.7.4" }, @@ -2149,9 +2043,9 @@ } }, "node_modules/@lezer/common": { - "version": "1.2.3", - "resolved": "https://registry.npmjs.org/@lezer/common/-/common-1.2.3.tgz", - "integrity": "sha512-w7ojc8ejBqr2REPsWxJjrMFsA/ysDCFICn8zEOR9mrqzOu2amhITYuLD8ag6XZf0CFXDrhKqw7+tW8cX66NaDA==" + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@lezer/common/-/common-1.3.0.tgz", + "integrity": "sha512-L9X8uHCYU310o99L3/MpJKYxPzXPOS7S0NmBaM7UO/x2Kb2WbmMLSkfvdr1KxRIFYOpbY0Jhn7CfLSUDzL8arQ==" }, "node_modules/@lezer/generator": { "version": "1.8.0", @@ -2168,21 +2062,97 @@ } }, "node_modules/@lezer/highlight": { - "version": "1.2.1", - "resolved": "https://registry.npmjs.org/@lezer/highlight/-/highlight-1.2.1.tgz", - "integrity": "sha512-Z5duk4RN/3zuVO7Jq0pGLJ3qynpxUVsh7IbUbGj88+uV2ApSAn6kWg2au3iJb+0Zi7kKtqffIESgNcRXWZWmSA==", - "license": "MIT", + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/@lezer/highlight/-/highlight-1.2.3.tgz", + "integrity": "sha512-qXdH7UqTvGfdVBINrgKhDsVTJTxactNNxLk7+UMwZhU13lMHaOBlJe9Vqp907ya56Y3+ed2tlqzys7jDkTmW0g==", + "dependencies": { + "@lezer/common": "^1.3.0" + } + }, + "node_modules/@lezer/lr": { + "version": "1.4.3", + "resolved": "https://registry.npmjs.org/@lezer/lr/-/lr-1.4.3.tgz", + "integrity": "sha512-yenN5SqAxAPv/qMnpWW0AT7l+SxVrgG+u0tNsRQWqbrz66HIl8DnEbBObvy21J5K7+I1v7gsAnlE2VQ5yYVSeA==", "dependencies": { "@lezer/common": "^1.0.0" } }, - "node_modules/@lezer/lr": { - "version": "1.4.2", - "resolved": "https://registry.npmjs.org/@lezer/lr/-/lr-1.4.2.tgz", - "integrity": "sha512-pu0K1jCIdnQ12aWNaAVU5bzi7Bd1w54J3ECgANPmYLtQKP0HBj2cE/5coBD66MT10xbtIuUr7tg0Shbsvk0mDA==", - "license": "MIT", + "node_modules/@mantine/code-highlight": { + "version": "8.3.6", + "resolved": "https://registry.npmjs.org/@mantine/code-highlight/-/code-highlight-8.3.6.tgz", + "integrity": "sha512-9jPrhchbfNCA73V3hMjXVcCBYL82/UOA9LiEs5LSwxr1q4JYBEBU8znMmVuxZlXA234Ci234AqxGNXdu9f+p4w==", "dependencies": { - "@lezer/common": "^1.0.0" + "clsx": "^2.1.1" + }, + "peerDependencies": { + "@mantine/core": "8.3.6", + "@mantine/hooks": "8.3.6", + "react": "^18.x || ^19.x", + "react-dom": "^18.x || ^19.x" + } + }, + "node_modules/@mantine/core": { + "version": "8.3.6", + "resolved": "https://registry.npmjs.org/@mantine/core/-/core-8.3.6.tgz", + "integrity": "sha512-paTl+0x+O/QtgMtqVJaG8maD8sfiOdgPmLOyG485FmeGZ1L3KMdEkhxZtmdGlDFsLXhmMGQ57ducT90bvhXX5A==", + "dependencies": { + "@floating-ui/react": "^0.27.16", + "clsx": "^2.1.1", + "react-number-format": "^5.4.4", + "react-remove-scroll": "^2.7.1", + "react-textarea-autosize": "8.5.9", + "type-fest": "^4.41.0" + }, + "peerDependencies": { + "@mantine/hooks": "8.3.6", + "react": "^18.x || ^19.x", + "react-dom": "^18.x || ^19.x" + } + }, + "node_modules/@mantine/dates": { + "version": "8.3.6", + "resolved": "https://registry.npmjs.org/@mantine/dates/-/dates-8.3.6.tgz", + "integrity": "sha512-lSi1zvyL86SKeePH0J3vOjAR7ZIVNOrZm6ja7jAH6IBdcpQOKH8TXbrcAi5okEStvmvkne7pVaGu0VkdE8KnAw==", + "dependencies": { + "clsx": "^2.1.1" + }, + "peerDependencies": { + "@mantine/core": "8.3.6", + "@mantine/hooks": "8.3.6", + "dayjs": ">=1.0.0", + "react": "^18.x || ^19.x", + "react-dom": "^18.x || ^19.x" + } + }, + "node_modules/@mantine/hooks": { + "version": "8.3.6", + "resolved": "https://registry.npmjs.org/@mantine/hooks/-/hooks-8.3.6.tgz", + "integrity": "sha512-liHfaWXHAkLjJy+Bkr29UsCwAoDQ/a64WrM67lksx8F0qqyjR5RQH8zVlhuOjdpQnwtlUkE/YiTvbJiPcoI0bw==", + "peerDependencies": { + "react": "^18.x || ^19.x" + } + }, + "node_modules/@mantine/notifications": { + "version": "8.3.6", + "resolved": "https://registry.npmjs.org/@mantine/notifications/-/notifications-8.3.6.tgz", + "integrity": "sha512-d3A96lyrFOVXtrwASEXALfzooKnnA60T2LclMXFF/4k27Ay5Hwza4D+ylqgxf0RkPfF9J6LhBXk72OjL5RH5Kg==", + "dependencies": { + "@mantine/store": "8.3.6", + "react-transition-group": "4.4.5" + }, + "peerDependencies": { + "@mantine/core": "8.3.6", + "@mantine/hooks": "8.3.6", + "react": "^18.x || ^19.x", + "react-dom": "^18.x || ^19.x" + } + }, + "node_modules/@mantine/store": { + "version": "8.3.6", + "resolved": "https://registry.npmjs.org/@mantine/store/-/store-8.3.6.tgz", + "integrity": "sha512-fo86wF6nL8RPukY8cseAFQKk+bRVv3Ga/WmHJMYRsCbNleZOEZMXXUf/OVhmr1D3t+xzCzAlJe/sQ8MIS+c+pA==", + "peerDependencies": { + "react": "^18.x || ^19.x" } }, "node_modules/@marijn/find-cluster-break": { @@ -2234,7 +2204,6 @@ "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", "integrity": "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g==", "dev": true, - "license": "MIT", "dependencies": { "@nodelib/fs.stat": "2.0.5", "run-parallel": "^1.1.9" @@ -2248,7 +2217,6 @@ "resolved": "https://registry.npmjs.org/@nodelib/fs.stat/-/fs.stat-2.0.5.tgz", "integrity": "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A==", "dev": true, - "license": "MIT", "engines": { "node": ">= 8" } @@ -2258,7 +2226,6 @@ "resolved": "https://registry.npmjs.org/@nodelib/fs.walk/-/fs.walk-1.2.8.tgz", "integrity": "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg==", "dev": true, - "license": "MIT", "dependencies": { "@nodelib/fs.scandir": "2.1.5", "fastq": "^1.6.0" @@ -2318,14 +2285,13 @@ "link": true }, "node_modules/@reduxjs/toolkit": { - "version": "2.9.0", - "resolved": "https://registry.npmjs.org/@reduxjs/toolkit/-/toolkit-2.9.0.tgz", - "integrity": "sha512-fSfQlSRu9Z5yBkvsNhYF2rPS8cGXn/TZVrlwN1948QyZ8xMZ0JvP50S2acZNaf+o63u6aEeMjipFyksjIcWrog==", - "license": "MIT", + "version": "2.10.1", + "resolved": "https://registry.npmjs.org/@reduxjs/toolkit/-/toolkit-2.10.1.tgz", + "integrity": "sha512-/U17EXQ9Do9Yx4DlNGU6eVNfZvFJfYpUtRRdLf19PbPjdWBxNlxGZXywQZ1p1Nz8nMkWplTI7iD/23m07nolDA==", "dependencies": { "@standard-schema/spec": "^1.0.0", "@standard-schema/utils": "^0.3.0", - "immer": "^10.0.3", + "immer": "^10.2.0", "redux": "^5.0.1", "redux-thunk": "^3.1.0", "reselect": "^5.1.0" @@ -2351,11 +2317,10 @@ "license": "MIT" }, "node_modules/@rollup/plugin-node-resolve": { - "version": "16.0.1", - "resolved": "https://registry.npmjs.org/@rollup/plugin-node-resolve/-/plugin-node-resolve-16.0.1.tgz", - "integrity": "sha512-tk5YCxJWIG81umIvNkSod2qK5KyQW19qcBF/B78n1bjtOON6gzKoVeSzAE8yHCZEDmqkHKkxplExA8KzdJLJpA==", + "version": "16.0.3", + "resolved": "https://registry.npmjs.org/@rollup/plugin-node-resolve/-/plugin-node-resolve-16.0.3.tgz", + "integrity": "sha512-lUYM3UBGuM93CnMPG1YocWu7X802BrNF3jW2zny5gQyLQgRFJhV1Sq0Zi74+dh/6NBx1DxFC4b4GXg9wUCG5Qg==", "dev": true, - "license": "MIT", "dependencies": { "@rollup/pluginutils": "^5.0.1", "@types/resolve": "1.20.2", @@ -2739,22 +2704,20 @@ } }, "node_modules/@tanstack/query-core": { - "version": "5.90.2", - "resolved": "https://registry.npmjs.org/@tanstack/query-core/-/query-core-5.90.2.tgz", - "integrity": "sha512-k/TcR3YalnzibscALLwxeiLUub6jN5EDLwKDiO7q5f4ICEoptJ+n9+7vcEFy5/x/i6Q+Lb/tXrsKCggf5uQJXQ==", - "license": "MIT", + "version": "5.90.7", + "resolved": "https://registry.npmjs.org/@tanstack/query-core/-/query-core-5.90.7.tgz", + "integrity": "sha512-6PN65csiuTNfBMXqQUxQhCNdtm1rV+9kC9YwWAIKcaxAauq3Wu7p18j3gQY3YIBJU70jT/wzCCZ2uqto/vQgiQ==", "funding": { "type": "github", "url": "https://github.com/sponsors/tannerlinsley" } }, "node_modules/@tanstack/react-query": { - "version": "5.90.2", - "resolved": "https://registry.npmjs.org/@tanstack/react-query/-/react-query-5.90.2.tgz", - "integrity": "sha512-CLABiR+h5PYfOWr/z+vWFt5VsOA2ekQeRQBFSKlcoW6Ndx/f8rfyVmq4LbgOM4GG2qtxAxjLYLOpCNTYm4uKzw==", - "license": "MIT", + "version": "5.90.7", + "resolved": "https://registry.npmjs.org/@tanstack/react-query/-/react-query-5.90.7.tgz", + "integrity": "sha512-wAHc/cgKzW7LZNFloThyHnV/AX9gTg3w5yAv0gvQHPZoCnepwqCMtzbuPbb2UvfvO32XZ46e8bPOYbfZhzVnnQ==", "dependencies": { - "@tanstack/query-core": "5.90.2" + "@tanstack/query-core": "5.90.7" }, "funding": { "type": "github", @@ -2785,10 +2748,9 @@ } }, "node_modules/@testing-library/jest-dom": { - "version": "6.9.0", - "resolved": "https://registry.npmjs.org/@testing-library/jest-dom/-/jest-dom-6.9.0.tgz", - "integrity": "sha512-QHdxYMJ0YPGKYofMc6zYvo7LOViVhdc6nPg/OtM2cf9MQrwEcTxFCs7d/GJ5eSyPkHzOiBkc/KfLdFJBHzldtQ==", - "license": "MIT", + "version": "6.9.1", + "resolved": "https://registry.npmjs.org/@testing-library/jest-dom/-/jest-dom-6.9.1.tgz", + "integrity": "sha512-zIcONa+hVtVSSep9UT3jZ5rizo2BsxgyDYU7WFD5eICBE7no3881HGeb/QkGfsJs6JTkY1aQhT7rIPC7e+0nnA==", "dependencies": { "@adobe/css-tools": "^4.4.0", "aria-query": "^5.0.0", @@ -2998,8 +2960,7 @@ "version": "7.0.15", "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.15.tgz", "integrity": "sha512-5+fP8P8MFNC+AyZCDxrB2pkZFPGzqQWUzpSeuuVLvm8VMcorNYavBqoFcxK8bQz4Qsbn4oUEEem4wDLfcysGHA==", - "dev": true, - "license": "MIT" + "dev": true }, "node_modules/@types/lodash": { "version": "4.17.20", @@ -3017,23 +2978,21 @@ } }, "node_modules/@types/react": { - "version": "19.1.16", - "resolved": "https://registry.npmjs.org/@types/react/-/react-19.1.16.tgz", - "integrity": "sha512-WBM/nDbEZmDUORKnh5i1bTnAz6vTohUf9b8esSMu+b24+srbaxa04UbJgWx78CVfNXA20sNu0odEIluZDFdCog==", + "version": "19.2.2", + "resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.2.tgz", + "integrity": "sha512-6mDvHUFSjyT2B2yeNx2nUgMxh9LtOWvkhIU3uePn2I2oyNymUAX1NIsdgviM4CH+JSrp2D2hsMvJOkxY+0wNRA==", "devOptional": true, - "license": "MIT", "dependencies": { "csstype": "^3.0.2" } }, "node_modules/@types/react-dom": { - "version": "19.1.9", - "resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.1.9.tgz", - "integrity": "sha512-qXRuZaOsAdXKFyOhRBg6Lqqc0yay13vN7KrIg4L7N4aaHN68ma9OK3NE1BoDFgFOTfM7zg+3/8+2n8rLUH3OKQ==", + "version": "19.2.2", + "resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.2.2.tgz", + "integrity": "sha512-9KQPoO6mZCi7jcIStSnlOWn2nEF3mNmyr3rIAsGnAbQKYbRLyqmeSc39EVgtxXVia+LMT8j3knZLAZAh+xLmrw==", "devOptional": true, - "license": "MIT", "peerDependencies": { - "@types/react": "^19.0.0" + "@types/react": "^19.2.0" } }, "node_modules/@types/resolve": { @@ -3083,17 +3042,16 @@ "license": "MIT" }, "node_modules/@typescript-eslint/eslint-plugin": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.45.0.tgz", - "integrity": "sha512-HC3y9CVuevvWCl/oyZuI47dOeDF9ztdMEfMH8/DW/Mhwa9cCLnK1oD7JoTVGW/u7kFzNZUKUoyJEqkaJh5y3Wg==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.46.3.tgz", + "integrity": "sha512-sbaQ27XBUopBkRiuY/P9sWGOWUW4rl8fDoHIUmLpZd8uldsTyB4/Zg6bWTegPoTLnKj9Hqgn3QD6cjPNB32Odw==", "dev": true, - "license": "MIT", "dependencies": { "@eslint-community/regexpp": "^4.10.0", - "@typescript-eslint/scope-manager": "8.45.0", - "@typescript-eslint/type-utils": "8.45.0", - "@typescript-eslint/utils": "8.45.0", - "@typescript-eslint/visitor-keys": "8.45.0", + "@typescript-eslint/scope-manager": "8.46.3", + "@typescript-eslint/type-utils": "8.46.3", + "@typescript-eslint/utils": "8.46.3", + "@typescript-eslint/visitor-keys": "8.46.3", "graphemer": "^1.4.0", "ignore": "^7.0.0", "natural-compare": "^1.4.0", @@ -3107,7 +3065,7 @@ "url": "https://opencollective.com/typescript-eslint" }, "peerDependencies": { - "@typescript-eslint/parser": "^8.45.0", + "@typescript-eslint/parser": "^8.46.3", "eslint": "^8.57.0 || ^9.0.0", "typescript": ">=4.8.4 <6.0.0" } @@ -3123,16 +3081,15 @@ } }, "node_modules/@typescript-eslint/parser": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.45.0.tgz", - "integrity": "sha512-TGf22kon8KW+DeKaUmOibKWktRY8b2NSAZNdtWh798COm1NWx8+xJ6iFBtk3IvLdv6+LGLJLRlyhrhEDZWargQ==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.46.3.tgz", + "integrity": "sha512-6m1I5RmHBGTnUGS113G04DMu3CpSdxCAU/UvtjNWL4Nuf3MW9tQhiJqRlHzChIkhy6kZSAQmc+I1bcGjE3yNKg==", "dev": true, - "license": "MIT", "dependencies": { - "@typescript-eslint/scope-manager": "8.45.0", - "@typescript-eslint/types": "8.45.0", - "@typescript-eslint/typescript-estree": "8.45.0", - "@typescript-eslint/visitor-keys": "8.45.0", + "@typescript-eslint/scope-manager": "8.46.3", + "@typescript-eslint/types": "8.46.3", + "@typescript-eslint/typescript-estree": "8.46.3", + "@typescript-eslint/visitor-keys": "8.46.3", "debug": "^4.3.4" }, "engines": { @@ -3148,14 +3105,13 @@ } }, "node_modules/@typescript-eslint/project-service": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.45.0.tgz", - "integrity": "sha512-3pcVHwMG/iA8afdGLMuTibGR7pDsn9RjDev6CCB+naRsSYs2pns5QbinF4Xqw6YC/Sj3lMrm/Im0eMfaa61WUg==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.46.3.tgz", + "integrity": "sha512-Fz8yFXsp2wDFeUElO88S9n4w1I4CWDTXDqDr9gYvZgUpwXQqmZBr9+NTTql5R3J7+hrJZPdpiWaB9VNhAKYLuQ==", "dev": true, - "license": "MIT", "dependencies": { - "@typescript-eslint/tsconfig-utils": "^8.45.0", - "@typescript-eslint/types": "^8.45.0", + "@typescript-eslint/tsconfig-utils": "^8.46.3", + "@typescript-eslint/types": "^8.46.3", "debug": "^4.3.4" }, "engines": { @@ -3170,14 +3126,13 @@ } }, "node_modules/@typescript-eslint/scope-manager": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.45.0.tgz", - "integrity": "sha512-clmm8XSNj/1dGvJeO6VGH7EUSeA0FMs+5au/u3lrA3KfG8iJ4u8ym9/j2tTEoacAffdW1TVUzXO30W1JTJS7dA==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.46.3.tgz", + "integrity": "sha512-FCi7Y1zgrmxp3DfWfr+3m9ansUUFoy8dkEdeQSgA9gbm8DaHYvZCdkFRQrtKiedFf3Ha6VmoqoAaP68+i+22kg==", "dev": true, - "license": "MIT", "dependencies": { - "@typescript-eslint/types": "8.45.0", - "@typescript-eslint/visitor-keys": "8.45.0" + "@typescript-eslint/types": "8.46.3", + "@typescript-eslint/visitor-keys": "8.46.3" }, "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" @@ -3188,11 +3143,10 @@ } }, "node_modules/@typescript-eslint/tsconfig-utils": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.45.0.tgz", - "integrity": "sha512-aFdr+c37sc+jqNMGhH+ajxPXwjv9UtFZk79k8pLoJ6p4y0snmYpPA52GuWHgt2ZF4gRRW6odsEj41uZLojDt5w==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.46.3.tgz", + "integrity": "sha512-GLupljMniHNIROP0zE7nCcybptolcH8QZfXOpCfhQDAdwJ/ZTlcaBOYebSOZotpti/3HrHSw7D3PZm75gYFsOA==", "dev": true, - "license": "MIT", "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" }, @@ -3205,15 +3159,14 @@ } }, "node_modules/@typescript-eslint/type-utils": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.45.0.tgz", - "integrity": "sha512-bpjepLlHceKgyMEPglAeULX1vixJDgaKocp0RVJ5u4wLJIMNuKtUXIczpJCPcn2waII0yuvks/5m5/h3ZQKs0A==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.46.3.tgz", + "integrity": "sha512-ZPCADbr+qfz3aiTTYNNkCbUt+cjNwI/5McyANNrFBpVxPt7GqpEYz5ZfdwuFyGUnJ9FdDXbGODUu6iRCI6XRXw==", "dev": true, - "license": "MIT", "dependencies": { - "@typescript-eslint/types": "8.45.0", - "@typescript-eslint/typescript-estree": "8.45.0", - "@typescript-eslint/utils": "8.45.0", + "@typescript-eslint/types": "8.46.3", + "@typescript-eslint/typescript-estree": "8.46.3", + "@typescript-eslint/utils": "8.46.3", "debug": "^4.3.4", "ts-api-utils": "^2.1.0" }, @@ -3230,11 +3183,10 @@ } }, "node_modules/@typescript-eslint/types": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.45.0.tgz", - "integrity": "sha512-WugXLuOIq67BMgQInIxxnsSyRLFxdkJEJu8r4ngLR56q/4Q5LrbfkFRH27vMTjxEK8Pyz7QfzuZe/G15qQnVRA==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.46.3.tgz", + "integrity": "sha512-G7Ok9WN/ggW7e/tOf8TQYMaxgID3Iujn231hfi0Pc7ZheztIJVpO44ekY00b7akqc6nZcvregk0Jpah3kep6hA==", "dev": true, - "license": "MIT", "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" }, @@ -3244,16 +3196,15 @@ } }, "node_modules/@typescript-eslint/typescript-estree": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.45.0.tgz", - "integrity": "sha512-GfE1NfVbLam6XQ0LcERKwdTTPlLvHvXXhOeUGC1OXi4eQBoyy1iVsW+uzJ/J9jtCz6/7GCQ9MtrQ0fml/jWCnA==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.46.3.tgz", + "integrity": "sha512-f/NvtRjOm80BtNM5OQtlaBdM5BRFUv7gf381j9wygDNL+qOYSNOgtQ/DCndiYi80iIOv76QqaTmp4fa9hwI0OA==", "dev": true, - "license": "MIT", "dependencies": { - "@typescript-eslint/project-service": "8.45.0", - "@typescript-eslint/tsconfig-utils": "8.45.0", - "@typescript-eslint/types": "8.45.0", - "@typescript-eslint/visitor-keys": "8.45.0", + "@typescript-eslint/project-service": "8.46.3", + "@typescript-eslint/tsconfig-utils": "8.46.3", + "@typescript-eslint/types": "8.46.3", + "@typescript-eslint/visitor-keys": "8.46.3", "debug": "^4.3.4", "fast-glob": "^3.3.2", "is-glob": "^4.0.3", @@ -3277,7 +3228,6 @@ "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz", "integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==", "dev": true, - "license": "MIT", "dependencies": { "balanced-match": "^1.0.0" } @@ -3287,7 +3237,6 @@ "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-9.0.5.tgz", "integrity": "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow==", "dev": true, - "license": "ISC", "dependencies": { "brace-expansion": "^2.0.1" }, @@ -3299,16 +3248,15 @@ } }, "node_modules/@typescript-eslint/utils": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.45.0.tgz", - "integrity": "sha512-bxi1ht+tLYg4+XV2knz/F7RVhU0k6VrSMc9sb8DQ6fyCTrGQLHfo7lDtN0QJjZjKkLA2ThrKuCdHEvLReqtIGg==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.46.3.tgz", + "integrity": "sha512-VXw7qmdkucEx9WkmR3ld/u6VhRyKeiF1uxWwCy/iuNfokjJ7VhsgLSOTjsol8BunSw190zABzpwdNsze2Kpo4g==", "dev": true, - "license": "MIT", "dependencies": { "@eslint-community/eslint-utils": "^4.7.0", - "@typescript-eslint/scope-manager": "8.45.0", - "@typescript-eslint/types": "8.45.0", - "@typescript-eslint/typescript-estree": "8.45.0" + "@typescript-eslint/scope-manager": "8.46.3", + "@typescript-eslint/types": "8.46.3", + "@typescript-eslint/typescript-estree": "8.46.3" }, "engines": { "node": "^18.18.0 || ^20.9.0 || >=21.1.0" @@ -3323,13 +3271,12 @@ } }, "node_modules/@typescript-eslint/visitor-keys": { - "version": "8.45.0", - "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.45.0.tgz", - "integrity": "sha512-qsaFBA3e09MIDAGFUrTk+dzqtfv1XPVz8t8d1f0ybTzrCY7BKiMC5cjrl1O/P7UmHsNyW90EYSkU/ZWpmXelag==", + "version": "8.46.3", + "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.46.3.tgz", + "integrity": "sha512-uk574k8IU0rOF/AjniX8qbLSGURJVUCeM5e4MIMKBFFi8weeiLrG1fyQejyLXQpRZbU/1BuQasleV/RfHC3hHg==", "dev": true, - "license": "MIT", "dependencies": { - "@typescript-eslint/types": "8.45.0", + "@typescript-eslint/types": "8.46.3", "eslint-visitor-keys": "^4.2.1" }, "engines": { @@ -3341,10 +3288,9 @@ } }, "node_modules/@uiw/codemirror-extensions-basic-setup": { - "version": "4.25.2", - "resolved": "https://registry.npmjs.org/@uiw/codemirror-extensions-basic-setup/-/codemirror-extensions-basic-setup-4.25.2.tgz", - "integrity": "sha512-s2fbpdXrSMWEc86moll/d007ZFhu6jzwNu5cWv/2o7egymvLeZO52LWkewgbr+BUCGWGPsoJVWeaejbsb/hLcw==", - "license": "MIT", + "version": "4.25.3", + "resolved": "https://registry.npmjs.org/@uiw/codemirror-extensions-basic-setup/-/codemirror-extensions-basic-setup-4.25.3.tgz", + "integrity": "sha512-F1doRyD50CWScwGHG2bBUtUpwnOv/zqSnzkZqJcX5YAHQx6Z1CuX8jdnFMH6qktRrPU1tfpNYftTWu3QIoHiMA==", "dependencies": { "@codemirror/autocomplete": "^6.0.0", "@codemirror/commands": "^6.0.0", @@ -3368,16 +3314,15 @@ } }, "node_modules/@uiw/react-codemirror": { - "version": "4.25.2", - "resolved": "https://registry.npmjs.org/@uiw/react-codemirror/-/react-codemirror-4.25.2.tgz", - "integrity": "sha512-XP3R1xyE0CP6Q0iR0xf3ed+cJzJnfmbLelgJR6osVVtMStGGZP3pGQjjwDRYptmjGHfEELUyyBLdY25h0BQg7w==", - "license": "MIT", + "version": "4.25.3", + "resolved": "https://registry.npmjs.org/@uiw/react-codemirror/-/react-codemirror-4.25.3.tgz", + "integrity": "sha512-1wtBZTXPIp8u6F/xjHvsUAYlEeF5Dic4xZBnqJyLzv7o7GjGYEUfSz9Z7bo9aK9GAx2uojG/AuBMfhA4uhvIVQ==", "dependencies": { "@babel/runtime": "^7.18.6", "@codemirror/commands": "^6.1.0", "@codemirror/state": "^6.1.1", "@codemirror/theme-one-dark": "^6.0.0", - "@uiw/codemirror-extensions-basic-setup": "4.25.2", + "@uiw/codemirror-extensions-basic-setup": "4.25.3", "codemirror": "^6.0.0" }, "funding": { @@ -4175,7 +4120,6 @@ "version": "1.0.2", "resolved": "https://registry.npmjs.org/cookie/-/cookie-1.0.2.tgz", "integrity": "sha512-9Kr/j4O16ISv8zBBhJoi4bXOYNTkFLOqSL3UDB0njXxCXNezjeyVrJyGOWtgfs/q2km1gwBcfH8q1yEGoMYunA==", - "license": "MIT", "engines": { "node": ">=18" } @@ -4276,10 +4220,9 @@ } }, "node_modules/dayjs": { - "version": "1.11.18", - "resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.11.18.tgz", - "integrity": "sha512-zFBQ7WFRvVRhKcWoUh+ZA1g2HVgUbsZm9sbddh8EC5iv93sui8DVVz1Npvz+r6meo9VKfa8NyLWBsQK1VvIKPA==", - "license": "MIT" + "version": "1.11.19", + "resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.11.19.tgz", + "integrity": "sha512-t5EcLVS6QPBNqM2z8fakk/NKel+Xzshgt8FFKAn+qwlD1pzZWxh0nVCrvFK7ZDb6XucZeF9z8C7CBWTRIVApAw==" }, "node_modules/debug": { "version": "4.4.3", @@ -4587,25 +4530,23 @@ } }, "node_modules/eslint": { - "version": "9.36.0", - "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.36.0.tgz", - "integrity": "sha512-hB4FIzXovouYzwzECDcUkJ4OcfOEkXTv2zRY6B9bkwjx/cprAq0uvm1nl7zvQ0/TsUk0zQiN4uPfJpB9m+rPMQ==", + "version": "9.39.1", + "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.1.tgz", + "integrity": "sha512-BhHmn2yNOFA9H9JmmIVKJmd288g9hrVRDkdoIgRCRuSySRUHH7r/DI6aAXW9T1WwUuY3DFgrcaqB+deURBLR5g==", "dev": true, - "license": "MIT", "dependencies": { "@eslint-community/eslint-utils": "^4.8.0", "@eslint-community/regexpp": "^4.12.1", - "@eslint/config-array": "^0.21.0", - "@eslint/config-helpers": "^0.3.1", - "@eslint/core": "^0.15.2", + "@eslint/config-array": "^0.21.1", + "@eslint/config-helpers": "^0.4.2", + "@eslint/core": "^0.17.0", "@eslint/eslintrc": "^3.3.1", - "@eslint/js": "9.36.0", - "@eslint/plugin-kit": "^0.3.5", + "@eslint/js": "9.39.1", + "@eslint/plugin-kit": "^0.4.1", "@humanfs/node": "^0.16.6", "@humanwhocodes/module-importer": "^1.0.1", "@humanwhocodes/retry": "^0.4.2", "@types/estree": "^1.0.6", - "@types/json-schema": "^7.0.15", "ajv": "^6.12.4", "chalk": "^4.0.0", "cross-spawn": "^7.0.6", @@ -4708,11 +4649,10 @@ } }, "node_modules/eslint-plugin-react-refresh": { - "version": "0.4.22", - "resolved": "https://registry.npmjs.org/eslint-plugin-react-refresh/-/eslint-plugin-react-refresh-0.4.22.tgz", - "integrity": "sha512-atkAG6QaJMGoTLc4MDAP+rqZcfwQuTIh2IqHWFLy2TEjxr0MOK+5BSG4RzL2564AAPpZkDRsZXAUz68kjnU6Ug==", + "version": "0.4.24", + "resolved": "https://registry.npmjs.org/eslint-plugin-react-refresh/-/eslint-plugin-react-refresh-0.4.24.tgz", + "integrity": "sha512-nLHIW7TEq3aLrEYWpVaJ1dRgFR+wLDPN8e8FpYAql/bMV2oBEfC37K0gLEGgv9fy66juNShSMV8OkTqzltcG/w==", "dev": true, - "license": "MIT", "peerDependencies": { "eslint": ">=8.40" } @@ -4747,19 +4687,6 @@ "url": "https://opencollective.com/eslint" } }, - "node_modules/eslint/node_modules/@eslint/core": { - "version": "0.15.2", - "resolved": "https://registry.npmjs.org/@eslint/core/-/core-0.15.2.tgz", - "integrity": "sha512-78Md3/Rrxh83gCxoUc0EiciuOHsIITzLy53m3d9UyiW8y9Dj2D29FeETqyKA+BRK76tnTp6RXWb3pCay8Oyomg==", - "dev": true, - "license": "Apache-2.0", - "dependencies": { - "@types/json-schema": "^7.0.15" - }, - "engines": { - "node": "^18.18.0 || ^20.9.0 || >=21.1.0" - } - }, "node_modules/espree": { "version": "10.4.0", "resolved": "https://registry.npmjs.org/espree/-/espree-10.4.0.tgz", @@ -4926,17 +4853,16 @@ "license": "Apache-2.0" }, "node_modules/fast-glob": { - "version": "3.3.2", - "resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.3.2.tgz", - "integrity": "sha512-oX2ruAFQwf/Orj8m737Y5adxDQO0LAB7/S5MnxCdTNDd4p6BsyIVsv9JQsATbTSq8KHRpLwIHbVlUNatxd+1Ow==", + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.3.3.tgz", + "integrity": "sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg==", "dev": true, - "license": "MIT", "dependencies": { "@nodelib/fs.stat": "^2.0.2", "@nodelib/fs.walk": "^1.2.3", "glob-parent": "^5.1.2", "merge2": "^1.3.0", - "micromatch": "^4.0.4" + "micromatch": "^4.0.8" }, "engines": { "node": ">=8.6.0" @@ -4947,7 +4873,6 @@ "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", "integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==", "dev": true, - "license": "ISC", "dependencies": { "is-glob": "^4.0.1" }, @@ -4970,11 +4895,10 @@ "license": "MIT" }, "node_modules/fastq": { - "version": "1.17.1", - "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.17.1.tgz", - "integrity": "sha512-sRVD3lWVIXWg6By68ZN7vho9a1pQcN/WBFaAAsDDFzlJjvoGx0P8z7V1t72grFJfJhu3YPZBuu25f7Kaw2jN1w==", + "version": "1.19.1", + "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.19.1.tgz", + "integrity": "sha512-GwLTyxkCXjXbxqIhTsMI2Nui8huMPtnxg7krajPJAjnEG/iiOS7i+zCtWGZR9G0NBKbXKh6X9m9UIsYX/N6vvQ==", "dev": true, - "license": "ISC", "dependencies": { "reusify": "^1.0.4" } @@ -5190,11 +5114,10 @@ } }, "node_modules/globals": { - "version": "16.4.0", - "resolved": "https://registry.npmjs.org/globals/-/globals-16.4.0.tgz", - "integrity": "sha512-ob/2LcVVaVGCYN+r14cnwnoDPUufjiYgSqRhiFD0Q1iI4Odora5RE8Iv1D24hAz5oMophRGkGz+yuvQmmUMnMw==", + "version": "16.5.0", + "resolved": "https://registry.npmjs.org/globals/-/globals-16.5.0.tgz", + "integrity": "sha512-c/c15i26VrJ4IRt5Z89DnIzCGDn9EcebibhAOjw5ibqEHsE1wLUgkPn9RDmNcUKyU87GeaL633nyJ+pplFR2ZQ==", "dev": true, - "license": "MIT", "engines": { "node": ">=18" }, @@ -5372,10 +5295,9 @@ } }, "node_modules/immer": { - "version": "10.1.1", - "resolved": "https://registry.npmjs.org/immer/-/immer-10.1.1.tgz", - "integrity": "sha512-s2MPrmjovJcoMaHtx6K11Ra7oD05NT97w1IC5zpMkT6Atjr7H8LjaDd81iIxUYpMKSRRNMJE703M1Fhr/TctHw==", - "license": "MIT", + "version": "10.2.0", + "resolved": "https://registry.npmjs.org/immer/-/immer-10.2.0.tgz", + "integrity": "sha512-d/+XTN3zfODyjr89gM3mPq1WNX2B8pYsu7eORitdwyA2sBubnTl3laYlBk4sXY5FUa5qTZGBDPJICVbvqzjlbw==", "funding": { "type": "opencollective", "url": "https://opencollective.com/immer" @@ -6918,7 +6840,6 @@ "resolved": "https://registry.npmjs.org/merge2/-/merge2-1.4.1.tgz", "integrity": "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg==", "dev": true, - "license": "MIT", "engines": { "node": ">= 8" } @@ -7772,28 +7693,25 @@ "type": "consulting", "url": "https://feross.org/support" } - ], - "license": "MIT" + ] }, "node_modules/react": { - "version": "19.1.1", - "resolved": "https://registry.npmjs.org/react/-/react-19.1.1.tgz", - "integrity": "sha512-w8nqGImo45dmMIfljjMwOGtbmC/mk4CMYhWIicdSflH91J9TyCyczcPFXJzrZ/ZXcgGRFeP6BU0BEJTw6tZdfQ==", - "license": "MIT", + "version": "19.2.0", + "resolved": "https://registry.npmjs.org/react/-/react-19.2.0.tgz", + "integrity": "sha512-tmbWg6W31tQLeB5cdIBOicJDJRR2KzXsV7uSK9iNfLWQ5bIZfxuPEHp7M8wiHyHnn0DD1i7w3Zmin0FtkrwoCQ==", "engines": { "node": ">=0.10.0" } }, "node_modules/react-dom": { - "version": "19.1.1", - "resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.1.1.tgz", - "integrity": "sha512-Dlq/5LAZgF0Gaz6yiqZCf6VCcZs1ghAJyrsu84Q/GT0gV+mCxbfmKNoGRKBYMJ8IEdGPqu49YWXD02GCknEDkw==", - "license": "MIT", + "version": "19.2.0", + "resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.0.tgz", + "integrity": "sha512-UlbRu4cAiGaIewkPyiRGJk0imDN2T3JjieT6spoL2UeSf5od4n5LB/mQ4ejmxhCFT1tYe8IvaFulzynWovsEFQ==", "dependencies": { - "scheduler": "^0.26.0" + "scheduler": "^0.27.0" }, "peerDependencies": { - "react": "^19.1.1" + "react": "^19.2.0" } }, "node_modules/react-infinite-scroll-component": { @@ -7905,10 +7823,9 @@ } }, "node_modules/react-router": { - "version": "7.9.3", - "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.9.3.tgz", - "integrity": "sha512-4o2iWCFIwhI/eYAIL43+cjORXYn/aRQPgtFRRZb3VzoyQ5Uej0Bmqj7437L97N9NJW4wnicSwLOLS+yCXfAPgg==", - "license": "MIT", + "version": "7.9.5", + "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.9.5.tgz", + "integrity": "sha512-JmxqrnBZ6E9hWmf02jzNn9Jm3UqyeimyiwzD69NjxGySG6lIz/1LVPsoTCwN7NBX2XjCEa1LIX5EMz1j2b6u6A==", "dependencies": { "cookie": "^1.0.1", "set-cookie-parser": "^2.6.0" @@ -7927,12 +7844,11 @@ } }, "node_modules/react-router-dom": { - "version": "7.9.3", - "resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-7.9.3.tgz", - "integrity": "sha512-1QSbA0TGGFKTAc/aWjpfW/zoEukYfU4dc1dLkT/vvf54JoGMkW+fNA+3oyo2gWVW1GM7BxjJVHz5GnPJv40rvg==", - "license": "MIT", + "version": "7.9.5", + "resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-7.9.5.tgz", + "integrity": "sha512-mkEmq/K8tKN63Ae2M7Xgz3c9l9YNbY+NHH6NNeUmLA3kDkhKXRsNb/ZpxaEunvGo2/3YXdk5EJU3Hxp3ocaBPw==", "dependencies": { - "react-router": "7.9.3" + "react-router": "7.9.5" }, "engines": { "node": ">=20.0.0" @@ -8111,11 +8027,10 @@ } }, "node_modules/reusify": { - "version": "1.0.4", - "resolved": "https://registry.npmjs.org/reusify/-/reusify-1.0.4.tgz", - "integrity": "sha512-U9nH88a3fc/ekCF1l0/UP1IosiuIjyTh7hBvXVMHYgVcfGvt897Xguj2UOLDeI5BG2m7/uwyaLVT6fbtCwTyzw==", + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/reusify/-/reusify-1.1.0.tgz", + "integrity": "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==", "dev": true, - "license": "MIT", "engines": { "iojs": ">=1.0.0", "node": ">=0.10.0" @@ -8186,7 +8101,6 @@ "url": "https://feross.org/support" } ], - "license": "MIT", "dependencies": { "queue-microtask": "^1.2.2" } @@ -8226,17 +8140,15 @@ } }, "node_modules/scheduler": { - "version": "0.26.0", - "resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.26.0.tgz", - "integrity": "sha512-NlHwttCI/l5gCPR3D1nNXtWABUmBwvZpEQiD4IXSbIDq8BzLIK/7Ir5gTFSGZDUu37K5cMNp0hFtzO38sC7gWA==", - "license": "MIT" + "version": "0.27.0", + "resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.27.0.tgz", + "integrity": "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==" }, "node_modules/semver": { - "version": "7.7.2", - "resolved": "https://registry.npmjs.org/semver/-/semver-7.7.2.tgz", - "integrity": "sha512-RF0Fw+rO5AMf9MAyaRXI4AV0Ulj5lMHqVxxdSgiVbixSCXoEmmX/jk0CuJw4+3SqroYO9VoUh+HcuJivvtJemA==", + "version": "7.7.3", + "resolved": "https://registry.npmjs.org/semver/-/semver-7.7.3.tgz", + "integrity": "sha512-SdsKMrI9TdgjdweUSR9MweHA4EJ8YxHn8DFaDisvhVlUOe4BF1tLD7GAj0lIqWVl+dPb/rExr0Btby5loQm20Q==", "dev": true, - "license": "ISC", "bin": { "semver": "bin/semver.js" }, @@ -8251,10 +8163,9 @@ "license": "ISC" }, "node_modules/set-cookie-parser": { - "version": "2.7.1", - "resolved": "https://registry.npmjs.org/set-cookie-parser/-/set-cookie-parser-2.7.1.tgz", - "integrity": "sha512-IOc8uWeOZgnb3ptbCURJWNjWUPcO3ZnTTdzsurqERrP6nPyv+paC55vJM0LpOlT2ne+Ix+9+CRG1MNLlyZ4GjQ==", - "license": "MIT" + "version": "2.7.2", + "resolved": "https://registry.npmjs.org/set-cookie-parser/-/set-cookie-parser-2.7.2.tgz", + "integrity": "sha512-oeM1lpU/UvhTxw+g3cIfxXHyJRc/uidd3yK1P242gzHds0udQBYzs3y8j4gCCW+ZJ7ad0yctld8RYO+bdurlvw==" }, "node_modules/shebang-command": { "version": "2.0.0", @@ -8584,9 +8495,9 @@ } }, "node_modules/tabbable": { - "version": "6.2.0", - "resolved": "https://registry.npmjs.org/tabbable/-/tabbable-6.2.0.tgz", - "integrity": "sha512-Cat63mxsVJlzYvN51JmVXIgNoUokrIaT2zLclCXjRd8boZ0004U4KCs/sToJ75C6sdlByWxpYnb5Boif1VSFew==" + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/tabbable/-/tabbable-6.3.0.tgz", + "integrity": "sha512-EIHvdY5bPLuWForiR/AN2Bxngzpuwn1is4asboytXtpTgsArc+WmSJKVLlhdh71u7jFcryDqB2A8lQvj78MkyQ==" }, "node_modules/test-exclude": { "version": "6.0.0", @@ -8774,7 +8685,6 @@ "resolved": "https://registry.npmjs.org/ts-api-utils/-/ts-api-utils-2.1.0.tgz", "integrity": "sha512-CUgTZL1irw8u29bzrOD/nH85jqyc74D6SshFgujOIA7osm2Rz7dYH77agkx7H4FBNxDq7Cjf+IjaX/8zwFW+ZQ==", "dev": true, - "license": "MIT", "engines": { "node": ">=18.12" }, @@ -8783,11 +8693,10 @@ } }, "node_modules/ts-jest": { - "version": "29.4.4", - "resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.4.4.tgz", - "integrity": "sha512-ccVcRABct5ZELCT5U0+DZwkXMCcOCLi2doHRrKy1nK/s7J7bch6TzJMsrY09WxgUUIP/ITfmcDS8D2yl63rnXw==", + "version": "29.4.5", + "resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.4.5.tgz", + "integrity": "sha512-HO3GyiWn2qvTQA4kTgjDcXiMwYQt68a1Y8+JuLRVpdIzm+UOLSHgl/XqR4c6nzJkq5rOkjc02O2I7P7l/Yof0Q==", "dev": true, - "license": "MIT", "dependencies": { "bs-logger": "^0.2.6", "fast-json-stable-stringify": "^2.1.0", @@ -8795,7 +8704,7 @@ "json5": "^2.2.3", "lodash.memoize": "^4.1.2", "make-error": "^1.3.6", - "semver": "^7.7.2", + "semver": "^7.7.3", "type-fest": "^4.41.0", "yargs-parser": "^21.1.1" }, @@ -9114,11 +9023,10 @@ } }, "node_modules/vite": { - "version": "6.3.6", - "resolved": "https://registry.npmjs.org/vite/-/vite-6.3.6.tgz", - "integrity": "sha512-0msEVHJEScQbhkbVTb/4iHZdJ6SXp/AvxL2sjwYQFfBqleHtnCqv1J3sa9zbWz/6kW1m9Tfzn92vW+kZ1WV6QA==", + "version": "6.4.1", + "resolved": "https://registry.npmjs.org/vite/-/vite-6.4.1.tgz", + "integrity": "sha512-+Oxm7q9hDoLMyJOYfUYBuHQo+dkAloi33apOPP56pzj+vsdJDzr+j1NISE5pyaAuKL4A3UD34qd0lx5+kfKp2g==", "dev": true, - "license": "MIT", "dependencies": { "esbuild": "^0.25.0", "fdir": "^6.4.4", diff --git a/web/ui/package.json b/web/ui/package.json index 4f4372a745..e237294df8 100644 --- a/web/ui/package.json +++ b/web/ui/package.json @@ -16,12 +16,12 @@ ], "devDependencies": { "@types/jest": "^29.5.14", - "@typescript-eslint/eslint-plugin": "^8.45.0", - "@typescript-eslint/parser": "^8.45.0", + "@typescript-eslint/eslint-plugin": "^8.46.3", + "@typescript-eslint/parser": "^8.46.3", "eslint-config-prettier": "^10.1.8", "prettier": "^3.6.2", - "ts-jest": "^29.4.4", + "ts-jest": "^29.4.5", "typescript": "^5.9.3", - "vite": "^6.3.6" + "vite": "^6.4.1" } } From 1b52ab9e3b9ecf5b969ff491288c45caabd5a3a4 Mon Sep 17 00:00:00 2001 From: matt-gp Date: Fri, 29 Aug 2025 22:01:18 +0100 Subject: [PATCH 022/439] feat: AWS ECS Service Discovery Signed-off-by: matt-gp --- discovery/aws/aws.go | 226 +++++++ discovery/aws/aws_test.go | 179 ++++++ discovery/aws/ecs.go | 663 +++++++++++++++++++ discovery/aws/ecs_test.go | 953 ++++++++++++++++++++++++++++ discovery/aws/metrics_aws.go | 32 + discovery/aws/metrics_ecs.go | 32 + docs/configuration/configuration.md | 157 +++++ go.mod | 1 + go.sum | 2 + 9 files changed, 2245 insertions(+) create mode 100644 discovery/aws/aws.go create mode 100644 discovery/aws/aws_test.go create mode 100644 discovery/aws/ecs.go create mode 100644 discovery/aws/ecs_test.go create mode 100644 discovery/aws/metrics_aws.go create mode 100644 discovery/aws/metrics_ecs.go diff --git a/discovery/aws/aws.go b/discovery/aws/aws.go new file mode 100644 index 0000000000..0fd5160b04 --- /dev/null +++ b/discovery/aws/aws.go @@ -0,0 +1,226 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package aws + +import ( + "errors" + "fmt" + "time" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/common/config" + "github.com/prometheus/common/model" + + "github.com/prometheus/prometheus/discovery" +) + +// DefaultSDConfig is the default AWS SD configuration. +var DefaultSDConfig = SDConfig{ + RefreshInterval: model.Duration(60 * time.Second), + HTTPClientConfig: config.DefaultHTTPClientConfig, +} + +func init() { + discovery.RegisterConfig(&SDConfig{}) +} + +// Role is role of the service in AWS. +type Role string + +// The valid options for Role. +const ( + RoleEC2 Role = "ec2" + RoleECS Role = "ecs" + RoleLightsail Role = "lightsail" +) + +// UnmarshalYAML implements the yaml.Unmarshaler interface. +func (c *Role) UnmarshalYAML(unmarshal func(any) error) error { + if err := unmarshal((*string)(c)); err != nil { + return err + } + switch *c { + case RoleEC2, RoleECS, RoleLightsail: + return nil + default: + return fmt.Errorf("unknown AWS SD role %q", *c) + } +} + +func (c Role) String() string { + return string(c) +} + +// SDConfig is the configuration for AWS service discovery. +type SDConfig struct { + Role Role `yaml:"role"` + Region string `yaml:"region,omitempty"` + Endpoint string `yaml:"endpoint,omitempty"` + AccessKey string `yaml:"access_key,omitempty"` + SecretKey config.Secret `yaml:"secret_key,omitempty"` + Profile string `yaml:"profile,omitempty"` + RoleARN string `yaml:"role_arn,omitempty"` + RefreshInterval model.Duration `yaml:"refresh_interval,omitempty"` + Port int `yaml:"port,omitempty"` + HTTPClientConfig config.HTTPClientConfig `yaml:",inline"` + + // ec2 specific + Filters []*EC2Filter `yaml:"filters,omitempty"` + + // ecs specific + Clusters []string `yaml:"clusters,omitempty"` + + // Embedded sub-configs (internal use only, not serialized) + *EC2SDConfig `yaml:"-"` + *ECSSDConfig `yaml:"-"` + *LightsailSDConfig `yaml:"-"` +} + +// UnmarshalYAML implements the yaml.Unmarshaler interface for SDConfig. +func (c *SDConfig) UnmarshalYAML(unmarshal func(any) error) error { + // Alias to avoid recursion + type plain SDConfig + var aux plain + // Unmarshal into aux + if err := unmarshal(&aux); err != nil { + return err + } + *c = SDConfig(aux) + + switch c.Role { + case RoleEC2: + if c.EC2SDConfig == nil { + c.EC2SDConfig = &DefaultEC2SDConfig + } + c.EC2SDConfig.HTTPClientConfig = c.HTTPClientConfig + if c.Region != "" { + c.EC2SDConfig.Region = c.Region + } + if c.Endpoint != "" { + c.EC2SDConfig.Endpoint = c.Endpoint + } + if c.AccessKey != "" { + c.EC2SDConfig.AccessKey = c.AccessKey + } + if c.SecretKey != "" { + c.EC2SDConfig.SecretKey = c.SecretKey + } + if c.Profile != "" { + c.EC2SDConfig.Profile = c.Profile + } + if c.RoleARN != "" { + c.EC2SDConfig.RoleARN = c.RoleARN + } + if c.Port != 0 { + c.EC2SDConfig.Port = c.Port + } + if c.RefreshInterval != 0 { + c.EC2SDConfig.RefreshInterval = c.RefreshInterval + } + if c.Filters != nil { + c.EC2SDConfig.Filters = c.Filters + } + case RoleECS: + if c.ECSSDConfig == nil { + c.ECSSDConfig = &DefaultECSSDConfig + } + c.ECSSDConfig.HTTPClientConfig = c.HTTPClientConfig + if c.Region != "" { + c.ECSSDConfig.Region = c.Region + } + if c.Endpoint != "" { + c.ECSSDConfig.Endpoint = c.Endpoint + } + if c.AccessKey != "" { + c.ECSSDConfig.AccessKey = c.AccessKey + } + if c.SecretKey != "" { + c.ECSSDConfig.SecretKey = c.SecretKey + } + if c.Profile != "" { + c.ECSSDConfig.Profile = c.Profile + } + if c.RoleARN != "" { + c.ECSSDConfig.RoleARN = c.RoleARN + } + if c.Port != 0 { + c.ECSSDConfig.Port = c.Port + } + if c.RefreshInterval != 0 { + c.ECSSDConfig.RefreshInterval = c.RefreshInterval + } + if c.Clusters != nil { + c.ECSSDConfig.Clusters = c.Clusters + } + case RoleLightsail: + if c.LightsailSDConfig == nil { + c.LightsailSDConfig = &DefaultLightsailSDConfig + } + c.LightsailSDConfig.HTTPClientConfig = c.HTTPClientConfig + if c.Region != "" { + c.LightsailSDConfig.Region = c.Region + } + if c.Endpoint != "" { + c.LightsailSDConfig.Endpoint = c.Endpoint + } + if c.AccessKey != "" { + c.LightsailSDConfig.AccessKey = c.AccessKey + } + if c.SecretKey != "" { + c.LightsailSDConfig.SecretKey = c.SecretKey + } + if c.Profile != "" { + c.LightsailSDConfig.Profile = c.Profile + } + if c.RoleARN != "" { + c.LightsailSDConfig.RoleARN = c.RoleARN + } + if c.Port != 0 { + c.LightsailSDConfig.Port = c.Port + } + if c.RefreshInterval != 0 { + c.LightsailSDConfig.RefreshInterval = c.RefreshInterval + } + default: + return fmt.Errorf("unknown AWS SD role %q", c.Role) + } + return nil +} + +// Name returns the name of the AWS Config. +func (*SDConfig) Name() string { return "aws" } + +// NewDiscovererMetrics implements discovery.Config. +func (*SDConfig) NewDiscovererMetrics(_ prometheus.Registerer, rmi discovery.RefreshMetricsInstantiator) discovery.DiscovererMetrics { + return &awsMetrics{refreshMetrics: rmi} +} + +// NewDiscoverer returns a Discoverer for the AWS Config. +func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { + awsMetrics, ok := opts.Metrics.(*awsMetrics) + if !ok { + return nil, errors.New("invalid discovery metrics type for AWS SD") + } + + switch c.Role { + case RoleEC2: + return NewEC2Discovery(c.EC2SDConfig, opts.Logger, &ec2Metrics{refreshMetrics: awsMetrics.refreshMetrics}) + case RoleECS: + return NewECSDiscovery(c.ECSSDConfig, opts.Logger, &ecsMetrics{refreshMetrics: awsMetrics.refreshMetrics}) + case RoleLightsail: + return NewLightsailDiscovery(c.LightsailSDConfig, opts.Logger, &lightsailMetrics{refreshMetrics: awsMetrics.refreshMetrics}) + default: + return nil, fmt.Errorf("unknown AWS SD role %q", c.Role) + } +} diff --git a/discovery/aws/aws_test.go b/discovery/aws/aws_test.go new file mode 100644 index 0000000000..a2f03a8b99 --- /dev/null +++ b/discovery/aws/aws_test.go @@ -0,0 +1,179 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package aws + +import ( + "errors" + "testing" + "time" + + "github.com/prometheus/common/model" + "github.com/stretchr/testify/require" + "gopkg.in/yaml.v3" +) + +func TestRoleUnmarshalYAML(t *testing.T) { + tests := []struct { + name string + input string + expected Role + wantErr bool + }{ + { + name: "EC2Role", + input: "ec2", + expected: RoleEC2, + wantErr: false, + }, + { + name: "LightsailRole", + input: "lightsail", + expected: RoleLightsail, + wantErr: false, + }, + { + name: "ECSRole", + input: "ecs", + expected: RoleECS, + wantErr: false, + }, + { + name: "InvalidRole", + input: "invalid", + expected: "invalid", + wantErr: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var r Role + err := r.UnmarshalYAML(func(v any) error { + ptr, ok := v.(*string) + if !ok { + return errors.New("not a string pointer") + } + *ptr = tt.input + return nil + }) + if tt.wantErr { + require.Error(t, err, "expected error for input %q", tt.input) + } else { + require.NoError(t, err, "unexpected error for input %q", tt.input) + require.Equal(t, tt.expected, r, "unexpected role for input %q", tt.input) + } + }) + } +} + +func TestRoleString(t *testing.T) { + tests := []struct { + name string + role Role + expected string + }{ + { + name: "EC2", + role: RoleEC2, + expected: "ec2", + }, + { + name: "Lightsail", + role: RoleLightsail, + expected: "lightsail", + }, + { + name: "ECS", + role: RoleECS, + expected: "ecs", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + require.Equal(t, tt.expected, tt.role.String()) + }) + } +} + +func TestSDConfigName(t *testing.T) { + cfg := &SDConfig{} + require.Equal(t, "aws", cfg.Name()) +} + +func TestDefaultSDConfig(t *testing.T) { + require.Equal(t, Role(""), DefaultSDConfig.Role) + require.Equal(t, model.Duration(60*time.Second), DefaultSDConfig.RefreshInterval) +} + +func TestSDConfigUnmarshalYAML(t *testing.T) { + tests := []struct { + name string + yaml string + validateFunc func(t *testing.T, cfg *SDConfig) + }{ + { + name: "EC2WithFlatFields", + yaml: `role: ec2 +region: us-west-2 +port: 9100 +filters: + - name: instance-state-name + values: [running]`, + validateFunc: func(t *testing.T, cfg *SDConfig) { + require.Equal(t, RoleEC2, cfg.Role) + require.NotNil(t, cfg.EC2SDConfig) + require.Equal(t, "us-west-2", cfg.EC2SDConfig.Region) + require.Equal(t, 9100, cfg.EC2SDConfig.Port) + require.Len(t, cfg.EC2SDConfig.Filters, 1) + require.Equal(t, "instance-state-name", cfg.EC2SDConfig.Filters[0].Name) + require.Equal(t, []string{"running"}, cfg.EC2SDConfig.Filters[0].Values) + }, + }, + { + name: "ECSWithFlatFields", + yaml: `role: ecs +region: us-east-1 +port: 9200 +clusters: ["some-cluster"]`, + validateFunc: func(t *testing.T, cfg *SDConfig) { + require.Equal(t, RoleECS, cfg.Role) + require.NotNil(t, cfg.ECSSDConfig) + require.Equal(t, "us-east-1", cfg.ECSSDConfig.Region) + require.Equal(t, 9200, cfg.ECSSDConfig.Port) + require.Equal(t, []string{"some-cluster"}, cfg.ECSSDConfig.Clusters) + }, + }, + { + name: "LightsailWithFlatFields", + yaml: `role: lightsail +region: eu-central-1 +port: 9300`, + validateFunc: func(t *testing.T, cfg *SDConfig) { + require.Equal(t, RoleLightsail, cfg.Role) + require.NotNil(t, cfg.LightsailSDConfig) + require.Equal(t, "eu-central-1", cfg.LightsailSDConfig.Region) + require.Equal(t, 9300, cfg.LightsailSDConfig.Port) + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var cfg SDConfig + require.NoError(t, yaml.Unmarshal([]byte(tt.yaml), &cfg)) + tt.validateFunc(t, &cfg) + }) + } +} diff --git a/discovery/aws/ecs.go b/discovery/aws/ecs.go new file mode 100644 index 0000000000..286c002a71 --- /dev/null +++ b/discovery/aws/ecs.go @@ -0,0 +1,663 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package aws + +import ( + "context" + "errors" + "fmt" + "log/slog" + "net" + "strconv" + "sync" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + awsConfig "github.com/aws/aws-sdk-go-v2/config" + "github.com/aws/aws-sdk-go-v2/credentials" + "github.com/aws/aws-sdk-go-v2/credentials/stscreds" + "github.com/aws/aws-sdk-go-v2/feature/ec2/imds" + "github.com/aws/aws-sdk-go-v2/service/ecs" + "github.com/aws/aws-sdk-go-v2/service/ecs/types" + "github.com/aws/aws-sdk-go-v2/service/sts" + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/common/config" + "github.com/prometheus/common/model" + "github.com/prometheus/common/promslog" + "golang.org/x/sync/errgroup" + + "github.com/prometheus/prometheus/discovery" + "github.com/prometheus/prometheus/discovery/refresh" + "github.com/prometheus/prometheus/discovery/targetgroup" + "github.com/prometheus/prometheus/util/strutil" +) + +const ( + ecsLabel = model.MetaLabelPrefix + "ecs_" + ecsLabelCluster = ecsLabel + "cluster" + ecsLabelClusterARN = ecsLabel + "cluster_arn" + ecsLabelService = ecsLabel + "service" + ecsLabelServiceARN = ecsLabel + "service_arn" + ecsLabelServiceStatus = ecsLabel + "service_status" + ecsLabelTaskGroup = ecsLabel + "task_group" + ecsLabelTaskARN = ecsLabel + "task_arn" + ecsLabelTaskDefinition = ecsLabel + "task_definition" + ecsLabelRegion = ecsLabel + "region" + ecsLabelAvailabilityZone = ecsLabel + "availability_zone" + ecsLabelAZID = ecsLabel + "availability_zone_id" + ecsLabelSubnetID = ecsLabel + "subnet_id" + ecsLabelIPAddress = ecsLabel + "ip_address" + ecsLabelLaunchType = ecsLabel + "launch_type" + ecsLabelDesiredStatus = ecsLabel + "desired_status" + ecsLabelLastStatus = ecsLabel + "last_status" + ecsLabelHealthStatus = ecsLabel + "health_status" + ecsLabelPlatformFamily = ecsLabel + "platform_family" + ecsLabelPlatformVersion = ecsLabel + "platform_version" + ecsLabelTag = ecsLabel + "tag_" + ecsLabelTagCluster = ecsLabelTag + "cluster_" + ecsLabelTagService = ecsLabelTag + "service_" + ecsLabelTagTask = ecsLabelTag + "task_" + ecsLabelSeparator = "," +) + +// DefaultECSSDConfig is the default ECS SD configuration. +var DefaultECSSDConfig = ECSSDConfig{ + Port: 80, + RefreshInterval: model.Duration(60 * time.Second), + RequestConcurrency: 20, // Aligned with AWS ECS API sustained rate limits (20 req/sec) + HTTPClientConfig: config.DefaultHTTPClientConfig, +} + +func init() { + discovery.RegisterConfig(&ECSSDConfig{}) +} + +// ECSSDConfig is the configuration for ECS based service discovery. +type ECSSDConfig struct { + Region string `yaml:"region"` + Endpoint string `yaml:"endpoint"` + AccessKey string `yaml:"access_key,omitempty"` + SecretKey config.Secret `yaml:"secret_key,omitempty"` + Profile string `yaml:"profile,omitempty"` + RoleARN string `yaml:"role_arn,omitempty"` + Clusters []string `yaml:"clusters,omitempty"` + Port int `yaml:"port"` + RefreshInterval model.Duration `yaml:"refresh_interval,omitempty"` + + // RequestConcurrency controls the maximum number of concurrent ECS API requests. + // Default is 20, which aligns with AWS ECS sustained rate limits: + // - Cluster read actions (DescribeClusters, ListClusters): 20 req/sec sustained + // - Service read actions (DescribeServices, ListServices): 20 req/sec sustained + // - Cluster resource read actions (DescribeTasks, ListTasks): 20 req/sec sustained + // See: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/request-throttling.html + RequestConcurrency int `yaml:"request_concurrency,omitempty"` + + HTTPClientConfig config.HTTPClientConfig `yaml:",inline"` +} + +// NewDiscovererMetrics implements discovery.Config. +func (*ECSSDConfig) NewDiscovererMetrics(_ prometheus.Registerer, rmi discovery.RefreshMetricsInstantiator) discovery.DiscovererMetrics { + return &ecsMetrics{ + refreshMetrics: rmi, + } +} + +// Name returns the name of the ECS Config. +func (*ECSSDConfig) Name() string { return "ecs" } + +// NewDiscoverer returns a Discoverer for the EC2 Config. +func (c *ECSSDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { + return NewECSDiscovery(c, opts.Logger, opts.Metrics) +} + +// UnmarshalYAML implements the yaml.Unmarshaler interface for the ECS Config. +func (c *ECSSDConfig) UnmarshalYAML(unmarshal func(interface{}) error) error { + *c = DefaultECSSDConfig + type plain ECSSDConfig + err := unmarshal((*plain)(c)) + if err != nil { + return err + } + + if c.Region == "" { + cfg, err := awsConfig.LoadDefaultConfig(context.TODO()) + if err != nil { + return err + } + client := imds.NewFromConfig(cfg) + result, err := client.GetRegion(context.Background(), &imds.GetRegionInput{}) + if err != nil { + return fmt.Errorf("ECS SD configuration requires a region. Tried to fetch it from the instance metadata: %w", err) + } + c.Region = result.Region + } + + return c.HTTPClientConfig.Validate() +} + +type ecsClient interface { + ListClusters(context.Context, *ecs.ListClustersInput, ...func(*ecs.Options)) (*ecs.ListClustersOutput, error) + DescribeClusters(context.Context, *ecs.DescribeClustersInput, ...func(*ecs.Options)) (*ecs.DescribeClustersOutput, error) + ListServices(context.Context, *ecs.ListServicesInput, ...func(*ecs.Options)) (*ecs.ListServicesOutput, error) + DescribeServices(context.Context, *ecs.DescribeServicesInput, ...func(*ecs.Options)) (*ecs.DescribeServicesOutput, error) + ListTasks(context.Context, *ecs.ListTasksInput, ...func(*ecs.Options)) (*ecs.ListTasksOutput, error) + DescribeTasks(context.Context, *ecs.DescribeTasksInput, ...func(*ecs.Options)) (*ecs.DescribeTasksOutput, error) +} + +// ECSDiscovery periodically performs ECS-SD requests. It implements +// the Discoverer interface. +type ECSDiscovery struct { + *refresh.Discovery + logger *slog.Logger + cfg *ECSSDConfig + ecs ecsClient +} + +// NewECSDiscovery returns a new ECSDiscovery which periodically refreshes its targets. +func NewECSDiscovery(conf *ECSSDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*ECSDiscovery, error) { + m, ok := metrics.(*ecsMetrics) + if !ok { + return nil, errors.New("invalid discovery metrics type") + } + + if logger == nil { + logger = promslog.NewNopLogger() + } + d := &ECSDiscovery{ + logger: logger, + cfg: conf, + } + d.Discovery = refresh.NewDiscovery( + refresh.Options{ + Logger: logger, + Mech: "ecs", + Interval: time.Duration(d.cfg.RefreshInterval), + RefreshF: d.refresh, + MetricsInstantiator: m.refreshMetrics, + }, + ) + return d, nil +} + +func (d *ECSDiscovery) initEcsClient(ctx context.Context) error { + if d.ecs != nil { + return nil + } + + if d.cfg.Region == "" { + return errors.New("region must be set for ECS service discovery") + } + + // Build the HTTP client from the provided HTTPClientConfig. + client, err := config.NewClientFromConfig(d.cfg.HTTPClientConfig, "ecs_sd") + if err != nil { + return err + } + + // Build the AWS config with the provided region. + var configOptions []func(*awsConfig.LoadOptions) error + configOptions = append(configOptions, awsConfig.WithRegion(d.cfg.Region)) + configOptions = append(configOptions, awsConfig.WithHTTPClient(client)) + + // Only set static credentials if both access key and secret key are provided + // Otherwise, let AWS SDK use its default credential chain + if d.cfg.AccessKey != "" && d.cfg.SecretKey != "" { + credProvider := credentials.NewStaticCredentialsProvider(d.cfg.AccessKey, string(d.cfg.SecretKey), "") + configOptions = append(configOptions, awsConfig.WithCredentialsProvider(credProvider)) + } + + if d.cfg.Profile != "" { + configOptions = append(configOptions, awsConfig.WithSharedConfigProfile(d.cfg.Profile)) + } + + cfg, err := awsConfig.LoadDefaultConfig(ctx, configOptions...) + if err != nil { + d.logger.Error("Failed to create AWS config", "error", err) + return fmt.Errorf("could not create aws config: %w", err) + } + + // If the role ARN is set, assume the role to get credentials and set the credentials provider in the config. + if d.cfg.RoleARN != "" { + assumeProvider := stscreds.NewAssumeRoleProvider(sts.NewFromConfig(cfg), d.cfg.RoleARN) + cfg.Credentials = aws.NewCredentialsCache(assumeProvider) + } + + d.ecs = ecs.NewFromConfig(cfg, func(options *ecs.Options) { + if d.cfg.Endpoint != "" { + options.BaseEndpoint = &d.cfg.Endpoint + } + options.HTTPClient = client + }) + + // Test credentials by making a simple API call + testCtx, cancel := context.WithTimeout(ctx, 10*time.Second) + defer cancel() + + _, err = d.ecs.DescribeClusters(testCtx, &ecs.DescribeClustersInput{}) + if err != nil { + d.logger.Error("Failed to test ECS credentials", "error", err) + return fmt.Errorf("ECS credential test failed: %w", err) + } + + return nil +} + +// listClusterARNs returns a slice of cluster arns. +// This method does not use concurrency as it's a simple paginated call. +// AWS ECS Cluster read actions have burst=50, sustained=20 req/sec limits. +func (d *ECSDiscovery) listClusterARNs(ctx context.Context) ([]string, error) { + var ( + clusterARNs []string + nextToken *string + ) + for { + resp, err := d.ecs.ListClusters(ctx, &ecs.ListClustersInput{ + NextToken: nextToken, + }) + if err != nil { + return nil, fmt.Errorf("could not list clusters: %w", err) + } + + clusterARNs = append(clusterARNs, resp.ClusterArns...) + + if resp.NextToken == nil { + break + } + nextToken = resp.NextToken + } + + return clusterARNs, nil +} + +// describeClusters returns a map of cluster ARN to a slice of clusters. +// This method processes clusters in batches without concurrency as it's typically +// a single call handling up to 100 clusters. AWS ECS Cluster read actions have +// burst=50, sustained=20 req/sec limits. +func (d *ECSDiscovery) describeClusters(ctx context.Context, clusters []string) (map[string]types.Cluster, error) { + clusterMap := make(map[string]types.Cluster) + + // AWS DescribeClusters can handle up to 100 clusters per call + batchSize := 100 + for _, batch := range batchSlice(clusters, batchSize) { + resp, err := d.ecs.DescribeClusters(ctx, &ecs.DescribeClustersInput{ + Clusters: batch, + Include: []types.ClusterField{"TAGS"}, + }) + if err != nil { + d.logger.Error("Failed to describe clusters", "clusters", batch, "error", err) + return nil, fmt.Errorf("could not describe clusters %v: %w", batch, err) + } + + for _, c := range resp.Clusters { + if c.ClusterArn != nil { + clusterMap[*c.ClusterArn] = c + } + } + } + + return clusterMap, nil +} + +// listServiceARNs returns a map of cluster ARN to a slice of service ARNs. +// Uses concurrent requests limited by RequestConcurrency to respect AWS API throttling. +// AWS ECS Service read actions have burst=100, sustained=20 req/sec limits. +func (d *ECSDiscovery) listServiceARNs(ctx context.Context, clusters []string) (map[string][]string, error) { + serviceARNsMu := sync.Mutex{} + serviceARNs := make(map[string][]string) + errg, ectx := errgroup.WithContext(ctx) + errg.SetLimit(d.cfg.RequestConcurrency) + for _, clusterARN := range clusters { + errg.Go(func() error { + var nextToken *string + var clusterServiceARNs []string + for { + resp, err := d.ecs.ListServices(ectx, &ecs.ListServicesInput{ + Cluster: aws.String(clusterARN), + NextToken: nextToken, + }) + if err != nil { + return fmt.Errorf("could not list services for cluster %q: %w", clusterARN, err) + } + + clusterServiceARNs = append(clusterServiceARNs, resp.ServiceArns...) + + if resp.NextToken == nil { + break + } + nextToken = resp.NextToken + } + + serviceARNsMu.Lock() + serviceARNs[clusterARN] = clusterServiceARNs + serviceARNsMu.Unlock() + return nil + }) + } + + return serviceARNs, errg.Wait() +} + +// describeServices returns a map of cluster ARN to services. +// Uses concurrent requests with batching (10 services per request) to respect AWS API limits. +// AWS ECS Service read actions have burst=100, sustained=20 req/sec limits. +func (d *ECSDiscovery) describeServices(ctx context.Context, clusterServiceARNsMap map[string][]string) (map[string][]types.Service, error) { + batchSize := 10 // AWS DescribeServices API limit is 10 services per request + serviceMu := sync.Mutex{} + services := make(map[string][]types.Service) + errg, ectx := errgroup.WithContext(ctx) + errg.SetLimit(d.cfg.RequestConcurrency) + for clusterARN, serviceARNs := range clusterServiceARNsMap { + for _, batch := range batchSlice(serviceARNs, batchSize) { + errg.Go(func() error { + resp, err := d.ecs.DescribeServices(ectx, &ecs.DescribeServicesInput{ + Services: batch, + Cluster: aws.String(clusterARN), + Include: []types.ServiceField{"TAGS"}, + }) + if err != nil { + d.logger.Error("Failed to describe services", "cluster", clusterARN, "batch", batch, "error", err) + return fmt.Errorf("could not describe services for cluster %q: %w", clusterARN, err) + } + + serviceMu.Lock() + services[clusterARN] = append(services[clusterARN], resp.Services...) + serviceMu.Unlock() + + return nil + }) + } + } + + return services, errg.Wait() +} + +// listTaskARNs returns a map of service ARN to a slice of task ARNs. +// Uses concurrent requests limited by RequestConcurrency to respect AWS API throttling. +// AWS ECS Cluster resource read actions have burst=100, sustained=20 req/sec limits. +func (d *ECSDiscovery) listTaskARNs(ctx context.Context, services []types.Service) (map[string][]string, error) { + taskARNsMu := sync.Mutex{} + taskARNs := make(map[string][]string) + errg, ectx := errgroup.WithContext(ctx) + errg.SetLimit(d.cfg.RequestConcurrency) + for _, service := range services { + errg.Go(func() error { + serviceArn := aws.ToString(service.ServiceArn) + + var nextToken *string + var serviceTaskARNs []string + for { + resp, err := d.ecs.ListTasks(ectx, &ecs.ListTasksInput{ + Cluster: aws.String(*service.ClusterArn), + ServiceName: aws.String(*service.ServiceName), + NextToken: nextToken, + }) + if err != nil { + return fmt.Errorf("could not list tasks for service %q: %w", serviceArn, err) + } + + serviceTaskARNs = append(serviceTaskARNs, resp.TaskArns...) + + if resp.NextToken == nil { + break + } + nextToken = resp.NextToken + } + + taskARNsMu.Lock() + taskARNs[serviceArn] = serviceTaskARNs + taskARNsMu.Unlock() + return nil + }) + } + + return taskARNs, errg.Wait() +} + +// describeTasks returns a map of task arn to a slice task. +// Uses concurrent requests with batching (100 tasks per request) to respect AWS API limits. +// AWS ECS Cluster resource read actions have burst=100, sustained=20 req/sec limits. +func (d *ECSDiscovery) describeTasks(ctx context.Context, clusterARN string, taskARNsMap map[string][]string) (map[string][]types.Task, error) { + batchSize := 100 // AWS DescribeTasks API limit is 100 tasks per request + taskMu := sync.Mutex{} + tasks := make(map[string][]types.Task) + errg, ectx := errgroup.WithContext(ctx) + errg.SetLimit(d.cfg.RequestConcurrency) + for serviceARN, taskARNs := range taskARNsMap { + for _, batch := range batchSlice(taskARNs, batchSize) { + errg.Go(func() error { + resp, err := d.ecs.DescribeTasks(ectx, &ecs.DescribeTasksInput{ + Cluster: aws.String(clusterARN), + Tasks: batch, + Include: []types.TaskField{"TAGS"}, + }) + if err != nil { + d.logger.Error("Failed to describe tasks", "service", serviceARN, "cluster", clusterARN, "batch", batch, "error", err) + return fmt.Errorf("could not describe tasks for service %q in cluster %q: %w", serviceARN, clusterARN, err) + } + + taskMu.Lock() + tasks[serviceARN] = append(tasks[serviceARN], resp.Tasks...) + taskMu.Unlock() + + return nil + }) + } + } + + return tasks, errg.Wait() +} + +func batchSlice[T any](a []T, size int) [][]T { + batches := make([][]T, 0, len(a)/size+1) + for i := 0; i < len(a); i += size { + end := i + size + if end > len(a) { + end = len(a) + } + batches = append(batches, a[i:end]) + } + return batches +} + +func (d *ECSDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, error) { + err := d.initEcsClient(ctx) + if err != nil { + return nil, err + } + + var clusters []string + if len(d.cfg.Clusters) == 0 { + clusters, err = d.listClusterARNs(ctx) + if err != nil { + return nil, err + } + } else { + clusters = d.cfg.Clusters + } + + if len(clusters) == 0 { + return []*targetgroup.Group{ + { + Source: d.cfg.Region, + }, + }, nil + } + + tg := &targetgroup.Group{ + Source: d.cfg.Region, + } + + clusterARNMap, err := d.describeClusters(ctx, clusters) + if err != nil { + return nil, err + } + + clusterServiceARNMap, err := d.listServiceARNs(ctx, clusters) + if err != nil { + return nil, err + } + + clusterServicesMap, err := d.describeServices(ctx, clusterServiceARNMap) + if err != nil { + return nil, err + } + + // Use goroutines to process clusters in parallel + var ( + targetsMu sync.Mutex + wg sync.WaitGroup + ) + + for clusterArn, clusterServices := range clusterServicesMap { + if len(clusterServices) == 0 { + continue + } + + wg.Add(1) + go func(clusterArn string, clusterServices []types.Service) { + defer wg.Done() + + serviceTaskARNMap, err := d.listTaskARNs(ctx, clusterServices) + if err != nil { + d.logger.Error("Failed to list task ARNs for cluster", "cluster", clusterArn, "error", err) + return + } + + serviceTaskMap, err := d.describeTasks(ctx, clusterArn, serviceTaskARNMap) + if err != nil { + d.logger.Error("Failed to describe tasks for cluster", "cluster", clusterArn, "error", err) + return + } + + // Process services within this cluster in parallel + var ( + serviceWg sync.WaitGroup + localTargets []model.LabelSet + localTargetsMu sync.Mutex + ) + + for _, clusterService := range clusterServices { + serviceWg.Add(1) + go func(clusterService types.Service) { + defer serviceWg.Done() + + serviceArn := *clusterService.ServiceArn + + if tasks, exists := serviceTaskMap[serviceArn]; exists { + var serviceTargets []model.LabelSet + + for _, task := range tasks { + // Find the ENI attachment to get the private IP address + var eniAttachment *types.Attachment + for _, attachment := range task.Attachments { + if attachment.Type != nil && *attachment.Type == "ElasticNetworkInterface" { + eniAttachment = &attachment + break + } + } + if eniAttachment == nil { + continue + } + + var ipAddress, subnetID string + for _, detail := range eniAttachment.Details { + switch *detail.Name { + case "privateIPv4Address": + ipAddress = *detail.Value + case "subnetId": + subnetID = *detail.Value + } + } + if ipAddress == "" { + continue + } + + labels := model.LabelSet{ + ecsLabelClusterARN: model.LabelValue(*clusterService.ClusterArn), + ecsLabelService: model.LabelValue(*clusterService.ServiceName), + ecsLabelServiceARN: model.LabelValue(*clusterService.ServiceArn), + ecsLabelServiceStatus: model.LabelValue(*clusterService.Status), + ecsLabelTaskGroup: model.LabelValue(*task.Group), + ecsLabelTaskARN: model.LabelValue(*task.TaskArn), + ecsLabelTaskDefinition: model.LabelValue(*task.TaskDefinitionArn), + ecsLabelIPAddress: model.LabelValue(ipAddress), + ecsLabelSubnetID: model.LabelValue(subnetID), + ecsLabelRegion: model.LabelValue(d.cfg.Region), + ecsLabelLaunchType: model.LabelValue(task.LaunchType), + ecsLabelAvailabilityZone: model.LabelValue(*task.AvailabilityZone), + ecsLabelDesiredStatus: model.LabelValue(*task.DesiredStatus), + ecsLabelLastStatus: model.LabelValue(*task.LastStatus), + ecsLabelHealthStatus: model.LabelValue(task.HealthStatus), + } + + if task.PlatformFamily != nil { + labels[ecsLabelPlatformFamily] = model.LabelValue(*task.PlatformFamily) + } + if task.PlatformVersion != nil { + labels[ecsLabelPlatformVersion] = model.LabelValue(*task.PlatformVersion) + } + + labels[model.AddressLabel] = model.LabelValue(net.JoinHostPort(ipAddress, strconv.Itoa(d.cfg.Port))) + + // Add cluster tags + if cluster, exists := clusterARNMap[*clusterService.ClusterArn]; exists { + if cluster.ClusterName != nil { + labels[ecsLabelCluster] = model.LabelValue(*cluster.ClusterName) + } + + for _, clusterTag := range cluster.Tags { + if clusterTag.Key != nil && clusterTag.Value != nil { + labels[model.LabelName(ecsLabelTagCluster+strutil.SanitizeLabelName(*clusterTag.Key))] = model.LabelValue(*clusterTag.Value) + } + } + } + + // Add service tags + for _, serviceTag := range clusterService.Tags { + if serviceTag.Key != nil && serviceTag.Value != nil { + labels[model.LabelName(ecsLabelTagService+strutil.SanitizeLabelName(*serviceTag.Key))] = model.LabelValue(*serviceTag.Value) + } + } + + // Add task tags + for _, taskTag := range task.Tags { + if taskTag.Key != nil && taskTag.Value != nil { + labels[model.LabelName(ecsLabelTagTask+strutil.SanitizeLabelName(*taskTag.Key))] = model.LabelValue(*taskTag.Value) + } + } + + serviceTargets = append(serviceTargets, labels) + } + + // Add service targets to local targets with mutex protection + localTargetsMu.Lock() + localTargets = append(localTargets, serviceTargets...) + localTargetsMu.Unlock() + } + }(clusterService) + } + + serviceWg.Wait() + + // Add all local targets to main target group with mutex protection + targetsMu.Lock() + tg.Targets = append(tg.Targets, localTargets...) + targetsMu.Unlock() + }(clusterArn, clusterServices) + } + + wg.Wait() + + return []*targetgroup.Group{tg}, nil +} diff --git a/discovery/aws/ecs_test.go b/discovery/aws/ecs_test.go new file mode 100644 index 0000000000..60138a01c7 --- /dev/null +++ b/discovery/aws/ecs_test.go @@ -0,0 +1,953 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package aws + +import ( + "context" + "testing" + + "github.com/aws/aws-sdk-go-v2/service/ecs" + ecsTypes "github.com/aws/aws-sdk-go-v2/service/ecs/types" + "github.com/prometheus/common/model" + "github.com/stretchr/testify/require" + + "github.com/prometheus/prometheus/discovery/targetgroup" +) + +// Struct for test data. +type ecsDataStore struct { + region string + + clusters []ecsTypes.Cluster + services []ecsTypes.Service + tasks []ecsTypes.Task +} + +func TestECSDiscoveryListClusterARNs(t *testing.T) { + ctx := context.Background() + + // iterate through the test cases + for _, tt := range []struct { + name string + ecsData *ecsDataStore + expected []string + }{ + { + name: "MultipleClusters", + ecsData: &ecsDataStore{ + region: "us-west-2", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("test-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + }, + { + ClusterName: strptr("prod-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/prod-cluster"), + Status: strptr("ACTIVE"), + }, + }, + }, + expected: []string{ + "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster", + "arn:aws:ecs:us-west-2:123456789012:cluster/prod-cluster", + }, + }, + { + name: "SingleCluster", + ecsData: &ecsDataStore{ + region: "us-east-1", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("single-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/single-cluster"), + Status: strptr("ACTIVE"), + }, + }, + }, + expected: []string{ + "arn:aws:ecs:us-east-1:123456789012:cluster/single-cluster", + }, + }, + { + name: "NoClusters", + ecsData: &ecsDataStore{ + region: "us-east-1", + clusters: []ecsTypes.Cluster{}, + }, + expected: nil, + }, + } { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + RequestConcurrency: 10, + }, + } + + clusters, err := d.listClusterARNs(ctx) + require.NoError(t, err) + require.Equal(t, tt.expected, clusters) + }) + } +} + +func TestECSDiscoveryDescribeClusters(t *testing.T) { + ctx := context.Background() + + // iterate through the test cases + for _, tt := range []struct { + name string + ecsData *ecsDataStore + clusterARNs []string + expected map[string]ecsTypes.Cluster + }{ + { + name: "SingleClusterWithTags", + ecsData: &ecsDataStore{ + region: "us-west-2", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("test-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("test")}, + {Key: strptr("Team"), Value: strptr("backend")}, + }, + }, + }, + }, + clusterARNs: []string{"arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"}, + expected: map[string]ecsTypes.Cluster{ + "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster": { + ClusterName: strptr("test-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("test")}, + {Key: strptr("Team"), Value: strptr("backend")}, + }, + }, + }, + }, + { + name: "MultipleClusters", + ecsData: &ecsDataStore{ + region: "us-east-1", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("cluster-1"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1"), + Status: strptr("ACTIVE"), + }, + { + ClusterName: strptr("cluster-2"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2"), + Status: strptr("DRAINING"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Stage"), Value: strptr("prod")}, + }, + }, + }, + }, + clusterARNs: []string{ + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1", + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2", + }, + expected: map[string]ecsTypes.Cluster{ + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1": { + ClusterName: strptr("cluster-1"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1"), + Status: strptr("ACTIVE"), + }, + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2": { + ClusterName: strptr("cluster-2"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2"), + Status: strptr("DRAINING"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Stage"), Value: strptr("prod")}, + }, + }, + }, + }, + } { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + RequestConcurrency: 10, + }, + } + + clusterMap, err := d.describeClusters(ctx, tt.clusterARNs) + require.NoError(t, err) + require.Equal(t, tt.expected, clusterMap) + }) + } +} + +func TestECSDiscoveryListServiceARNs(t *testing.T) { + ctx := context.Background() + + // iterate through the test cases + for _, tt := range []struct { + name string + ecsData *ecsDataStore + clusterARNs []string + expected map[string][]string + }{ + { + name: "SingleClusterWithServices", + ecsData: &ecsDataStore{ + region: "us-west-2", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("test-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + }, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("web-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + }, + { + ServiceName: strptr("api-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + }, + { + // this is to test the old arn format without the cluster name in the service arn + // https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-arn-migration.html + ServiceName: strptr("old-api-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/old-api-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + }, + }, + }, + clusterARNs: []string{"arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"}, + expected: map[string][]string{ + "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster": { + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service", + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service", + "arn:aws:ecs:us-west-2:123456789012:service/old-api-service", + }, + }, + }, + { + name: "MultipleClustesWithServices", + ecsData: &ecsDataStore{ + region: "us-east-1", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("cluster-1"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1"), + Status: strptr("ACTIVE"), + }, + { + ClusterName: strptr("cluster-2"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2"), + Status: strptr("ACTIVE"), + }, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("service-1"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/cluster-1/service-1"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1"), + Status: strptr("RUNNING"), + }, + { + ServiceName: strptr("service-2"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/cluster-2/service-2"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2"), + Status: strptr("RUNNING"), + }, + }, + }, + clusterARNs: []string{ + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1", + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2", + }, + expected: map[string][]string{ + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1": { + "arn:aws:ecs:us-east-1:123456789012:service/cluster-1/service-1", + }, + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2": { + "arn:aws:ecs:us-east-1:123456789012:service/cluster-2/service-2", + }, + }, + }, + { + name: "ClusterWithNoServices", + ecsData: &ecsDataStore{ + region: "us-west-2", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("empty-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/empty-cluster"), + Status: strptr("ACTIVE"), + }, + }, + services: []ecsTypes.Service{}, + }, + clusterARNs: []string{"arn:aws:ecs:us-west-2:123456789012:cluster/empty-cluster"}, + expected: map[string][]string{ + "arn:aws:ecs:us-west-2:123456789012:cluster/empty-cluster": nil, + }, + }, + } { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + RequestConcurrency: 1, + }, + } + + serviceMap, err := d.listServiceARNs(ctx, tt.clusterARNs) + require.NoError(t, err) + require.Equal(t, tt.expected, serviceMap) + }) + } +} + +func TestECSDiscoveryDescribeServices(t *testing.T) { + ctx := context.Background() + + // iterate through the test cases + for _, tt := range []struct { + name string + ecsData *ecsDataStore + clusterServiceARNsMap map[string][]string + expected map[string][]ecsTypes.Service + }{ + { + name: "SingleClusterServices", + ecsData: &ecsDataStore{ + region: "us-west-2", + services: []ecsTypes.Service{ + { + ServiceName: strptr("web-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + TaskDefinition: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("production")}, + }, + }, + { + ServiceName: strptr("api-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + TaskDefinition: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/api-task:2"), + }, + }, + }, + clusterServiceARNsMap: map[string][]string{ + "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster": { + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service", + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service", + }, + }, + expected: map[string][]ecsTypes.Service{ + "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster": { + { + ServiceName: strptr("web-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + TaskDefinition: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("production")}, + }, + }, + { + ServiceName: strptr("api-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + TaskDefinition: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/api-task:2"), + }, + }, + }, + }, + { + name: "MultipleClustersServices", + ecsData: &ecsDataStore{ + region: "us-east-1", + services: []ecsTypes.Service{ + { + ServiceName: strptr("service-1"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/cluster-1/service-1"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1"), + Status: strptr("RUNNING"), + TaskDefinition: strptr("arn:aws:ecs:us-east-1:123456789012:task-definition/task-1:1"), + }, + { + ServiceName: strptr("service-2"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/cluster-2/service-2"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2"), + Status: strptr("DRAINING"), + TaskDefinition: strptr("arn:aws:ecs:us-east-1:123456789012:task-definition/task-2:1"), + }, + }, + }, + clusterServiceARNsMap: map[string][]string{ + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1": { + "arn:aws:ecs:us-east-1:123456789012:service/cluster-1/service-1", + }, + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2": { + "arn:aws:ecs:us-east-1:123456789012:service/cluster-2/service-2", + }, + }, + expected: map[string][]ecsTypes.Service{ + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1": { + { + ServiceName: strptr("service-1"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/cluster-1/service-1"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-1"), + Status: strptr("RUNNING"), + TaskDefinition: strptr("arn:aws:ecs:us-east-1:123456789012:task-definition/task-1:1"), + }, + }, + "arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2": { + { + ServiceName: strptr("service-2"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/cluster-2/service-2"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/cluster-2"), + Status: strptr("DRAINING"), + TaskDefinition: strptr("arn:aws:ecs:us-east-1:123456789012:task-definition/task-2:1"), + }, + }, + }, + }, + } { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + RequestConcurrency: 1, + }, + } + + serviceMap, err := d.describeServices(ctx, tt.clusterServiceARNsMap) + require.NoError(t, err) + require.Equal(t, tt.expected, serviceMap) + }) + } +} + +func TestECSDiscoveryListTaskARNs(t *testing.T) { + ctx := context.Background() + + // iterate through the test cases + for _, tt := range []struct { + name string + ecsData *ecsDataStore + services []ecsTypes.Service + expected map[string][]string + }{ + { + name: "ServicesWithTasks", + ecsData: &ecsDataStore{ + region: "us-west-2", + tasks: []ecsTypes.Task{ + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:web-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + }, + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-2"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:web-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + }, + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-3"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:api-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/api-task:2"), + }, + }, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("web-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + }, + { + ServiceName: strptr("api-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + }, + }, + expected: map[string][]string{ + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service": { + "arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1", + "arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-2", + }, + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service": { + "arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-3", + }, + }, + }, + { + name: "ServiceWithNoTasks", + ecsData: &ecsDataStore{ + region: "us-west-2", + tasks: []ecsTypes.Task{}, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("empty-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/empty-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("RUNNING"), + }, + }, + expected: map[string][]string{ + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/empty-service": nil, + }, + }, + } { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + RequestConcurrency: 1, + }, + } + + taskMap, err := d.listTaskARNs(ctx, tt.services) + require.NoError(t, err) + require.Equal(t, tt.expected, taskMap) + }) + } +} + +func TestECSDiscoveryDescribeTasks(t *testing.T) { + ctx := context.Background() + + // iterate through the test cases + for _, tt := range []struct { + name string + ecsData *ecsDataStore + clusterARN string + taskARNsMap map[string][]string + expected map[string][]ecsTypes.Task + }{ + { + name: "TasksInCluster", + ecsData: &ecsDataStore{ + region: "us-west-2", + tasks: []ecsTypes.Task{ + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:web-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + LastStatus: strptr("RUNNING"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("production")}, + }, + }, + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-2"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:api-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/api-task:2"), + LastStatus: strptr("RUNNING"), + }, + }, + }, + clusterARN: "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster", + taskARNsMap: map[string][]string{ + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service": { + "arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1", + }, + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service": { + "arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-2", + }, + }, + expected: map[string][]ecsTypes.Task{ + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service": { + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:web-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + LastStatus: strptr("RUNNING"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("production")}, + }, + }, + }, + "arn:aws:ecs:us-west-2:123456789012:service/test-cluster/api-service": { + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-2"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Group: strptr("service:api-service"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/api-task:2"), + LastStatus: strptr("RUNNING"), + }, + }, + }, + }, + { + name: "EmptyTaskARNsMap", + ecsData: &ecsDataStore{ + region: "us-west-2", + tasks: []ecsTypes.Task{}, + }, + clusterARN: "arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster", + taskARNsMap: map[string][]string{}, + expected: map[string][]ecsTypes.Task{}, + }, + } { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + RequestConcurrency: 1, + }, + } + + taskMap, err := d.describeTasks(ctx, tt.clusterARN, tt.taskARNsMap) + require.NoError(t, err) + require.Equal(t, tt.expected, taskMap) + }) + } +} + +func TestECSDiscoveryRefresh(t *testing.T) { + ctx := context.Background() + + tests := []struct { + name string + ecsData *ecsDataStore + expected []*targetgroup.Group + }{ + { + name: "SingleClusterWithTasks", + ecsData: &ecsDataStore{ + region: "us-west-2", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("test-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + Tags: []ecsTypes.Tag{ + {Key: strptr("Environment"), Value: strptr("test")}, + }, + }, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("web-service"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + Tags: []ecsTypes.Tag{ + {Key: strptr("App"), Value: strptr("web")}, + }, + }, + }, + tasks: []ecsTypes.Task{ + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + Group: strptr("service:web-service"), + LaunchType: ecsTypes.LaunchTypeFargate, + LastStatus: strptr("RUNNING"), + DesiredStatus: strptr("RUNNING"), + HealthStatus: ecsTypes.HealthStatusHealthy, + AvailabilityZone: strptr("us-west-2a"), + PlatformFamily: strptr("Linux"), + PlatformVersion: strptr("1.4.0"), + Attachments: []ecsTypes.Attachment{ + { + Type: strptr("ElasticNetworkInterface"), + Details: []ecsTypes.KeyValuePair{ + {Name: strptr("subnetId"), Value: strptr("subnet-12345")}, + {Name: strptr("privateIPv4Address"), Value: strptr("10.0.1.100")}, + }, + }, + }, + Tags: []ecsTypes.Tag{ + {Key: strptr("Version"), Value: strptr("v1.0")}, + }, + }, + }, + }, + expected: []*targetgroup.Group{ + { + Source: "us-west-2", + Targets: []model.LabelSet{ + { + model.AddressLabel: model.LabelValue("10.0.1.100:80"), + "__meta_ecs_cluster": model.LabelValue("test-cluster"), + "__meta_ecs_cluster_arn": model.LabelValue("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + "__meta_ecs_service": model.LabelValue("web-service"), + "__meta_ecs_service_arn": model.LabelValue("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/web-service"), + "__meta_ecs_service_status": model.LabelValue("ACTIVE"), + "__meta_ecs_task_group": model.LabelValue("service:web-service"), + "__meta_ecs_task_arn": model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1"), + "__meta_ecs_task_definition": model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task-definition/web-task:1"), + "__meta_ecs_region": model.LabelValue("us-west-2"), + "__meta_ecs_availability_zone": model.LabelValue("us-west-2a"), + "__meta_ecs_subnet_id": model.LabelValue("subnet-12345"), + "__meta_ecs_ip_address": model.LabelValue("10.0.1.100"), + "__meta_ecs_launch_type": model.LabelValue("FARGATE"), + "__meta_ecs_desired_status": model.LabelValue("RUNNING"), + "__meta_ecs_last_status": model.LabelValue("RUNNING"), + "__meta_ecs_health_status": model.LabelValue("HEALTHY"), + "__meta_ecs_platform_family": model.LabelValue("Linux"), + "__meta_ecs_platform_version": model.LabelValue("1.4.0"), + "__meta_ecs_tag_cluster_Environment": model.LabelValue("test"), + "__meta_ecs_tag_service_App": model.LabelValue("web"), + "__meta_ecs_tag_task_Version": model.LabelValue("v1.0"), + }, + }, + }, + }, + }, + { + name: "NoTasks", + ecsData: &ecsDataStore{ + region: "us-east-1", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("empty-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/empty-cluster"), + Status: strptr("ACTIVE"), + }, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("empty-service"), + ServiceArn: strptr("arn:aws:ecs:us-east-1:123456789012:service/empty-cluster/empty-service"), + ClusterArn: strptr("arn:aws:ecs:us-east-1:123456789012:cluster/empty-cluster"), + Status: strptr("ACTIVE"), + }, + }, + tasks: []ecsTypes.Task{}, + }, + expected: []*targetgroup.Group{ + { + Source: "us-east-1", + }, + }, + }, + { + name: "TaskWithoutENI", + ecsData: &ecsDataStore{ + region: "us-west-2", + clusters: []ecsTypes.Cluster{ + { + ClusterName: strptr("test-cluster"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + }, + }, + services: []ecsTypes.Service{ + { + ServiceName: strptr("service-1"), + ServiceArn: strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/service-1"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + Status: strptr("ACTIVE"), + }, + }, + tasks: []ecsTypes.Task{ + { + TaskArn: strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-1"), + ClusterArn: strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"), + TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/task-def:1"), + Group: strptr("service:service-1"), + LaunchType: ecsTypes.LaunchTypeEc2, + LastStatus: strptr("RUNNING"), + DesiredStatus: strptr("RUNNING"), + HealthStatus: ecsTypes.HealthStatusHealthy, + AvailabilityZone: strptr("us-west-2a"), + // No attachments - should be skipped + Attachments: []ecsTypes.Attachment{}, + }, + }, + }, + expected: []*targetgroup.Group{ + { + Source: "us-west-2", + }, + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + client := newMockECSClient(tt.ecsData) + + d := &ECSDiscovery{ + ecs: client, + cfg: &ECSSDConfig{ + Region: tt.ecsData.region, + Port: 80, + RequestConcurrency: 1, + }, + } + + groups, err := d.refresh(ctx) + require.NoError(t, err) + require.Equal(t, tt.expected, groups) + }) + } +} + +// ECS client mock. +type mockECSClient struct { + ecsData ecsDataStore +} + +func newMockECSClient(ecsData *ecsDataStore) *mockECSClient { + client := mockECSClient{ + ecsData: *ecsData, + } + return &client +} + +func (m *mockECSClient) ListClusters(_ context.Context, _ *ecs.ListClustersInput, _ ...func(*ecs.Options)) (*ecs.ListClustersOutput, error) { + clusterArns := make([]string, 0, len(m.ecsData.clusters)) + for _, cluster := range m.ecsData.clusters { + clusterArns = append(clusterArns, *cluster.ClusterArn) + } + + return &ecs.ListClustersOutput{ + ClusterArns: clusterArns, + }, nil +} + +func (m *mockECSClient) DescribeClusters(_ context.Context, input *ecs.DescribeClustersInput, _ ...func(*ecs.Options)) (*ecs.DescribeClustersOutput, error) { + var clusters []ecsTypes.Cluster + for _, clusterArn := range input.Clusters { + for _, cluster := range m.ecsData.clusters { + if *cluster.ClusterArn == clusterArn { + clusters = append(clusters, cluster) + break + } + } + } + + return &ecs.DescribeClustersOutput{ + Clusters: clusters, + }, nil +} + +func (m *mockECSClient) ListServices(_ context.Context, input *ecs.ListServicesInput, _ ...func(*ecs.Options)) (*ecs.ListServicesOutput, error) { + var serviceArns []string + for _, service := range m.ecsData.services { + if *service.ClusterArn == *input.Cluster { + serviceArns = append(serviceArns, *service.ServiceArn) + } + } + + return &ecs.ListServicesOutput{ + ServiceArns: serviceArns, + }, nil +} + +func (m *mockECSClient) DescribeServices(_ context.Context, input *ecs.DescribeServicesInput, _ ...func(*ecs.Options)) (*ecs.DescribeServicesOutput, error) { + var services []ecsTypes.Service + for _, serviceArn := range input.Services { + for _, service := range m.ecsData.services { + if *service.ServiceArn == serviceArn && *service.ClusterArn == *input.Cluster { + services = append(services, service) + break + } + } + } + + return &ecs.DescribeServicesOutput{ + Services: services, + }, nil +} + +func (m *mockECSClient) ListTasks(_ context.Context, input *ecs.ListTasksInput, _ ...func(*ecs.Options)) (*ecs.ListTasksOutput, error) { + var taskArns []string + for _, task := range m.ecsData.tasks { + if *task.ClusterArn == *input.Cluster { + // If ServiceName is specified, filter by service + if input.ServiceName != nil { + expectedGroup := "service:" + *input.ServiceName + if task.Group != nil && *task.Group == expectedGroup { + taskArns = append(taskArns, *task.TaskArn) + } + } else { + taskArns = append(taskArns, *task.TaskArn) + } + } + } + + return &ecs.ListTasksOutput{ + TaskArns: taskArns, + }, nil +} + +func (m *mockECSClient) DescribeTasks(_ context.Context, input *ecs.DescribeTasksInput, _ ...func(*ecs.Options)) (*ecs.DescribeTasksOutput, error) { + var tasks []ecsTypes.Task + for _, taskArn := range input.Tasks { + for _, task := range m.ecsData.tasks { + if *task.TaskArn == taskArn && *task.ClusterArn == *input.Cluster { + tasks = append(tasks, task) + break + } + } + } + + return &ecs.DescribeTasksOutput{ + Tasks: tasks, + }, nil +} diff --git a/discovery/aws/metrics_aws.go b/discovery/aws/metrics_aws.go new file mode 100644 index 0000000000..4cb9b25041 --- /dev/null +++ b/discovery/aws/metrics_aws.go @@ -0,0 +1,32 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package aws + +import ( + "github.com/prometheus/prometheus/discovery" +) + +type awsMetrics struct { + refreshMetrics discovery.RefreshMetricsInstantiator +} + +var _ discovery.DiscovererMetrics = (*awsMetrics)(nil) + +// Register implements discovery.DiscovererMetrics. +func (*awsMetrics) Register() error { + return nil +} + +// Unregister implements discovery.DiscovererMetrics. +func (*awsMetrics) Unregister() {} diff --git a/discovery/aws/metrics_ecs.go b/discovery/aws/metrics_ecs.go new file mode 100644 index 0000000000..dde3483c06 --- /dev/null +++ b/discovery/aws/metrics_ecs.go @@ -0,0 +1,32 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package aws + +import ( + "github.com/prometheus/prometheus/discovery" +) + +type ecsMetrics struct { + refreshMetrics discovery.RefreshMetricsInstantiator +} + +var _ discovery.DiscovererMetrics = (*ecsMetrics)(nil) + +// Register implements discovery.DiscovererMetrics. +func (*ecsMetrics) Register() error { + return nil +} + +// Unregister implements discovery.DiscovererMetrics. +func (*ecsMetrics) Unregister() {} diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md index 7eab633987..cefc84e087 100644 --- a/docs/configuration/configuration.md +++ b/docs/configuration/configuration.md @@ -395,6 +395,10 @@ params: # authorization), proxy configurations, TLS options, custom HTTP headers, etc. [ ] +# List of AWS service discovery configurations. +aws_sd_configs: + [ - ... ] + # List of Azure service discovery configurations. azure_sd_configs: [ - ... ] @@ -812,6 +816,155 @@ http_headers: [ files: [, ...] ] ] ``` +### `` + +AWS SD configurations allow retrieving scrape targets from AWS services. +This is a unified service discovery that supports multiple AWS service types through the `role` parameter. + +One of the following `role` types can be configured to discover targets: + +#### `ec2` + +The `ec2` role discovers targets from AWS EC2 instances. The private IP address is used by default, but may be changed to +the public IP address with relabeling. + +The IAM credentials used must have the `ec2:DescribeInstances` permission to +discover scrape targets, and may optionally have the +`ec2:DescribeAvailabilityZones` permission if you want the availability zone ID +available as a label (see below). + +The following meta labels are available on targets during [relabeling](#relabel_config): + +* `__meta_ec2_ami`: the EC2 Amazon Machine Image +* `__meta_ec2_architecture`: the architecture of the instance +* `__meta_ec2_availability_zone`: the availability zone in which the instance is running +* `__meta_ec2_availability_zone_id`: the [availability zone ID](https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html) in which the instance is running (requires `ec2:DescribeAvailabilityZones`) +* `__meta_ec2_instance_id`: the EC2 instance ID +* `__meta_ec2_instance_lifecycle`: the lifecycle of the EC2 instance, set only for 'spot' or 'scheduled' instances, absent otherwise +* `__meta_ec2_instance_state`: the state of the EC2 instance +* `__meta_ec2_instance_type`: the type of the EC2 instance +* `__meta_ec2_ipv6_addresses`: comma separated list of IPv6 addresses assigned to the instance's network interfaces, if present +* `__meta_ec2_owner_id`: the ID of the AWS account that owns the EC2 instance +* `__meta_ec2_platform`: the Operating System platform, set to 'windows' on Windows servers, absent otherwise +* `__meta_ec2_primary_ipv6_addresses`: comma separated list of the Primary IPv6 addresses of the instance, if present. The list is ordered based on the position of each corresponding network interface in the attachment order. +* `__meta_ec2_primary_subnet_id`: the subnet ID of the primary network interface, if available +* `__meta_ec2_private_dns_name`: the private DNS name of the instance, if available +* `__meta_ec2_private_ip`: the private IP address of the instance, if present +* `__meta_ec2_public_dns_name`: the public DNS name of the instance, if available +* `__meta_ec2_public_ip`: the public IP address of the instance, if available +* `__meta_ec2_region`: the region of the instance +* `__meta_ec2_subnet_id`: comma separated list of subnets IDs in which the instance is running, if available +* `__meta_ec2_tag_`: each tag value of the instance +* `__meta_ec2_vpc_id`: the ID of the VPC in which the instance is running, if available + +#### `lightsail` + +The `lightsail` role discovers targets from [AWS Lightsail](https://aws.amazon.com/lightsail/) +instances. The private IP address is used by default, but may be changed to +the public IP address with relabeling. + +The following meta labels are available on targets during [relabeling](#relabel_config): + +* `__meta_lightsail_availability_zone`: the availability zone in which the instance is running +* `__meta_lightsail_blueprint_id`: the Lightsail blueprint ID +* `__meta_lightsail_bundle_id`: the Lightsail bundle ID +* `__meta_lightsail_instance_name`: the name of the Lightsail instance +* `__meta_lightsail_instance_state`: the state of the Lightsail instance +* `__meta_lightsail_instance_support_code`: the support code of the Lightsail instance +* `__meta_lightsail_ipv6_addresses`: comma separated list of IPv6 addresses assigned to the instance's network interfaces, if present +* `__meta_lightsail_private_ip`: the private IP address of the instance +* `__meta_lightsail_public_ip`: the public IP address of the instance, if available +* `__meta_lightsail_region`: the region of the instance +* `__meta_lightsail_tag_`: each tag value of the instance + +#### `ecs` + +The `ecs` role discovers targets from AWS ECS containers. The private IP address is used by default, but may be changed to +the public IP address with relabeling. + +The IAM credentials used must have the following permissions to discover +scrape targets: + +- `ecs:ListClusters` +- `ecs:DescribeClusters` +- `ecs:ListServices` +- `ecs:DescribeServices` +- `ecs:ListTasks` +- `ecs:DescribeTasks` + +The following meta labels are available on targets during [relabeling](#relabel_config): + +* `__meta_ecs_cluster`: the name of the ECS cluster +* `__meta_ecs_cluster_arn`: the ARN of the ECS cluster +* `__meta_ecs_service`: the name of the ECS service +* `__meta_ecs_service_arn`: the ARN of the ECS service +* `__meta_ecs_service_status`: the status of the ECS service +* `__meta_ecs_task_group`: the ECS task group (typically service:service-name) +* `__meta_ecs_task_arn`: the ARN of the ECS task +* `__meta_ecs_task_definition`: the ARN of the ECS task definition +* `__meta_ecs_ip_address`: the private IP address of the task +* `__meta_ecs_launch_type`: the launch type of the task (EC2 or Fargate) +* `__meta_ecs_desired_status`: the desired status of the task +* `__meta_ecs_last_status`: the last known status of the task +* `__meta_ecs_health_status`: the health status of the task +* `__meta_ecs_platform_family`: the platform family (e.g., Linux, Windows) +* `__meta_ecs_platform_version`: the platform version +* `__meta_ecs_subnet_id`: the subnet ID where the task is running +* `__meta_ecs_availability_zone`: the availability zone where the task is running +* `__meta_ecs_region`: the AWS region +* `__meta_ecs_tag_cluster_`: each cluster tag value, keyed by tag name +* `__meta_ecs_tag_service_`: each service tag value, keyed by tag name +* `__meta_ecs_tag_task_`: each task tag value, keyed by tag name + +See below for the configuration options for AWS discovery: + +```yaml +# The AWS role to use for service discovery. +# Must be one of: ec2, lightsail, or ecs. +role: + +# The AWS region. If blank, the region from the instance metadata is used. +[ region: ] + +# Custom endpoint to be used. +[ endpoint: ] + +# AWS access key ID. If blank, the environment variable AWS_ACCESS_KEY_ID is used. +[ access_key: ] + +# AWS secret access key. If blank, the environment variable AWS_SECRET_ACCESS_KEY is used. +[ secret_key: ] + +# Named AWS profile used to authenticate. +[ profile: ] + +# AWS Role ARN, an alternative to using AWS API keys. +[ role_arn: ] + +# Refresh interval to re-read the targets list. +[ refresh_interval: | default = 60s ] + +# The port to scrape metrics from. If using the public IP address, this must +# instead be specified in the relabeling rule. +[ port: | default = 80 ] + +# Filters can be used optionally to filter the instance list by other criteria (ec2 role only). +# Available filter criteria can be found here: +# https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html +# Filter API documentation: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_Filter.html +filters: + [ - name: + values: , [...] ] + +# List of ECS cluster ARNs to discover (ecs role only). If empty, all clusters in the region are discovered. +# This can significantly improve performance when you only need to monitor specific clusters. +[ clusters: [, ...] ] + +# HTTP client settings, including authentication methods (such as basic auth and +# authorization), proxy configurations, TLS options, custom HTTP headers, etc. +[ ] +``` + ### `` Azure SD configurations allow retrieving scrape targets from Azure VMs. @@ -2885,6 +3038,10 @@ sigv4: # authorization), proxy configurations, TLS options, custom HTTP headers, etc. [ ] +# List of AWS service discovery configurations. +aws_sd_configs: + [ - ... ] + # List of Azure service discovery configurations. azure_sd_configs: [ - ... ] diff --git a/go.mod b/go.mod index 2f1b4e5039..465aa52b85 100644 --- a/go.mod +++ b/go.mod @@ -15,6 +15,7 @@ require ( github.com/aws/aws-sdk-go-v2/config v1.31.17 github.com/aws/aws-sdk-go-v2/credentials v1.18.21 github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0 + github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2 github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4 github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 github.com/aws/smithy-go v1.23.2 diff --git a/go.sum b/go.sum index 64bc639857..70ee342e10 100644 --- a/go.sum +++ b/go.sum @@ -65,6 +65,8 @@ github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEG github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc= github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0 h1:5qBb1XV/D18qtCHd3bmmxoVglI+fZ4QWuS/EB8kIXYQ= github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0/go.mod h1:NDdDLLW5PtLLXN661gKcvJvqAH5OBXsfhMlmKVu1/pY= +github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2 h1:oeICOX/+D0XXV1aMYJPXVe3CO37zYr7fB6HFgxchleU= +github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2/go.mod h1:rrhqfkXfa2DSNq0RyFhnnFEAyI+yJB4+2QlZKeJvMjs= github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o= github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo= github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM= From b60e33ae4ab870570f5b6241063e344cf2bbca7f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 7 Nov 2025 11:12:50 +0000 Subject: [PATCH 023/439] chore(deps): bump github.com/prometheus/sigv4 from 0.2.1 to 0.3.0 Bumps [github.com/prometheus/sigv4](https://github.com/prometheus/sigv4) from 0.2.1 to 0.3.0. - [Release notes](https://github.com/prometheus/sigv4/releases) - [Changelog](https://github.com/prometheus/sigv4/blob/main/RELEASE.md) - [Commits](https://github.com/prometheus/sigv4/compare/v0.2.1...v0.3.0) --- updated-dependencies: - dependency-name: github.com/prometheus/sigv4 dependency-version: 0.3.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 465aa52b85..55b8d2ce1f 100644 --- a/go.mod +++ b/go.mod @@ -61,7 +61,7 @@ require ( github.com/prometheus/common v0.67.2 github.com/prometheus/common/assets v0.2.0 github.com/prometheus/exporter-toolkit v0.15.0 - github.com/prometheus/sigv4 v0.2.1 + github.com/prometheus/sigv4 v0.3.0 github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35 github.com/shurcooL/httpfs v0.0.0-20230704072500-f1e31cf0ba5c github.com/stackitcloud/stackit-sdk-go/core v0.17.3 diff --git a/go.sum b/go.sum index 70ee342e10..2c0042edbb 100644 --- a/go.sum +++ b/go.sum @@ -460,8 +460,8 @@ github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsT github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= -github.com/prometheus/sigv4 v0.2.1 h1:hl8D3+QEzU9rRmbKIRwMKRwaFGyLkbPdH5ZerglRHY0= -github.com/prometheus/sigv4 v0.2.1/go.mod h1:ySk6TahIlsR2sxADuHy4IBFhwEjRGGsfbbLGhFYFj6Q= +github.com/prometheus/sigv4 v0.3.0 h1:QIG7nTbu0JTnNidGI1Uwl5AGVIChWUACxn2B/BQ1kms= +github.com/prometheus/sigv4 v0.3.0/go.mod h1:fKtFYDus2M43CWKMNtGvFNHGXnAJJEGZbiYCmVp/F8I= github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg= github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA= github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= From 0fbe5af9618d728d56ccf8bfc06179c816306857 Mon Sep 17 00:00:00 2001 From: Charles Korn Date: Mon, 10 Nov 2025 11:55:30 +1100 Subject: [PATCH 024/439] Fix heading for `limitk` docs Signed-off-by: Charles Korn --- docs/querying/operators.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/operators.md b/docs/querying/operators.md index 9aaae3fbcf..b320d8e86e 100644 --- a/docs/querying/operators.md +++ b/docs/querying/operators.md @@ -425,7 +425,7 @@ To get the 5 instances with the highest memory consumption across all instances topk(5, memory_consumption_bytes) -#### `limitk` and `limit_ratio` +#### `limitk` `limitk(k, v)` returns a subset of `k` input samples, including the original labels in the result vector. From 393ab9e12e302a334e69ba066e61275ad2d4a5f2 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 31 Oct 2025 13:02:24 +0000 Subject: [PATCH 025/439] [TEST] TSDB: More realistic BenchmarkIntersect 100,000 matchers is not something that could happen while using Prometheus. Signed-off-by: Bryan Boreham --- tsdb/index/postings_test.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tsdb/index/postings_test.go b/tsdb/index/postings_test.go index 3ba523c22f..fbbfc95a47 100644 --- a/tsdb/index/postings_test.go +++ b/tsdb/index/postings_test.go @@ -347,13 +347,13 @@ func BenchmarkIntersect(t *testing.B) { } }) - // Many matchers(k >> n). + // Many matchers. t.Run("ManyPostings", func(bench *testing.B) { var lps []*ListPostings var refs [][]storage.SeriesRef - // Create 100000 matchers(k=100000), making sure all memory allocation is done before starting the loop. - for range 100000 { + // Create 100 matchers, making sure all memory allocation is done before starting the loop. + for range 100 { var temp []storage.SeriesRef for j := storage.SeriesRef(1); j < 100; j++ { temp = append(temp, j) From be8307db58ac310f49fb07404a30f1f33061ef97 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 31 Oct 2025 16:20:18 +0000 Subject: [PATCH 026/439] [TEST] Refactor BenchmarkIntersect to reduce memory allocations Extract functions which pre-create all the memory for the benchmark itself. Signed-off-by: Bryan Boreham --- tsdb/index/postings_test.go | 96 ++++++++++++++++--------------------- 1 file changed, 41 insertions(+), 55 deletions(-) diff --git a/tsdb/index/postings_test.go b/tsdb/index/postings_test.go index fbbfc95a47..56c0f02455 100644 --- a/tsdb/index/postings_test.go +++ b/tsdb/index/postings_test.go @@ -285,92 +285,78 @@ func consumePostings(p Postings) error { return p.Err() } +// Create ListPostings for a benchmark, collecting the original sets of references +// so they can be reset without additional memory allocations. +func createPostings(lps *[]*ListPostings, refs *[][]storage.SeriesRef, params ...storage.SeriesRef) { + var temp []storage.SeriesRef + for i := 0; i < len(params); i += 3 { + for j := params[i]; j < params[i+1]; j += params[i+2] { + temp = append(temp, j) + } + } + *lps = append(*lps, newListPostings(temp...)) + *refs = append(*refs, temp) +} + +// Reset the ListPostings to their original values each time round the benchmark loop. +func resetPostings(its []Postings, lps []*ListPostings, refs [][]storage.SeriesRef) { + for j := range refs { + lps[j].list = refs[j] + its[j] = lps[j] + } +} + func BenchmarkIntersect(t *testing.B) { t.Run("LongPostings1", func(bench *testing.B) { - var a, b, c, d []storage.SeriesRef - - for i := 0; i < 10000000; i += 2 { - a = append(a, storage.SeriesRef(i)) - } - for i := 5000000; i < 5000100; i += 4 { - b = append(b, storage.SeriesRef(i)) - } - for i := 5090000; i < 5090600; i += 4 { - b = append(b, storage.SeriesRef(i)) - } - for i := 4990000; i < 5100000; i++ { - c = append(c, storage.SeriesRef(i)) - } - for i := 4000000; i < 6000000; i++ { - d = append(d, storage.SeriesRef(i)) - } + var lps []*ListPostings + var refs [][]storage.SeriesRef + createPostings(&lps, &refs, 0, 10000000, 2) + createPostings(&lps, &refs, 5000000, 5000100, 4, 5090000, 5090600, 4) + createPostings(&lps, &refs, 4990000, 5100000, 1) + createPostings(&lps, &refs, 4000000, 6000000, 1) + its := make([]Postings, len(refs)) bench.ResetTimer() bench.ReportAllocs() for bench.Loop() { - i1 := newListPostings(a...) - i2 := newListPostings(b...) - i3 := newListPostings(c...) - i4 := newListPostings(d...) - if err := consumePostings(Intersect(i1, i2, i3, i4)); err != nil { + resetPostings(its, lps, refs) + if err := consumePostings(Intersect(its...)); err != nil { bench.Fatal(err) } } }) t.Run("LongPostings2", func(bench *testing.B) { - var a, b, c, d []storage.SeriesRef - - for i := range 12500000 { - a = append(a, storage.SeriesRef(i)) - } - for i := 7500000; i < 12500000; i++ { - b = append(b, storage.SeriesRef(i)) - } - for i := 9000000; i < 20000000; i++ { - c = append(c, storage.SeriesRef(i)) - } - for i := 10000000; i < 12000000; i++ { - d = append(d, storage.SeriesRef(i)) - } + var lps []*ListPostings + var refs [][]storage.SeriesRef + createPostings(&lps, &refs, 0, 12500000, 1) + createPostings(&lps, &refs, 7500000, 12500000, 1) + createPostings(&lps, &refs, 9000000, 20000000, 1) + createPostings(&lps, &refs, 10000000, 12000000, 1) + its := make([]Postings, len(refs)) bench.ResetTimer() bench.ReportAllocs() for bench.Loop() { - i1 := newListPostings(a...) - i2 := newListPostings(b...) - i3 := newListPostings(c...) - i4 := newListPostings(d...) - if err := consumePostings(Intersect(i1, i2, i3, i4)); err != nil { + resetPostings(its, lps, refs) + if err := consumePostings(Intersect(its...)); err != nil { bench.Fatal(err) } } }) - // Many matchers. t.Run("ManyPostings", func(bench *testing.B) { var lps []*ListPostings var refs [][]storage.SeriesRef - - // Create 100 matchers, making sure all memory allocation is done before starting the loop. for range 100 { - var temp []storage.SeriesRef - for j := storage.SeriesRef(1); j < 100; j++ { - temp = append(temp, j) - } - lps = append(lps, newListPostings(temp...)) - refs = append(refs, temp) + createPostings(&lps, &refs, 1, 100, 1) } its := make([]Postings, len(refs)) bench.ResetTimer() bench.ReportAllocs() for bench.Loop() { - // Reset the ListPostings to their original values each time round the loop. - for j := range refs { - lps[j].list = refs[j] - its[j] = lps[j] - } + resetPostings(its, lps, refs) if err := consumePostings(Intersect(its...)); err != nil { bench.Fatal(err) } From 0e1e7441e45f5ea208629541c5db455a3f80eaae Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 31 Oct 2025 15:28:15 +0000 Subject: [PATCH 027/439] [PERF] TSDB: ListPostings: check next item before binary search It is fairly common that the next item is the one we want, and cheap to check. We could also start the binary search one position on, but strangely that slows it down. Signed-off-by: Bryan Boreham --- tsdb/index/postings.go | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/tsdb/index/postings.go b/tsdb/index/postings.go index d5a17c3daa..0a0ac6a1fd 100644 --- a/tsdb/index/postings.go +++ b/tsdb/index/postings.go @@ -851,15 +851,17 @@ func (it *ListPostings) Seek(x storage.SeriesRef) bool { return false } - // Do binary search between current position and end. - i, _ := slices.BinarySearch(it.list, x) - if i < len(it.list) { - it.cur = it.list[i] - it.list = it.list[i+1:] - return true + i := 0 // Check the next item in the list, otherwise binary search between current position and end. + if it.list[0] < x { + i, _ = slices.BinarySearch(it.list, x) + if i >= len(it.list) { // Off the end - terminate the iterator. + it.list = nil + return false + } } - it.list = nil - return false + it.cur = it.list[i] + it.list = it.list[i+1:] + return true } func (*ListPostings) Err() error { From c1e0ab11c67c9ddc105a8d3479503a6dc05c812e Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 31 Oct 2025 15:39:23 +0000 Subject: [PATCH 028/439] [PERF] TSDB: Speed up intersectPostings.Next Check if the next position is already a match, in which case we don't have to call `Seek`. Signed-off-by: Bryan Boreham --- tsdb/index/postings.go | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/tsdb/index/postings.go b/tsdb/index/postings.go index 0a0ac6a1fd..665a241c34 100644 --- a/tsdb/index/postings.go +++ b/tsdb/index/postings.go @@ -641,15 +641,28 @@ func (it *intersectPostings) Seek(target storage.SeriesRef) bool { } func (it *intersectPostings) Next() bool { - target := it.current - for _, p := range it.postings { + // Move forward the first Postings and take its value as the target to match. + if !it.postings[0].Next() { + return false + } + target := it.postings[0].At() + allEqual := true + for _, p := range it.postings[1:] { // Now move forward all the other ones and check if they match. if !p.Next() { return false } - if p.At() > target { - target = p.At() + at := p.At() + if at > target { // This one is past the target, so pick up a new target to Seek at the end. + target = at + allEqual = false + } else if at < target { // This one needs to Seek to the target, but carry on with other postings in case they have an even higher target. + allEqual = false } } + if allEqual { + it.current = target + return true + } return it.Seek(target) } From 5560397a7001b49829bc3a4272f7fceaa1d7ca13 Mon Sep 17 00:00:00 2001 From: smallfish Date: Fri, 21 Feb 2025 13:29:09 +0800 Subject: [PATCH 029/439] promtool: add dump-series Signed-off-by: smallfish --- cmd/promtool/main.go | 11 +++++ cmd/promtool/tsdb.go | 25 ++++++++++ cmd/promtool/tsdb_test.go | 92 +++++++++++++++++++++++++++++++++++ docs/command-line/promtool.md | 27 ++++++++++ 4 files changed, 155 insertions(+) diff --git a/cmd/promtool/main.go b/cmd/promtool/main.go index 00280500ed..434ef7e48e 100644 --- a/cmd/promtool/main.go +++ b/cmd/promtool/main.go @@ -259,6 +259,13 @@ func main() { dumpMaxTime := tsdbDumpCmd.Flag("max-time", "Maximum timestamp to dump.").Default(strconv.FormatInt(math.MaxInt64, 10)).Int64() dumpMatch := tsdbDumpCmd.Flag("match", "Series selector. Can be specified multiple times.").Default("{__name__=~'(?s:.*)'}").Strings() + tsdbDumpSeriesCmd := tsdbCmd.Command("dump-series", "Dump series (identified by a unique set of labels) from a TSDB into JSON format.") + dumpSeriesPath := tsdbDumpSeriesCmd.Arg("db path", "Database path (default is "+defaultDBPath+").").Default(defaultDBPath).String() + dumpSeriesSandboxDirRoot := tsdbDumpSeriesCmd.Flag("sandbox-dir-root", "Root directory where a sandbox directory would be created in case WAL replay generates chunks. The sandbox directory is cleaned up at the end.").Default(defaultDBPath).String() + dumpSeriesMinTime := tsdbDumpSeriesCmd.Flag("min-time", "Minimum timestamp to dump.").Default(strconv.FormatInt(math.MinInt64, 10)).Int64() + dumpSeriesMaxTime := tsdbDumpSeriesCmd.Flag("max-time", "Maximum timestamp to dump.").Default(strconv.FormatInt(math.MaxInt64, 10)).Int64() + dumpSeriesMatch := tsdbDumpSeriesCmd.Flag("match", "Series selector. Can be specified multiple times.").Default("{__name__=~'(?s:.*)'}").Strings() + tsdbDumpOpenMetricsCmd := tsdbCmd.Command("dump-openmetrics", "[Experimental] Dump samples from a TSDB into OpenMetrics text format, excluding native histograms and staleness markers, which are not representable in OpenMetrics.") dumpOpenMetricsPath := tsdbDumpOpenMetricsCmd.Arg("db path", "Database path (default is "+defaultDBPath+").").Default(defaultDBPath).String() dumpOpenMetricsSandboxDirRoot := tsdbDumpOpenMetricsCmd.Flag("sandbox-dir-root", "Root directory where a sandbox directory will be created, this sandbox is used in case WAL replay generates chunks (default is the database path). The sandbox is cleaned up at the end.").String() @@ -420,6 +427,10 @@ func main() { case tsdbDumpCmd.FullCommand(): os.Exit(checkErr(dumpSamples(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSet))) + + case tsdbDumpSeriesCmd.FullCommand(): + os.Exit(checkErr(dumpSamples(ctx, *dumpSeriesPath, *dumpSeriesSandboxDirRoot, *dumpSeriesMinTime, *dumpSeriesMaxTime, *dumpSeriesMatch, formatSeriesSetToJSON))) + case tsdbDumpOpenMetricsCmd.FullCommand(): os.Exit(checkErr(dumpSamples(ctx, *dumpOpenMetricsPath, *dumpOpenMetricsSandboxDirRoot, *dumpOpenMetricsMinTime, *dumpOpenMetricsMaxTime, *dumpOpenMetricsMatch, formatSeriesSetOpenMetrics))) // TODO(aSquare14): Work on adding support for custom block size. diff --git a/cmd/promtool/tsdb.go b/cmd/promtool/tsdb.go index 6a62e2e8bc..996bbc390f 100644 --- a/cmd/promtool/tsdb.go +++ b/cmd/promtool/tsdb.go @@ -17,6 +17,7 @@ import ( "bufio" "bytes" "context" + "encoding/json" "errors" "fmt" "io" @@ -798,6 +799,30 @@ func CondensedString(ls labels.Labels) string { return b.String() } +func formatSeriesSetToJSON(ss storage.SeriesSet) error { + seriesCache := make(map[string]struct{}) + for ss.Next() { + series := ss.At() + lbs := series.Labels() + + b, err := json.Marshal(lbs) + if err != nil { + return err + } + + if len(b) == 0 { + continue + } + + s := string(b) + if _, ok := seriesCache[s]; !ok { + fmt.Println(s) + seriesCache[s] = struct{}{} + } + } + return nil +} + func formatSeriesSetOpenMetrics(ss storage.SeriesSet) error { for ss.Next() { series := ss.At() diff --git a/cmd/promtool/tsdb_test.go b/cmd/promtool/tsdb_test.go index e745a3fe7a..53d7a637dd 100644 --- a/cmd/promtool/tsdb_test.go +++ b/cmd/promtool/tsdb_test.go @@ -176,6 +176,98 @@ func sortLines(buf string) string { return strings.Join(lines, "\n") } +// getDumpedSeries dumps series and returns them. +func getDumpedSeries(t *testing.T, path string, mint, maxt int64, match []string, formatter SeriesSetFormatter) string { + t.Helper() + + oldStdout := os.Stdout + r, w, _ := os.Pipe() + os.Stdout = w + + err := dumpSamples( + context.Background(), + path, + t.TempDir(), + mint, + maxt, + match, + formatter, + ) + require.NoError(t, err) + + w.Close() + os.Stdout = oldStdout + + var buf bytes.Buffer + io.Copy(&buf, r) + return buf.String() +} + +func TestTSDBDumpSeries(t *testing.T) { + storage := promqltest.LoadedStorage(t, ` + load 1m + metric{foo="bar", baz="abc"} 1 2 3 4 5 + heavy_metric{foo="bar"} 5 4 3 2 1 + heavy_metric{foo="foo"} 5 4 3 2 1 + `) + + expectedArray := []string{ + `{"__name__":"heavy_metric","foo":"bar"} +`, + `{"__name__":"heavy_metric","foo":"foo"} +`, + `{"__name__":"metric","baz":"abc","foo":"bar"} +`, + } + + tests := []struct { + name string + mint int64 + maxt int64 + match []string + expected string + }{ + { + name: "default match", + match: []string{"{__name__=~'(?s:.*)'}"}, + expected: strings.Join(expectedArray, ""), + }, + { + name: "same matcher twice", + match: []string{"{foo=~'.+'}", "{foo=~'.+'}"}, + expected: strings.Join(expectedArray, ""), + }, + { + name: "no duplication", + match: []string{"{__name__=~'(?s:.*)'}", "{baz='abc'}"}, + expected: strings.Join(expectedArray, ""), + }, + { + name: "well merged", + match: []string{"{__name__='heavy_metric'}", "{baz='abc'}"}, + expected: strings.Join(expectedArray, ""), + }, + { + name: "multi matchers", + match: []string{"{__name__='heavy_metric',foo='foo'}", "{__name__='metric'}"}, + expected: expectedArray[1] + expectedArray[2], + }, + { + name: "with reduced mint and maxt", + match: []string{"{__name__='metric'}"}, + expected: expectedArray[2], + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + dumpedSeries := getDumpedSeries(t, storage.Dir(), tt.mint, tt.maxt, tt.match, formatSeriesSetToJSON) + expectedSeries := normalizeNewLine([]byte(tt.expected)) + // Sort both, because Prometheus does not guarantee the output order. + require.Equal(t, sortLines(string(expectedSeries)), sortLines(dumpedSeries)) + }) + } +} + func TestTSDBDumpOpenMetrics(t *testing.T) { storage := promqltest.LoadedStorage(t, ` load 1m diff --git a/docs/command-line/promtool.md b/docs/command-line/promtool.md index ab675e6345..2fa7976a76 100644 --- a/docs/command-line/promtool.md +++ b/docs/command-line/promtool.md @@ -597,6 +597,33 @@ Dump samples from a TSDB. +##### `promtool tsdb dump-series` + +Dump series (identified by a unique set of labels) from a TSDB into JSON format. + + + +###### Flags + +| Flag | Description | Default | +| --- | --- | --- | +| --sandbox-dir-root | Root directory where a sandbox directory would be created in case WAL replay generates chunks. The sandbox directory is cleaned up at the end. | `data/` | +| --min-time | Minimum timestamp to dump. | `-9223372036854775808` | +| --max-time | Maximum timestamp to dump. | `9223372036854775807` | +| --match ... | Series selector. Can be specified multiple times. | `{__name__=~'(?s:.*)'}` | + + + + +###### Arguments + +| Argument | Description | Default | +| --- | --- | --- | +| db path | Database path (default is data/). | `data/` | + + + + ##### `promtool tsdb dump-openmetrics` [Experimental] Dump samples from a TSDB into OpenMetrics text format, excluding native histograms and staleness markers, which are not representable in OpenMetrics. From beea578b20da796ce9d6ad48b38b6c8181d440d8 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Mon, 10 Nov 2025 17:26:31 +0000 Subject: [PATCH 030/439] Make dump-series a --format flag on the dump command Signed-off-by: Bryan Boreham --- cmd/promtool/main.go | 21 ++++++++------------- cmd/promtool/tsdb.go | 2 +- cmd/promtool/tsdb_test.go | 4 ++-- docs/command-line/promtool.md | 30 ++---------------------------- 4 files changed, 13 insertions(+), 44 deletions(-) diff --git a/cmd/promtool/main.go b/cmd/promtool/main.go index 434ef7e48e..1b36d518ee 100644 --- a/cmd/promtool/main.go +++ b/cmd/promtool/main.go @@ -252,19 +252,13 @@ func main() { listHumanReadable := tsdbListCmd.Flag("human-readable", "Print human readable values.").Short('r').Bool() listPath := tsdbListCmd.Arg("db path", "Database path (default is "+defaultDBPath+").").Default(defaultDBPath).String() - tsdbDumpCmd := tsdbCmd.Command("dump", "Dump samples from a TSDB.") + tsdbDumpCmd := tsdbCmd.Command("dump", "Dump data (series+samples or optionally just series) from a TSDB.") dumpPath := tsdbDumpCmd.Arg("db path", "Database path (default is "+defaultDBPath+").").Default(defaultDBPath).String() dumpSandboxDirRoot := tsdbDumpCmd.Flag("sandbox-dir-root", "Root directory where a sandbox directory will be created, this sandbox is used in case WAL replay generates chunks (default is the database path). The sandbox is cleaned up at the end.").String() dumpMinTime := tsdbDumpCmd.Flag("min-time", "Minimum timestamp to dump.").Default(strconv.FormatInt(math.MinInt64, 10)).Int64() dumpMaxTime := tsdbDumpCmd.Flag("max-time", "Maximum timestamp to dump.").Default(strconv.FormatInt(math.MaxInt64, 10)).Int64() dumpMatch := tsdbDumpCmd.Flag("match", "Series selector. Can be specified multiple times.").Default("{__name__=~'(?s:.*)'}").Strings() - - tsdbDumpSeriesCmd := tsdbCmd.Command("dump-series", "Dump series (identified by a unique set of labels) from a TSDB into JSON format.") - dumpSeriesPath := tsdbDumpSeriesCmd.Arg("db path", "Database path (default is "+defaultDBPath+").").Default(defaultDBPath).String() - dumpSeriesSandboxDirRoot := tsdbDumpSeriesCmd.Flag("sandbox-dir-root", "Root directory where a sandbox directory would be created in case WAL replay generates chunks. The sandbox directory is cleaned up at the end.").Default(defaultDBPath).String() - dumpSeriesMinTime := tsdbDumpSeriesCmd.Flag("min-time", "Minimum timestamp to dump.").Default(strconv.FormatInt(math.MinInt64, 10)).Int64() - dumpSeriesMaxTime := tsdbDumpSeriesCmd.Flag("max-time", "Maximum timestamp to dump.").Default(strconv.FormatInt(math.MaxInt64, 10)).Int64() - dumpSeriesMatch := tsdbDumpSeriesCmd.Flag("match", "Series selector. Can be specified multiple times.").Default("{__name__=~'(?s:.*)'}").Strings() + dumpFormat := tsdbDumpCmd.Flag("format", "Output format of the dump (prom (default) or seriesjson).").Default("prom").Enum("prom", "seriesjson") tsdbDumpOpenMetricsCmd := tsdbCmd.Command("dump-openmetrics", "[Experimental] Dump samples from a TSDB into OpenMetrics text format, excluding native histograms and staleness markers, which are not representable in OpenMetrics.") dumpOpenMetricsPath := tsdbDumpOpenMetricsCmd.Arg("db path", "Database path (default is "+defaultDBPath+").").Default(defaultDBPath).String() @@ -426,13 +420,14 @@ func main() { os.Exit(checkErr(listBlocks(*listPath, *listHumanReadable))) case tsdbDumpCmd.FullCommand(): - os.Exit(checkErr(dumpSamples(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSet))) - - case tsdbDumpSeriesCmd.FullCommand(): - os.Exit(checkErr(dumpSamples(ctx, *dumpSeriesPath, *dumpSeriesSandboxDirRoot, *dumpSeriesMinTime, *dumpSeriesMaxTime, *dumpSeriesMatch, formatSeriesSetToJSON))) + if *dumpFormat == "seriesjson" { + os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSetToJSON))) + } else { + os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSet))) + } case tsdbDumpOpenMetricsCmd.FullCommand(): - os.Exit(checkErr(dumpSamples(ctx, *dumpOpenMetricsPath, *dumpOpenMetricsSandboxDirRoot, *dumpOpenMetricsMinTime, *dumpOpenMetricsMaxTime, *dumpOpenMetricsMatch, formatSeriesSetOpenMetrics))) + os.Exit(checkErr(dumpTSDBData(ctx, *dumpOpenMetricsPath, *dumpOpenMetricsSandboxDirRoot, *dumpOpenMetricsMinTime, *dumpOpenMetricsMaxTime, *dumpOpenMetricsMatch, formatSeriesSetOpenMetrics))) // TODO(aSquare14): Work on adding support for custom block size. case openMetricsImportCmd.FullCommand(): os.Exit(backfillOpenMetrics(*importFilePath, *importDBPath, *importHumanReadable, *importQuiet, *maxBlockDuration, *openMetricsLabels)) diff --git a/cmd/promtool/tsdb.go b/cmd/promtool/tsdb.go index 996bbc390f..ab822f2e7e 100644 --- a/cmd/promtool/tsdb.go +++ b/cmd/promtool/tsdb.go @@ -711,7 +711,7 @@ func analyzeCompaction(ctx context.Context, block tsdb.BlockReader, indexr tsdb. type SeriesSetFormatter func(series storage.SeriesSet) error -func dumpSamples(ctx context.Context, dbDir, sandboxDirRoot string, mint, maxt int64, match []string, formatter SeriesSetFormatter) (err error) { +func dumpTSDBData(ctx context.Context, dbDir, sandboxDirRoot string, mint, maxt int64, match []string, formatter SeriesSetFormatter) (err error) { db, err := tsdb.OpenDBReadOnly(dbDir, sandboxDirRoot, nil) if err != nil { return err diff --git a/cmd/promtool/tsdb_test.go b/cmd/promtool/tsdb_test.go index 53d7a637dd..5b4d62b75b 100644 --- a/cmd/promtool/tsdb_test.go +++ b/cmd/promtool/tsdb_test.go @@ -63,7 +63,7 @@ func getDumpedSamples(t *testing.T, databasePath, sandboxDirRoot string, mint, m r, w, _ := os.Pipe() os.Stdout = w - err := dumpSamples( + err := dumpTSDBData( context.Background(), databasePath, sandboxDirRoot, @@ -184,7 +184,7 @@ func getDumpedSeries(t *testing.T, path string, mint, maxt int64, match []string r, w, _ := os.Pipe() os.Stdout = w - err := dumpSamples( + err := dumpTSDBData( context.Background(), path, t.TempDir(), diff --git a/docs/command-line/promtool.md b/docs/command-line/promtool.md index 2fa7976a76..328f88d247 100644 --- a/docs/command-line/promtool.md +++ b/docs/command-line/promtool.md @@ -572,7 +572,7 @@ List tsdb blocks. ##### `promtool tsdb dump` -Dump samples from a TSDB. +Dump data (series+samples or optionally just series) from a TSDB. @@ -584,33 +584,7 @@ Dump samples from a TSDB. | --min-time | Minimum timestamp to dump. | `-9223372036854775808` | | --max-time | Maximum timestamp to dump. | `9223372036854775807` | | --match ... | Series selector. Can be specified multiple times. | `{__name__=~'(?s:.*)'}` | - - - - -###### Arguments - -| Argument | Description | Default | -| --- | --- | --- | -| db path | Database path (default is data/). | `data/` | - - - - -##### `promtool tsdb dump-series` - -Dump series (identified by a unique set of labels) from a TSDB into JSON format. - - - -###### Flags - -| Flag | Description | Default | -| --- | --- | --- | -| --sandbox-dir-root | Root directory where a sandbox directory would be created in case WAL replay generates chunks. The sandbox directory is cleaned up at the end. | `data/` | -| --min-time | Minimum timestamp to dump. | `-9223372036854775808` | -| --max-time | Maximum timestamp to dump. | `9223372036854775807` | -| --match ... | Series selector. Can be specified multiple times. | `{__name__=~'(?s:.*)'}` | +| --format | Output format of the dump (prom (default) or seriesjson). | `prom` | From 89b92f7880864a4e2d814a5301bc08f94368c521 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Mon, 10 Nov 2025 17:48:01 +0000 Subject: [PATCH 031/439] [TEST] Promtool: test dump series together with dump samples Add data files for series tests. Signed-off-by: Bryan Boreham --- cmd/promtool/testdata/dump-series-1.prom | 3 + cmd/promtool/testdata/dump-series-2.prom | 2 + cmd/promtool/testdata/dump-series-3.prom | 1 + cmd/promtool/tsdb_test.go | 166 ++++++----------------- 4 files changed, 50 insertions(+), 122 deletions(-) create mode 100644 cmd/promtool/testdata/dump-series-1.prom create mode 100644 cmd/promtool/testdata/dump-series-2.prom create mode 100644 cmd/promtool/testdata/dump-series-3.prom diff --git a/cmd/promtool/testdata/dump-series-1.prom b/cmd/promtool/testdata/dump-series-1.prom new file mode 100644 index 0000000000..5e44c0bf1b --- /dev/null +++ b/cmd/promtool/testdata/dump-series-1.prom @@ -0,0 +1,3 @@ +{"__name__":"heavy_metric","foo":"bar"} +{"__name__":"heavy_metric","foo":"foo"} +{"__name__":"metric","baz":"abc","foo":"bar"} diff --git a/cmd/promtool/testdata/dump-series-2.prom b/cmd/promtool/testdata/dump-series-2.prom new file mode 100644 index 0000000000..fefefa6d1b --- /dev/null +++ b/cmd/promtool/testdata/dump-series-2.prom @@ -0,0 +1,2 @@ +{"__name__":"heavy_metric","foo":"foo"} +{"__name__":"metric","baz":"abc","foo":"bar"} diff --git a/cmd/promtool/testdata/dump-series-3.prom b/cmd/promtool/testdata/dump-series-3.prom new file mode 100644 index 0000000000..dd98e8707d --- /dev/null +++ b/cmd/promtool/testdata/dump-series-3.prom @@ -0,0 +1 @@ +{"__name__":"metric","baz":"abc","foo":"bar"} diff --git a/cmd/promtool/tsdb_test.go b/cmd/promtool/tsdb_test.go index 5b4d62b75b..ea2f28083d 100644 --- a/cmd/promtool/tsdb_test.go +++ b/cmd/promtool/tsdb_test.go @@ -106,13 +106,15 @@ func TestTSDBDump(t *testing.T) { sandboxDirRoot string match []string expectedDump string + expectedSeries string }{ { - name: "default match", - mint: math.MinInt64, - maxt: math.MaxInt64, - match: []string{"{__name__=~'(?s:.*)'}"}, - expectedDump: "testdata/dump-test-1.prom", + name: "default match", + mint: math.MinInt64, + maxt: math.MaxInt64, + match: []string{"{__name__=~'(?s:.*)'}"}, + expectedDump: "testdata/dump-test-1.prom", + expectedSeries: "testdata/dump-series-1.prom", }, { name: "default match with sandbox dir root set", @@ -121,41 +123,47 @@ func TestTSDBDump(t *testing.T) { sandboxDirRoot: t.TempDir(), match: []string{"{__name__=~'(?s:.*)'}"}, expectedDump: "testdata/dump-test-1.prom", + expectedSeries: "testdata/dump-series-1.prom", }, { - name: "same matcher twice", - mint: math.MinInt64, - maxt: math.MaxInt64, - match: []string{"{foo=~'.+'}", "{foo=~'.+'}"}, - expectedDump: "testdata/dump-test-1.prom", + name: "same matcher twice", + mint: math.MinInt64, + maxt: math.MaxInt64, + match: []string{"{foo=~'.+'}", "{foo=~'.+'}"}, + expectedDump: "testdata/dump-test-1.prom", + expectedSeries: "testdata/dump-series-1.prom", }, { - name: "no duplication", - mint: math.MinInt64, - maxt: math.MaxInt64, - match: []string{"{__name__=~'(?s:.*)'}", "{baz='abc'}"}, - expectedDump: "testdata/dump-test-1.prom", + name: "no duplication", + mint: math.MinInt64, + maxt: math.MaxInt64, + match: []string{"{__name__=~'(?s:.*)'}", "{baz='abc'}"}, + expectedDump: "testdata/dump-test-1.prom", + expectedSeries: "testdata/dump-series-1.prom", }, { - name: "well merged", - mint: math.MinInt64, - maxt: math.MaxInt64, - match: []string{"{__name__='heavy_metric'}", "{baz='abc'}"}, - expectedDump: "testdata/dump-test-1.prom", + name: "well merged", + mint: math.MinInt64, + maxt: math.MaxInt64, + match: []string{"{__name__='heavy_metric'}", "{baz='abc'}"}, + expectedDump: "testdata/dump-test-1.prom", + expectedSeries: "testdata/dump-series-1.prom", }, { - name: "multi matchers", - mint: math.MinInt64, - maxt: math.MaxInt64, - match: []string{"{__name__='heavy_metric',foo='foo'}", "{__name__='metric'}"}, - expectedDump: "testdata/dump-test-2.prom", + name: "multi matchers", + mint: math.MinInt64, + maxt: math.MaxInt64, + match: []string{"{__name__='heavy_metric',foo='foo'}", "{__name__='metric'}"}, + expectedDump: "testdata/dump-test-2.prom", + expectedSeries: "testdata/dump-series-2.prom", }, { - name: "with reduced mint and maxt", - mint: int64(60000), - maxt: int64(120000), - match: []string{"{__name__='metric'}"}, - expectedDump: "testdata/dump-test-3.prom", + name: "with reduced mint and maxt", + mint: int64(60000), + maxt: int64(120000), + match: []string{"{__name__='metric'}"}, + expectedDump: "testdata/dump-test-3.prom", + expectedSeries: "testdata/dump-series-3.prom", }, } for _, tt := range tests { @@ -166,6 +174,12 @@ func TestTSDBDump(t *testing.T) { expectedMetrics = normalizeNewLine(expectedMetrics) // Sort both, because Prometheus does not guarantee the output order. require.Equal(t, sortLines(string(expectedMetrics)), sortLines(dumpedMetrics)) + + dumpedSeries := getDumpedSamples(t, storage.Dir(), tt.sandboxDirRoot, tt.mint, tt.maxt, tt.match, formatSeriesSetToJSON) + expectedSeries, err := os.ReadFile(tt.expectedSeries) + require.NoError(t, err) + expectedSeries = normalizeNewLine(expectedSeries) + require.Equal(t, sortLines(string(expectedSeries)), sortLines(dumpedSeries)) }) } } @@ -176,98 +190,6 @@ func sortLines(buf string) string { return strings.Join(lines, "\n") } -// getDumpedSeries dumps series and returns them. -func getDumpedSeries(t *testing.T, path string, mint, maxt int64, match []string, formatter SeriesSetFormatter) string { - t.Helper() - - oldStdout := os.Stdout - r, w, _ := os.Pipe() - os.Stdout = w - - err := dumpTSDBData( - context.Background(), - path, - t.TempDir(), - mint, - maxt, - match, - formatter, - ) - require.NoError(t, err) - - w.Close() - os.Stdout = oldStdout - - var buf bytes.Buffer - io.Copy(&buf, r) - return buf.String() -} - -func TestTSDBDumpSeries(t *testing.T) { - storage := promqltest.LoadedStorage(t, ` - load 1m - metric{foo="bar", baz="abc"} 1 2 3 4 5 - heavy_metric{foo="bar"} 5 4 3 2 1 - heavy_metric{foo="foo"} 5 4 3 2 1 - `) - - expectedArray := []string{ - `{"__name__":"heavy_metric","foo":"bar"} -`, - `{"__name__":"heavy_metric","foo":"foo"} -`, - `{"__name__":"metric","baz":"abc","foo":"bar"} -`, - } - - tests := []struct { - name string - mint int64 - maxt int64 - match []string - expected string - }{ - { - name: "default match", - match: []string{"{__name__=~'(?s:.*)'}"}, - expected: strings.Join(expectedArray, ""), - }, - { - name: "same matcher twice", - match: []string{"{foo=~'.+'}", "{foo=~'.+'}"}, - expected: strings.Join(expectedArray, ""), - }, - { - name: "no duplication", - match: []string{"{__name__=~'(?s:.*)'}", "{baz='abc'}"}, - expected: strings.Join(expectedArray, ""), - }, - { - name: "well merged", - match: []string{"{__name__='heavy_metric'}", "{baz='abc'}"}, - expected: strings.Join(expectedArray, ""), - }, - { - name: "multi matchers", - match: []string{"{__name__='heavy_metric',foo='foo'}", "{__name__='metric'}"}, - expected: expectedArray[1] + expectedArray[2], - }, - { - name: "with reduced mint and maxt", - match: []string{"{__name__='metric'}"}, - expected: expectedArray[2], - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - dumpedSeries := getDumpedSeries(t, storage.Dir(), tt.mint, tt.maxt, tt.match, formatSeriesSetToJSON) - expectedSeries := normalizeNewLine([]byte(tt.expected)) - // Sort both, because Prometheus does not guarantee the output order. - require.Equal(t, sortLines(string(expectedSeries)), sortLines(dumpedSeries)) - }) - } -} - func TestTSDBDumpOpenMetrics(t *testing.T) { storage := promqltest.LoadedStorage(t, ` load 1m From 11cf858166ec8511d86c1bf24185bddc23eb5b97 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Mon, 10 Nov 2025 17:55:30 +0000 Subject: [PATCH 032/439] Small refactor: use clearer name formatSeriesSetLabelsToJSON Signed-off-by: Bryan Boreham --- cmd/promtool/main.go | 2 +- cmd/promtool/tsdb.go | 2 +- cmd/promtool/tsdb_test.go | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/cmd/promtool/main.go b/cmd/promtool/main.go index 1b36d518ee..ea4e066586 100644 --- a/cmd/promtool/main.go +++ b/cmd/promtool/main.go @@ -421,7 +421,7 @@ func main() { case tsdbDumpCmd.FullCommand(): if *dumpFormat == "seriesjson" { - os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSetToJSON))) + os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSetLabelsToJSON))) } else { os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSet))) } diff --git a/cmd/promtool/tsdb.go b/cmd/promtool/tsdb.go index ab822f2e7e..14ec051df2 100644 --- a/cmd/promtool/tsdb.go +++ b/cmd/promtool/tsdb.go @@ -799,7 +799,7 @@ func CondensedString(ls labels.Labels) string { return b.String() } -func formatSeriesSetToJSON(ss storage.SeriesSet) error { +func formatSeriesSetLabelsToJSON(ss storage.SeriesSet) error { seriesCache := make(map[string]struct{}) for ss.Next() { series := ss.At() diff --git a/cmd/promtool/tsdb_test.go b/cmd/promtool/tsdb_test.go index ea2f28083d..286456fee3 100644 --- a/cmd/promtool/tsdb_test.go +++ b/cmd/promtool/tsdb_test.go @@ -175,7 +175,7 @@ func TestTSDBDump(t *testing.T) { // Sort both, because Prometheus does not guarantee the output order. require.Equal(t, sortLines(string(expectedMetrics)), sortLines(dumpedMetrics)) - dumpedSeries := getDumpedSamples(t, storage.Dir(), tt.sandboxDirRoot, tt.mint, tt.maxt, tt.match, formatSeriesSetToJSON) + dumpedSeries := getDumpedSamples(t, storage.Dir(), tt.sandboxDirRoot, tt.mint, tt.maxt, tt.match, formatSeriesSetLabelsToJSON) expectedSeries, err := os.ReadFile(tt.expectedSeries) require.NoError(t, err) expectedSeries = normalizeNewLine(expectedSeries) From e17742902b162f6c53bfe81695805d7e728b8f74 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Mon, 10 Nov 2025 18:28:13 +0000 Subject: [PATCH 033/439] lint Signed-off-by: Bryan Boreham --- cmd/promtool/main.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/cmd/promtool/main.go b/cmd/promtool/main.go index ea4e066586..42eb88dabe 100644 --- a/cmd/promtool/main.go +++ b/cmd/promtool/main.go @@ -420,11 +420,11 @@ func main() { os.Exit(checkErr(listBlocks(*listPath, *listHumanReadable))) case tsdbDumpCmd.FullCommand(): + format := formatSeriesSet if *dumpFormat == "seriesjson" { - os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSetLabelsToJSON))) - } else { - os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, formatSeriesSet))) + format = formatSeriesSetLabelsToJSON } + os.Exit(checkErr(dumpTSDBData(ctx, *dumpPath, *dumpSandboxDirRoot, *dumpMinTime, *dumpMaxTime, *dumpMatch, format))) case tsdbDumpOpenMetricsCmd.FullCommand(): os.Exit(checkErr(dumpTSDBData(ctx, *dumpOpenMetricsPath, *dumpOpenMetricsSandboxDirRoot, *dumpOpenMetricsMinTime, *dumpOpenMetricsMaxTime, *dumpOpenMetricsMatch, formatSeriesSetOpenMetrics))) From cc23e3760df2abfaedf9506bfa01d89d47634cf6 Mon Sep 17 00:00:00 2001 From: Andrew Hall Date: Tue, 11 Nov 2025 17:03:35 +0800 Subject: [PATCH 034/439] Allow for promql tests to compare expected fail message during query preparation Signed-off-by: Andrew Hall --- promql/promqltest/test.go | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go index 41d8cdde20..b16433c14e 100644 --- a/promql/promqltest/test.go +++ b/promql/promqltest/test.go @@ -1430,6 +1430,11 @@ func (t *test) execEval(cmd *evalCmd, engine promql.QueryEngine) error { func (t *test) execRangeEval(cmd *evalCmd, engine promql.QueryEngine) error { q, err := engine.NewRangeQuery(t.context, t.storage, nil, cmd.expr, cmd.start, cmd.end, cmd.step) if err != nil { + if cmd.isFail() { + if err := cmd.checkExpectedFailure(err); err == nil { + return nil + } + } return fmt.Errorf("error creating range query for %q (line %d): %w", cmd.expr, cmd.line, err) } defer q.Close() @@ -1473,6 +1478,11 @@ func (t *test) execInstantEval(cmd *evalCmd, engine promql.QueryEngine) error { func (t *test) runInstantQuery(iq atModifierTestCase, cmd *evalCmd, engine promql.QueryEngine) error { q, err := engine.NewInstantQuery(t.context, t.storage, nil, iq.expr, iq.evalTime) if err != nil { + if cmd.isFail() { + if err := cmd.checkExpectedFailure(err); err == nil { + return nil + } + } return fmt.Errorf("error creating instant query for %q (line %d): %w", cmd.expr, cmd.line, err) } defer q.Close() From 2e609511bbef1c6aee38f248b600cf0c05df3de2 Mon Sep 17 00:00:00 2001 From: Ben Ye Date: Tue, 11 Nov 2025 02:07:08 -0800 Subject: [PATCH 035/439] Register missing metric prometheus_tsdb_sample_ooo_delta (#17477) * register missing metric prometheus_tsdb_sample_ooo_delta Signed-off-by: yeya24 * changelog Signed-off-by: yeya24 --------- Signed-off-by: yeya24 --- CHANGELOG.md | 1 + tsdb/head.go | 1 + 2 files changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 11ec004cb6..01da079725 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ ## main / unreleased * [FEATURE] Templates: Add urlQueryEscape to template functions. #17403 +* [BUGFIX] TSDB: Register `prometheus_tsdb_sample_ooo_delta` metric properly. #17477 ## 3.7.3 / 2025-10-29 diff --git a/tsdb/head.go b/tsdb/head.go index 2c71977b1a..cf773e82b0 100644 --- a/tsdb/head.go +++ b/tsdb/head.go @@ -562,6 +562,7 @@ func newHeadMetrics(h *Head, r prometheus.Registerer) *headMetrics { m.checkpointDeleteTotal, m.checkpointCreationFail, m.checkpointCreationTotal, + m.oooHistogram, m.mmapChunksTotal, m.mmapChunkCorruptionTotal, m.snapshotReplayErrorTotal, From 9d508a4888edc2d9a7fd41e071482018003ae25d Mon Sep 17 00:00:00 2001 From: Clark McCauley Date: Tue, 11 Nov 2025 04:43:04 -0700 Subject: [PATCH 036/439] util: add +Inf bucket in MetricFamiliesToWriteRequest when not present (#15864) * Add +Inf bucket in MetricFamiliesToWriteRequest if not present Signed-off-by: Clark McCauley --- util/fmtutil/format.go | 13 ++++++++++ util/fmtutil/format_test.go | 50 +++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/util/fmtutil/format.go b/util/fmtutil/format.go index 7a78df849c..377f4ece05 100644 --- a/util/fmtutil/format.go +++ b/util/fmtutil/format.go @@ -18,6 +18,7 @@ import ( "fmt" "io" "maps" + "math" "sort" "time" @@ -140,11 +141,23 @@ func makeTimeseries(wr *prompb.WriteRequest, labels map[string]string, m *dto.Me // Add Histogram bucket timeseries bucketLabels := make(map[string]string, len(labels)+1) maps.Copy(bucketLabels, labels) + var hasInf bool for _, b := range m.GetHistogram().Bucket { + if b.GetUpperBound() == math.Inf(1) { + hasInf = true + } bucketLabels[model.MetricNameLabel] = metricName + bucketStr bucketLabels[model.BucketLabel] = fmt.Sprint(b.GetUpperBound()) toTimeseries(wr, bucketLabels, timestamp, float64(b.GetCumulativeCount())) } + + // Add +Inf bucket if not present + if !hasInf { + bucketLabels[model.MetricNameLabel] = metricName + bucketStr + bucketLabels[model.BucketLabel] = "+Inf" + toTimeseries(wr, bucketLabels, timestamp, float64(m.GetHistogram().GetSampleCount())) + } + // Overwrite label model.MetricNameLabel for count and sum metrics // Add Histogram sum timeseries labels[model.MetricNameLabel] = metricName + sumStr diff --git a/util/fmtutil/format_test.go b/util/fmtutil/format_test.go index c592630fe8..f1d025806e 100644 --- a/util/fmtutil/format_test.go +++ b/util/fmtutil/format_test.go @@ -15,8 +15,11 @@ package fmtutil import ( "bytes" + "math" "testing" + "time" + dto "github.com/prometheus/client_model/go" "github.com/stretchr/testify/require" "github.com/prometheus/prometheus/prompb" @@ -231,3 +234,50 @@ func TestMetricTextToWriteRequestErrorParsingMetricType(t *testing.T) { _, err := MetricTextToWriteRequest(input, labels) require.Equal(t, "text format parsing error in line 3: unknown metric type \"info\"", err.Error()) } + +func TestMakeTimeseries_HistogramInfBucket(t *testing.T) { + tests := map[string]*dto.Histogram{ + "Histogram missing +Inf bucket": { + Bucket: []*dto.Bucket{ + {CumulativeCount: p[uint64](5), UpperBound: p(1.0)}, + {CumulativeCount: p[uint64](10), UpperBound: p(5.0)}, + }, + SampleCount: p[uint64](15), + }, + "Histogram already has +Inf bucket": { + Bucket: []*dto.Bucket{ + {CumulativeCount: p[uint64](5), UpperBound: p(1.0)}, + {CumulativeCount: p[uint64](10), UpperBound: p(5.0)}, + {CumulativeCount: p[uint64](15), UpperBound: p(math.Inf(1))}, + }, + SampleCount: p[uint64](15), + }, + } + + for name, histogram := range tests { + t.Run(name, func(t *testing.T) { + wr := &prompb.WriteRequest{} + labels := map[string]string{"__name__": "test_histogram"} + metric := &dto.Metric{ + Histogram: histogram, + TimestampMs: p(time.Now().UnixMilli()), + } + + require.NoError(t, makeTimeseries(wr, labels, metric)) + + var hasInf bool + for _, ts := range wr.Timeseries { + for _, lbl := range ts.Labels { + if lbl.Name == "le" && lbl.Value == "+Inf" { + hasInf = true + } + } + } + require.Truef(t, hasInf, "expected +Inf bucket in histogram") + }) + } +} + +func p[T any](v T) *T { + return &v +} From 33082be0e26da431b9f724350bc4d9465561fc46 Mon Sep 17 00:00:00 2001 From: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com> Date: Tue, 11 Nov 2025 13:47:37 +0100 Subject: [PATCH 037/439] feat: add histogram metric for notification_latency_seconds (#16637) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This metric can be used to create alerting based on how many notifications finish or do not finish within a certain amount of time. Change-Id: afbf3d8ceb3994c7d6220389353cff92 Signed-Off-By: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com> Co-authored-by: Björn Rabenstein --------- Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com> Co-authored-by: Björn Rabenstein --- notifier/alertmanagerset.go | 3 ++- notifier/manager.go | 4 +++- notifier/metric.go | 33 ++++++++++++++++++++++++++++----- 3 files changed, 33 insertions(+), 7 deletions(-) diff --git a/notifier/alertmanagerset.go b/notifier/alertmanagerset.go index c47c9ea23a..b6d1b8c4aa 100644 --- a/notifier/alertmanagerset.go +++ b/notifier/alertmanagerset.go @@ -111,7 +111,8 @@ func (s *alertmanagerSet) sync(tgs []*targetgroup.Group) { if _, ok := seen[us]; ok { continue } - s.metrics.latency.DeleteLabelValues(us) + s.metrics.latencySummary.DeleteLabelValues(us) + s.metrics.latencyHistogram.DeleteLabelValues(us) s.metrics.sent.DeleteLabelValues(us) s.metrics.errors.DeleteLabelValues(us) seen[us] = struct{}{} diff --git a/notifier/manager.go b/notifier/manager.go index 65adfd5c3e..e37f59a250 100644 --- a/notifier/manager.go +++ b/notifier/manager.go @@ -498,7 +498,9 @@ func (n *Manager) sendAll(alerts ...*Alert) bool { amSetCovered.CompareAndSwap(k, false, true) } - n.metrics.latency.WithLabelValues(url).Observe(time.Since(begin).Seconds()) + durationSeconds := time.Since(begin).Seconds() + n.metrics.latencySummary.WithLabelValues(url).Observe(durationSeconds) + n.metrics.latencyHistogram.WithLabelValues(url).Observe(durationSeconds) n.metrics.sent.WithLabelValues(url).Add(float64(count)) wg.Done() diff --git a/notifier/metric.go b/notifier/metric.go index b9a55b3ec7..3f4abdda93 100644 --- a/notifier/metric.go +++ b/notifier/metric.go @@ -13,10 +13,15 @@ package notifier -import "github.com/prometheus/client_golang/prometheus" +import ( + "time" + + "github.com/prometheus/client_golang/prometheus" +) type alertMetrics struct { - latency *prometheus.SummaryVec + latencySummary *prometheus.SummaryVec + latencyHistogram *prometheus.HistogramVec errors *prometheus.CounterVec sent *prometheus.CounterVec dropped prometheus.Counter @@ -25,9 +30,13 @@ type alertMetrics struct { alertmanagersDiscovered prometheus.GaugeFunc } -func newAlertMetrics(r prometheus.Registerer, queueCap int, queueLen, alertmanagersDiscovered func() float64) *alertMetrics { +func newAlertMetrics( + r prometheus.Registerer, + queueCap int, + queueLen, alertmanagersDiscovered func() float64, +) *alertMetrics { m := &alertMetrics{ - latency: prometheus.NewSummaryVec(prometheus.SummaryOpts{ + latencySummary: prometheus.NewSummaryVec(prometheus.SummaryOpts{ Namespace: namespace, Subsystem: subsystem, Name: "latency_seconds", @@ -36,6 +45,19 @@ func newAlertMetrics(r prometheus.Registerer, queueCap int, queueLen, alertmanag }, []string{alertmanagerLabel}, ), + latencyHistogram: prometheus.NewHistogramVec(prometheus.HistogramOpts{ + Namespace: namespace, + Subsystem: subsystem, + Name: "latency_histogram_seconds", + Help: "Latency histogram for sending alert notifications.", + + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }, + []string{alertmanagerLabel}, + ), errors: prometheus.NewCounterVec(prometheus.CounterOpts{ Namespace: namespace, Subsystem: subsystem, @@ -80,7 +102,8 @@ func newAlertMetrics(r prometheus.Registerer, queueCap int, queueLen, alertmanag if r != nil { r.MustRegister( - m.latency, + m.latencySummary, + m.latencyHistogram, m.errors, m.sent, m.dropped, From f330ccaf2fee672127049bd867a9331b941c5c8c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linas=20Med=C5=BEi=C5=ABnas?= Date: Wed, 12 Nov 2025 12:43:05 +0200 Subject: [PATCH 038/439] [PERF] PromQL: eliminate string-keyed maps in binary vector matching (#17131) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In this PR, we are eliminating expensive string-keyed (by signature) maps that are accessed for every sample processed. During preprocessing in rangeEval, we assign a unique number from 0 to n-1 to each of the n string signature values, and later only use this number as a label set signature. Signed-off-by: Linas Medžiūnas Co-authored-by: George Krajcsovits --- promql/engine.go | 144 ++++++++++++++++++++++++++--------------------- promql/info.go | 14 ++--- 2 files changed, 86 insertions(+), 72 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 75fc9b05d3..146a93ec58 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -1144,8 +1144,13 @@ func (ev *evaluator) Eval(ctx context.Context, expr parser.Expr) (v parser.Value // EvalSeriesHelper stores extra information about a series. type EvalSeriesHelper struct { - // Used to map left-hand to right-hand in binary operations. - signature string + // Ordinal number of join signature, used to map left-hand to right-hand in + // binary operations. For example given the following series, if + // the join signature is job, instance then: + // metric{job="a", instance="1", other="x"} -> sigOrdinal 0 + // metric{job="a", instance="1", other="y"} -> sigOrdinal 0 + // metric{job="a", instance="2", other="x"} -> sigOrdinal 1 + sigOrdinal int } // EvalNodeHelper stores extra information and caches for evaluating a single node across steps. @@ -1165,9 +1170,13 @@ type EvalNodeHelper struct { lblResultBuf []byte // For binary vector matching. - rightSigs map[string]Sample - matchedSigs map[string]map[uint64]struct{} + rightSigs map[int]Sample + matchedSigs map[int]map[uint64]struct{} resultMetric map[string]labels.Labels + numSigs int + + // For info series matching. + rightStrSigs map[string]Sample // Additional options for the evaluation. enableDelayedNameRemoval bool @@ -1252,11 +1261,12 @@ func (enh *EvalNodeHelper) resetHistograms(inVec Vector, arg parser.Expr) annota // function call results. // The prepSeries function (if provided) can be used to prepare the helper // for each series, then passed to each call funcCall. -func (ev *evaluator) rangeEval(ctx context.Context, prepSeries func(labels.Labels, *EvalSeriesHelper), funcCall func([]Vector, Matrix, [][]EvalSeriesHelper, *EvalNodeHelper) (Vector, annotations.Annotations), exprs ...parser.Expr) (Matrix, annotations.Annotations) { +func (ev *evaluator) rangeEval(ctx context.Context, matching *parser.VectorMatching, funcCall func([]Vector, Matrix, [][]EvalSeriesHelper, *EvalNodeHelper) (Vector, annotations.Annotations), exprs ...parser.Expr) (Matrix, annotations.Annotations) { numSteps := int((ev.endTimestamp-ev.startTimestamp)/ev.interval) + 1 matrixes := make([]Matrix, len(exprs)) origMatrixes := make([]Matrix, len(exprs)) originalNumSamples := ev.currentSamples + useSignatures := matching != nil var warnings annotations.Annotations for i, e := range exprs { @@ -1297,18 +1307,46 @@ func (ev *evaluator) rangeEval(ctx context.Context, prepSeries func(labels.Label bufHelpers [][]EvalSeriesHelper // Buffer updated on each step ) - // If the series preparation function is provided, we should run it for - // every single series in the matrix. - if prepSeries != nil { + if useSignatures { + var ( + // Function to compute the join signature for each series. + sigf func(labels.Labels) string + buf = make([]byte, 0, 1024) + names = slices.Clone(matching.MatchingLabels) + ) + if matching.On { + slices.Sort(names) + sigf = func(lset labels.Labels) string { + return string(lset.BytesWithLabels(buf, names...)) + } + } else { // "without" + names = append([]string{labels.MetricName}, names...) + slices.Sort(names) + sigf = func(lset labels.Labels) string { + return string(lset.BytesWithoutLabels(buf, names...)) + } + } + seriesHelpers = make([][]EvalSeriesHelper, len(exprs)) bufHelpers = make([][]EvalSeriesHelper, len(exprs)) + signatureToOrdinal := make(map[string]int) + for i := range exprs { seriesHelpers[i] = make([]EvalSeriesHelper, len(matrixes[i])) bufHelpers[i] = make([]EvalSeriesHelper, len(matrixes[i])) for si, series := range matrixes[i] { - prepSeries(series.Metric, &seriesHelpers[i][si]) + strSig := sigf(series.Metric) + + if sigOrd, ok := signatureToOrdinal[strSig]; ok { + seriesHelpers[i][si] = EvalSeriesHelper{sigOrdinal: sigOrd} + continue + } + + signatureToOrdinal[strSig] = enh.numSigs + seriesHelpers[i][si] = EvalSeriesHelper{sigOrdinal: enh.numSigs} + enh.numSigs++ } } } @@ -1323,12 +1361,12 @@ func (ev *evaluator) rangeEval(ctx context.Context, prepSeries func(labels.Label for i := range exprs { var bh []EvalSeriesHelper var sh []EvalSeriesHelper - if prepSeries != nil { + if useSignatures { bh = bufHelpers[i][:0] sh = seriesHelpers[i] } vectors[i], bh = ev.gatherVector(ts, matrixes[i], vectors[i], bh, sh) - if prepSeries != nil { + if useSignatures { bufHelpers[i] = bh } } @@ -2129,27 +2167,21 @@ func (ev *evaluator) eval(ctx context.Context, expr parser.Expr) (parser.Value, return append(enh.Out, Sample{F: val}), nil }, e.LHS, e.RHS) case lt == parser.ValueTypeVector && rt == parser.ValueTypeVector: - // Function to compute the join signature for each series. - buf := make([]byte, 0, 1024) - sigf := signatureFunc(e.VectorMatching.On, buf, e.VectorMatching.MatchingLabels...) - initSignatures := func(series labels.Labels, h *EvalSeriesHelper) { - h.signature = sigf(series) - } switch e.Op { case parser.LAND: - return ev.rangeEval(ctx, initSignatures, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { + return ev.rangeEval(ctx, e.VectorMatching, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { return ev.VectorAnd(v[0], v[1], e.VectorMatching, sh[0], sh[1], enh), nil }, e.LHS, e.RHS) case parser.LOR: - return ev.rangeEval(ctx, initSignatures, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { + return ev.rangeEval(ctx, e.VectorMatching, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { return ev.VectorOr(v[0], v[1], e.VectorMatching, sh[0], sh[1], enh), nil }, e.LHS, e.RHS) case parser.LUNLESS: - return ev.rangeEval(ctx, initSignatures, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { + return ev.rangeEval(ctx, e.VectorMatching, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { return ev.VectorUnless(v[0], v[1], e.VectorMatching, sh[0], sh[1], enh), nil }, e.LHS, e.RHS) default: - return ev.rangeEval(ctx, initSignatures, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { + return ev.rangeEval(ctx, e.VectorMatching, func(v []Vector, _ Matrix, sh [][]EvalSeriesHelper, enh *EvalNodeHelper) (Vector, annotations.Annotations) { vec, err := ev.VectorBinop(e.Op, v[0], v[1], e.VectorMatching, e.ReturnBool, sh[0], sh[1], enh, e.PositionRange()) return vec, handleVectorBinopError(err, e) }, e.LHS, e.RHS) @@ -2720,16 +2752,15 @@ func (*evaluator) VectorAnd(lhs, rhs Vector, matching *parser.VectorMatching, lh return nil // Short-circuit: AND with nothing is nothing. } - // The set of signatures for the right-hand side Vector. - rightSigs := map[string]struct{}{} - // Add all rhs samples to a map so we can easily find matches later. + // Ordinals of signatures present on the right-hand side. + rightSigOrdinalsPresent := make([]bool, enh.numSigs) for _, sh := range rhsh { - rightSigs[sh.signature] = struct{}{} + rightSigOrdinalsPresent[sh.sigOrdinal] = true } for i, ls := range lhs { // If there's a matching entry in the right-hand side Vector, add the sample. - if _, ok := rightSigs[lhsh[i].signature]; ok { + if rightSigOrdinalsPresent[lhsh[i].sigOrdinal] { enh.Out = append(enh.Out, ls) } } @@ -2748,15 +2779,15 @@ func (*evaluator) VectorOr(lhs, rhs Vector, matching *parser.VectorMatching, lhs return enh.Out } - leftSigs := map[string]struct{}{} + leftSigOrdinalsPresent := make([]bool, enh.numSigs) // Add everything from the left-hand-side Vector. for i, ls := range lhs { - leftSigs[lhsh[i].signature] = struct{}{} + leftSigOrdinalsPresent[lhsh[i].sigOrdinal] = true enh.Out = append(enh.Out, ls) } // Add all right-hand side elements which have not been added from the left-hand side. for j, rs := range rhs { - if _, ok := leftSigs[rhsh[j].signature]; !ok { + if !leftSigOrdinalsPresent[rhsh[j].sigOrdinal] { enh.Out = append(enh.Out, rs) } } @@ -2774,13 +2805,14 @@ func (*evaluator) VectorUnless(lhs, rhs Vector, matching *parser.VectorMatching, return enh.Out } - rightSigs := map[string]struct{}{} + // Ordinals of signatures present on the right-hand side. + rightSigOrdinalsPresent := make([]bool, enh.numSigs) for _, sh := range rhsh { - rightSigs[sh.signature] = struct{}{} + rightSigOrdinalsPresent[sh.sigOrdinal] = true } for i, ls := range lhs { - if _, ok := rightSigs[lhsh[i].signature]; !ok { + if !rightSigOrdinalsPresent[lhsh[i].sigOrdinal] { enh.Out = append(enh.Out, ls) } } @@ -2804,22 +2836,20 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * lhsh, rhsh = rhsh, lhsh } - // All samples from the rhs hashed by the matching label/values. + // All samples from the rhs by their join signature ordinal. if enh.rightSigs == nil { - enh.rightSigs = make(map[string]Sample, len(enh.Out)) + enh.rightSigs = make(map[int]Sample, len(enh.Out)) } else { - for k := range enh.rightSigs { - delete(enh.rightSigs, k) - } + clear(enh.rightSigs) } rightSigs := enh.rightSigs // Add all rhs samples to a map so we can easily find matches later. for i, rs := range rhs { - sig := rhsh[i].signature + sigOrd := rhsh[i].sigOrdinal // The rhs is guaranteed to be the 'one' side. Having multiple samples // with the same signature means that the matching is many-to-many. - if duplSample, found := rightSigs[sig]; found { + if duplSample, found := rightSigs[sigOrd]; found { // oneSide represents which side of the vector represents the 'one' in the many-to-one relationship. oneSide := "right" if matching.Card == parser.CardOneToMany { @@ -2830,17 +2860,15 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * ev.errorf("found duplicate series for the match group %s on the %s hand-side of the operation: [%s, %s]"+ ";many-to-many matching not allowed: matching labels must be unique on one side", matchedLabels.String(), oneSide, rs.Metric.String(), duplSample.Metric.String()) } - rightSigs[sig] = rs + rightSigs[sigOrd] = rs } - // Tracks the match-signature. For one-to-one operations the value is nil. For many-to-one - // the value is a set of signatures to detect duplicated result elements. + // Tracks the matching by signature ordinals. For one-to-one operations the value is nil. + // For many-to-one the value is a set of hashes to detect duplicated result elements. if enh.matchedSigs == nil { - enh.matchedSigs = make(map[string]map[uint64]struct{}, len(rightSigs)) + enh.matchedSigs = make(map[int]map[uint64]struct{}, len(rightSigs)) } else { - for k := range enh.matchedSigs { - delete(enh.matchedSigs, k) - } + clear(enh.matchedSigs) } matchedSigs := enh.matchedSigs @@ -2848,9 +2876,9 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * // the binary operation. var lastErr error for i, ls := range lhs { - sig := lhsh[i].signature + sigOrd := lhsh[i].sigOrdinal - rs, found := rightSigs[sig] // Look for a match in the rhs Vector. + rs, found := rightSigs[sigOrd] // Look for a match in the rhs Vector. if !found { continue } @@ -2885,12 +2913,12 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * if !ev.enableDelayedNameRemoval && returnBool { metric = metric.DropReserved(schema.IsMetadataLabel) } - insertedSigs, exists := matchedSigs[sig] + insertedSigs, exists := matchedSigs[sigOrd] if matching.Card == parser.CardOneToOne { if exists { ev.errorf("multiple matches for labels: many-to-one matching must be explicit (group_left/group_right)") } - matchedSigs[sig] = nil // Set existence to true. + matchedSigs[sigOrd] = nil // Set existence to true. } else { // In many-to-one matching the grouping labels have to ensure a unique metric // for the result Vector. Check whether those labels have already been added for @@ -2899,7 +2927,7 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * if !exists { insertedSigs = map[uint64]struct{}{} - matchedSigs[sig] = insertedSigs + matchedSigs[sigOrd] = insertedSigs } else if _, duplicate := insertedSigs[insertSig]; duplicate { ev.errorf("multiple matches for labels: grouping labels must ensure unique matches") } @@ -2916,20 +2944,6 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * return enh.Out, lastErr } -func signatureFunc(on bool, b []byte, names ...string) func(labels.Labels) string { - if on { - slices.Sort(names) - return func(lset labels.Labels) string { - return string(lset.BytesWithLabels(b, names...)) - } - } - names = append([]string{labels.MetricName}, names...) - slices.Sort(names) - return func(lset labels.Labels) string { - return string(lset.BytesWithoutLabels(b, names...)) - } -} - // resultMetric returns the metric for the given sample(s) based on the Vector // binary operation and the matching options. func resultMetric(lhs, rhs labels.Labels, op parser.ItemType, matching *parser.VectorMatching, enh *EvalNodeHelper) labels.Labels { diff --git a/promql/info.go b/promql/info.go index 0067fce845..d5ffda6af2 100644 --- a/promql/info.go +++ b/promql/info.go @@ -337,10 +337,10 @@ func (ev *evaluator) combineWithInfoVector(base, info Vector, ignoreSeries map[u } // All samples from the info Vector hashed by the matching label/values. - if enh.rightSigs == nil { - enh.rightSigs = make(map[string]Sample, len(enh.Out)) + if enh.rightStrSigs == nil { + enh.rightStrSigs = make(map[string]Sample, len(enh.Out)) } else { - clear(enh.rightSigs) + clear(enh.rightStrSigs) } for _, s := range info { @@ -351,7 +351,7 @@ func (ev *evaluator) combineWithInfoVector(base, info Vector, ignoreSeries map[u origT := int64(s.F) sig := infoSigs[s.Metric.Hash()] - if existing, exists := enh.rightSigs[sig]; exists { + if existing, exists := enh.rightStrSigs[sig]; exists { // We encode original info sample timestamps via the float value. existingOrigT := int64(existing.F) switch { @@ -359,14 +359,14 @@ func (ev *evaluator) combineWithInfoVector(base, info Vector, ignoreSeries map[u // Keep the other info sample, since it's newer. case existingOrigT < origT: // Keep this info sample, since it's newer. - enh.rightSigs[sig] = s + enh.rightStrSigs[sig] = s default: // The two info samples have the same timestamp - conflict. ev.errorf("found duplicate series for info metric: existing %s @ %d, new %s @ %d", existing.Metric.String(), existingOrigT, s.Metric.String(), origT) } } else { - enh.rightSigs[sig] = s + enh.rightStrSigs[sig] = s } } @@ -389,7 +389,7 @@ func (ev *evaluator) combineWithInfoVector(base, info Vector, ignoreSeries map[u // For every info metric name, try to find an info series with the same signature. seenInfoMetrics := map[string]struct{}{} for infoName, sig := range baseSigs[hash] { - is, exists := enh.rightSigs[sig] + is, exists := enh.rightStrSigs[sig] if !exists { continue } From 37d153e5b5c014031a17b0a766b2ff92d0ff24a4 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 12 Nov 2025 12:05:25 +0000 Subject: [PATCH 039/439] [PERF] PromQL: Only look up operation name if we need it Signed-off-by: Bryan Boreham --- promql/engine.go | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 146a93ec58..44cef1c1c6 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -3072,7 +3072,6 @@ func scalarBinop(op parser.ItemType, lhs, rhs float64) float64 { // vectorElemBinop evaluates a binary operation between two Vector elements. func vectorElemBinop(op parser.ItemType, lhs, rhs float64, hlhs, hrhs *histogram.FloatHistogram, pos posrange.PositionRange) (res float64, resH *histogram.FloatHistogram, keep bool, info, err error) { - opName := parser.ItemTypeStr[op] switch { case hlhs == nil && hrhs == nil: { @@ -3111,7 +3110,7 @@ func vectorElemBinop(op parser.ItemType, lhs, rhs float64, hlhs, hrhs *histogram case parser.MUL: return 0, hrhs.Copy().Mul(lhs).Compact(0), true, nil, nil case parser.ADD, parser.SUB, parser.DIV, parser.POW, parser.MOD, parser.EQLC, parser.NEQ, parser.GTR, parser.LSS, parser.GTE, parser.LTE, parser.ATAN2: - return 0, nil, false, nil, annotations.NewIncompatibleTypesInBinOpInfo("float", opName, "histogram", pos) + return 0, nil, false, nil, annotations.NewIncompatibleTypesInBinOpInfo("float", parser.ItemTypeStr[op], "histogram", pos) } } case hlhs != nil && hrhs == nil: @@ -3122,7 +3121,7 @@ func vectorElemBinop(op parser.ItemType, lhs, rhs float64, hlhs, hrhs *histogram case parser.DIV: return 0, hlhs.Copy().Div(rhs).Compact(0), true, nil, nil case parser.ADD, parser.SUB, parser.POW, parser.MOD, parser.EQLC, parser.NEQ, parser.GTR, parser.LSS, parser.GTE, parser.LTE, parser.ATAN2: - return 0, nil, false, nil, annotations.NewIncompatibleTypesInBinOpInfo("histogram", opName, "float", pos) + return 0, nil, false, nil, annotations.NewIncompatibleTypesInBinOpInfo("histogram", parser.ItemTypeStr[op], "float", pos) } } case hlhs != nil && hrhs != nil: @@ -3162,7 +3161,7 @@ func vectorElemBinop(op parser.ItemType, lhs, rhs float64, hlhs, hrhs *histogram // This operation expects that both histograms are compacted. return 0, hlhs, !hlhs.Equals(hrhs), nil, nil case parser.MUL, parser.DIV, parser.POW, parser.MOD, parser.GTR, parser.LSS, parser.GTE, parser.LTE, parser.ATAN2: - return 0, nil, false, nil, annotations.NewIncompatibleTypesInBinOpInfo("histogram", opName, "histogram", pos) + return 0, nil, false, nil, annotations.NewIncompatibleTypesInBinOpInfo("histogram", parser.ItemTypeStr[op], "histogram", pos) } } } @@ -3852,13 +3851,13 @@ func handleVectorBinopError(err error, e *parser.BinaryExpr) annotations.Annotat if err == nil { return nil } - op := parser.ItemTypeStr[e.Op] - pos := e.PositionRange() if errors.Is(err, annotations.PromQLInfo) || errors.Is(err, annotations.PromQLWarning) { return annotations.New().Add(err) } // TODO(NeerajGartia21): Test the exact annotation output once the testing framework can do so. if errors.Is(err, histogram.ErrHistogramsIncompatibleSchema) { + op := parser.ItemTypeStr[e.Op] + pos := e.PositionRange() return annotations.New().Add(annotations.NewIncompatibleBucketLayoutInBinOpWarning(op, pos)) } return nil From 7ebff91cfd019fd30abbf2ae38a67b33968ffc42 Mon Sep 17 00:00:00 2001 From: Minh Nguyen <148210689+pipiland2612@users.noreply.github.com> Date: Thu, 13 Nov 2025 10:53:12 +0200 Subject: [PATCH 040/439] OTLP Receiver: Only update metadata to WAL when metadata-wal-records feature is enabled (#17472) OTLP Receiver: Only update metadata to WAL when metadata-wal-records feature is enabled. --------- Signed-off-by: pipiland2612 --- .../combined_appender.go | 28 +- .../combined_appender_test.go | 290 ++++++++++++------ .../metrics_to_prw_test.go | 2 +- storage/remote/write_handler.go | 6 +- storage/remote/write_test.go | 1 + web/api/v1/api.go | 1 + 6 files changed, 221 insertions(+), 107 deletions(-) diff --git a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go index 1441aecb6d..9ed114567d 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go @@ -82,11 +82,12 @@ func NewCombinedAppenderMetrics(reg prometheus.Registerer) CombinedAppenderMetri // NewCombinedAppender creates a combined appender that sets start times and // updates metadata for each series only once, and appends samples and // exemplars for each call. -func NewCombinedAppender(app storage.Appender, logger *slog.Logger, ingestCTZeroSample bool, metrics CombinedAppenderMetrics) CombinedAppender { +func NewCombinedAppender(app storage.Appender, logger *slog.Logger, ingestCTZeroSample, appendMetadata bool, metrics CombinedAppenderMetrics) CombinedAppender { return &combinedAppender{ app: app, logger: logger, ingestCTZeroSample: ingestCTZeroSample, + appendMetadata: appendMetadata, refs: make(map[uint64]seriesRef), samplesAppendedWithoutMetadata: metrics.samplesAppendedWithoutMetadata, outOfOrderExemplars: metrics.outOfOrderExemplars, @@ -106,6 +107,7 @@ type combinedAppender struct { samplesAppendedWithoutMetadata prometheus.Counter outOfOrderExemplars prometheus.Counter ingestCTZeroSample bool + appendMetadata bool // Used to ensure we only update metadata and created timestamps once, and to share storage.SeriesRefs. // To detect hash collision it also stores the labels. // There is no overflow/conflict list, the TSDB will handle that part. @@ -189,17 +191,10 @@ func (b *combinedAppender) appendFloatOrHistogram(ls labels.Labels, meta metadat return err } - if !exists || series.meta.Help != meta.Help || series.meta.Type != meta.Type || series.meta.Unit != meta.Unit { - updateRefs = true - // If this is the first time we see this series, set the metadata. - _, err := b.app.UpdateMetadata(ref, ls, meta) - if err != nil { - b.samplesAppendedWithoutMetadata.Add(1) - b.logger.Warn("Error while updating metadata from OTLP", "err", err) - } - } + metadataChanged := exists && (series.meta.Help != meta.Help || series.meta.Type != meta.Type || series.meta.Unit != meta.Unit) - if updateRefs { + // Update cache if references changed or metadata changed. + if updateRefs || metadataChanged { b.refs[hash] = seriesRef{ ref: ref, ct: ct, @@ -208,6 +203,17 @@ func (b *combinedAppender) appendFloatOrHistogram(ls labels.Labels, meta metadat } } + // Update metadata in storage if enabled and needed. + if b.appendMetadata && (!exists || metadataChanged) { + // Only update metadata in WAL if the metadata-wal-records feature is enabled. + // Without this feature, metadata is not persisted to WAL. + _, err := b.app.UpdateMetadata(ref, ls, meta) + if err != nil { + b.samplesAppendedWithoutMetadata.Add(1) + b.logger.Warn("Error while updating metadata from OTLP", "err", err) + } + } + b.appendExemplars(ref, ls, es) return err diff --git a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go index a914277f92..7d79637803 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go @@ -17,6 +17,7 @@ import ( "bytes" "context" "errors" + "fmt" "math" "testing" "time" @@ -412,13 +413,13 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { reg := prometheus.NewRegistry() cappMetrics := NewCombinedAppenderMetrics(reg) app := db.Appender(ctx) - capp := NewCombinedAppender(app, logger, ingestCTZeroSample, cappMetrics) + capp := NewCombinedAppender(app, logger, ingestCTZeroSample, false, cappMetrics) tc.appendFunc(t, capp) require.NoError(t, app.Commit()) if tc.extraAppendFunc != nil { app = db.Appender(ctx) - capp = NewCombinedAppender(app, logger, ingestCTZeroSample, cappMetrics) + capp = NewCombinedAppender(app, logger, ingestCTZeroSample, false, cappMetrics) tc.extraAppendFunc(t, capp) require.NoError(t, app.Commit()) } @@ -501,7 +502,7 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { t.Run("happy case with CT zero, reference is passed and reused", func(t *testing.T) { app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) @@ -512,109 +513,38 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { }, })) - require.Len(t, app.records, 6) + require.Len(t, app.records, 5) requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) - requireEqualOpAndRef(t, "Append", ref, app.records[4]) - requireEqualOpAndRef(t, "AppendExemplar", ref, app.records[5]) + requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[2]) + requireEqualOpAndRef(t, "Append", ref, app.records[3]) + requireEqualOpAndRef(t, "AppendExemplar", ref, app.records[4]) }) t.Run("error on second CT ingest doesn't update the reference", func(t *testing.T) { app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) app.appendCTZeroSampleError = errors.New("test error") require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 3, 4, 62.0, nil)) - require.Len(t, app.records, 5) + require.Len(t, app.records, 4) requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) - require.Zero(t, app.records[3].outRef, "the second AppendCTZeroSample returned 0") - requireEqualOpAndRef(t, "Append", ref, app.records[4]) - }) - - t.Run("updateMetadata called when meta help changes", func(t *testing.T) { - app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) - - newMetadata := floatMetadata - newMetadata.Help = "some other help" - - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 4, 62.0, nil)) - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 5, 162.0, nil)) - - require.Len(t, app.records, 7) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) - ref := app.records[0].outRef - require.NotZero(t, ref) - requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) - requireEqualOpAndRef(t, "Append", ref, app.records[4]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[5]) - requireEqualOpAndRef(t, "Append", ref, app.records[6]) - }) - - t.Run("updateMetadata called when meta unit changes", func(t *testing.T) { - app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) - - newMetadata := floatMetadata - newMetadata.Unit = "seconds" - - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 4, 62.0, nil)) - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 5, 162.0, nil)) - - require.Len(t, app.records, 7) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) - ref := app.records[0].outRef - require.NotZero(t, ref) - requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) - requireEqualOpAndRef(t, "Append", ref, app.records[4]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[5]) - requireEqualOpAndRef(t, "Append", ref, app.records[6]) - }) - - t.Run("updateMetadata called when meta type changes", func(t *testing.T) { - app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) - - newMetadata := floatMetadata - newMetadata.Type = model.MetricTypeGauge - - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 4, 62.0, nil)) - require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 5, 162.0, nil)) - - require.Len(t, app.records, 7) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) - ref := app.records[0].outRef - require.NotZero(t, ref) - requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) - requireEqualOpAndRef(t, "Append", ref, app.records[4]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[5]) - requireEqualOpAndRef(t, "Append", ref, app.records[6]) + requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[2]) + require.Zero(t, app.records[2].outRef, "the second AppendCTZeroSample returned 0") + requireEqualOpAndRef(t, "Append", ref, app.records[3]) }) t.Run("metadata, exemplars are not updated if append failed", func(t *testing.T) { app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) app.appendError = errors.New("test error") require.Error(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 0, 1, 42.0, []exemplar.Exemplar{ { @@ -632,7 +562,7 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { t.Run("metadata, exemplars are updated if append failed but reference is valid", func(t *testing.T) { app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) newMetadata := floatMetadata newMetadata.Help = "some other help" @@ -661,7 +591,7 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { t.Run("simulate conflict with existing series", func(t *testing.T) { app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) ls := labels.FromStrings( model.MetricNameLabel, "test_bytes_total", @@ -688,23 +618,21 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { }, })) - require.Len(t, app.records, 7) + require.Len(t, app.records, 5) requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[3]) - newRef := app.records[3].outRef + requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[2]) + newRef := app.records[2].outRef require.NotEqual(t, ref, newRef, "the second AppendCTZeroSample returned a different reference") - requireEqualOpAndRef(t, "Append", newRef, app.records[4]) - requireEqualOpAndRef(t, "UpdateMetadata", newRef, app.records[5]) - requireEqualOpAndRef(t, "AppendExemplar", newRef, app.records[6]) + requireEqualOpAndRef(t, "Append", newRef, app.records[3]) + requireEqualOpAndRef(t, "AppendExemplar", newRef, app.records[4]) }) t.Run("check that invoking AppendHistogram returns an error for nil histogram", func(t *testing.T) { app := &appenderRecorder{} - capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) ls := labels.FromStrings( model.MetricNameLabel, "test_bytes_total", @@ -713,6 +641,101 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { err := capp.AppendHistogram(ls, Metadata{}, 4, 2, nil, nil) require.Error(t, err) }) + + for _, appendMetadata := range []bool{false, true} { + t.Run(fmt.Sprintf("appendMetadata=%t", appendMetadata), func(t *testing.T) { + app := &appenderRecorder{} + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, appendMetadata, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + + require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) + + if appendMetadata { + require.Len(t, app.records, 3) + requireEqualOp(t, "AppendCTZeroSample", app.records[0]) + requireEqualOp(t, "Append", app.records[1]) + requireEqualOp(t, "UpdateMetadata", app.records[2]) + } else { + require.Len(t, app.records, 2) + requireEqualOp(t, "AppendCTZeroSample", app.records[0]) + requireEqualOp(t, "Append", app.records[1]) + } + }) + } +} + +// TestCombinedAppenderMetadataChanges verifies that UpdateMetadata is called +// when metadata fields change (help, unit, or type). +func TestCombinedAppenderMetadataChanges(t *testing.T) { + seriesLabels := labels.FromStrings( + model.MetricNameLabel, "test_metric", + "foo", "bar", + ) + + baseMetadata := Metadata{ + Metadata: metadata.Metadata{ + Type: model.MetricTypeCounter, + Unit: "bytes", + Help: "original help", + }, + MetricFamilyName: "test_metric", + } + + tests := []struct { + name string + modifyMetadata func(Metadata) Metadata + }{ + { + name: "help changes", + modifyMetadata: func(m Metadata) Metadata { + m.Help = "new help text" + return m + }, + }, + { + name: "unit changes", + modifyMetadata: func(m Metadata) Metadata { + m.Unit = "seconds" + return m + }, + }, + { + name: "type changes", + modifyMetadata: func(m Metadata) Metadata { + m.Type = model.MetricTypeGauge + return m + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + app := &appenderRecorder{} + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, true, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + + newMetadata := tt.modifyMetadata(baseMetadata) + + require.NoError(t, capp.AppendSample(seriesLabels.Copy(), baseMetadata, 1, 2, 42.0, nil)) + require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 4, 62.0, nil)) + require.NoError(t, capp.AppendSample(seriesLabels.Copy(), newMetadata, 3, 5, 162.0, nil)) + + // Verify expected operations. + require.Len(t, app.records, 7) + requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) + ref := app.records[0].outRef + require.NotZero(t, ref) + requireEqualOpAndRef(t, "Append", ref, app.records[1]) + requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) + requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) + requireEqualOpAndRef(t, "Append", ref, app.records[4]) + requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[5]) + requireEqualOpAndRef(t, "Append", ref, app.records[6]) + }) + } +} + +func requireEqualOp(t *testing.T, expectedOp string, actual appenderRecord) { + t.Helper() + require.Equal(t, expectedOp, actual.op) } func requireEqualOpAndRef(t *testing.T, expectedOp string, expectedRef storage.SeriesRef, actual appenderRecord) { @@ -833,3 +856,82 @@ func (a *appenderRecorder) Rollback() error { func (*appenderRecorder) SetOptions(_ *storage.AppendOptions) { panic("not implemented") } + +func TestMetadataChangedLogic(t *testing.T) { + seriesLabels := labels.FromStrings(model.MetricNameLabel, "test_metric", "foo", "bar") + baseMetadata := Metadata{ + Metadata: metadata.Metadata{Type: model.MetricTypeCounter, Unit: "bytes", Help: "original"}, + MetricFamilyName: "test_metric", + } + + tests := []struct { + name string + appendMetadata bool + modifyMetadata func(Metadata) Metadata + expectWALCall bool + verifyCached func(*testing.T, metadata.Metadata) + }{ + { + name: "appendMetadata=false, no change", + appendMetadata: false, + modifyMetadata: func(m Metadata) Metadata { return m }, + expectWALCall: false, + verifyCached: func(t *testing.T, m metadata.Metadata) { require.Equal(t, "original", m.Help) }, + }, + { + name: "appendMetadata=false, help changes - cache updated, no WAL", + appendMetadata: false, + modifyMetadata: func(m Metadata) Metadata { m.Help = "changed"; return m }, + expectWALCall: false, + verifyCached: func(t *testing.T, m metadata.Metadata) { require.Equal(t, "changed", m.Help) }, + }, + { + name: "appendMetadata=true, help changes - cache and WAL updated", + appendMetadata: true, + modifyMetadata: func(m Metadata) Metadata { m.Help = "changed"; return m }, + expectWALCall: true, + verifyCached: func(t *testing.T, m metadata.Metadata) { require.Equal(t, "changed", m.Help) }, + }, + { + name: "appendMetadata=true, unit changes", + appendMetadata: true, + modifyMetadata: func(m Metadata) Metadata { m.Unit = "seconds"; return m }, + expectWALCall: true, + verifyCached: func(t *testing.T, m metadata.Metadata) { require.Equal(t, "seconds", m.Unit) }, + }, + { + name: "appendMetadata=true, type changes", + appendMetadata: true, + modifyMetadata: func(m Metadata) Metadata { m.Type = model.MetricTypeGauge; return m }, + expectWALCall: true, + verifyCached: func(t *testing.T, m metadata.Metadata) { require.Equal(t, model.MetricTypeGauge, m.Type) }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + app := &appenderRecorder{} + capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, tt.appendMetadata, NewCombinedAppenderMetrics(prometheus.NewRegistry())) + + require.NoError(t, capp.AppendSample(seriesLabels.Copy(), baseMetadata, 1, 2, 42.0, nil)) + + modifiedMetadata := tt.modifyMetadata(baseMetadata) + app.records = nil + require.NoError(t, capp.AppendSample(seriesLabels.Copy(), modifiedMetadata, 1, 3, 43.0, nil)) + + hash := seriesLabels.Hash() + cached, exists := capp.(*combinedAppender).refs[hash] + require.True(t, exists) + tt.verifyCached(t, cached.meta) + + updateMetadataCalled := false + for _, record := range app.records { + if record.op == "UpdateMetadata" { + updateMetadataCalled = true + break + } + } + require.Equal(t, tt.expectWALCall, updateMetadataCalled) + }) + } +} diff --git a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go index 341ee797cf..5675249153 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go @@ -1067,7 +1067,7 @@ func BenchmarkPrometheusConverter_FromMetrics(b *testing.B) { for b.Loop() { app := &noOpAppender{} - mockAppender := NewCombinedAppender(app, noOpLogger, false, appMetrics) + mockAppender := NewCombinedAppender(app, noOpLogger, false, false, appMetrics) converter := NewPrometheusConverter(mockAppender) annots, err := converter.FromMetrics(context.Background(), payload.Metrics(), settings) require.NoError(b, err) diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index e8559dd00e..5d1c561802 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -501,6 +501,8 @@ type OTLPOptions struct { // IngestCTZeroSample enables writing zero samples based on the start time // of metrics. IngestCTZeroSample bool + // AppendMetadata enables writing metadata to WAL when metadata-wal-records feature is enabled. + AppendMetadata bool } // NewOTLPWriteHandler creates a http.Handler that accepts OTLP write requests and @@ -519,6 +521,7 @@ func NewOTLPWriteHandler(logger *slog.Logger, reg prometheus.Registerer, appenda lookbackDelta: opts.LookbackDelta, ingestCTZeroSample: opts.IngestCTZeroSample, enableTypeAndUnitLabels: opts.EnableTypeAndUnitLabels, + appendMetadata: opts.AppendMetadata, // Register metrics. metrics: otlptranslator.NewCombinedAppenderMetrics(reg), } @@ -561,6 +564,7 @@ type rwExporter struct { lookbackDelta time.Duration ingestCTZeroSample bool enableTypeAndUnitLabels bool + appendMetadata bool // Metrics. metrics otlptranslator.CombinedAppenderMetrics @@ -572,7 +576,7 @@ func (rw *rwExporter) ConsumeMetrics(ctx context.Context, md pmetric.Metrics) er Appender: rw.appendable.Appender(ctx), maxTime: timestamp.FromTime(time.Now().Add(maxAheadTime)), } - combinedAppender := otlptranslator.NewCombinedAppender(app, rw.logger, rw.ingestCTZeroSample, rw.metrics) + combinedAppender := otlptranslator.NewCombinedAppender(app, rw.logger, rw.ingestCTZeroSample, rw.appendMetadata, rw.metrics) converter := otlptranslator.NewPrometheusConverter(combinedAppender) annots, err := converter.FromMetrics(ctx, md, otlptranslator.Settings{ AddMetricSuffixes: otlpCfg.TranslationStrategy.ShouldAddSuffixes(), diff --git a/storage/remote/write_test.go b/storage/remote/write_test.go index 6103a7f262..975caccd6c 100644 --- a/storage/remote/write_test.go +++ b/storage/remote/write_test.go @@ -921,6 +921,7 @@ func TestOTLPWriteHandler(t *testing.T) { t.Run(testCase.name, func(t *testing.T) { otlpOpts := OTLPOptions{ EnableTypeAndUnitLabels: testCase.typeAndUnitLabels, + AppendMetadata: true, } appendable := handleOTLP(t, exportRequest, testCase.otlpCfg, otlpOpts) for _, sample := range testCase.expectedSamples { diff --git a/web/api/v1/api.go b/web/api/v1/api.go index baddedd495..793e5a0075 100644 --- a/web/api/v1/api.go +++ b/web/api/v1/api.go @@ -348,6 +348,7 @@ func NewAPI( LookbackDelta: lookbackDelta, IngestCTZeroSample: ctZeroIngestionEnabled, EnableTypeAndUnitLabels: enableTypeAndUnitLabels, + AppendMetadata: appendMetadata, }) } From 85150f9dec6329a0566bea2fa1f71c963452bde3 Mon Sep 17 00:00:00 2001 From: Linas Medziunas Date: Thu, 13 Nov 2025 11:37:53 +0200 Subject: [PATCH 041/439] [PERF] PromQL: only reset labels builder when needed Signed-off-by: Linas Medziunas --- promql/engine.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/promql/engine.go b/promql/engine.go index 44cef1c1c6..74864bdcae 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -2951,7 +2951,6 @@ func resultMetric(lhs, rhs labels.Labels, op parser.ItemType, matching *parser.V enh.resultMetric = make(map[string]labels.Labels, len(enh.Out)) } - enh.resetBuilder(lhs) buf := bytes.NewBuffer(enh.lblResultBuf[:0]) enh.lblBuf = lhs.Bytes(enh.lblBuf) buf.Write(enh.lblBuf) @@ -2964,6 +2963,7 @@ func resultMetric(lhs, rhs labels.Labels, op parser.ItemType, matching *parser.V } str := string(enh.lblResultBuf) + enh.resetBuilder(lhs) if changesMetricSchema(op) { // Setting empty Metadata causes the deletion of those if they exists. schema.Metadata{}.SetToLabels(enh.lb) From 0f5f1955e5bf9ce11975b74e1e34a121097f90d7 Mon Sep 17 00:00:00 2001 From: Mohammad Alavi Date: Thu, 13 Nov 2025 17:17:51 +0700 Subject: [PATCH 042/439] promql: fix histogram_fraction issue when lower falls within the first bucket (#17424) Signed-off-by: Mohammad Alavi --- promql/promqltest/testdata/histograms.test | 377 +++++++++++++++++++++ promql/quantile.go | 34 +- 2 files changed, 410 insertions(+), 1 deletion(-) diff --git a/promql/promqltest/testdata/histograms.test b/promql/promqltest/testdata/histograms.test index 84a467a314..436390ee41 100644 --- a/promql/promqltest/testdata/histograms.test +++ b/promql/promqltest/testdata/histograms.test @@ -158,6 +158,383 @@ eval instant at 50m histogram_fraction(0, 0.2, rate(testhistogram3_bucket[10m])) {start="positive"} 0.6363636363636364 {start="negative"} 0 +# Positive buckets, lower falls in the first bucket. +load_with_nhcb 5m + positive_buckets_lower_falls_in_the_first_bucket_bucket{le="1"} 1+0x10 + positive_buckets_lower_falls_in_the_first_bucket_bucket{le="2"} 3+0x10 + positive_buckets_lower_falls_in_the_first_bucket_bucket{le="3"} 6+0x10 + positive_buckets_lower_falls_in_the_first_bucket_bucket{le="+Inf"} 100+0x10 + +# - Bucket [0, 1]: contributes 1.0 observation (full bucket). +# - Bucket [1, 2]: contributes (1.5-1)/(2-1) * (3-1) = 0.5 * 2 = 1.0 observations. +# Total: (1.0 + 1.0) / 100.0 = 0.02 + +eval instant at 50m histogram_fraction(0, 1.5, positive_buckets_lower_falls_in_the_first_bucket_bucket) + expect no_warn + {} 0.02 + +eval instant at 50m histogram_fraction(0, 1.5, positive_buckets_lower_falls_in_the_first_bucket) + expect no_warn + {} 0.02 + +# Negative buckets, lower falls in the first bucket. +load_with_nhcb 5m + negative_buckets_lower_falls_in_the_first_bucket_bucket{le="-3"} 10+0x10 + negative_buckets_lower_falls_in_the_first_bucket_bucket{le="-2"} 12+0x10 + negative_buckets_lower_falls_in_the_first_bucket_bucket{le="-1"} 15+0x10 + negative_buckets_lower_falls_in_the_first_bucket_bucket{le="+Inf"} 100+0x10 + +# - Bucket [-Inf, -3]: contributes zero observations (no interpolation with infinite width bucket). +# - Bucket [-3, -2]: contributes 12-10 = 2.0 observations (full bucket). +# Total: 2.0 / 100.0 = 0.02 + +eval instant at 50m histogram_fraction(-4, -2, negative_buckets_lower_falls_in_the_first_bucket_bucket) + expect no_warn + {} 0.02 + +eval instant at 50m histogram_fraction(-4, -2, negative_buckets_lower_falls_in_the_first_bucket) + expect no_warn + {} 0.02 + +# Lower is -Inf. +load_with_nhcb 5m + lower_is_negative_Inf_bucket{le="-3"} 10+0x10 + lower_is_negative_Inf_bucket{le="-2"} 12+0x10 + lower_is_negative_Inf_bucket{le="-1"} 15+0x10 + lower_is_negative_Inf_bucket{le="+Inf"} 100+0x10 + +# - Bucket [-Inf, -3]: contributes 10.0 observations (full bucket). +# - Bucket [-3, -2]: contributes 12-10 = 2.0 observations (full bucket). +# - Bucket [-2, -1]: contributes (-1.5-(-2))/(-1-(-2)) * (15-12) = 0.5 * 3 = 1.5 observations. +# Total: (10.0 + 2.0 + 1.5) / 100.0 = 0.135 + +eval instant at 50m histogram_fraction(-Inf, -1.5, lower_is_negative_Inf_bucket) + expect no_warn + {} 0.135 + +eval instant at 50m histogram_fraction(-Inf, -1.5, lower_is_negative_Inf) + expect no_warn + {} 0.135 + +# Lower is -Inf and upper is +Inf (positive buckets). +load_with_nhcb 5m + lower_is_negative_Inf_and_upper_is_positive_Inf__positive_buckets__bucket{le="1"} 1+0x10 + lower_is_negative_Inf_and_upper_is_positive_Inf__positive_buckets__bucket{le="2"} 3+0x10 + lower_is_negative_Inf_and_upper_is_positive_Inf__positive_buckets__bucket{le="3"} 6+0x10 + lower_is_negative_Inf_and_upper_is_positive_Inf__positive_buckets__bucket{le="+Inf"} 100+0x10 + +# Range [-Inf, +Inf] captures all observations. + +eval instant at 50m histogram_fraction(-Inf, +Inf, lower_is_negative_Inf_and_upper_is_positive_Inf__positive_buckets__bucket) + expect no_warn + {} 1.0 + +eval instant at 50m histogram_fraction(-Inf, +Inf, lower_is_negative_Inf_and_upper_is_positive_Inf__positive_buckets_) + expect no_warn + {} 1.0 + +# Lower is -Inf and upper is +Inf (negative buckets). +load_with_nhcb 5m + lower_is_negative_Inf_and_upper_is_positive_Inf__negative_buckets__bucket{le="-3"} 10+0x10 + lower_is_negative_Inf_and_upper_is_positive_Inf__negative_buckets__bucket{le="-2"} 12+0x10 + lower_is_negative_Inf_and_upper_is_positive_Inf__negative_buckets__bucket{le="-1"} 15+0x10 + lower_is_negative_Inf_and_upper_is_positive_Inf__negative_buckets__bucket{le="+Inf"} 100+0x10 + +# Range [-Inf, +Inf] captures all observations. + +eval instant at 50m histogram_fraction(-Inf, +Inf, lower_is_negative_Inf_and_upper_is_positive_Inf__negative_buckets__bucket) + expect no_warn + {} 1.0 + +eval instant at 50m histogram_fraction(-Inf, +Inf, lower_is_negative_Inf_and_upper_is_positive_Inf__negative_buckets_) + expect no_warn + {} 1.0 + +# Lower and upper fall in last bucket (positive buckets). +load_with_nhcb 5m + lower_and_upper_fall_in_last_bucket__positive_buckets__bucket{le="1"} 1+0x10 + lower_and_upper_fall_in_last_bucket__positive_buckets__bucket{le="2"} 3+0x10 + lower_and_upper_fall_in_last_bucket__positive_buckets__bucket{le="3"} 6+0x10 + lower_and_upper_fall_in_last_bucket__positive_buckets__bucket{le="+Inf"} 100+0x10 + +# - Bucket [3, +Inf]: contributes zero observations (no interpolation with infinite width bucket). +# Total: 0.0 / 100.0 = 0.0 + +eval instant at 50m histogram_fraction(4, 5, lower_and_upper_fall_in_last_bucket__positive_buckets__bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(4, 5, lower_and_upper_fall_in_last_bucket__positive_buckets_) + expect no_warn + {} 0.0 + +# Lower and upper fall in last bucket (negative buckets). +load_with_nhcb 5m + lower_and_upper_fall_in_last_bucket__negative_buckets__bucket{le="-3"} 10+0x10 + lower_and_upper_fall_in_last_bucket__negative_buckets__bucket{le="-2"} 12+0x10 + lower_and_upper_fall_in_last_bucket__negative_buckets__bucket{le="-1"} 15+0x10 + lower_and_upper_fall_in_last_bucket__negative_buckets__bucket{le="+Inf"} 100+0x10 + +# - Bucket [-1, +Inf]: contributes zero observations (no interpolation with infinite width bucket). +# Total: 0.0 / 100.0 = 0.0 + +eval instant at 50m histogram_fraction(0, 1, lower_and_upper_fall_in_last_bucket__negative_buckets__bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(0, 1, lower_and_upper_fall_in_last_bucket__negative_buckets_) + expect no_warn + {} 0.0 + +# Upper falls in last bucket. +load_with_nhcb 5m + upper_falls_in_last_bucket_bucket{le="1"} 1+0x10 + upper_falls_in_last_bucket_bucket{le="2"} 3+0x10 + upper_falls_in_last_bucket_bucket{le="3"} 6+0x10 + upper_falls_in_last_bucket_bucket{le="+Inf"} 100+0x10 + +# - Bucket [2, 3]: 6-3 = 3.0 observations (full bucket). +# - Bucket [3, +Inf]: contributes zero observations (no interpolation with infinite width bucket). +# Total: 3.0 / 100.0 = 0.03 + +eval instant at 50m histogram_fraction(2, 5, upper_falls_in_last_bucket_bucket) + expect no_warn + {} 0.03 + +eval instant at 50m histogram_fraction(2, 5, upper_falls_in_last_bucket) + expect no_warn + {} 0.03 + +# Upper is +Inf. +load_with_nhcb 5m + upper_is_positive_Inf_bucket{le="1"} 1+0x10 + upper_is_positive_Inf_bucket{le="2"} 3+0x10 + upper_is_positive_Inf_bucket{le="3"} 6+0x10 + upper_is_positive_Inf_bucket{le="+Inf"} 100+0x10 + +# All observations in +Inf bucket: 100-6 = 94.0 observations. +# Total: 94.0 / 100.0 = 0.94 + +eval instant at 50m histogram_fraction(400, +Inf, upper_is_positive_Inf_bucket) + expect no_warn + {} 0.94 + +eval instant at 50m histogram_fraction(400, +Inf, upper_is_positive_Inf) + expect no_warn + {} 0.94 + +# Lower equals upper. +load_with_nhcb 5m + lower_equals_upper_bucket{le="1"} 1+0x10 + lower_equals_upper_bucket{le="2"} 3+0x10 + lower_equals_upper_bucket{le="3"} 6+0x10 + lower_equals_upper_bucket{le="+Inf"} 100+0x10 + +# No observations can be captured in a zero-width range. + +eval instant at 50m histogram_fraction(2, 2, lower_equals_upper_bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(2, 2, lower_equals_upper) + expect no_warn + {} 0.0 + +# Lower greater than upper. +load_with_nhcb 5m + lower_greater_than_upper_bucket{le="1"} 1+0x10 + lower_greater_than_upper_bucket{le="2"} 3+0x10 + lower_greater_than_upper_bucket{le="3"} 6+0x10 + lower_greater_than_upper_bucket{le="+Inf"} 100+0x10 + +eval instant at 50m histogram_fraction(3, 2, lower_greater_than_upper_bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(3, 2, lower_greater_than_upper) + expect no_warn + {} 0.0 + +# Single bucket. +load_with_nhcb 5m + single_bucket_bucket{le="+Inf"} 100+0x10 + +# - Bucket [0, +Inf]: contributes zero observations (no interpolation with infinite width bucket). +# Total: 0.0 / 100.0 = 0.0 + +eval instant at 50m histogram_fraction(0, 1, single_bucket_bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(0, 1, single_bucket) + expect no_warn + {} 0.0 + +# All zero counts. +load_with_nhcb 5m + all_zero_counts_bucket{le="1"} 0+0x10 + all_zero_counts_bucket{le="2"} 0+0x10 + all_zero_counts_bucket{le="3"} 0+0x10 + all_zero_counts_bucket{le="+Inf"} 0+0x10 + +eval instant at 50m histogram_fraction(0, 5, all_zero_counts_bucket) + expect no_warn + {} NaN + +eval instant at 50m histogram_fraction(0, 5, all_zero_counts) + expect no_warn + {} NaN + +# Lower exactly on bucket boundary. +load_with_nhcb 5m + lower_exactly_on_bucket_boundary_bucket{le="1"} 1+0x10 + lower_exactly_on_bucket_boundary_bucket{le="2"} 3+0x10 + lower_exactly_on_bucket_boundary_bucket{le="3"} 6+0x10 + lower_exactly_on_bucket_boundary_bucket{le="+Inf"} 100+0x10 + +# - Bucket [2, 3]: 6-3 = 3.0 observations (full bucket). +# - Bucket [3, +Inf]: contributes zero observations (no interpolation with infinite width bucket). +# Total: 3.0 / 100.0 = 0.03 + +eval instant at 50m histogram_fraction(2, 3.5, lower_exactly_on_bucket_boundary_bucket) + expect no_warn + {} 0.03 + +eval instant at 50m histogram_fraction(2, 3.5, lower_exactly_on_bucket_boundary) + expect no_warn + {} 0.03 + +# Upper exactly on bucket boundary. +load_with_nhcb 5m + upper_exactly_on_bucket_boundary_bucket{le="1"} 1+0x10 + upper_exactly_on_bucket_boundary_bucket{le="2"} 3+0x10 + upper_exactly_on_bucket_boundary_bucket{le="3"} 6+0x10 + upper_exactly_on_bucket_boundary_bucket{le="+Inf"} 100+0x10 + +# - Bucket [0, 1]: (1.0-0.5)/(1.0-0.0) * 1.0 = 0.5 * 1.0 = 0.5 observations. +# - Bucket [1, 2]: 3-1 = 2.0 observations (full bucket). +# Total: (0.5 + 2.0) / 100.0 = 0.025 + +eval instant at 50m histogram_fraction(0.5, 2, upper_exactly_on_bucket_boundary_bucket) + expect no_warn + {} 0.025 + +eval instant at 50m histogram_fraction(0.5, 2, upper_exactly_on_bucket_boundary) + expect no_warn + {} 0.025 + +# Both bounds exactly on bucket boundaries. +load_with_nhcb 5m + both_bounds_exactly_on_bucket_boundaries_bucket{le="1"} 1+0x10 + both_bounds_exactly_on_bucket_boundaries_bucket{le="2"} 3+0x10 + both_bounds_exactly_on_bucket_boundaries_bucket{le="3"} 6+0x10 + both_bounds_exactly_on_bucket_boundaries_bucket{le="+Inf"} 100+0x10 + +# - Bucket [1, 2]: 3-1 = 2.0 observations (full bucket). +# - Bucket [2, 3]: 6-3 = 3.0 observations (full bucket). +# Total: (2.0 + 3.0) / 100.0 = 0.05 + +eval instant at 50m histogram_fraction(1, 3, both_bounds_exactly_on_bucket_boundaries_bucket) + expect no_warn + {} 0.05 + +eval instant at 50m histogram_fraction(1, 3, both_bounds_exactly_on_bucket_boundaries) + expect no_warn + {} 0.05 + +# Fractional bucket bounds. +load_with_nhcb 5m + fractional_bucket_bounds_bucket{le="0.5"} 2.5+0x10 + fractional_bucket_bounds_bucket{le="1"} 7.5+0x10 + fractional_bucket_bounds_bucket{le="+Inf"} 100+0x10 + +# - Bucket [0, 0.5]: (0.5-0.1)/(0.5-0.0) * 2.5 = 0.8 * 2.5 = 2.0 observations. +# - Bucket [0.5, 1.0]: (0.75-0.5)/(1.0-0.5) * (7.5-2.5) = 0.5 * 5.0 = 2.5 observations. +# Total: (2.0 + 2.5) / 100.0 = 0.045 + +eval instant at 50m histogram_fraction(0.1, 0.75, fractional_bucket_bounds_bucket) + expect no_warn + {} 0.045 + +eval instant at 50m histogram_fraction(0.1, 0.75, fractional_bucket_bounds) + expect no_warn + {} 0.045 + +# Range crosses zero. +load_with_nhcb 5m + range_crosses_zero_bucket{le="-2"} 5+0x10 + range_crosses_zero_bucket{le="-1"} 10+0x10 + range_crosses_zero_bucket{le="0"} 15+0x10 + range_crosses_zero_bucket{le="1"} 20+0x10 + range_crosses_zero_bucket{le="+Inf"} 100+0x10 + +# - Bucket [-1, 0]: 15-10 = 5.0 observations (full bucket). +# - Bucket [0, 1]: 20-15 = 5.0 observations (full bucket). +# Total: (5.0 + 5.0) / 100.0 = 0.1 + +eval instant at 50m histogram_fraction(-1, 1, range_crosses_zero_bucket) + expect no_warn + {} 0.1 + +eval instant at 50m histogram_fraction(-1, 1, range_crosses_zero) + expect no_warn + {} 0.1 + +# Lower is NaN. +load_with_nhcb 5m + lower_is_NaN_bucket{le="1"} 1+0x10 + lower_is_NaN_bucket{le="+Inf"} 100+0x10 + +eval instant at 50m histogram_fraction(NaN, 1, lower_is_NaN_bucket) + expect no_warn + {} NaN + +eval instant at 50m histogram_fraction(NaN, 1, lower_is_NaN) + expect no_warn + {} NaN + +# Upper is NaN. +load_with_nhcb 5m + upper_is_NaN_bucket{le="1"} 1+0x10 + upper_is_NaN_bucket{le="+Inf"} 100+0x10 + +eval instant at 50m histogram_fraction(0, NaN, upper_is_NaN_bucket) + expect no_warn + {} NaN + +eval instant at 50m histogram_fraction(0, NaN, upper_is_NaN) + expect no_warn + {} NaN + +# Range entirely below all buckets. +load_with_nhcb 5m + range_entirely_below_all_buckets_bucket{le="1"} 1+0x10 + range_entirely_below_all_buckets_bucket{le="2"} 3+0x10 + range_entirely_below_all_buckets_bucket{le="+Inf"} 10+0x10 + +eval instant at 50m histogram_fraction(-10, -5, range_entirely_below_all_buckets_bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(-10, -5, range_entirely_below_all_buckets) + expect no_warn + {} 0.0 + +# Range entirely above all buckets. +load_with_nhcb 5m + range_entirely_above_all_buckets_bucket{le="1"} 1+0x10 + range_entirely_above_all_buckets_bucket{le="2"} 3+0x10 + range_entirely_above_all_buckets_bucket{le="+Inf"} 10+0x10 + +eval instant at 50m histogram_fraction(5, 10, range_entirely_above_all_buckets_bucket) + expect no_warn + {} 0.0 + +eval instant at 50m histogram_fraction(5, 10, range_entirely_above_all_buckets) + expect no_warn + {} 0.0 + + # In the classic histogram, we can access the corresponding bucket (if # it exists) and divide by the count to get the same result. diff --git a/promql/quantile.go b/promql/quantile.go index 1454974107..78df925c51 100644 --- a/promql/quantile.go +++ b/promql/quantile.go @@ -406,6 +406,18 @@ func HistogramFraction(lower, upper float64, h *histogram.FloatHistogram, metric // consistent with the linear interpolation known from classic // histograms. It is also used for the zero bucket. interpolateLinearly := func(v float64) float64 { + // Note: `v` is a finite value. + // For buckets with infinite bounds, we cannot interpolate meaningfully. + // For +Inf upper bound, interpolation returns the cumulative count of the previous bucket + // as the second term in the interpolation formula yields 0 (finite/Inf). + // In other words, no observations from the last bucket are considered in the fraction calculation. + // For -Inf lower bound, however, the second term would be (v-(-Inf))/(upperBound-(-Inf)) = Inf/Inf = NaN. + // To achieve the same effect of no contribution as the +Inf bucket, handle the -Inf case by returning + // the cumulative count at the first bucket (which equals the bucket's count). + // In both cases, we effectively skip interpolation within the infinite-width bucket. + if b.Lower == math.Inf(-1) { + return b.Count + } return rank + b.Count*(v-b.Lower)/(b.Upper-b.Lower) } @@ -531,14 +543,34 @@ func BucketFraction(lower, upper float64, buckets Buckets) float64 { rank, lowerRank, upperRank float64 lowerSet, upperSet bool ) + + // If the upper bound of the first bucket is greater than 0, we assume + // we are dealing with positive buckets only and lowerBound for the + // first bucket is set to 0; otherwise it is set to -Inf. + lowerBound := 0.0 + if buckets[0].UpperBound <= 0 { + lowerBound = math.Inf(-1) + } + for i, b := range buckets { - lowerBound := math.Inf(-1) if i > 0 { lowerBound = buckets[i-1].UpperBound } upperBound := b.UpperBound interpolateLinearly := func(v float64) float64 { + // Note: `v` is a finite value. + // For buckets with infinite bounds, we cannot interpolate meaningfully. + // For +Inf upper bound, interpolation returns the cumulative count of the previous bucket + // as the second term in the interpolation formula yields 0 (finite/Inf). + // In other words, no observations from the last bucket are considered in the fraction calculation. + // For -Inf lower bound, however, the second term would be (v-(-Inf))/(upperBound-(-Inf)) = Inf/Inf = NaN. + // To achieve the same effect of no contribution as the +Inf bucket, handle the -Inf case by returning + // the cumulative count at the first bucket. + // In both cases, we effectively skip interpolation within the infinite-width bucket. + if lowerBound == math.Inf(-1) { + return b.Count + } return rank + (b.Count-rank)*(v-lowerBound)/(upperBound-lowerBound) } From f50ff0a40ad4ef24d9bb8e81a6546c8c994a924a Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Thu, 13 Nov 2025 15:17:51 +0100 Subject: [PATCH 043/439] feat: rename CreatedTimestamp to StartTimestamp (#17523) Partially fixes https://github.com/prometheus/prometheus/issues/17416 by renaming all CT* names to ST* in the whole codebase except RW2 (this is done in separate [PR](https://github.com/prometheus/prometheus/pull/17411)) and PrometheusProto exposition proto. ``` CreatedTimestamp -> StartTimestamp CreatedTimeStamp -> StartTimestamp created_timestamp -> start_timestamp CT -> ST ct -> st ``` Signed-off-by: bwplotka --- cmd/prometheus/main.go | 8 +- docs/feature_flags.md | 25 ++-- model/textparse/benchmark_test.go | 14 +- model/textparse/interface.go | 14 +- model/textparse/interface_test.go | 4 +- model/textparse/nhcbparse.go | 18 +-- model/textparse/nhcbparse_test.go | 82 +++++------ model/textparse/openmetricsparse.go | 92 ++++++------- model/textparse/openmetricsparse_test.go | 92 ++++++------- model/textparse/promparse.go | 4 +- model/textparse/protobufparse.go | 16 +-- model/textparse/protobufparse_test.go | 48 +++---- scrape/helpers_test.go | 14 +- scrape/manager.go | 2 +- scrape/manager_test.go | 86 ++++++------ scrape/scrape.go | 28 ++-- scrape/scrape_test.go | 4 +- storage/fanout.go | 12 +- storage/interface.go | 42 +++--- storage/remote/codec_test.go | 2 +- .../combined_appender.go | 40 +++--- .../combined_appender_test.go | 128 +++++++++--------- .../prometheusremotewrite/helper_test.go | 16 +-- .../prometheusremotewrite/histograms.go | 8 +- .../prometheusremotewrite/histograms_test.go | 24 ++-- .../metrics_to_prw_test.go | 4 +- .../number_data_points.go | 8 +- .../number_data_points_test.go | 2 +- storage/remote/write.go | 16 +-- storage/remote/write_handler.go | 48 +++---- storage/remote/write_handler_test.go | 56 ++++---- storage/remote/write_test.go | 36 ++--- tsdb/agent/db.go | 34 ++--- tsdb/agent/db_test.go | 44 +++--- tsdb/head_append.go | 84 ++++++------ tsdb/head_test.go | 98 +++++++------- web/api/v1/api.go | 6 +- web/web.go | 4 +- 38 files changed, 635 insertions(+), 628 deletions(-) diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go index d108e4c7a2..75b268322a 100644 --- a/cmd/prometheus/main.go +++ b/cmd/prometheus/main.go @@ -263,8 +263,8 @@ func (c *flagConfig) setFeatureListOptions(logger *slog.Logger) error { case "ooo-native-histograms": logger.Warn("This option for --enable-feature is now permanently enabled and therefore a no-op.", "option", o) case "created-timestamp-zero-ingestion": - c.scrape.EnableCreatedTimestampZeroIngestion = true - c.web.CTZeroIngestionEnabled = true + c.scrape.EnableStartTimestampZeroIngestion = true + c.web.STZeroIngestionEnabled = true // Change relevant global variables. Hacky, but it's hard to pass a new option or default to unmarshallers. config.DefaultConfig.GlobalConfig.ScrapeProtocols = config.DefaultProtoFirstScrapeProtocols config.DefaultGlobalConfig.ScrapeProtocols = config.DefaultProtoFirstScrapeProtocols @@ -1729,7 +1729,7 @@ func (notReadyAppender) AppendHistogram(storage.SeriesRef, labels.Labels, int64, return 0, tsdb.ErrNotReady } -func (notReadyAppender) AppendHistogramCTZeroSample(storage.SeriesRef, labels.Labels, int64, int64, *histogram.Histogram, *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (notReadyAppender) AppendHistogramSTZeroSample(storage.SeriesRef, labels.Labels, int64, int64, *histogram.Histogram, *histogram.FloatHistogram) (storage.SeriesRef, error) { return 0, tsdb.ErrNotReady } @@ -1737,7 +1737,7 @@ func (notReadyAppender) UpdateMetadata(storage.SeriesRef, labels.Labels, metadat return 0, tsdb.ErrNotReady } -func (notReadyAppender) AppendCTZeroSample(storage.SeriesRef, labels.Labels, int64, int64) (storage.SeriesRef, error) { +func (notReadyAppender) AppendSTZeroSample(storage.SeriesRef, labels.Labels, int64, int64) (storage.SeriesRef, error) { return 0, tsdb.ErrNotReady } diff --git a/docs/feature_flags.md b/docs/feature_flags.md index 1b3c21aae8..384b124c6a 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -67,20 +67,27 @@ Enables PromQL functions that are considered experimental. These functions might change their name, syntax, or semantics. They might also get removed entirely. -## Created Timestamps Zero Injection +## Start (Created) Timestamps Zero Injection `--enable-feature=created-timestamp-zero-ingestion` -Enables ingestion of created timestamp. Created timestamps are injected as 0 valued samples when appropriate. See [PromCon talk](https://youtu.be/nWf0BfQ5EEA) for details. +> NOTE: CreatedTimestamp feature was renamed to StartTimestamp for consistency. The above flag uses old name for stability. -Currently Prometheus supports created timestamps only on the traditional -Prometheus Protobuf protocol (WIP for other protocols). Therefore, enabling -this feature pre-sets the global `scrape_protocols` configuration option to -`[ PrometheusProto, OpenMetricsText1.0.0, OpenMetricsText0.0.1, PrometheusText0.0.4 ]`, -resulting in negotiating the Prometheus Protobuf protocol with first priority -(unless the `scrape_protocols` option is set to a different value explicitly). +Enables ingestion of start timestamp. Start timestamps are injected as 0 valued samples when appropriate. See [PromCon talk](https://youtu.be/nWf0BfQ5EEA) for details. -Besides enabling this feature in Prometheus, created timestamps need to be exposed by the application being scraped. +Currently, Prometheus supports start timestamps on the + +* `PrometheusProto` +* `OpenMetrics1.0.0` + + +From the above, Prometheus recommends `PrometheusProto`. This is because OpenMetrics 1.0 Start Timestamp information is shared as a `_created` metric and parsing those +are prone to errors and expensive (thus, adding an overhead). You also need to be careful to not pollute your Prometheus with extra `_created` metrics. + +Therefore, when `created-timestamp-zero-ingestion` is enabled Prometheus changes the global `scrape_protocols` default configuration option to +`[ PrometheusProto, OpenMetricsText1.0.0, OpenMetricsText0.0.1, PrometheusText0.0.4 ]`, resulting in negotiating the Prometheus Protobuf protocol first (unless the `scrape_protocols` option is set to a different value explicitly). + +Besides enabling this feature in Prometheus, start timestamps need to be exposed by the application being scraped. ## Concurrent evaluation of independent rules diff --git a/model/textparse/benchmark_test.go b/model/textparse/benchmark_test.go index 8445017ddf..510da72c6c 100644 --- a/model/textparse/benchmark_test.go +++ b/model/textparse/benchmark_test.go @@ -36,7 +36,7 @@ import ( // and allows comparison with expfmt decoders if applicable. // // NOTE(bwplotka): Previous iterations of this benchmark had different cases for isolated -// Series, Series+Metrics with and without reuse, Series+CT. Those cases are sometimes +// Series, Series+Metrics with and without reuse, Series+ST. Those cases are sometimes // good to know if you are working on a certain optimization, but it does not // make sense to persist such cases for everybody (e.g. for CI one day). // For local iteration, feel free to adjust cases/comment out code etc. @@ -153,7 +153,7 @@ func benchParse(b *testing.B, data []byte, parser string) { } case "omtext": newParserFn = func(b []byte, st *labels.SymbolTable) Parser { - return NewOpenMetricsParser(b, st, WithOMParserCTSeriesSkipped()) + return NewOpenMetricsParser(b, st, WithOMParserSTSeriesSkipped()) } case "omtext_with_nhcb": newParserFn = func(buf []byte, st *labels.SymbolTable) Parser { @@ -206,7 +206,7 @@ func benchParse(b *testing.B, data []byte, parser string) { } p.Labels(&res) - _ = p.CreatedTimestamp() + _ = p.StartTimestamp() for hasExemplar := p.Exemplar(&e); hasExemplar; hasExemplar = p.Exemplar(&e) { } } @@ -266,11 +266,11 @@ func readTestdataFile(tb testing.TB, file string) []byte { /* export bench=v1 && go test ./model/textparse/... \ - -run '^$' -bench '^BenchmarkCreatedTimestampPromProto' \ + -run '^$' -bench '^BenchmarkStartTimestampPromProto' \ -benchtime 2s -count 6 -cpu 2 -benchmem -timeout 999m \ | tee ${bench}.txt */ -func BenchmarkCreatedTimestampPromProto(b *testing.B) { +func BenchmarkStartTimestampPromProto(b *testing.B) { data := createTestProtoBuf(b).Bytes() st := labels.NewSymbolTable() @@ -301,7 +301,7 @@ Inner: b.ReportAllocs() b.ResetTimer() for b.Loop() { - if p.CreatedTimestamp() != 0 { + if p.StartTimestamp() != 0 { b.Fatal("should be nil") } } @@ -331,7 +331,7 @@ Inner2: b.ReportAllocs() b.ResetTimer() for b.Loop() { - if p.CreatedTimestamp() == 0 { + if p.StartTimestamp() == 0 { b.Fatal("should be not nil") } } diff --git a/model/textparse/interface.go b/model/textparse/interface.go index 37b1b761a0..bbc52290ad 100644 --- a/model/textparse/interface.go +++ b/model/textparse/interface.go @@ -29,7 +29,7 @@ import ( type Parser interface { // Series returns the bytes of a series with a simple float64 as a // value, the timestamp if set, and the value of the current sample. - // TODO(bwplotka): Similar to CreatedTimestamp, have ts == 0 meaning no timestamp provided. + // TODO(bwplotka): Similar to StartTimestamp, have ts == 0 meaning no timestamp provided. // We already accepted in many places (PRW, proto parsing histograms) that 0 timestamp is not a // a valid timestamp. If needed it can be represented as 0+1ms. Series() ([]byte, *int64, float64) @@ -38,7 +38,7 @@ type Parser interface { // value, the timestamp if set, and the histogram in the current sample. // Depending on the parsed input, the function returns an (integer) Histogram // or a FloatHistogram, with the respective other return value being nil. - // TODO(bwplotka): Similar to CreatedTimestamp, have ts == 0 meaning no timestamp provided. + // TODO(bwplotka): Similar to StartTimestamp, have ts == 0 meaning no timestamp provided. // We already accepted in many places (PRW, proto parsing histograms) that 0 timestamp is not a // a valid timestamp. If needed it can be represented as 0+1ms. Histogram() ([]byte, *int64, *histogram.Histogram, *histogram.FloatHistogram) @@ -76,10 +76,10 @@ type Parser interface { // retrieved (including the case where no exemplars exist at all). Exemplar(l *exemplar.Exemplar) bool - // CreatedTimestamp returns the created timestamp (in milliseconds) for the + // StartTimestamp returns the created timestamp (in milliseconds) for the // current sample. It returns 0 if it is unknown e.g. if it wasn't set or // if the scrape protocol or metric type does not support created timestamps. - CreatedTimestamp() int64 + StartTimestamp() int64 // Next advances the parser to the next sample. // It returns (EntryInvalid, io.EOF) if no samples were read. @@ -146,9 +146,9 @@ type ParserOptions struct { // that is also present as a native histogram. (Proto parsing only). KeepClassicOnClassicAndNativeHistograms bool - // OpenMetricsSkipCTSeries determines whether to skip `_created` timestamp series + // OpenMetricsSkipSTSeries determines whether to skip `_created` timestamp series // during (OpenMetrics parsing only). - OpenMetricsSkipCTSeries bool + OpenMetricsSkipSTSeries bool // FallbackContentType specifies the fallback content type to use when the provided // Content-Type header cannot be parsed or is not supported. @@ -175,7 +175,7 @@ func New(b []byte, contentType string, st *labels.SymbolTable, opts ParserOption switch mediaType { case "application/openmetrics-text": baseParser = NewOpenMetricsParser(b, st, func(o *openMetricsParserOptions) { - o.skipCTSeries = opts.OpenMetricsSkipCTSeries + o.skipSTSeries = opts.OpenMetricsSkipSTSeries o.enableTypeAndUnitLabels = opts.EnableTypeAndUnitLabels }) case "application/vnd.google.protobuf": diff --git a/model/textparse/interface_test.go b/model/textparse/interface_test.go index 532c474845..7030544793 100644 --- a/model/textparse/interface_test.go +++ b/model/textparse/interface_test.go @@ -195,7 +195,7 @@ type parsedEntry struct { lset labels.Labels t *int64 es []exemplar.Exemplar - ct int64 + st int64 // In EntryType. typ model.MetricType @@ -255,7 +255,7 @@ func testParse(t *testing.T, p Parser) (ret []parsedEntry) { } got.m = string(m) p.Labels(&got.lset) - got.ct = p.CreatedTimestamp() + got.st = p.StartTimestamp() for e := (exemplar.Exemplar{}); p.Exemplar(&e); { got.es = append(got.es, e) diff --git a/model/textparse/nhcbparse.go b/model/textparse/nhcbparse.go index 8ec541de8a..ab821f0e63 100644 --- a/model/textparse/nhcbparse.go +++ b/model/textparse/nhcbparse.go @@ -83,7 +83,7 @@ type NHCBParser struct { fhNHCB *histogram.FloatHistogram lsetNHCB labels.Labels exemplars []exemplar.Exemplar - ctNHCB int64 + stNHCB int64 metricStringNHCB string // Collates values from the classic histogram series to build @@ -92,7 +92,7 @@ type NHCBParser struct { tempNHCB convertnhcb.TempHistogram tempExemplars []exemplar.Exemplar tempExemplarCount int - tempCT int64 + tempST int64 // Remembers the last base histogram metric name (assuming it's // a classic histogram) so we can tell if the next float series @@ -159,16 +159,16 @@ func (p *NHCBParser) Exemplar(ex *exemplar.Exemplar) bool { return p.parser.Exemplar(ex) } -func (p *NHCBParser) CreatedTimestamp() int64 { +func (p *NHCBParser) StartTimestamp() int64 { switch p.state { case stateStart, stateInhibiting: if p.entry == EntrySeries || p.entry == EntryHistogram { - return p.parser.CreatedTimestamp() + return p.parser.StartTimestamp() } case stateCollecting: - return p.tempCT + return p.tempST case stateEmitting: - return p.ctNHCB + return p.stNHCB } return 0 } @@ -318,7 +318,7 @@ func (p *NHCBParser) handleClassicHistogramSeries(lset labels.Labels) bool { func (p *NHCBParser) processClassicHistogramSeries(lset labels.Labels, name string, updateHist func(*convertnhcb.TempHistogram)) { if p.state != stateCollecting { p.storeClassicLabels(name) - p.tempCT = p.parser.CreatedTimestamp() + p.tempST = p.parser.StartTimestamp() p.state = stateCollecting p.tempLsetNHCB = convertnhcb.GetHistogramMetricBase(lset, name) } @@ -385,13 +385,13 @@ func (p *NHCBParser) processNHCB() bool { p.bytesNHCB = []byte(p.metricStringNHCB) p.lsetNHCB = p.tempLsetNHCB p.swapExemplars() - p.ctNHCB = p.tempCT + p.stNHCB = p.tempST p.state = stateEmitting } else { p.state = stateStart } p.tempNHCB.Reset() p.tempExemplarCount = 0 - p.tempCT = 0 + p.tempST = 0 return err == nil } diff --git a/model/textparse/nhcbparse_test.go b/model/textparse/nhcbparse_test.go index f5836b5f7f..7e2f75ae63 100644 --- a/model/textparse/nhcbparse_test.go +++ b/model/textparse/nhcbparse_test.go @@ -67,13 +67,13 @@ ss{A="a"} 0 _metric_starting_with_underscore 1 testmetric{_label_starting_with_underscore="foo"} 1 testmetric{label="\"bar\""} 1 -# HELP foo Counter with and without labels to certify CT is parsed for both cases +# HELP foo Counter with and without labels to certify ST is parsed for both cases # TYPE foo counter foo_total 17.0 1520879607.789 # {id="counter-test"} 5 foo_created 1520872607.123 foo_total{a="b"} 17.0 1520879607.789 # {id="counter-test"} 5 foo_created{a="b"} 1520872607.123 -# HELP bar Summary with CT at the end, making sure we find CT even if it's multiple lines a far +# HELP bar Summary with ST at the end, making sure we find ST even if it's multiple lines a far # TYPE bar summary bar_count 17.0 bar_sum 324789.3 @@ -87,7 +87,7 @@ baz_bucket{le="+Inf"} 17 baz_count 17 baz_sum 324789.3 baz_created 1520872609.125 -# HELP fizz_created Gauge which shouldn't be parsed as CT +# HELP fizz_created Gauge which shouldn't be parsed as ST # TYPE fizz_created gauge fizz_created 17.0 # HELP something Histogram with _created between buckets and summary @@ -279,7 +279,7 @@ foobar{quantile="0.99"} 150.1` lset: labels.FromStrings("__name__", "testmetric", "label", `"bar"`), }, { m: "foo", - help: "Counter with and without labels to certify CT is parsed for both cases", + help: "Counter with and without labels to certify ST is parsed for both cases", }, { m: "foo", typ: model.MetricTypeCounter, @@ -289,17 +289,17 @@ foobar{quantile="0.99"} 150.1` lset: labels.FromStrings("__name__", "foo_total"), t: int64p(1520879607789), es: []exemplar.Exemplar{{Labels: labels.FromStrings("id", "counter-test"), Value: 5}}, - ct: 1520872607123, + st: 1520872607123, }, { m: `foo_total{a="b"}`, v: 17.0, lset: labels.FromStrings("__name__", "foo_total", "a", "b"), t: int64p(1520879607789), es: []exemplar.Exemplar{{Labels: labels.FromStrings("id", "counter-test"), Value: 5}}, - ct: 1520872607123, + st: 1520872607123, }, { m: "bar", - help: "Summary with CT at the end, making sure we find CT even if it's multiple lines a far", + help: "Summary with ST at the end, making sure we find ST even if it's multiple lines a far", }, { m: "bar", typ: model.MetricTypeSummary, @@ -307,22 +307,22 @@ foobar{quantile="0.99"} 150.1` m: "bar_count", v: 17.0, lset: labels.FromStrings("__name__", "bar_count"), - ct: 1520872608124, + st: 1520872608124, }, { m: "bar_sum", v: 324789.3, lset: labels.FromStrings("__name__", "bar_sum"), - ct: 1520872608124, + st: 1520872608124, }, { m: `bar{quantile="0.95"}`, v: 123.7, lset: labels.FromStrings("__name__", "bar", "quantile", "0.95"), - ct: 1520872608124, + st: 1520872608124, }, { m: `bar{quantile="0.99"}`, v: 150.0, lset: labels.FromStrings("__name__", "bar", "quantile", "0.99"), - ct: 1520872608124, + st: 1520872608124, }, { m: "baz", help: "Histogram with the same objective as above's summary", @@ -340,10 +340,10 @@ foobar{quantile="0.99"} 150.1` CustomValues: []float64{0.0}, // We do not store the +Inf boundary. }, lset: labels.FromStrings("__name__", "baz"), - ct: 1520872609125, + st: 1520872609125, }, { m: "fizz_created", - help: "Gauge which shouldn't be parsed as CT", + help: "Gauge which shouldn't be parsed as ST", }, { m: "fizz_created", typ: model.MetricTypeGauge, @@ -368,7 +368,7 @@ foobar{quantile="0.99"} 150.1` CustomValues: []float64{0.0}, // We do not store the +Inf boundary. }, lset: labels.FromStrings("__name__", "something"), - ct: 1520430001000, + st: 1520430001000, }, { m: `something{a="b"}`, shs: &histogram.Histogram{ @@ -380,7 +380,7 @@ foobar{quantile="0.99"} 150.1` CustomValues: []float64{0.0}, // We do not store the +Inf boundary. }, lset: labels.FromStrings("__name__", "something", "a", "b"), - ct: 1520430002000, + st: 1520430002000, }, { m: "yum", help: "Summary with _created between sum and quantiles", @@ -391,22 +391,22 @@ foobar{quantile="0.99"} 150.1` m: `yum_count`, v: 20, lset: labels.FromStrings("__name__", "yum_count"), - ct: 1520430003000, + st: 1520430003000, }, { m: `yum_sum`, v: 324789.5, lset: labels.FromStrings("__name__", "yum_sum"), - ct: 1520430003000, + st: 1520430003000, }, { m: `yum{quantile="0.95"}`, v: 123.7, lset: labels.FromStrings("__name__", "yum", "quantile", "0.95"), - ct: 1520430003000, + st: 1520430003000, }, { m: `yum{quantile="0.99"}`, v: 150.0, lset: labels.FromStrings("__name__", "yum", "quantile", "0.99"), - ct: 1520430003000, + st: 1520430003000, }, { m: "foobar", help: "Summary with _created as the first line", @@ -417,22 +417,22 @@ foobar{quantile="0.99"} 150.1` m: `foobar_count`, v: 21, lset: labels.FromStrings("__name__", "foobar_count"), - ct: 1520430004000, + st: 1520430004000, }, { m: `foobar_sum`, v: 324789.6, lset: labels.FromStrings("__name__", "foobar_sum"), - ct: 1520430004000, + st: 1520430004000, }, { m: `foobar{quantile="0.95"}`, v: 123.8, lset: labels.FromStrings("__name__", "foobar", "quantile", "0.95"), - ct: 1520430004000, + st: 1520430004000, }, { m: `foobar{quantile="0.99"}`, v: 150.1, lset: labels.FromStrings("__name__", "foobar", "quantile", "0.99"), - ct: 1520430004000, + st: 1520430004000, }, { m: "metric", help: "foo\x00bar", @@ -587,8 +587,8 @@ func TestNHCBParser_NoNHCBWhenExponential(t *testing.T) { } type parserOptions struct { - useUTF8sep bool - hasCreatedTimeStamp bool + useUTF8sep bool + hasStartTimestamp bool } // Defines the parser name, the Parser factory and the test cases // supported by the parser and parser options. @@ -598,14 +598,14 @@ func TestNHCBParser_NoNHCBWhenExponential(t *testing.T) { inputBuf := createTestProtoBufHistogram(t) return New(inputBuf.Bytes(), "application/vnd.google.protobuf", labels.NewSymbolTable(), ParserOptions{KeepClassicOnClassicAndNativeHistograms: keepClassic, ConvertClassicHistogramsToNHCB: nhcb}) } - return "ProtoBuf", factory, []int{1, 2, 3}, parserOptions{useUTF8sep: true, hasCreatedTimeStamp: true} + return "ProtoBuf", factory, []int{1, 2, 3}, parserOptions{useUTF8sep: true, hasStartTimestamp: true} }, func() (string, parserFactory, []int, parserOptions) { factory := func(keepClassic, nhcb bool) (Parser, error) { input := createTestOpenMetricsHistogram() return New([]byte(input), "application/openmetrics-text", labels.NewSymbolTable(), ParserOptions{KeepClassicOnClassicAndNativeHistograms: keepClassic, ConvertClassicHistogramsToNHCB: nhcb}) } - return "OpenMetrics", factory, []int{1}, parserOptions{hasCreatedTimeStamp: true} + return "OpenMetrics", factory, []int{1}, parserOptions{hasStartTimestamp: true} }, func() (string, parserFactory, []int, parserOptions) { factory := func(keepClassic, nhcb bool) (Parser, error) { @@ -643,9 +643,9 @@ func TestNHCBParser_NoNHCBWhenExponential(t *testing.T) { typ: model.MetricTypeHistogram, }) - var ct int64 - if options.hasCreatedTimeStamp { - ct = 1000 + var st int64 + if options.hasStartTimestamp { + st = 1000 } var bucketForMetric func(string) string @@ -677,7 +677,7 @@ func TestNHCBParser_NoNHCBWhenExponential(t *testing.T) { }, lset: labels.FromStrings("__name__", metric), t: int64p(1234568), - ct: ct, + st: st, }, } tc.exp = append(tc.exp, exponentialSeries...) @@ -690,42 +690,42 @@ func TestNHCBParser_NoNHCBWhenExponential(t *testing.T) { v: 175, lset: labels.FromStrings("__name__", metric+"_count"), t: int64p(1234568), - ct: ct, + st: st, }, { m: metric + "_sum", v: 0.0008280461746287094, lset: labels.FromStrings("__name__", metric+"_sum"), t: int64p(1234568), - ct: ct, + st: st, }, { m: metric + bucketForMetric("-0.0004899999999999998"), v: 2, lset: labels.FromStrings("__name__", metric+"_bucket", "le", "-0.0004899999999999998"), t: int64p(1234568), - ct: ct, + st: st, }, { m: metric + bucketForMetric("-0.0003899999999999998"), v: 4, lset: labels.FromStrings("__name__", metric+"_bucket", "le", "-0.0003899999999999998"), t: int64p(1234568), - ct: ct, + st: st, }, { m: metric + bucketForMetric("-0.0002899999999999998"), v: 16, lset: labels.FromStrings("__name__", metric+"_bucket", "le", "-0.0002899999999999998"), t: int64p(1234568), - ct: ct, + st: st, }, { m: metric + bucketForMetric("+Inf"), v: 175, lset: labels.FromStrings("__name__", metric+"_bucket", "le", "+Inf"), t: int64p(1234568), - ct: ct, + st: st, }, } tc.exp = append(tc.exp, classicSeries...) @@ -745,7 +745,7 @@ func TestNHCBParser_NoNHCBWhenExponential(t *testing.T) { }, lset: labels.FromStrings("__name__", metric), t: int64p(1234568), - ct: ct, + st: st, }, } tc.exp = append(tc.exp, nhcbSeries...) @@ -952,7 +952,7 @@ something_bucket{a="b",le="+Inf"} 9 CustomValues: []float64{0.0}, // We do not store the +Inf boundary. }, lset: labels.FromStrings("__name__", "something", "a", "b"), - ct: 1520430002000, + st: 1520430002000, }, } @@ -1061,7 +1061,7 @@ metric: < }, lset: labels.FromStrings("__name__", "test_histogram1"), t: int64p(1234568), - ct: 1000, + st: 1000, }, { m: "test_histogram2", @@ -1083,7 +1083,7 @@ metric: < }, lset: labels.FromStrings("__name__", "test_histogram2"), t: int64p(1234568), - ct: 1000, + st: 1000, }, } diff --git a/model/textparse/openmetricsparse.go b/model/textparse/openmetricsparse.go index 505e45fc40..207ceb4573 100644 --- a/model/textparse/openmetricsparse.go +++ b/model/textparse/openmetricsparse.go @@ -103,34 +103,34 @@ type OpenMetricsParser struct { hasExemplarTs bool // Created timestamp parsing state. - ct int64 - ctHashSet uint64 + st int64 + stHashSet uint64 // ignoreExemplar instructs the parser to not overwrite exemplars (to keep them while peeking ahead). ignoreExemplar bool // visitedMFName is the metric family name of the last visited metric when peeking ahead - // for _created series during the execution of the CreatedTimestamp method. + // for _created series during the execution of the StartTimestamp method. visitedMFName []byte - skipCTSeries bool + skipSTSeries bool enableTypeAndUnitLabels bool } type openMetricsParserOptions struct { - skipCTSeries bool + skipSTSeries bool enableTypeAndUnitLabels bool } type OpenMetricsOption func(*openMetricsParserOptions) -// WithOMParserCTSeriesSkipped turns off exposing _created lines +// WithOMParserSTSeriesSkipped turns off exposing _created lines // as series, which makes those only used for parsing created timestamp -// for `CreatedTimestamp` method purposes. +// for `StartTimestamp` method purposes. // // It's recommended to use this option to avoid using _created lines for other // purposes than created timestamp, but leave false by default for the // best-effort compatibility. -func WithOMParserCTSeriesSkipped() OpenMetricsOption { +func WithOMParserSTSeriesSkipped() OpenMetricsOption { return func(o *openMetricsParserOptions) { - o.skipCTSeries = true + o.skipSTSeries = true } } @@ -142,7 +142,7 @@ func WithOMParserTypeAndUnitLabels() OpenMetricsOption { } } -// NewOpenMetricsParser returns a new parser for the byte slice with option to skip CT series parsing. +// NewOpenMetricsParser returns a new parser for the byte slice with option to skip ST series parsing. func NewOpenMetricsParser(b []byte, st *labels.SymbolTable, opts ...OpenMetricsOption) Parser { options := &openMetricsParserOptions{} @@ -153,7 +153,7 @@ func NewOpenMetricsParser(b []byte, st *labels.SymbolTable, opts ...OpenMetricsO parser := &OpenMetricsParser{ l: &openMetricsLexer{b: b}, builder: labels.NewScratchBuilderWithSymbolTable(st, 16), - skipCTSeries: options.skipCTSeries, + skipSTSeries: options.skipSTSeries, enableTypeAndUnitLabels: options.enableTypeAndUnitLabels, } @@ -285,12 +285,12 @@ func (p *OpenMetricsParser) Exemplar(e *exemplar.Exemplar) bool { return true } -// CreatedTimestamp returns the created timestamp for a current Metric if exists or nil. +// StartTimestamp returns the created timestamp for a current Metric if exists or nil. // NOTE(Maniktherana): Might use additional CPU/mem resources due to deep copy of parser required for peeking given 1.0 OM specification on _created series. -func (p *OpenMetricsParser) CreatedTimestamp() int64 { - if !typeRequiresCT(p.mtype) { - // Not a CT supported metric type, fast path. - p.ctHashSet = 0 // Use ctHashSet as a single way of telling "empty cache" +func (p *OpenMetricsParser) StartTimestamp() int64 { + if !typeRequiresST(p.mtype) { + // Not a ST supported metric type, fast path. + p.stHashSet = 0 // Use stHashSet as a single way of telling "empty cache" return 0 } @@ -307,8 +307,8 @@ func (p *OpenMetricsParser) CreatedTimestamp() int64 { currHash := p.seriesHash(&buf, currName) // Check cache, perhaps we fetched something already. - if currHash == p.ctHashSet && p.ct > 0 { - return p.ct + if currHash == p.stHashSet && p.st > 0 { + return p.st } // Create a new lexer and other core state details to reset the parser once this function is done executing. @@ -322,7 +322,7 @@ func (p *OpenMetricsParser) CreatedTimestamp() int64 { resetStart := p.start resetMType := p.mtype - p.skipCTSeries = false + p.skipSTSeries = false p.ignoreExemplar = true defer func() { p.l = resetLexer @@ -334,38 +334,38 @@ func (p *OpenMetricsParser) CreatedTimestamp() int64 { for { eType, err := p.Next() if err != nil { - // This means p.Next() will give error too later on, so def no CT line found. - // This might result in partial scrape with wrong/missing CT, but only + // This means p.Next() will give error too later on, so def no ST line found. + // This might result in partial scrape with wrong/missing ST, but only // spec improvement would help. - // TODO: Make sure OM 1.1/2.0 pass CT via metadata or exemplar-like to avoid this. - p.resetCTParseValues() + // TODO: Make sure OM 1.1/2.0 pass ST via metadata or exemplar-like to avoid this. + p.resetSTParseValues() return 0 } if eType != EntrySeries { - // Assume we hit different family, no CT line found. - p.resetCTParseValues() + // Assume we hit different family, no ST line found. + p.resetSTParseValues() return 0 } peekedName := p.series[p.offsets[0]-p.start : p.offsets[1]-p.start] if len(peekedName) < 8 || string(peekedName[len(peekedName)-8:]) != "_created" { - // Not a CT line, search more. + // Not a ST line, search more. continue } // Remove _created suffix. peekedHash := p.seriesHash(&buf, peekedName[:len(peekedName)-8]) if peekedHash != currHash { - // Found CT line for a different series, for our series no CT. - p.resetCTParseValues() + // Found ST line for a different series, for our series no ST. + p.resetSTParseValues() return 0 } // All timestamps in OpenMetrics are Unix Epoch in seconds. Convert to milliseconds. // https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#timestamps - ct := int64(p.val * 1000.0) - p.setCTParseValues(ct, currHash, currName, true) - return ct + st := int64(p.val * 1000.0) + p.setSTParseValues(st, currHash, currName, true) + return st } } @@ -404,23 +404,23 @@ func (p *OpenMetricsParser) seriesHash(offsetsArr *[]byte, metricFamilyName []by return hashedOffsets } -// setCTParseValues sets the parser to the state after CreatedTimestamp method was called and CT was found. -// This is useful to prevent re-parsing the same series again and early return the CT value. -func (p *OpenMetricsParser) setCTParseValues(ct int64, ctHashSet uint64, mfName []byte, skipCTSeries bool) { - p.ct = ct - p.ctHashSet = ctHashSet +// setSTParseValues sets the parser to the state after StartTimestamp method was called and ST was found. +// This is useful to prevent re-parsing the same series again and early return the ST value. +func (p *OpenMetricsParser) setSTParseValues(st int64, stHashSet uint64, mfName []byte, skipSTSeries bool) { + p.st = st + p.stHashSet = stHashSet p.visitedMFName = mfName - p.skipCTSeries = skipCTSeries // Do we need to set it? + p.skipSTSeries = skipSTSeries // Do we need to set it? } -// resetCTParseValues resets the parser to the state before CreatedTimestamp method was called. -func (p *OpenMetricsParser) resetCTParseValues() { - p.ctHashSet = 0 - p.skipCTSeries = true +// resetSTParseValues resets the parser to the state before StartTimestamp method was called. +func (p *OpenMetricsParser) resetSTParseValues() { + p.stHashSet = 0 + p.skipSTSeries = true } -// typeRequiresCT returns true if the metric type requires a _created timestamp. -func typeRequiresCT(t model.MetricType) bool { +// typeRequiresST returns true if the metric type requires a _created timestamp. +func typeRequiresST(t model.MetricType) bool { switch t { case model.MetricTypeCounter, model.MetricTypeSummary, model.MetricTypeHistogram: return true @@ -544,7 +544,7 @@ func (p *OpenMetricsParser) Next() (Entry, error) { if err := p.parseSeriesEndOfLine(p.nextToken()); err != nil { return EntryInvalid, err } - if p.skipCTSeries && p.isCreatedSeries() { + if p.skipSTSeries && p.isCreatedSeries() { return p.Next() } return EntrySeries, nil @@ -565,7 +565,7 @@ func (p *OpenMetricsParser) Next() (Entry, error) { if err := p.parseSeriesEndOfLine(t2); err != nil { return EntryInvalid, err } - if p.skipCTSeries && p.isCreatedSeries() { + if p.skipSTSeries && p.isCreatedSeries() { return p.Next() } return EntrySeries, nil @@ -697,7 +697,7 @@ func (p *OpenMetricsParser) parseLVals(offsets []int, isExemplar bool) ([]int, e func (p *OpenMetricsParser) isCreatedSeries() bool { metricName := p.series[p.offsets[0]-p.start : p.offsets[1]-p.start] // check length so the metric is longer than len("_created") - if typeRequiresCT(p.mtype) && len(metricName) >= 8 && string(metricName[len(metricName)-8:]) == "_created" { + if typeRequiresST(p.mtype) && len(metricName) >= 8 && string(metricName[len(metricName)-8:]) == "_created" { return true } return false diff --git a/model/textparse/openmetricsparse_test.go b/model/textparse/openmetricsparse_test.go index 35536e7861..f0bbab309e 100644 --- a/model/textparse/openmetricsparse_test.go +++ b/model/textparse/openmetricsparse_test.go @@ -66,7 +66,7 @@ ss{A="a"} 0 _metric_starting_with_underscore 1 testmetric{_label_starting_with_underscore="foo"} 1 testmetric{label="\"bar\""} 1 -# HELP foo Counter with and without labels to certify CT is parsed for both cases +# HELP foo Counter with and without labels to certify ST is parsed for both cases # TYPE foo counter foo_total 17.0 1520879607.789 # {id="counter-test"} 5 foo_created 1520872607.123 @@ -75,7 +75,7 @@ foo_created{a="b"} 1520872607.123 foo_total{le="c"} 21.0 foo_created{le="c"} 1520872621.123 foo_total{le="1"} 10.0 -# HELP bar Summary with CT at the end, making sure we find CT even if it's multiple lines a far +# HELP bar Summary with ST at the end, making sure we find ST even if it's multiple lines a far # TYPE bar summary bar_count 17.0 bar_sum 324789.3 @@ -89,7 +89,7 @@ baz_bucket{le="+Inf"} 17 baz_count 17 baz_sum 324789.3 baz_created 1520872609.125 -# HELP fizz_created Gauge which shouldn't be parsed as CT +# HELP fizz_created Gauge which shouldn't be parsed as ST # TYPE fizz_created gauge fizz_created 17.0 # HELP something Histogram with _created between buckets and summary @@ -351,7 +351,7 @@ foobar{quantile="0.99"} 150.1` lset: labels.FromStrings("__name__", "testmetric", "label", `"bar"`), }, { m: "foo", - help: "Counter with and without labels to certify CT is parsed for both cases", + help: "Counter with and without labels to certify ST is parsed for both cases", }, { m: "foo", typ: model.MetricTypeCounter, @@ -367,7 +367,7 @@ foobar{quantile="0.99"} 150.1` es: []exemplar.Exemplar{ {Labels: labels.FromStrings("id", "counter-test"), Value: 5}, }, - ct: 1520872607123, + st: 1520872607123, }, { m: `foo_total{a="b"}`, v: 17.0, @@ -380,7 +380,7 @@ foobar{quantile="0.99"} 150.1` es: []exemplar.Exemplar{ {Labels: labels.FromStrings("id", "counter-test"), Value: 5}, }, - ct: 1520872607123, + st: 1520872607123, }, { m: `foo_total{le="c"}`, v: 21.0, @@ -389,7 +389,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "foo_total", "__type__", string(model.MetricTypeCounter), "le", "c"), labels.FromStrings("__name__", "foo_total", "le", "c"), ), - ct: 1520872621123, + st: 1520872621123, }, { m: `foo_total{le="1"}`, v: 10.0, @@ -400,7 +400,7 @@ foobar{quantile="0.99"} 150.1` ), }, { m: "bar", - help: "Summary with CT at the end, making sure we find CT even if it's multiple lines a far", + help: "Summary with ST at the end, making sure we find ST even if it's multiple lines a far", }, { m: "bar", typ: model.MetricTypeSummary, @@ -412,7 +412,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "bar_count", "__type__", string(model.MetricTypeSummary)), labels.FromStrings("__name__", "bar_count"), ), - ct: 1520872608124, + st: 1520872608124, }, { m: "bar_sum", v: 324789.3, @@ -421,7 +421,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "bar_sum", "__type__", string(model.MetricTypeSummary)), labels.FromStrings("__name__", "bar_sum"), ), - ct: 1520872608124, + st: 1520872608124, }, { m: `bar{quantile="0.95"}`, v: 123.7, @@ -430,7 +430,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "bar", "__type__", string(model.MetricTypeSummary), "quantile", "0.95"), labels.FromStrings("__name__", "bar", "quantile", "0.95"), ), - ct: 1520872608124, + st: 1520872608124, }, { m: `bar{quantile="0.99"}`, v: 150.0, @@ -439,7 +439,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "bar", "__type__", string(model.MetricTypeSummary), "quantile", "0.99"), labels.FromStrings("__name__", "bar", "quantile", "0.99"), ), - ct: 1520872608124, + st: 1520872608124, }, { m: "baz", help: "Histogram with the same objective as above's summary", @@ -454,7 +454,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "baz_bucket", "__type__", string(model.MetricTypeHistogram), "le", "0.0"), labels.FromStrings("__name__", "baz_bucket", "le", "0.0"), ), - ct: 1520872609125, + st: 1520872609125, }, { m: `baz_bucket{le="+Inf"}`, v: 17, @@ -463,7 +463,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "baz_bucket", "__type__", string(model.MetricTypeHistogram), "le", "+Inf"), labels.FromStrings("__name__", "baz_bucket", "le", "+Inf"), ), - ct: 1520872609125, + st: 1520872609125, }, { m: `baz_count`, v: 17, @@ -472,7 +472,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "baz_count", "__type__", string(model.MetricTypeHistogram)), labels.FromStrings("__name__", "baz_count"), ), - ct: 1520872609125, + st: 1520872609125, }, { m: `baz_sum`, v: 324789.3, @@ -481,10 +481,10 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "baz_sum", "__type__", string(model.MetricTypeHistogram)), labels.FromStrings("__name__", "baz_sum"), ), - ct: 1520872609125, + st: 1520872609125, }, { m: "fizz_created", - help: "Gauge which shouldn't be parsed as CT", + help: "Gauge which shouldn't be parsed as ST", }, { m: "fizz_created", typ: model.MetricTypeGauge, @@ -510,7 +510,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "something_count", "__type__", string(model.MetricTypeHistogram)), labels.FromStrings("__name__", "something_count"), ), - ct: 1520430001000, + st: 1520430001000, }, { m: `something_sum`, v: 324789.4, @@ -519,7 +519,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "something_sum", "__type__", string(model.MetricTypeHistogram)), labels.FromStrings("__name__", "something_sum"), ), - ct: 1520430001000, + st: 1520430001000, }, { m: `something_bucket{le="0.0"}`, v: 1, @@ -528,7 +528,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "something_bucket", "__type__", string(model.MetricTypeHistogram), "le", "0.0"), labels.FromStrings("__name__", "something_bucket", "le", "0.0"), ), - ct: 1520430001000, + st: 1520430001000, }, { m: `something_bucket{le="1"}`, v: 2, @@ -537,7 +537,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "something_bucket", "__type__", string(model.MetricTypeHistogram), "le", "1.0"), labels.FromStrings("__name__", "something_bucket", "le", "1.0"), ), - ct: 1520430001000, + st: 1520430001000, }, { m: `something_bucket{le="+Inf"}`, v: 18, @@ -546,7 +546,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "something_bucket", "__type__", string(model.MetricTypeHistogram), "le", "+Inf"), labels.FromStrings("__name__", "something_bucket", "le", "+Inf"), ), - ct: 1520430001000, + st: 1520430001000, }, { m: "yum", help: "Summary with _created between sum and quantiles", @@ -561,7 +561,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "yum_count", "__type__", string(model.MetricTypeSummary)), labels.FromStrings("__name__", "yum_count"), ), - ct: 1520430003000, + st: 1520430003000, }, { m: `yum_sum`, v: 324789.5, @@ -570,7 +570,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "yum_sum", "__type__", string(model.MetricTypeSummary)), labels.FromStrings("__name__", "yum_sum"), ), - ct: 1520430003000, + st: 1520430003000, }, { m: `yum{quantile="0.95"}`, v: 123.7, @@ -579,7 +579,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "yum", "__type__", string(model.MetricTypeSummary), "quantile", "0.95"), labels.FromStrings("__name__", "yum", "quantile", "0.95"), ), - ct: 1520430003000, + st: 1520430003000, }, { m: `yum{quantile="0.99"}`, v: 150.0, @@ -588,7 +588,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "yum", "__type__", string(model.MetricTypeSummary), "quantile", "0.99"), labels.FromStrings("__name__", "yum", "quantile", "0.99"), ), - ct: 1520430003000, + st: 1520430003000, }, { m: "foobar", help: "Summary with _created as the first line", @@ -603,7 +603,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "foobar_count", "__type__", string(model.MetricTypeSummary)), labels.FromStrings("__name__", "foobar_count"), ), - ct: 1520430004000, + st: 1520430004000, }, { m: `foobar_sum`, v: 324789.6, @@ -612,7 +612,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "foobar_sum", "__type__", string(model.MetricTypeSummary)), labels.FromStrings("__name__", "foobar_sum"), ), - ct: 1520430004000, + st: 1520430004000, }, { m: `foobar{quantile="0.95"}`, v: 123.8, @@ -621,7 +621,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "foobar", "__type__", string(model.MetricTypeSummary), "quantile", "0.95"), labels.FromStrings("__name__", "foobar", "quantile", "0.95"), ), - ct: 1520430004000, + st: 1520430004000, }, { m: `foobar{quantile="0.99"}`, v: 150.1, @@ -630,7 +630,7 @@ foobar{quantile="0.99"} 150.1` labels.FromStrings("__name__", "foobar", "__type__", string(model.MetricTypeSummary), "quantile", "0.99"), labels.FromStrings("__name__", "foobar", "quantile", "0.99"), ), - ct: 1520430004000, + st: 1520430004000, }, { m: "metric", help: "foo\x00bar", @@ -640,7 +640,7 @@ foobar{quantile="0.99"} 150.1` lset: todoDetectFamilySwitch(typeAndUnitEnabled, labels.FromStrings("__name__", "null_byte_metric", "a", "abc\x00"), model.MetricTypeSummary), }, } - opts := []OpenMetricsOption{WithOMParserCTSeriesSkipped()} + opts := []OpenMetricsOption{WithOMParserSTSeriesSkipped()} if typeAndUnitEnabled { opts = append(opts, WithOMParserTypeAndUnitLabels()) } @@ -684,12 +684,12 @@ quotedexemplar2_count 1 # {"id.thing"="histogram-count-test",other="hello"} 4 m: `{"go.gc_duration_seconds",quantile="0"}`, v: 4.9351e-05, lset: labels.FromStrings("__name__", "go.gc_duration_seconds", "quantile", "0.0"), - ct: 1520872607123, + st: 1520872607123, }, { m: `{"go.gc_duration_seconds",quantile="0.25"}`, v: 7.424100000000001e-05, lset: labels.FromStrings("__name__", "go.gc_duration_seconds", "quantile", "0.25"), - ct: 1520872607123, + st: 1520872607123, }, { m: `{"go.gc_duration_seconds",quantile="0.5",a="b"}`, v: 8.3835e-05, @@ -732,7 +732,7 @@ choices}`, "strange©™\n'quoted' \"name\"", "6"), }, } - p := NewOpenMetricsParser([]byte(input), labels.NewSymbolTable(), WithOMParserCTSeriesSkipped()) + p := NewOpenMetricsParser([]byte(input), labels.NewSymbolTable(), WithOMParserSTSeriesSkipped()) got := testParse(t, p) requireEntries(t, exp, got) } @@ -1028,7 +1028,7 @@ func TestOpenMetricsParseErrors(t *testing.T) { } for i, c := range cases { - p := NewOpenMetricsParser([]byte(c.input), labels.NewSymbolTable(), WithOMParserCTSeriesSkipped()) + p := NewOpenMetricsParser([]byte(c.input), labels.NewSymbolTable(), WithOMParserSTSeriesSkipped()) var err error for err == nil { _, err = p.Next() @@ -1093,7 +1093,7 @@ func TestOMNullByteHandling(t *testing.T) { } for i, c := range cases { - p := NewOpenMetricsParser([]byte(c.input), labels.NewSymbolTable(), WithOMParserCTSeriesSkipped()) + p := NewOpenMetricsParser([]byte(c.input), labels.NewSymbolTable(), WithOMParserSTSeriesSkipped()) var err error for err == nil { _, err = p.Next() @@ -1108,10 +1108,10 @@ func TestOMNullByteHandling(t *testing.T) { } } -// TestCTParseFailures tests known failure edge cases, we know does not work due +// TestSTParseFailures tests known failure edge cases, we know does not work due // current OM spec limitations or clients with broken OM format. -// TODO(maniktherana): Make sure OM 1.1/2.0 pass CT via metadata or exemplar-like to avoid this. -func TestCTParseFailures(t *testing.T) { +// TODO(maniktherana): Make sure OM 1.1/2.0 pass ST via metadata or exemplar-like to avoid this. +func TestSTParseFailures(t *testing.T) { for _, tcase := range []struct { name string input string @@ -1143,19 +1143,19 @@ thing_c_total 14123.232 }, { m: `thing_count`, - ct: 0, // Should be int64p(1520872607123). + st: 0, // Should be int64p(1520872607123). }, { m: `thing_sum`, - ct: 0, // Should be int64p(1520872607123). + st: 0, // Should be int64p(1520872607123). }, { m: `thing_bucket{le="0.0"}`, - ct: 0, // Should be int64p(1520872607123). + st: 0, // Should be int64p(1520872607123). }, { m: `thing_bucket{le="+Inf"}`, - ct: 0, // Should be int64p(1520872607123), + st: 0, // Should be int64p(1520872607123), }, { m: "thing_c", @@ -1167,7 +1167,7 @@ thing_c_total 14123.232 }, { m: `thing_c_total`, - ct: 0, // Should be int64p(1520872607123). + st: 0, // Should be int64p(1520872607123). }, }, }, @@ -1197,9 +1197,9 @@ foo_created{a="b"} 1520872608.123 }, } { t.Run(fmt.Sprintf("case=%v", tcase.name), func(t *testing.T) { - p := NewOpenMetricsParser([]byte(tcase.input), labels.NewSymbolTable(), WithOMParserCTSeriesSkipped()) + p := NewOpenMetricsParser([]byte(tcase.input), labels.NewSymbolTable(), WithOMParserSTSeriesSkipped()) got := testParse(t, p) - resetValAndLset(got) // Keep this test focused on metric, basic entries and CT only. + resetValAndLset(got) // Keep this test focused on metric, basic entries and ST only. requireEntries(t, tcase.expected, got) }) } diff --git a/model/textparse/promparse.go b/model/textparse/promparse.go index 2b4b750b4d..4a75bcd8d8 100644 --- a/model/textparse/promparse.go +++ b/model/textparse/promparse.go @@ -274,9 +274,9 @@ func (*PromParser) Exemplar(*exemplar.Exemplar) bool { return false } -// CreatedTimestamp returns 0 as it's not implemented yet. +// StartTimestamp returns 0 as it's not implemented yet. // TODO(bwplotka): https://github.com/prometheus/prometheus/issues/12980 -func (*PromParser) CreatedTimestamp() int64 { +func (*PromParser) StartTimestamp() int64 { return 0 } diff --git a/model/textparse/protobufparse.go b/model/textparse/protobufparse.go index 800f02085e..8b517f4c49 100644 --- a/model/textparse/protobufparse.go +++ b/model/textparse/protobufparse.go @@ -400,24 +400,24 @@ func (p *ProtobufParser) Exemplar(ex *exemplar.Exemplar) bool { return true } -// CreatedTimestamp returns CT or 0 if CT is not present on counters, summaries or histograms. -func (p *ProtobufParser) CreatedTimestamp() int64 { - var ct *types.Timestamp +// StartTimestamp returns ST or 0 if ST is not present on counters, summaries or histograms. +func (p *ProtobufParser) StartTimestamp() int64 { + var st *types.Timestamp switch p.dec.GetType() { case dto.MetricType_COUNTER: - ct = p.dec.GetCounter().GetCreatedTimestamp() + st = p.dec.GetCounter().GetCreatedTimestamp() case dto.MetricType_SUMMARY: - ct = p.dec.GetSummary().GetCreatedTimestamp() + st = p.dec.GetSummary().GetCreatedTimestamp() case dto.MetricType_HISTOGRAM, dto.MetricType_GAUGE_HISTOGRAM: - ct = p.dec.GetHistogram().GetCreatedTimestamp() + st = p.dec.GetHistogram().GetCreatedTimestamp() default: } - if ct == nil { + if st == nil { return 0 } // Same as the gogo proto types.TimestampFromProto but straight to integer. // and without validation. - return ct.GetSeconds()*1e3 + int64(ct.GetNanos())/1e6 + return st.GetSeconds()*1e3 + int64(st.GetNanos())/1e6 } // Next advances the parser to the next "sample" (emulating the behavior of a diff --git a/model/textparse/protobufparse_test.go b/model/textparse/protobufparse_test.go index 9e8ca3a6f2..6a16258f00 100644 --- a/model/textparse/protobufparse_test.go +++ b/model/textparse/protobufparse_test.go @@ -1334,7 +1334,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_counter_with_createdtimestamp", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_counter_with_createdtimestamp", ), @@ -1350,7 +1350,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_count", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_count", ), @@ -1358,7 +1358,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_sum", v: 1.234, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_sum", ), @@ -1373,7 +1373,7 @@ func TestProtobufParse(t *testing.T) { }, { m: "test_histogram_with_createdtimestamp", - ct: 1625851153146, + st: 1625851153146, shs: &histogram.Histogram{ CounterResetHint: histogram.UnknownCounterReset, PositiveSpans: []histogram.Span{}, @@ -1393,7 +1393,7 @@ func TestProtobufParse(t *testing.T) { }, { m: "test_gaugehistogram_with_createdtimestamp", - ct: 1625851153146, + st: 1625851153146, shs: &histogram.Histogram{ CounterResetHint: histogram.GaugeType, PositiveSpans: []histogram.Span{}, @@ -1999,7 +1999,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_counter_with_createdtimestamp\xff__type__\xffcounter", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_counter_with_createdtimestamp", "__type__", string(model.MetricTypeCounter), @@ -2016,7 +2016,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_count\xff__type__\xffsummary", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_count", "__type__", string(model.MetricTypeSummary), @@ -2025,7 +2025,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_sum\xff__type__\xffsummary", v: 1.234, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_sum", "__type__", string(model.MetricTypeSummary), @@ -2041,7 +2041,7 @@ func TestProtobufParse(t *testing.T) { }, { m: "test_histogram_with_createdtimestamp\xff__type__\xffhistogram", - ct: 1625851153146, + st: 1625851153146, shs: &histogram.Histogram{ CounterResetHint: histogram.UnknownCounterReset, PositiveSpans: []histogram.Span{}, @@ -2062,7 +2062,7 @@ func TestProtobufParse(t *testing.T) { }, { m: "test_gaugehistogram_with_createdtimestamp\xff__type__\xffgaugehistogram", - ct: 1625851153146, + st: 1625851153146, shs: &histogram.Histogram{ CounterResetHint: histogram.GaugeType, PositiveSpans: []histogram.Span{}, @@ -2959,7 +2959,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_counter_with_createdtimestamp", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_counter_with_createdtimestamp", ), @@ -2975,7 +2975,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_count", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_count", ), @@ -2983,7 +2983,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_sum", v: 1.234, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_sum", ), @@ -2998,7 +2998,7 @@ func TestProtobufParse(t *testing.T) { }, { m: "test_histogram_with_createdtimestamp", - ct: 1625851153146, + st: 1625851153146, shs: &histogram.Histogram{ CounterResetHint: histogram.UnknownCounterReset, PositiveSpans: []histogram.Span{}, @@ -3018,7 +3018,7 @@ func TestProtobufParse(t *testing.T) { }, { m: "test_gaugehistogram_with_createdtimestamp", - ct: 1625851153146, + st: 1625851153146, shs: &histogram.Histogram{ CounterResetHint: histogram.GaugeType, PositiveSpans: []histogram.Span{}, @@ -3893,7 +3893,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_counter_with_createdtimestamp", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_counter_with_createdtimestamp", ), @@ -3909,7 +3909,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_count", v: 42, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_count", ), @@ -3917,7 +3917,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_summary_with_createdtimestamp_sum", v: 1.234, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_summary_with_createdtimestamp_sum", ), @@ -3933,7 +3933,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_histogram_with_createdtimestamp_count", v: 0, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_histogram_with_createdtimestamp_count", ), @@ -3941,7 +3941,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_histogram_with_createdtimestamp_sum", v: 0, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_histogram_with_createdtimestamp_sum", ), @@ -3949,7 +3949,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_histogram_with_createdtimestamp_bucket\xffle\xff+Inf", v: 0, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_histogram_with_createdtimestamp_bucket", "le", "+Inf", @@ -3966,7 +3966,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_gaugehistogram_with_createdtimestamp_count", v: 0, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_gaugehistogram_with_createdtimestamp_count", ), @@ -3974,7 +3974,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_gaugehistogram_with_createdtimestamp_sum", v: 0, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_gaugehistogram_with_createdtimestamp_sum", ), @@ -3982,7 +3982,7 @@ func TestProtobufParse(t *testing.T) { { m: "test_gaugehistogram_with_createdtimestamp_bucket\xffle\xff+Inf", v: 0, - ct: 1625851153146, + st: 1625851153146, lset: labels.FromStrings( "__name__", "test_gaugehistogram_with_createdtimestamp_bucket", "le", "+Inf", diff --git a/scrape/helpers_test.go b/scrape/helpers_test.go index abc2011bef..f847dbac76 100644 --- a/scrape/helpers_test.go +++ b/scrape/helpers_test.go @@ -57,7 +57,7 @@ func (nopAppender) AppendHistogram(storage.SeriesRef, labels.Labels, int64, *his return 3, nil } -func (nopAppender) AppendHistogramCTZeroSample(storage.SeriesRef, labels.Labels, int64, int64, *histogram.Histogram, *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (nopAppender) AppendHistogramSTZeroSample(storage.SeriesRef, labels.Labels, int64, int64, *histogram.Histogram, *histogram.FloatHistogram) (storage.SeriesRef, error) { return 0, nil } @@ -65,7 +65,7 @@ func (nopAppender) UpdateMetadata(storage.SeriesRef, labels.Labels, metadata.Met return 4, nil } -func (nopAppender) AppendCTZeroSample(storage.SeriesRef, labels.Labels, int64, int64) (storage.SeriesRef, error) { +func (nopAppender) AppendSTZeroSample(storage.SeriesRef, labels.Labels, int64, int64) (storage.SeriesRef, error) { return 5, nil } @@ -184,11 +184,11 @@ func (a *collectResultAppender) AppendHistogram(ref storage.SeriesRef, l labels. return a.next.AppendHistogram(ref, l, t, h, fh) } -func (a *collectResultAppender) AppendHistogramCTZeroSample(ref storage.SeriesRef, l labels.Labels, _, ct int64, h *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (a *collectResultAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, _, st int64, h *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { if h != nil { - return a.AppendHistogram(ref, l, ct, &histogram.Histogram{}, nil) + return a.AppendHistogram(ref, l, st, &histogram.Histogram{}, nil) } - return a.AppendHistogram(ref, l, ct, nil, &histogram.FloatHistogram{}) + return a.AppendHistogram(ref, l, st, nil, &histogram.FloatHistogram{}) } func (a *collectResultAppender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) { @@ -205,8 +205,8 @@ func (a *collectResultAppender) UpdateMetadata(ref storage.SeriesRef, l labels.L return a.next.UpdateMetadata(ref, l, m) } -func (a *collectResultAppender) AppendCTZeroSample(ref storage.SeriesRef, l labels.Labels, _, ct int64) (storage.SeriesRef, error) { - return a.Append(ref, l, ct, 0.0) +func (a *collectResultAppender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, _, st int64) (storage.SeriesRef, error) { + return a.Append(ref, l, st, 0.0) } func (a *collectResultAppender) Commit() error { diff --git a/scrape/manager.go b/scrape/manager.go index 7389f24b52..c63d7d0eae 100644 --- a/scrape/manager.go +++ b/scrape/manager.go @@ -85,7 +85,7 @@ type Options struct { DiscoveryReloadInterval model.Duration // Option to enable the ingestion of the created timestamp as a synthetic zero sample. // See: https://github.com/prometheus/proposals/blob/main/proposals/2023-06-13_created-timestamp.md - EnableCreatedTimestampZeroIngestion bool + EnableStartTimestampZeroIngestion bool // EnableTypeAndUnitLabels EnableTypeAndUnitLabels bool diff --git a/scrape/manager_test.go b/scrape/manager_test.go index a4f3552f82..1ec4875d19 100644 --- a/scrape/manager_test.go +++ b/scrape/manager_test.go @@ -749,8 +749,8 @@ func setupTestServer(t *testing.T, typ string, toWrite []byte) *httptest.Server return server } -// TestManagerCTZeroIngestion tests scrape manager for various CT cases. -func TestManagerCTZeroIngestion(t *testing.T) { +// TestManagerSTZeroIngestion tests scrape manager for various ST cases. +func TestManagerSTZeroIngestion(t *testing.T) { t.Parallel() const ( // _total suffix is required, otherwise expfmt with OMText will mark metric as "unknown" @@ -761,26 +761,26 @@ func TestManagerCTZeroIngestion(t *testing.T) { for _, testFormat := range []config.ScrapeProtocol{config.PrometheusProto, config.OpenMetricsText1_0_0} { t.Run(fmt.Sprintf("format=%s", testFormat), func(t *testing.T) { - for _, testWithCT := range []bool{false, true} { - t.Run(fmt.Sprintf("withCT=%v", testWithCT), func(t *testing.T) { - for _, testCTZeroIngest := range []bool{false, true} { - t.Run(fmt.Sprintf("ctZeroIngest=%v", testCTZeroIngest), func(t *testing.T) { + for _, testWithST := range []bool{false, true} { + t.Run(fmt.Sprintf("withST=%v", testWithST), func(t *testing.T) { + for _, testSTZeroIngest := range []bool{false, true} { + t.Run(fmt.Sprintf("ctZeroIngest=%v", testSTZeroIngest), func(t *testing.T) { ctx, cancel := context.WithCancel(context.Background()) defer cancel() sampleTs := time.Now() - ctTs := time.Time{} - if testWithCT { - ctTs = sampleTs.Add(-2 * time.Minute) + stTs := time.Time{} + if testWithST { + stTs = sampleTs.Add(-2 * time.Minute) } // TODO(bwplotka): Add more types than just counter? - encoded := prepareTestEncodedCounter(t, testFormat, expectedMetricName, expectedSampleValue, sampleTs, ctTs) + encoded := prepareTestEncodedCounter(t, testFormat, expectedMetricName, expectedSampleValue, sampleTs, stTs) app := &collectResultAppender{} discoveryManager, scrapeManager := runManagers(t, ctx, &Options{ - EnableCreatedTimestampZeroIngestion: testCTZeroIngest, - skipOffsetting: true, + EnableStartTimestampZeroIngestion: testSTZeroIngest, + skipOffsetting: true, }, &collectResultAppendable{app}) defer scrapeManager.Stop() @@ -817,12 +817,12 @@ scrape_configs: }), "after 1 minute") // Verify results. - // Verify what we got vs expectations around CT injection. + // Verify what we got vs expectations around ST injection. samples := findSamplesForMetric(app.resultFloats, expectedMetricName) - if testWithCT && testCTZeroIngest { + if testWithST && testSTZeroIngest { require.Len(t, samples, 2) require.Equal(t, 0.0, samples[0].f) - require.Equal(t, timestamp.FromTime(ctTs), samples[0].t) + require.Equal(t, timestamp.FromTime(stTs), samples[0].t) require.Equal(t, expectedSampleValue, samples[1].f) require.Equal(t, timestamp.FromTime(sampleTs), samples[1].t) } else { @@ -832,16 +832,16 @@ scrape_configs: } // Verify what we got vs expectations around additional _created series for OM text. - // enableCTZeroInjection also kills that _created line. + // enableSTZeroInjection also kills that _created line. createdSeriesSamples := findSamplesForMetric(app.resultFloats, expectedCreatedMetricName) - if testFormat == config.OpenMetricsText1_0_0 && testWithCT && !testCTZeroIngest { - // For OM Text, when counter has CT, and feature flag disabled we should see _created lines. + if testFormat == config.OpenMetricsText1_0_0 && testWithST && !testSTZeroIngest { + // For OM Text, when counter has ST, and feature flag disabled we should see _created lines. require.Len(t, createdSeriesSamples, 1) // Conversion taken from common/expfmt.writeOpenMetricsFloat. - // We don't check the ct timestamp as explicit ts was not implemented in expfmt.Encoder, + // We don't check the st timestamp as explicit ts was not implemented in expfmt.Encoder, // but exists in OM https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#:~:text=An%20example%20with%20a%20Metric%20with%20no%20labels%2C%20and%20a%20MetricPoint%20with%20a%20timestamp%20and%20a%20created - // We can implement this, but we want to potentially get rid of OM 1.0 CT lines - require.Equal(t, float64(timestamppb.New(ctTs).AsTime().UnixNano())/1e9, createdSeriesSamples[0].f) + // We can implement this, but we want to potentially get rid of OM 1.0 ST lines + require.Equal(t, float64(timestamppb.New(stTs).AsTime().UnixNano())/1e9, createdSeriesSamples[0].f) } else { require.Empty(t, createdSeriesSamples) } @@ -853,12 +853,12 @@ scrape_configs: } } -func prepareTestEncodedCounter(t *testing.T, format config.ScrapeProtocol, mName string, v float64, ts, ct time.Time) (encoded []byte) { +func prepareTestEncodedCounter(t *testing.T, format config.ScrapeProtocol, mName string, v float64, ts, st time.Time) (encoded []byte) { t.Helper() counter := &dto.Counter{Value: proto.Float64(v)} - if !ct.IsZero() { - counter.CreatedTimestamp = timestamppb.New(ct) + if !st.IsZero() { + counter.CreatedTimestamp = timestamppb.New(st) } ctrType := dto.MetricType_COUNTER inputMetric := &dto.MetricFamily{ @@ -923,40 +923,40 @@ func generateTestHistogram(i int) *dto.Histogram { return h } -func TestManagerCTZeroIngestionHistogram(t *testing.T) { +func TestManagerSTZeroIngestionHistogram(t *testing.T) { t.Parallel() const mName = "expected_histogram" for _, tc := range []struct { name string inputHistSample *dto.Histogram - enableCTZeroIngestion bool + enableSTZeroIngestion bool }{ { - name: "disabled with CT on histogram", + name: "disabled with ST on histogram", inputHistSample: func() *dto.Histogram { h := generateTestHistogram(0) h.CreatedTimestamp = timestamppb.Now() return h }(), - enableCTZeroIngestion: false, + enableSTZeroIngestion: false, }, { - name: "enabled with CT on histogram", + name: "enabled with ST on histogram", inputHistSample: func() *dto.Histogram { h := generateTestHistogram(0) h.CreatedTimestamp = timestamppb.Now() return h }(), - enableCTZeroIngestion: true, + enableSTZeroIngestion: true, }, { - name: "enabled without CT on histogram", + name: "enabled without ST on histogram", inputHistSample: func() *dto.Histogram { h := generateTestHistogram(0) return h }(), - enableCTZeroIngestion: true, + enableSTZeroIngestion: true, }, } { t.Run(tc.name, func(t *testing.T) { @@ -966,8 +966,8 @@ func TestManagerCTZeroIngestionHistogram(t *testing.T) { app := &collectResultAppender{} discoveryManager, scrapeManager := runManagers(t, ctx, &Options{ - EnableCreatedTimestampZeroIngestion: tc.enableCTZeroIngestion, - skipOffsetting: true, + EnableStartTimestampZeroIngestion: tc.enableSTZeroIngestion, + skipOffsetting: true, }, &collectResultAppendable{app}) defer scrapeManager.Stop() @@ -1035,8 +1035,8 @@ scrape_configs: }), "after 1 minute") // Check for zero samples, assuming we only injected always one histogram sample. - // Did it contain CT to inject? If yes, was CT zero enabled? - if tc.inputHistSample.CreatedTimestamp.IsValid() && tc.enableCTZeroIngestion { + // Did it contain ST to inject? If yes, was ST zero enabled? + if tc.inputHistSample.CreatedTimestamp.IsValid() && tc.enableSTZeroIngestion { require.Len(t, got, 2) // Zero sample. require.Equal(t, histogram.Histogram{}, *got[0].h) @@ -1066,12 +1066,12 @@ func TestUnregisterMetrics(t *testing.T) { } } -// TestNHCBAndCTZeroIngestion verifies that both ConvertClassicHistogramsToNHCBEnabled -// and EnableCreatedTimestampZeroIngestion can be used simultaneously without errors. +// TestNHCBAndSTZeroIngestion verifies that both ConvertClassicHistogramsToNHCBEnabled +// and EnableStartTimestampZeroIngestion can be used simultaneously without errors. // This test addresses issue #17216 by ensuring the previously blocking check has been removed. // The test verifies that the presence of exemplars in the input does not cause errors, // although exemplars are not preserved during NHCB conversion (as documented below). -func TestNHCBAndCTZeroIngestion(t *testing.T) { +func TestNHCBAndSTZeroIngestion(t *testing.T) { t.Parallel() const ( @@ -1085,8 +1085,8 @@ func TestNHCBAndCTZeroIngestion(t *testing.T) { app := &collectResultAppender{} discoveryManager, scrapeManager := runManagers(t, ctx, &Options{ - EnableCreatedTimestampZeroIngestion: true, - skipOffsetting: true, + EnableStartTimestampZeroIngestion: true, + skipOffsetting: true, }, &collectResultAppendable{app}) defer scrapeManager.Stop() @@ -1122,7 +1122,7 @@ test_histogram_created 1520430001 serverURL, err := url.Parse(server.URL) require.NoError(t, err) - // Configuration with both convert_classic_histograms_to_nhcb enabled and CT zero ingestion enabled. + // Configuration with both convert_classic_histograms_to_nhcb enabled and ST zero ingestion enabled. testConfig := fmt.Sprintf(` global: # Use a very long scrape_interval to prevent automatic scraping during the test. @@ -1167,7 +1167,7 @@ scrape_configs: // Verify that samples were ingested (proving both features work together). got := getMatchingHistograms() - // With CT zero ingestion enabled and a created timestamp present, we expect 2 samples: + // With ST zero ingestion enabled and a created timestamp present, we expect 2 samples: // one zero sample and one actual sample. require.Len(t, got, 2, "expected 2 histogram samples (zero sample + actual sample)") require.Equal(t, histogram.Histogram{}, *got[0].h, "first sample should be zero sample") diff --git a/scrape/scrape.go b/scrape/scrape.go index 09652d0484..d10858d8ae 100644 --- a/scrape/scrape.go +++ b/scrape/scrape.go @@ -215,7 +215,7 @@ func newScrapePool(cfg *config.ScrapeConfig, app storage.Appendable, offsetSeed opts.alwaysScrapeClassicHist, opts.convertClassicHistToNHCB, cfg.ScrapeNativeHistogramsEnabled(), - options.EnableCreatedTimestampZeroIngestion, + options.EnableStartTimestampZeroIngestion, options.EnableTypeAndUnitLabels, options.ExtraMetrics, options.AppendMetadata, @@ -951,7 +951,7 @@ type scrapeLoop struct { alwaysScrapeClassicHist bool convertClassicHistToNHCB bool - enableCTZeroIngestion bool + enableSTZeroIngestion bool enableTypeAndUnitLabels bool fallbackScrapeProtocol string @@ -1264,7 +1264,7 @@ func newScrapeLoop(ctx context.Context, alwaysScrapeClassicHist bool, convertClassicHistToNHCB bool, enableNativeHistogramScraping bool, - enableCTZeroIngestion bool, + enableSTZeroIngestion bool, enableTypeAndUnitLabels bool, reportExtraMetrics bool, appendMetadataToWAL bool, @@ -1321,7 +1321,7 @@ func newScrapeLoop(ctx context.Context, timeout: timeout, alwaysScrapeClassicHist: alwaysScrapeClassicHist, convertClassicHistToNHCB: convertClassicHistToNHCB, - enableCTZeroIngestion: enableCTZeroIngestion, + enableSTZeroIngestion: enableSTZeroIngestion, enableTypeAndUnitLabels: enableTypeAndUnitLabels, fallbackScrapeProtocol: fallbackScrapeProtocol, enableNativeHistogramScraping: enableNativeHistogramScraping, @@ -1660,7 +1660,7 @@ func (sl *scrapeLoop) append(app storage.Appender, b []byte, contentType string, IgnoreNativeHistograms: !sl.enableNativeHistogramScraping, ConvertClassicHistogramsToNHCB: sl.convertClassicHistToNHCB, KeepClassicOnClassicAndNativeHistograms: sl.alwaysScrapeClassicHist, - OpenMetricsSkipCTSeries: sl.enableCTZeroIngestion, + OpenMetricsSkipSTSeries: sl.enableSTZeroIngestion, FallbackContentType: sl.fallbackScrapeProtocol, }) if p == nil { @@ -1801,21 +1801,21 @@ loop: if seriesAlreadyScraped && parsedTimestamp == nil { err = storage.ErrDuplicateSampleForTimestamp } else { - if sl.enableCTZeroIngestion { - if ctMs := p.CreatedTimestamp(); ctMs != 0 { + if sl.enableSTZeroIngestion { + if stMs := p.StartTimestamp(); stMs != 0 { if isHistogram { if h != nil { - ref, err = app.AppendHistogramCTZeroSample(ref, lset, t, ctMs, h, nil) + ref, err = app.AppendHistogramSTZeroSample(ref, lset, t, stMs, h, nil) } else { - ref, err = app.AppendHistogramCTZeroSample(ref, lset, t, ctMs, nil, fh) + ref, err = app.AppendHistogramSTZeroSample(ref, lset, t, stMs, nil, fh) } } else { - ref, err = app.AppendCTZeroSample(ref, lset, t, ctMs) + ref, err = app.AppendSTZeroSample(ref, lset, t, stMs) } - if err != nil && !errors.Is(err, storage.ErrOutOfOrderCT) { // OOO is a common case, ignoring completely for now. - // CT is an experimental feature. For now, we don't need to fail the + if err != nil && !errors.Is(err, storage.ErrOutOfOrderST) { // OOO is a common case, ignoring completely for now. + // ST is an experimental feature. For now, we don't need to fail the // scrape on errors updating the created timestamp, log debug. - sl.l.Debug("Error when appending CT in scrape loop", "series", string(met), "ct", ctMs, "t", t, "err", err) + sl.l.Debug("Error when appending ST in scrape loop", "series", string(met), "ct", stMs, "t", t, "err", err) } } } @@ -1913,7 +1913,7 @@ loop: if !seriesCached || lastMeta.lastIterChange == sl.cache.iter { // In majority cases we can trust that the current series/histogram is matching the lastMeta and lastMFName. // However, optional TYPE etc metadata and broken OM text can break this, detect those cases here. - // TODO(bwplotka): Consider moving this to parser as many parser users end up doing this (e.g. CT and NHCB parsing). + // TODO(bwplotka): Consider moving this to parser as many parser users end up doing this (e.g. ST and NHCB parsing). if isSeriesPartOfFamily(lset.Get(labels.MetricName), lastMFName, lastMeta.Type) { if _, merr := app.UpdateMetadata(ref, lset, lastMeta.Metadata); merr != nil { // No need to fail the scrape on errors appending metadata. diff --git a/scrape/scrape_test.go b/scrape/scrape_test.go index c7412365d0..5ccdb80019 100644 --- a/scrape/scrape_test.go +++ b/scrape/scrape_test.go @@ -3281,8 +3281,8 @@ func TestTargetScraperScrapeOK(t *testing.T) { } contentTypes := strings.SplitSeq(accept, ",") - for ct := range contentTypes { - match := qValuePattern.FindStringSubmatch(ct) + for st := range contentTypes { + match := qValuePattern.FindStringSubmatch(st) require.Len(t, match, 3) qValue, err := strconv.ParseFloat(match[1], 64) require.NoError(t, err, "Error parsing q value") diff --git a/storage/fanout.go b/storage/fanout.go index f99edb473a..a699a97b02 100644 --- a/storage/fanout.go +++ b/storage/fanout.go @@ -199,14 +199,14 @@ func (f *fanoutAppender) AppendHistogram(ref SeriesRef, l labels.Labels, t int64 return ref, nil } -func (f *fanoutAppender) AppendHistogramCTZeroSample(ref SeriesRef, l labels.Labels, t, ct int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) { - ref, err := f.primary.AppendHistogramCTZeroSample(ref, l, t, ct, h, fh) +func (f *fanoutAppender) AppendHistogramSTZeroSample(ref SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) { + ref, err := f.primary.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) if err != nil { return ref, err } for _, appender := range f.secondaries { - if _, err := appender.AppendHistogramCTZeroSample(ref, l, t, ct, h, fh); err != nil { + if _, err := appender.AppendHistogramSTZeroSample(ref, l, t, st, h, fh); err != nil { return 0, err } } @@ -227,14 +227,14 @@ func (f *fanoutAppender) UpdateMetadata(ref SeriesRef, l labels.Labels, m metada return ref, nil } -func (f *fanoutAppender) AppendCTZeroSample(ref SeriesRef, l labels.Labels, t, ct int64) (SeriesRef, error) { - ref, err := f.primary.AppendCTZeroSample(ref, l, t, ct) +func (f *fanoutAppender) AppendSTZeroSample(ref SeriesRef, l labels.Labels, t, st int64) (SeriesRef, error) { + ref, err := f.primary.AppendSTZeroSample(ref, l, t, st) if err != nil { return ref, err } for _, appender := range f.secondaries { - if _, err := appender.AppendCTZeroSample(ref, l, t, ct); err != nil { + if _, err := appender.AppendSTZeroSample(ref, l, t, st); err != nil { return 0, err } } diff --git a/storage/interface.go b/storage/interface.go index 9d7e5d93a6..d4c98e0710 100644 --- a/storage/interface.go +++ b/storage/interface.go @@ -44,13 +44,13 @@ var ( ErrExemplarsDisabled = errors.New("exemplar storage is disabled or max exemplars is less than or equal to 0") ErrNativeHistogramsDisabled = errors.New("native histograms are disabled") - // ErrOutOfOrderCT indicates failed append of CT to the storage - // due to CT being older the then newer sample. + // ErrOutOfOrderST indicates failed append of ST to the storage + // due to ST being older the then newer sample. // NOTE(bwplotka): This can be both an instrumentation failure or commonly expected // behaviour, and we currently don't have a way to determine this. As a result // it's recommended to ignore this error for now. - ErrOutOfOrderCT = errors.New("created timestamp out of order, ignoring") - ErrCTNewerThanSample = errors.New("CT is newer or the same as sample's timestamp, ignoring") + ErrOutOfOrderST = errors.New("created timestamp out of order, ignoring") + ErrSTNewerThanSample = errors.New("ST is newer or the same as sample's timestamp, ignoring") ) // SeriesRef is a generic series reference. In prometheus it is either a @@ -294,7 +294,7 @@ type Appender interface { ExemplarAppender HistogramAppender MetadataUpdater - CreatedTimestampAppender + StartTimestampAppender } // GetRef is an extra interface on Appenders used by downstream projects @@ -338,20 +338,20 @@ type HistogramAppender interface { // pointer. AppendHistogram won't mutate the histogram, but in turn // depends on the caller to not mutate it either. AppendHistogram(ref SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) - // AppendHistogramCTZeroSample adds synthetic zero sample for the given ct timestamp, + // AppendHistogramSTZeroSample adds synthetic zero sample for the given st timestamp, // which will be associated with given series, labels and the incoming - // sample's t (timestamp). AppendHistogramCTZeroSample returns error if zero sample can't be - // appended, for example when ct is too old, or when it would collide with + // sample's t (timestamp). AppendHistogramSTZeroSample returns error if zero sample can't be + // appended, for example when st is too old, or when it would collide with // incoming sample (sample has priority). // - // AppendHistogramCTZeroSample has to be called before the corresponding histogram AppendHistogram. + // AppendHistogramSTZeroSample has to be called before the corresponding histogram AppendHistogram. // A series reference number is returned which can be used to modify the - // CT for the given series in the same or later transactions. + // ST for the given series in the same or later transactions. // Returned reference numbers are ephemeral and may be rejected in calls - // to AppendHistogramCTZeroSample() at any point. + // to AppendHistogramSTZeroSample() at any point. // // If the reference is 0 it must not be used for caching. - AppendHistogramCTZeroSample(ref SeriesRef, l labels.Labels, t, ct int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) + AppendHistogramSTZeroSample(ref SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) } // MetadataUpdater provides an interface for associating metadata to stored series. @@ -366,22 +366,22 @@ type MetadataUpdater interface { UpdateMetadata(ref SeriesRef, l labels.Labels, m metadata.Metadata) (SeriesRef, error) } -// CreatedTimestampAppender provides an interface for appending CT to storage. -type CreatedTimestampAppender interface { - // AppendCTZeroSample adds synthetic zero sample for the given ct timestamp, +// StartTimestampAppender provides an interface for appending ST to storage. +type StartTimestampAppender interface { + // AppendSTZeroSample adds synthetic zero sample for the given st timestamp, // which will be associated with given series, labels and the incoming - // sample's t (timestamp). AppendCTZeroSample returns error if zero sample can't be - // appended, for example when ct is too old, or when it would collide with + // sample's t (timestamp). AppendSTZeroSample returns error if zero sample can't be + // appended, for example when st is too old, or when it would collide with // incoming sample (sample has priority). // - // AppendCTZeroSample has to be called before the corresponding sample Append. + // AppendSTZeroSample has to be called before the corresponding sample Append. // A series reference number is returned which can be used to modify the - // CT for the given series in the same or later transactions. + // ST for the given series in the same or later transactions. // Returned reference numbers are ephemeral and may be rejected in calls - // to AppendCTZeroSample() at any point. + // to AppendSTZeroSample() at any point. // // If the reference is 0 it must not be used for caching. - AppendCTZeroSample(ref SeriesRef, l labels.Labels, t, ct int64) (SeriesRef, error) + AppendSTZeroSample(ref SeriesRef, l labels.Labels, t, st int64) (SeriesRef, error) } // SeriesSet contains a set of series. diff --git a/storage/remote/codec_test.go b/storage/remote/codec_test.go index ddf8f76cf6..8bbe3c2813 100644 --- a/storage/remote/codec_test.go +++ b/storage/remote/codec_test.go @@ -122,7 +122,7 @@ var ( writev2.FromIntHistogram(30, &testHistogramCustomBuckets), writev2.FromFloatHistogram(40, testHistogramCustomBuckets.ToFloat(nil)), }, - CreatedTimestamp: 1, // CT needs to be lower than the sample's timestamp. + CreatedTimestamp: 1, // ST needs to be lower than the sample's timestamp. }, { LabelsRefs: []uint32{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, // Same series as first. diff --git a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go index 9ed114567d..883b8d3142 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender.go @@ -49,10 +49,10 @@ type Metadata struct { type CombinedAppender interface { // AppendSample appends a sample and related exemplars, metadata, and // created timestamp to the storage. - AppendSample(ls labels.Labels, meta Metadata, ct, t int64, v float64, es []exemplar.Exemplar) error + AppendSample(ls labels.Labels, meta Metadata, st, t int64, v float64, es []exemplar.Exemplar) error // AppendHistogram appends a histogram and related exemplars, metadata, and // created timestamp to the storage. - AppendHistogram(ls labels.Labels, meta Metadata, ct, t int64, h *histogram.Histogram, es []exemplar.Exemplar) error + AppendHistogram(ls labels.Labels, meta Metadata, st, t int64, h *histogram.Histogram, es []exemplar.Exemplar) error } // CombinedAppenderMetrics is for the metrics observed by the @@ -82,11 +82,11 @@ func NewCombinedAppenderMetrics(reg prometheus.Registerer) CombinedAppenderMetri // NewCombinedAppender creates a combined appender that sets start times and // updates metadata for each series only once, and appends samples and // exemplars for each call. -func NewCombinedAppender(app storage.Appender, logger *slog.Logger, ingestCTZeroSample, appendMetadata bool, metrics CombinedAppenderMetrics) CombinedAppender { +func NewCombinedAppender(app storage.Appender, logger *slog.Logger, ingestSTZeroSample, appendMetadata bool, metrics CombinedAppenderMetrics) CombinedAppender { return &combinedAppender{ app: app, logger: logger, - ingestCTZeroSample: ingestCTZeroSample, + ingestSTZeroSample: ingestSTZeroSample, appendMetadata: appendMetadata, refs: make(map[uint64]seriesRef), samplesAppendedWithoutMetadata: metrics.samplesAppendedWithoutMetadata, @@ -96,7 +96,7 @@ func NewCombinedAppender(app storage.Appender, logger *slog.Logger, ingestCTZero type seriesRef struct { ref storage.SeriesRef - ct int64 + st int64 ls labels.Labels meta metadata.Metadata } @@ -106,7 +106,7 @@ type combinedAppender struct { logger *slog.Logger samplesAppendedWithoutMetadata prometheus.Counter outOfOrderExemplars prometheus.Counter - ingestCTZeroSample bool + ingestSTZeroSample bool appendMetadata bool // Used to ensure we only update metadata and created timestamps once, and to share storage.SeriesRefs. // To detect hash collision it also stores the labels. @@ -114,20 +114,20 @@ type combinedAppender struct { refs map[uint64]seriesRef } -func (b *combinedAppender) AppendSample(ls labels.Labels, meta Metadata, ct, t int64, v float64, es []exemplar.Exemplar) (err error) { - return b.appendFloatOrHistogram(ls, meta.Metadata, ct, t, v, nil, es) +func (b *combinedAppender) AppendSample(ls labels.Labels, meta Metadata, st, t int64, v float64, es []exemplar.Exemplar) (err error) { + return b.appendFloatOrHistogram(ls, meta.Metadata, st, t, v, nil, es) } -func (b *combinedAppender) AppendHistogram(ls labels.Labels, meta Metadata, ct, t int64, h *histogram.Histogram, es []exemplar.Exemplar) (err error) { +func (b *combinedAppender) AppendHistogram(ls labels.Labels, meta Metadata, st, t int64, h *histogram.Histogram, es []exemplar.Exemplar) (err error) { if h == nil { // Sanity check, we should never get here with a nil histogram. b.logger.Error("Received nil histogram in CombinedAppender.AppendHistogram", "series", ls.String()) return errors.New("internal error, attempted to append nil histogram") } - return b.appendFloatOrHistogram(ls, meta.Metadata, ct, t, 0, h, es) + return b.appendFloatOrHistogram(ls, meta.Metadata, st, t, 0, h, es) } -func (b *combinedAppender) appendFloatOrHistogram(ls labels.Labels, meta metadata.Metadata, ct, t int64, v float64, h *histogram.Histogram, es []exemplar.Exemplar) (err error) { +func (b *combinedAppender) appendFloatOrHistogram(ls labels.Labels, meta metadata.Metadata, st, t int64, v float64, h *histogram.Histogram, es []exemplar.Exemplar) (err error) { hash := ls.Hash() series, exists := b.refs[hash] ref := series.ref @@ -140,28 +140,28 @@ func (b *combinedAppender) appendFloatOrHistogram(ls labels.Labels, meta metadat exists = false ref = 0 } - updateRefs := !exists || series.ct != ct - if updateRefs && ct != 0 && ct < t && b.ingestCTZeroSample { + updateRefs := !exists || series.st != st + if updateRefs && st != 0 && st < t && b.ingestSTZeroSample { var newRef storage.SeriesRef if h != nil { - newRef, err = b.app.AppendHistogramCTZeroSample(ref, ls, t, ct, h, nil) + newRef, err = b.app.AppendHistogramSTZeroSample(ref, ls, t, st, h, nil) } else { - newRef, err = b.app.AppendCTZeroSample(ref, ls, t, ct) + newRef, err = b.app.AppendSTZeroSample(ref, ls, t, st) } if err != nil { - if !errors.Is(err, storage.ErrOutOfOrderCT) && !errors.Is(err, storage.ErrDuplicateSampleForTimestamp) { + if !errors.Is(err, storage.ErrOutOfOrderST) && !errors.Is(err, storage.ErrDuplicateSampleForTimestamp) { // Even for the first sample OOO is a common scenario because - // we can't tell if a CT was already ingested in a previous request. + // we can't tell if a ST was already ingested in a previous request. // We ignore the error. // ErrDuplicateSampleForTimestamp is also a common scenario because // unknown start times in Opentelemetry are indicated by setting // the start time to the same as the first sample time. // https://opentelemetry.io/docs/specs/otel/metrics/data-model/#cumulative-streams-handling-unknown-start-time - b.logger.Warn("Error when appending CT from OTLP", "err", err, "series", ls.String(), "created_timestamp", ct, "timestamp", t, "sample_type", sampleType(h)) + b.logger.Warn("Error when appending ST from OTLP", "err", err, "series", ls.String(), "start_timestamp", st, "timestamp", t, "sample_type", sampleType(h)) } } else { // We only use the returned reference on success as otherwise an - // error of CT append could invalidate the series reference. + // error of ST append could invalidate the series reference. ref = newRef } } @@ -197,7 +197,7 @@ func (b *combinedAppender) appendFloatOrHistogram(ls labels.Labels, meta metadat if updateRefs || metadataChanged { b.refs[hash] = seriesRef{ ref: ref, - ct: ct, + st: st, ls: ls, meta: meta, } diff --git a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go index 7d79637803..753112cf82 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go @@ -52,7 +52,7 @@ type combinedSample struct { ls labels.Labels meta metadata.Metadata t int64 - ct int64 + st int64 v float64 es []exemplar.Exemplar } @@ -62,31 +62,31 @@ type combinedHistogram struct { ls labels.Labels meta metadata.Metadata t int64 - ct int64 + st int64 h *histogram.Histogram es []exemplar.Exemplar } -func (m *mockCombinedAppender) AppendSample(ls labels.Labels, meta Metadata, ct, t int64, v float64, es []exemplar.Exemplar) error { +func (m *mockCombinedAppender) AppendSample(ls labels.Labels, meta Metadata, st, t int64, v float64, es []exemplar.Exemplar) error { m.pendingSamples = append(m.pendingSamples, combinedSample{ metricFamilyName: meta.MetricFamilyName, ls: ls, meta: meta.Metadata, t: t, - ct: ct, + st: st, v: v, es: es, }) return nil } -func (m *mockCombinedAppender) AppendHistogram(ls labels.Labels, meta Metadata, ct, t int64, h *histogram.Histogram, es []exemplar.Exemplar) error { +func (m *mockCombinedAppender) AppendHistogram(ls labels.Labels, meta Metadata, st, t int64, h *histogram.Histogram, es []exemplar.Exemplar) error { m.pendingHistograms = append(m.pendingHistograms, combinedHistogram{ metricFamilyName: meta.MetricFamilyName, ls: ls, meta: meta.Metadata, t: t, - ct: ct, + st: st, h: h, es: es, }) @@ -108,12 +108,12 @@ func requireEqual(t testing.TB, expected, actual any, msgAndArgs ...any) { // TestCombinedAppenderOnTSDB runs some basic tests on a real TSDB to check // that the combinedAppender works on a real TSDB. func TestCombinedAppenderOnTSDB(t *testing.T) { - t.Run("ingestCTZeroSample=false", func(t *testing.T) { testCombinedAppenderOnTSDB(t, false) }) + t.Run("ingestSTZeroSample=false", func(t *testing.T) { testCombinedAppenderOnTSDB(t, false) }) - t.Run("ingestCTZeroSample=true", func(t *testing.T) { testCombinedAppenderOnTSDB(t, true) }) + t.Run("ingestSTZeroSample=true", func(t *testing.T) { testCombinedAppenderOnTSDB(t, true) }) } -func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { +func testCombinedAppenderOnTSDB(t *testing.T, ingestSTZeroSample bool) { t.Helper() now := time.Now() @@ -165,9 +165,9 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { extraAppendFunc func(*testing.T, CombinedAppender) expectedSamples []sample expectedExemplars []exemplar.QueryResult - expectedLogsForCT []string + expectedLogsForST []string }{ - "single float sample, zero CT": { + "single float sample, zero ST": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, 0, now.UnixMilli(), 42.0, testExemplars)) }, @@ -179,7 +179,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, expectedExemplars: expectedExemplars, }, - "single float sample, very old CT": { + "single float sample, very old ST": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, 1, now.UnixMilli(), 42.0, nil)) }, @@ -189,18 +189,18 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { f: 42.0, }, }, - expectedLogsForCT: []string{ - "Error when appending CT from OTLP", + expectedLogsForST: []string{ + "Error when appending ST from OTLP", "out of bound", }, }, - "single float sample, normal CT": { + "single float sample, normal ST": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, now.Add(-2*time.Minute).UnixMilli(), now.UnixMilli(), 42.0, nil)) }, expectedSamples: []sample{ { - ctZero: true, + stZero: true, t: now.Add(-2 * time.Minute).UnixMilli(), }, { @@ -209,7 +209,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "single float sample, CT same time as sample": { + "single float sample, ST same time as sample": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, now.UnixMilli(), now.UnixMilli(), 42.0, nil)) }, @@ -220,7 +220,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "two float samples in different messages, CT same time as first sample": { + "two float samples in different messages, ST same time as first sample": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, now.UnixMilli(), now.UnixMilli(), 42.0, nil)) }, @@ -238,7 +238,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "single float sample, CT in the future of the sample": { + "single float sample, ST in the future of the sample": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, now.Add(time.Minute).UnixMilli(), now.UnixMilli(), 42.0, nil)) }, @@ -249,7 +249,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "single histogram sample, zero CT": { + "single histogram sample, zero ST": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendHistogram(seriesLabels.Copy(), histogramMetadata, 0, now.UnixMilli(), tsdbutil.GenerateTestHistogram(42), testExemplars)) }, @@ -261,7 +261,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, expectedExemplars: expectedExemplars, }, - "single histogram sample, very old CT": { + "single histogram sample, very old ST": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendHistogram(seriesLabels.Copy(), histogramMetadata, 1, now.UnixMilli(), tsdbutil.GenerateTestHistogram(42), nil)) }, @@ -271,18 +271,18 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { h: tsdbutil.GenerateTestHistogram(42), }, }, - expectedLogsForCT: []string{ - "Error when appending CT from OTLP", + expectedLogsForST: []string{ + "Error when appending ST from OTLP", "out of bound", }, }, - "single histogram sample, normal CT": { + "single histogram sample, normal ST": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendHistogram(seriesLabels.Copy(), histogramMetadata, now.Add(-2*time.Minute).UnixMilli(), now.UnixMilli(), tsdbutil.GenerateTestHistogram(42), nil)) }, expectedSamples: []sample{ { - ctZero: true, + stZero: true, t: now.Add(-2 * time.Minute).UnixMilli(), h: &histogram.Histogram{}, }, @@ -292,7 +292,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "single histogram sample, CT same time as sample": { + "single histogram sample, ST same time as sample": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendHistogram(seriesLabels.Copy(), histogramMetadata, now.UnixMilli(), now.UnixMilli(), tsdbutil.GenerateTestHistogram(42), nil)) }, @@ -303,7 +303,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "two histogram samples in different messages, CT same time as first sample": { + "two histogram samples in different messages, ST same time as first sample": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendHistogram(seriesLabels.Copy(), floatMetadata, now.UnixMilli(), now.UnixMilli(), tsdbutil.GenerateTestHistogram(42), nil)) }, @@ -321,7 +321,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "single histogram sample, CT in the future of the sample": { + "single histogram sample, ST in the future of the sample": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendHistogram(seriesLabels.Copy(), histogramMetadata, now.Add(time.Minute).UnixMilli(), now.UnixMilli(), tsdbutil.GenerateTestHistogram(42), nil)) }, @@ -364,14 +364,14 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { }, }, }, - "float samples with CT changing": { + "float samples with ST changing": { appendFunc: func(t *testing.T, app CombinedAppender) { require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, now.Add(-4*time.Second).UnixMilli(), now.Add(-3*time.Second).UnixMilli(), 42.0, nil)) require.NoError(t, app.AppendSample(seriesLabels.Copy(), floatMetadata, now.Add(-1*time.Second).UnixMilli(), now.UnixMilli(), 62.0, nil)) }, expectedSamples: []sample{ { - ctZero: true, + stZero: true, t: now.Add(-4 * time.Second).UnixMilli(), }, { @@ -379,7 +379,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { f: 42.0, }, { - ctZero: true, + stZero: true, t: now.Add(-1 * time.Second).UnixMilli(), }, { @@ -393,8 +393,8 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { for name, tc := range testCases { t.Run(name, func(t *testing.T) { var expectedLogs []string - if ingestCTZeroSample { - expectedLogs = append(expectedLogs, tc.expectedLogsForCT...) + if ingestSTZeroSample { + expectedLogs = append(expectedLogs, tc.expectedLogsForST...) } dir := t.TempDir() @@ -413,13 +413,13 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { reg := prometheus.NewRegistry() cappMetrics := NewCombinedAppenderMetrics(reg) app := db.Appender(ctx) - capp := NewCombinedAppender(app, logger, ingestCTZeroSample, false, cappMetrics) + capp := NewCombinedAppender(app, logger, ingestSTZeroSample, false, cappMetrics) tc.appendFunc(t, capp) require.NoError(t, app.Commit()) if tc.extraAppendFunc != nil { app = db.Appender(ctx) - capp = NewCombinedAppender(app, logger, ingestCTZeroSample, false, cappMetrics) + capp = NewCombinedAppender(app, logger, ingestSTZeroSample, false, cappMetrics) tc.extraAppendFunc(t, capp) require.NoError(t, app.Commit()) } @@ -446,7 +446,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { series := ss.At() it := series.Iterator(nil) for i, sample := range tc.expectedSamples { - if !ingestCTZeroSample && sample.ctZero { + if !ingestSTZeroSample && sample.stZero { continue } if sample.h == nil { @@ -476,7 +476,7 @@ func testCombinedAppenderOnTSDB(t *testing.T, ingestCTZeroSample bool) { } type sample struct { - ctZero bool + stZero bool t int64 f float64 @@ -500,7 +500,7 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { MetricFamilyName: "test_bytes_total", } - t.Run("happy case with CT zero, reference is passed and reused", func(t *testing.T) { + t.Run("happy case with ST zero, reference is passed and reused", func(t *testing.T) { app := &appenderRecorder{} capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) @@ -514,31 +514,31 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { })) require.Len(t, app.records, 5) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) + requireEqualOpAndRef(t, "AppendSTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[2]) + requireEqualOpAndRef(t, "AppendSTZeroSample", ref, app.records[2]) requireEqualOpAndRef(t, "Append", ref, app.records[3]) requireEqualOpAndRef(t, "AppendExemplar", ref, app.records[4]) }) - t.Run("error on second CT ingest doesn't update the reference", func(t *testing.T) { + t.Run("error on second ST ingest doesn't update the reference", func(t *testing.T) { app := &appenderRecorder{} capp := NewCombinedAppender(app, promslog.NewNopLogger(), true, false, NewCombinedAppenderMetrics(prometheus.NewRegistry())) require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 1, 2, 42.0, nil)) - app.appendCTZeroSampleError = errors.New("test error") + app.appendSTZeroSampleError = errors.New("test error") require.NoError(t, capp.AppendSample(seriesLabels.Copy(), floatMetadata, 3, 4, 62.0, nil)) require.Len(t, app.records, 4) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) + requireEqualOpAndRef(t, "AppendSTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[2]) - require.Zero(t, app.records[2].outRef, "the second AppendCTZeroSample returned 0") + requireEqualOpAndRef(t, "AppendSTZeroSample", ref, app.records[2]) + require.Zero(t, app.records[2].outRef, "the second AppendSTZeroSample returned 0") requireEqualOpAndRef(t, "Append", ref, app.records[3]) }) @@ -577,12 +577,12 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { })) require.Len(t, app.records, 7) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) + requireEqualOpAndRef(t, "AppendSTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) + requireEqualOpAndRef(t, "AppendSTZeroSample", ref, app.records[3]) requireEqualOpAndRef(t, "Append", ref, app.records[4]) require.Zero(t, app.records[4].outRef, "the second Append returned 0") requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[5]) @@ -619,13 +619,13 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { })) require.Len(t, app.records, 5) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) + requireEqualOpAndRef(t, "AppendSTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[2]) + requireEqualOpAndRef(t, "AppendSTZeroSample", 0, app.records[2]) newRef := app.records[2].outRef - require.NotEqual(t, ref, newRef, "the second AppendCTZeroSample returned a different reference") + require.NotEqual(t, ref, newRef, "the second AppendSTZeroSample returned a different reference") requireEqualOpAndRef(t, "Append", newRef, app.records[3]) requireEqualOpAndRef(t, "AppendExemplar", newRef, app.records[4]) }) @@ -651,12 +651,12 @@ func TestCombinedAppenderSeriesRefs(t *testing.T) { if appendMetadata { require.Len(t, app.records, 3) - requireEqualOp(t, "AppendCTZeroSample", app.records[0]) + requireEqualOp(t, "AppendSTZeroSample", app.records[0]) requireEqualOp(t, "Append", app.records[1]) requireEqualOp(t, "UpdateMetadata", app.records[2]) } else { require.Len(t, app.records, 2) - requireEqualOp(t, "AppendCTZeroSample", app.records[0]) + requireEqualOp(t, "AppendSTZeroSample", app.records[0]) requireEqualOp(t, "Append", app.records[1]) } }) @@ -720,12 +720,12 @@ func TestCombinedAppenderMetadataChanges(t *testing.T) { // Verify expected operations. require.Len(t, app.records, 7) - requireEqualOpAndRef(t, "AppendCTZeroSample", 0, app.records[0]) + requireEqualOpAndRef(t, "AppendSTZeroSample", 0, app.records[0]) ref := app.records[0].outRef require.NotZero(t, ref) requireEqualOpAndRef(t, "Append", ref, app.records[1]) requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[2]) - requireEqualOpAndRef(t, "AppendCTZeroSample", ref, app.records[3]) + requireEqualOpAndRef(t, "AppendSTZeroSample", ref, app.records[3]) requireEqualOpAndRef(t, "Append", ref, app.records[4]) requireEqualOpAndRef(t, "UpdateMetadata", ref, app.records[5]) requireEqualOpAndRef(t, "Append", ref, app.records[6]) @@ -756,9 +756,9 @@ type appenderRecorder struct { records []appenderRecord appendError error - appendCTZeroSampleError error + appendSTZeroSampleError error appendHistogramError error - appendHistogramCTZeroSampleError error + appendHistogramSTZeroSampleError error updateMetadataError error appendExemplarError error } @@ -789,10 +789,10 @@ func (a *appenderRecorder) Append(ref storage.SeriesRef, ls labels.Labels, _ int return ref, nil } -func (a *appenderRecorder) AppendCTZeroSample(ref storage.SeriesRef, ls labels.Labels, _, _ int64) (storage.SeriesRef, error) { - a.records = append(a.records, appenderRecord{op: "AppendCTZeroSample", ref: ref, ls: ls}) - if a.appendCTZeroSampleError != nil { - return 0, a.appendCTZeroSampleError +func (a *appenderRecorder) AppendSTZeroSample(ref storage.SeriesRef, ls labels.Labels, _, _ int64) (storage.SeriesRef, error) { + a.records = append(a.records, appenderRecord{op: "AppendSTZeroSample", ref: ref, ls: ls}) + if a.appendSTZeroSampleError != nil { + return 0, a.appendSTZeroSampleError } if ref == 0 { ref = a.newRef() @@ -813,10 +813,10 @@ func (a *appenderRecorder) AppendHistogram(ref storage.SeriesRef, ls labels.Labe return ref, nil } -func (a *appenderRecorder) AppendHistogramCTZeroSample(ref storage.SeriesRef, ls labels.Labels, _, _ int64, _ *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { - a.records = append(a.records, appenderRecord{op: "AppendHistogramCTZeroSample", ref: ref, ls: ls}) - if a.appendHistogramCTZeroSampleError != nil { - return 0, a.appendHistogramCTZeroSampleError +func (a *appenderRecorder) AppendHistogramSTZeroSample(ref storage.SeriesRef, ls labels.Labels, _, _ int64, _ *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { + a.records = append(a.records, appenderRecord{op: "AppendHistogramSTZeroSample", ref: ref, ls: ls}) + if a.appendHistogramSTZeroSampleError != nil { + return 0, a.appendHistogramSTZeroSampleError } if ref == 0 { ref = a.newRef() diff --git a/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go b/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go index fc120c0b6a..893fe97ec4 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go @@ -482,7 +482,7 @@ func TestPrometheusConverter_AddSummaryDataPoints(t *testing.T) { model.MetricNameLabel, "test_summary"+sumStr, ), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, { @@ -491,7 +491,7 @@ func TestPrometheusConverter_AddSummaryDataPoints(t *testing.T) { model.MetricNameLabel, "test_summary"+countStr, ), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, } @@ -526,7 +526,7 @@ func TestPrometheusConverter_AddSummaryDataPoints(t *testing.T) { ls: labels.FromStrings(append(scopeLabels, model.MetricNameLabel, "test_summary"+sumStr)...), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, { @@ -534,7 +534,7 @@ func TestPrometheusConverter_AddSummaryDataPoints(t *testing.T) { ls: labels.FromStrings(append(scopeLabels, model.MetricNameLabel, "test_summary"+countStr)...), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, } @@ -706,7 +706,7 @@ func TestPrometheusConverter_AddHistogramDataPoints(t *testing.T) { model.MetricNameLabel, "test_hist"+countStr, ), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, { @@ -716,7 +716,7 @@ func TestPrometheusConverter_AddHistogramDataPoints(t *testing.T) { model.BucketLabel, "+Inf", ), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, } @@ -751,7 +751,7 @@ func TestPrometheusConverter_AddHistogramDataPoints(t *testing.T) { ls: labels.FromStrings(append(scopeLabels, model.MetricNameLabel, "test_hist"+countStr)...), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, { @@ -760,7 +760,7 @@ func TestPrometheusConverter_AddHistogramDataPoints(t *testing.T) { model.MetricNameLabel, "test_hist_bucket", model.BucketLabel, "+Inf")...), t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 0, }, } diff --git a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go index 0bc8a876e4..ecf7338c96 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go @@ -67,13 +67,13 @@ func (c *PrometheusConverter) addExponentialHistogramDataPoints(ctx context.Cont return annots, err } ts := convertTimeStamp(pt.Timestamp()) - ct := convertTimeStamp(pt.StartTimestamp()) + st := convertTimeStamp(pt.StartTimestamp()) exemplars, err := c.getPromExemplars(ctx, pt.Exemplars()) if err != nil { return annots, err } // OTel exponential histograms are always Int Histograms. - if err = c.appender.AppendHistogram(lbls, meta, ct, ts, hp, exemplars); err != nil { + if err = c.appender.AppendHistogram(lbls, meta, st, ts, hp, exemplars); err != nil { return annots, err } } @@ -286,12 +286,12 @@ func (c *PrometheusConverter) addCustomBucketsHistogramDataPoints(ctx context.Co return annots, err } ts := convertTimeStamp(pt.Timestamp()) - ct := convertTimeStamp(pt.StartTimestamp()) + st := convertTimeStamp(pt.StartTimestamp()) exemplars, err := c.getPromExemplars(ctx, pt.Exemplars()) if err != nil { return annots, err } - if err = c.appender.AppendHistogram(lbls, meta, ct, ts, hp, exemplars); err != nil { + if err = c.appender.AppendHistogram(lbls, meta, st, ts, hp, exemplars); err != nil { return annots, err } } diff --git a/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go b/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go index adb0cf8eee..22e654ab9c 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go @@ -673,7 +673,7 @@ func TestPrometheusConverter_addExponentialHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 7, Schema: 1, @@ -689,7 +689,7 @@ func TestPrometheusConverter_addExponentialHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 4, Schema: 1, @@ -746,7 +746,7 @@ func TestPrometheusConverter_addExponentialHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 7, Schema: 1, @@ -762,7 +762,7 @@ func TestPrometheusConverter_addExponentialHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 4, Schema: 1, @@ -819,7 +819,7 @@ func TestPrometheusConverter_addExponentialHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 7, Schema: 1, @@ -835,7 +835,7 @@ func TestPrometheusConverter_addExponentialHistogramDataPoints(t *testing.T) { ls: labelsAnother, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 4, Schema: 1, @@ -1146,7 +1146,7 @@ func TestPrometheusConverter_addCustomBucketsHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 3, Sum: 3, @@ -1162,7 +1162,7 @@ func TestPrometheusConverter_addCustomBucketsHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 11, Sum: 5, @@ -1219,7 +1219,7 @@ func TestPrometheusConverter_addCustomBucketsHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 3, Sum: 3, @@ -1235,7 +1235,7 @@ func TestPrometheusConverter_addCustomBucketsHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 11, Sum: 5, @@ -1292,7 +1292,7 @@ func TestPrometheusConverter_addCustomBucketsHistogramDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 6, Sum: 3, @@ -1308,7 +1308,7 @@ func TestPrometheusConverter_addCustomBucketsHistogramDataPoints(t *testing.T) { ls: labelsAnother, meta: metadata.Metadata{}, t: 0, - ct: 0, + st: 0, h: &histogram.Histogram{ Count: 11, Sum: 5, diff --git a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go index 5675249153..e409b4e8b5 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go @@ -1105,7 +1105,7 @@ func (a *noOpAppender) Append(_ storage.SeriesRef, _ labels.Labels, _ int64, _ f return 1, nil } -func (*noOpAppender) AppendCTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, _ int64) (storage.SeriesRef, error) { +func (*noOpAppender) AppendSTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, _ int64) (storage.SeriesRef, error) { return 1, nil } @@ -1114,7 +1114,7 @@ func (a *noOpAppender) AppendHistogram(_ storage.SeriesRef, _ labels.Labels, _ i return 1, nil } -func (*noOpAppender) AppendHistogramCTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, _ int64, _ *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (*noOpAppender) AppendHistogramSTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, _ int64, _ *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { return 1, nil } diff --git a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go index cdae978736..8f30dbb6b6 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go @@ -61,8 +61,8 @@ func (c *PrometheusConverter) addGaugeNumberDataPoints(ctx context.Context, data val = math.Float64frombits(value.StaleNaN) } ts := convertTimeStamp(pt.Timestamp()) - ct := convertTimeStamp(pt.StartTimestamp()) - if err := c.appender.AppendSample(labels, meta, ct, ts, val, nil); err != nil { + st := convertTimeStamp(pt.StartTimestamp()) + if err := c.appender.AppendSample(labels, meta, st, ts, val, nil); err != nil { return err } } @@ -104,12 +104,12 @@ func (c *PrometheusConverter) addSumNumberDataPoints(ctx context.Context, dataPo val = math.Float64frombits(value.StaleNaN) } ts := convertTimeStamp(pt.Timestamp()) - ct := convertTimeStamp(pt.StartTimestamp()) + st := convertTimeStamp(pt.StartTimestamp()) exemplars, err := c.getPromExemplars(ctx, pt.Exemplars()) if err != nil { return err } - if err := c.appender.AppendSample(lbls, meta, ct, ts, val, exemplars); err != nil { + if err := c.appender.AppendSample(lbls, meta, st, ts, val, exemplars); err != nil { return err } } diff --git a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go index 3e918eecbd..32435020c5 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go @@ -272,7 +272,7 @@ func TestPrometheusConverter_addSumNumberDataPoints(t *testing.T) { ls: lbls, meta: metadata.Metadata{}, t: convertTimeStamp(ts), - ct: convertTimeStamp(ts), + st: convertTimeStamp(ts), v: 1, }, } diff --git a/storage/remote/write.go b/storage/remote/write.go index 6bc02bd6fe..1a036c1795 100644 --- a/storage/remote/write.go +++ b/storage/remote/write.go @@ -318,22 +318,22 @@ func (t *timestampTracker) AppendHistogram(_ storage.SeriesRef, _ labels.Labels, return 0, nil } -func (t *timestampTracker) AppendCTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, ct int64) (storage.SeriesRef, error) { +func (t *timestampTracker) AppendSTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, st int64) (storage.SeriesRef, error) { t.samples++ - if ct > t.highestTimestamp { - // Theoretically, we should never see a CT zero sample with a timestamp higher than the highest timestamp we've seen so far. + if st > t.highestTimestamp { + // Theoretically, we should never see a ST zero sample with a timestamp higher than the highest timestamp we've seen so far. // However, we're not going to enforce that here, as it is not the responsibility of the tracker to enforce this. - t.highestTimestamp = ct + t.highestTimestamp = st } return 0, nil } -func (t *timestampTracker) AppendHistogramCTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, ct int64, _ *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (t *timestampTracker) AppendHistogramSTZeroSample(_ storage.SeriesRef, _ labels.Labels, _, st int64, _ *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { t.histograms++ - if ct > t.highestTimestamp { - // Theoretically, we should never see a CT zero sample with a timestamp higher than the highest timestamp we've seen so far. + if st > t.highestTimestamp { + // Theoretically, we should never see a ST zero sample with a timestamp higher than the highest timestamp we've seen so far. // However, we're not going to enforce that here, as it is not the responsibility of the tracker to enforce this. - t.highestTimestamp = ct + t.highestTimestamp = st } return 0, nil } diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index 5d1c561802..579c7a794f 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -53,7 +53,7 @@ type writeHandler struct { samplesWithInvalidLabelsTotal prometheus.Counter samplesAppendedWithoutMetadata prometheus.Counter - ingestCTZeroSample bool + ingestSTZeroSample bool enableTypeAndUnitLabels bool appendMetadata bool } @@ -65,7 +65,7 @@ const maxAheadTime = 10 * time.Minute // // NOTE(bwplotka): When accepting v2 proto and spec, partial writes are possible // as per https://prometheus.io/docs/specs/remote_write_spec_2_0/#partial-write. -func NewWriteHandler(logger *slog.Logger, reg prometheus.Registerer, appendable storage.Appendable, acceptedMsgs remoteapi.MessageTypes, ingestCTZeroSample, enableTypeAndUnitLabels, appendMetadata bool) http.Handler { +func NewWriteHandler(logger *slog.Logger, reg prometheus.Registerer, appendable storage.Appendable, acceptedMsgs remoteapi.MessageTypes, ingestSTZeroSample, enableTypeAndUnitLabels, appendMetadata bool) http.Handler { h := &writeHandler{ logger: logger, appendable: appendable, @@ -82,7 +82,7 @@ func NewWriteHandler(logger *slog.Logger, reg prometheus.Registerer, appendable Help: "The total number of received remote write samples (and histogram samples) which were ingested without corresponding metadata.", }), - ingestCTZeroSample: ingestCTZeroSample, + ingestSTZeroSample: ingestSTZeroSample, enableTypeAndUnitLabels: enableTypeAndUnitLabels, appendMetadata: appendMetadata, } @@ -355,15 +355,15 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * var ref storage.SeriesRef // Samples. - if h.ingestCTZeroSample && len(ts.Samples) > 0 && ts.Samples[0].Timestamp != 0 && ts.CreatedTimestamp != 0 { - // CT only needs to be ingested for the first sample, it will be considered + if h.ingestSTZeroSample && len(ts.Samples) > 0 && ts.Samples[0].Timestamp != 0 && ts.CreatedTimestamp != 0 { + // ST only needs to be ingested for the first sample, it will be considered // out of order for the rest. - ref, err = app.AppendCTZeroSample(ref, ls, ts.Samples[0].Timestamp, ts.CreatedTimestamp) - if err != nil && !errors.Is(err, storage.ErrOutOfOrderCT) { + ref, err = app.AppendSTZeroSample(ref, ls, ts.Samples[0].Timestamp, ts.CreatedTimestamp) + if err != nil && !errors.Is(err, storage.ErrOutOfOrderST) { // Even for the first sample OOO is a common scenario because - // we can't tell if a CT was already ingested in a previous request. + // we can't tell if a ST was already ingested in a previous request. // We ignore the error. - h.logger.Debug("Error when appending CT in remote write request", "err", err, "series", ls.String(), "created_timestamp", ts.CreatedTimestamp, "timestamp", ts.Samples[0].Timestamp) + h.logger.Debug("Error when appending ST in remote write request", "err", err, "series", ls.String(), "start_timestamp", ts.CreatedTimestamp, "timestamp", ts.Samples[0].Timestamp) } } for _, s := range ts.Samples { @@ -387,15 +387,15 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * // Native Histograms. for _, hp := range ts.Histograms { - if h.ingestCTZeroSample && hp.Timestamp != 0 && ts.CreatedTimestamp != 0 { - // Differently from samples, we need to handle CT for each histogram instead of just the first one. + if h.ingestSTZeroSample && hp.Timestamp != 0 && ts.CreatedTimestamp != 0 { + // Differently from samples, we need to handle ST for each histogram instead of just the first one. // This is because histograms and float histograms are stored separately, even if they have the same labels. ref, err = h.handleHistogramZeroSample(app, ref, ls, hp, ts.CreatedTimestamp) - if err != nil && !errors.Is(err, storage.ErrOutOfOrderCT) { + if err != nil && !errors.Is(err, storage.ErrOutOfOrderST) { // Even for the first sample OOO is a common scenario because - // we can't tell if a CT was already ingested in a previous request. + // we can't tell if a ST was already ingested in a previous request. // We ignore the error. - h.logger.Debug("Error when appending CT in remote write request", "err", err, "series", ls.String(), "created_timestamp", ts.CreatedTimestamp, "timestamp", hp.Timestamp) + h.logger.Debug("Error when appending ST in remote write request", "err", err, "series", ls.String(), "start_timestamp", ts.CreatedTimestamp, "timestamp", hp.Timestamp) } } if hp.IsFloatHistogram() { @@ -474,14 +474,14 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * return samplesWithoutMetadata, http.StatusBadRequest, errors.Join(badRequestErrs...) } -// handleHistogramZeroSample appends CT as a zero-value sample with CT value as the sample timestamp. -// It doesn't return errors in case of out of order CT. -func (*writeHandler) handleHistogramZeroSample(app storage.Appender, ref storage.SeriesRef, l labels.Labels, hist writev2.Histogram, ct int64) (storage.SeriesRef, error) { +// handleHistogramZeroSample appends ST as a zero-value sample with ST value as the sample timestamp. +// It doesn't return errors in case of out of order ST. +func (*writeHandler) handleHistogramZeroSample(app storage.Appender, ref storage.SeriesRef, l labels.Labels, hist writev2.Histogram, st int64) (storage.SeriesRef, error) { var err error if hist.IsFloatHistogram() { - ref, err = app.AppendHistogramCTZeroSample(ref, l, hist.Timestamp, ct, nil, hist.ToFloatHistogram()) + ref, err = app.AppendHistogramSTZeroSample(ref, l, hist.Timestamp, st, nil, hist.ToFloatHistogram()) } else { - ref, err = app.AppendHistogramCTZeroSample(ref, l, hist.Timestamp, ct, hist.ToIntHistogram(), nil) + ref, err = app.AppendHistogramSTZeroSample(ref, l, hist.Timestamp, st, hist.ToIntHistogram(), nil) } return ref, err } @@ -498,9 +498,9 @@ type OTLPOptions struct { LookbackDelta time.Duration // Add type and unit labels to the metrics. EnableTypeAndUnitLabels bool - // IngestCTZeroSample enables writing zero samples based on the start time + // IngestSTZeroSample enables writing zero samples based on the start time // of metrics. - IngestCTZeroSample bool + IngestSTZeroSample bool // AppendMetadata enables writing metadata to WAL when metadata-wal-records feature is enabled. AppendMetadata bool } @@ -519,7 +519,7 @@ func NewOTLPWriteHandler(logger *slog.Logger, reg prometheus.Registerer, appenda config: configFunc, allowDeltaTemporality: opts.NativeDelta, lookbackDelta: opts.LookbackDelta, - ingestCTZeroSample: opts.IngestCTZeroSample, + ingestSTZeroSample: opts.IngestSTZeroSample, enableTypeAndUnitLabels: opts.EnableTypeAndUnitLabels, appendMetadata: opts.AppendMetadata, // Register metrics. @@ -562,7 +562,7 @@ type rwExporter struct { config func() config.Config allowDeltaTemporality bool lookbackDelta time.Duration - ingestCTZeroSample bool + ingestSTZeroSample bool enableTypeAndUnitLabels bool appendMetadata bool @@ -576,7 +576,7 @@ func (rw *rwExporter) ConsumeMetrics(ctx context.Context, md pmetric.Metrics) er Appender: rw.appendable.Appender(ctx), maxTime: timestamp.FromTime(time.Now().Add(maxAheadTime)), } - combinedAppender := otlptranslator.NewCombinedAppender(app, rw.logger, rw.ingestCTZeroSample, rw.appendMetadata, rw.metrics) + combinedAppender := otlptranslator.NewCombinedAppender(app, rw.logger, rw.ingestSTZeroSample, rw.appendMetadata, rw.metrics) converter := otlptranslator.NewPrometheusConverter(combinedAppender) annots, err := converter.FromMetrics(ctx, md, otlptranslator.Settings{ AddMetricSuffixes: otlpCfg.TranslationStrategy.ShouldAddSuffixes(), diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go index 536fba63cd..40f1bdff0f 100644 --- a/storage/remote/write_handler_test.go +++ b/storage/remote/write_handler_test.go @@ -358,12 +358,12 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { commitErr error appendSampleErr error - appendCTZeroSampleErr error + appendSTZeroSampleErr error appendHistogramErr error appendExemplarErr error updateMetadataErr error - ingestCTZeroSample bool + ingestSTZeroSample bool enableTypeAndUnitLabels bool appendMetadata bool expectedLabels labels.Labels // For verifying type/unit labels @@ -372,7 +372,7 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { desc: "All timeseries accepted/ct_enabled", input: writeV2RequestFixture.Timeseries, expectedCode: http.StatusNoContent, - ingestCTZeroSample: true, + ingestSTZeroSample: true, }, { desc: "All timeseries accepted/ct_disabled", @@ -701,12 +701,12 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { appendable := &mockAppendable{ commitErr: tc.commitErr, appendSampleErr: tc.appendSampleErr, - appendCTZeroSampleErr: tc.appendCTZeroSampleErr, + appendSTZeroSampleErr: tc.appendSTZeroSampleErr, appendHistogramErr: tc.appendHistogramErr, appendExemplarErr: tc.appendExemplarErr, updateMetadataErr: tc.updateMetadataErr, } - handler := NewWriteHandler(promslog.NewNopLogger(), nil, appendable, []remoteapi.WriteMessageType{remoteapi.WriteV2MessageType}, tc.ingestCTZeroSample, tc.enableTypeAndUnitLabels, tc.appendMetadata) + handler := NewWriteHandler(promslog.NewNopLogger(), nil, appendable, []remoteapi.WriteMessageType{remoteapi.WriteV2MessageType}, tc.ingestSTZeroSample, tc.enableTypeAndUnitLabels, tc.appendMetadata) recorder := httptest.NewRecorder() handler.ServeHTTP(recorder, req) @@ -758,7 +758,7 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { require.NoError(t, err) for _, s := range ts.Samples { - if ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { + if ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { requireEqual(t, mockSample{ls, ts.CreatedTimestamp, 0}, appendable.samples[i]) i++ } @@ -768,7 +768,7 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { for _, hp := range ts.Histograms { if hp.IsFloatHistogram() { fh := hp.ToFloatHistogram() - if !zeroFloatHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { + if !zeroFloatHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { requireEqual(t, mockHistogram{ls, ts.CreatedTimestamp, nil, &histogram.FloatHistogram{}}, appendable.histograms[k]) k++ zeroFloatHistogramIngested = true @@ -776,7 +776,7 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { requireEqual(t, mockHistogram{ls, hp.Timestamp, nil, fh}, appendable.histograms[k]) } else { h := hp.ToIntHistogram() - if !zeroHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { + if !zeroHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { requireEqual(t, mockHistogram{ls, ts.CreatedTimestamp, &histogram.Histogram{}, nil}, appendable.histograms[k]) k++ zeroHistogramIngested = true @@ -785,7 +785,7 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { } k++ } - if ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { + if ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { require.True(t, zeroHistogramIngested) require.True(t, zeroFloatHistogramIngested) } @@ -1190,7 +1190,7 @@ type mockAppendable struct { // optional errors to inject. commitErr error appendSampleErr error - appendCTZeroSampleErr error + appendSTZeroSampleErr error appendHistogramErr error appendExemplarErr error updateMetadataErr error @@ -1342,13 +1342,13 @@ func (m *mockAppendable) AppendHistogram(_ storage.SeriesRef, l labels.Labels, t return storage.SeriesRef(hash), nil } -func (m *mockAppendable) AppendHistogramCTZeroSample(_ storage.SeriesRef, l labels.Labels, t, ct int64, h *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { - if m.appendCTZeroSampleErr != nil { - return 0, m.appendCTZeroSampleErr +func (m *mockAppendable) AppendHistogramSTZeroSample(_ storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) { + if m.appendSTZeroSampleErr != nil { + return 0, m.appendSTZeroSampleErr } // Created Timestamp can't be higher than the original sample's timestamp. - if ct > t { + if st > t { return 0, storage.ErrOutOfOrderSample } hash := l.Hash() @@ -1358,10 +1358,10 @@ func (m *mockAppendable) AppendHistogramCTZeroSample(_ storage.SeriesRef, l labe } else { latestTs = m.latestFloatHist[hash] } - if ct < latestTs { + if st < latestTs { return 0, storage.ErrOutOfOrderSample } - if ct == latestTs { + if st == latestTs { return 0, storage.ErrDuplicateSampleForTimestamp } @@ -1374,11 +1374,11 @@ func (m *mockAppendable) AppendHistogramCTZeroSample(_ storage.SeriesRef, l labe } if h != nil { - m.latestHistogram[hash] = ct - m.histograms = append(m.histograms, mockHistogram{l, ct, &histogram.Histogram{}, nil}) + m.latestHistogram[hash] = st + m.histograms = append(m.histograms, mockHistogram{l, st, &histogram.Histogram{}, nil}) } else { - m.latestFloatHist[hash] = ct - m.histograms = append(m.histograms, mockHistogram{l, ct, nil, &histogram.FloatHistogram{}}) + m.latestFloatHist[hash] = st + m.histograms = append(m.histograms, mockHistogram{l, st, nil, &histogram.FloatHistogram{}}) } return storage.SeriesRef(hash), nil } @@ -1392,21 +1392,21 @@ func (m *mockAppendable) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, return ref, nil } -func (m *mockAppendable) AppendCTZeroSample(_ storage.SeriesRef, l labels.Labels, t, ct int64) (storage.SeriesRef, error) { - if m.appendCTZeroSampleErr != nil { - return 0, m.appendCTZeroSampleErr +func (m *mockAppendable) AppendSTZeroSample(_ storage.SeriesRef, l labels.Labels, t, st int64) (storage.SeriesRef, error) { + if m.appendSTZeroSampleErr != nil { + return 0, m.appendSTZeroSampleErr } // Created Timestamp can't be higher than the original sample's timestamp. - if ct > t { + if st > t { return 0, storage.ErrOutOfOrderSample } hash := l.Hash() latestTs := m.latestSample[hash] - if ct < latestTs { + if st < latestTs { return 0, storage.ErrOutOfOrderSample } - if ct == latestTs { + if st == latestTs { return 0, storage.ErrDuplicateSampleForTimestamp } @@ -1417,8 +1417,8 @@ func (m *mockAppendable) AppendCTZeroSample(_ storage.SeriesRef, l labels.Labels return 0, tsdb.ErrInvalidSample } - m.latestSample[hash] = ct - m.samples = append(m.samples, mockSample{l, ct, 0}) + m.latestSample[hash] = st + m.samples = append(m.samples, mockSample{l, st, 0}) return storage.SeriesRef(hash), nil } diff --git a/storage/remote/write_test.go b/storage/remote/write_test.go index 975caccd6c..2bf317465c 100644 --- a/storage/remote/write_test.go +++ b/storage/remote/write_test.go @@ -938,7 +938,7 @@ func TestOTLPWriteHandler(t *testing.T) { } } -// Check that start time is ingested if ingestCTZeroSample is enabled +// Check that start time is ingested if ingestSTZeroSample is enabled // and the start time is actually set (non-zero). func TestOTLPWriteHandler_StartTime(t *testing.T) { timestamp := time.Now() @@ -1023,72 +1023,72 @@ func TestOTLPWriteHandler_StartTime(t *testing.T) { }, } - expectedSamplesWithCTZero := make([]mockSample, 0, len(expectedSamples)*2-1) // All samples will get CT zero, except target_info. + expectedSamplesWithSTZero := make([]mockSample, 0, len(expectedSamples)*2-1) // All samples will get ST zero, except target_info. for _, s := range expectedSamples { if s.l.Get(model.MetricNameLabel) != "target_info" { - expectedSamplesWithCTZero = append(expectedSamplesWithCTZero, mockSample{ + expectedSamplesWithSTZero = append(expectedSamplesWithSTZero, mockSample{ l: s.l.Copy(), t: startTime.UnixMilli(), v: 0, }) } - expectedSamplesWithCTZero = append(expectedSamplesWithCTZero, s) + expectedSamplesWithSTZero = append(expectedSamplesWithSTZero, s) } - expectedHistogramsWithCTZero := make([]mockHistogram, 0, len(expectedHistograms)*2) + expectedHistogramsWithSTZero := make([]mockHistogram, 0, len(expectedHistograms)*2) for _, s := range expectedHistograms { if s.l.Get(model.MetricNameLabel) != "target_info" { - expectedHistogramsWithCTZero = append(expectedHistogramsWithCTZero, mockHistogram{ + expectedHistogramsWithSTZero = append(expectedHistogramsWithSTZero, mockHistogram{ l: s.l.Copy(), t: startTime.UnixMilli(), h: &histogram.Histogram{}, }) } - expectedHistogramsWithCTZero = append(expectedHistogramsWithCTZero, s) + expectedHistogramsWithSTZero = append(expectedHistogramsWithSTZero, s) } for _, testCase := range []struct { name string otlpOpts OTLPOptions startTime time.Time - expectCTZero bool + expectSTZero bool expectedSamples []mockSample expectedHistograms []mockHistogram }{ { - name: "IngestCTZero=false/startTime=0", + name: "IngestSTZero=false/startTime=0", otlpOpts: OTLPOptions{ - IngestCTZeroSample: false, + IngestSTZeroSample: false, }, startTime: zeroTime, expectedSamples: expectedSamples, expectedHistograms: expectedHistograms, }, { - name: "IngestCTZero=true/startTime=0", + name: "IngestSTZero=true/startTime=0", otlpOpts: OTLPOptions{ - IngestCTZeroSample: true, + IngestSTZeroSample: true, }, startTime: zeroTime, expectedSamples: expectedSamples, expectedHistograms: expectedHistograms, }, { - name: "IngestCTZero=false/startTime=ts-1ms", + name: "IngestSTZero=false/startTime=ts-1ms", otlpOpts: OTLPOptions{ - IngestCTZeroSample: false, + IngestSTZeroSample: false, }, startTime: startTime, expectedSamples: expectedSamples, expectedHistograms: expectedHistograms, }, { - name: "IngestCTZero=true/startTime=ts-1ms", + name: "IngestSTZero=true/startTime=ts-1ms", otlpOpts: OTLPOptions{ - IngestCTZeroSample: true, + IngestSTZeroSample: true, }, startTime: startTime, - expectedSamples: expectedSamplesWithCTZero, - expectedHistograms: expectedHistogramsWithCTZero, + expectedSamples: expectedSamplesWithSTZero, + expectedHistograms: expectedHistogramsWithSTZero, }, } { t.Run(testCase.name, func(t *testing.T) { diff --git a/tsdb/agent/db.go b/tsdb/agent/db.go index 7884366ebe..5c9774cd58 100644 --- a/tsdb/agent/db.go +++ b/tsdb/agent/db.go @@ -997,7 +997,7 @@ func (*appender) UpdateMetadata(storage.SeriesRef, labels.Labels, metadata.Metad return 0, nil } -func (a *appender) AppendHistogramCTZeroSample(ref storage.SeriesRef, l labels.Labels, t, ct int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (a *appender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { if h != nil { if err := h.Validate(); err != nil { return 0, err @@ -1008,8 +1008,8 @@ func (a *appender) AppendHistogramCTZeroSample(ref storage.SeriesRef, l labels.L return 0, err } } - if ct >= t { - return 0, storage.ErrCTNewerThanSample + if st >= t { + return 0, storage.ErrSTNewerThanSample } series := a.series.GetByID(chunks.HeadSeriesRef(ref)) @@ -1038,29 +1038,29 @@ func (a *appender) AppendHistogramCTZeroSample(ref storage.SeriesRef, l labels.L series.Lock() defer series.Unlock() - if ct <= a.minValidTime(series.lastTs) { - return 0, storage.ErrOutOfOrderCT + if st <= a.minValidTime(series.lastTs) { + return 0, storage.ErrOutOfOrderST } - if ct <= series.lastTs { + if st <= series.lastTs { // discard the sample if it's out of order. - return 0, storage.ErrOutOfOrderCT + return 0, storage.ErrOutOfOrderST } - series.lastTs = ct + series.lastTs = st switch { case h != nil: zeroHistogram := &histogram.Histogram{} a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{ Ref: series.ref, - T: ct, + T: st, H: zeroHistogram, }) a.histogramSeries = append(a.histogramSeries, series) case fh != nil: a.pendingFloatHistograms = append(a.pendingFloatHistograms, record.RefFloatHistogramSample{ Ref: series.ref, - T: ct, + T: st, FH: &histogram.FloatHistogram{}, }) a.floatHistogramSeries = append(a.floatHistogramSeries, series) @@ -1070,9 +1070,9 @@ func (a *appender) AppendHistogramCTZeroSample(ref storage.SeriesRef, l labels.L return storage.SeriesRef(series.ref), nil } -func (a *appender) AppendCTZeroSample(ref storage.SeriesRef, l labels.Labels, t, ct int64) (storage.SeriesRef, error) { - if ct >= t { - return 0, storage.ErrCTNewerThanSample +func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64) (storage.SeriesRef, error) { + if st >= t { + return 0, storage.ErrSTNewerThanSample } series := a.series.GetByID(chunks.HeadSeriesRef(ref)) @@ -1106,16 +1106,16 @@ func (a *appender) AppendCTZeroSample(ref storage.SeriesRef, l labels.Labels, t, return 0, storage.ErrOutOfOrderSample } - if ct <= series.lastTs { + if st <= series.lastTs { // discard the sample if it's out of order. - return 0, storage.ErrOutOfOrderCT + return 0, storage.ErrOutOfOrderST } - series.lastTs = ct + series.lastTs = st // NOTE: always modify pendingSamples and sampleSeries together. a.pendingSamples = append(a.pendingSamples, record.RefSample{ Ref: series.ref, - T: ct, + T: st, V: 0, }) a.sampleSeries = append(a.sampleSeries, series) diff --git a/tsdb/agent/db_test.go b/tsdb/agent/db_test.go index c2674c8871..7409f79ec5 100644 --- a/tsdb/agent/db_test.go +++ b/tsdb/agent/db_test.go @@ -1142,12 +1142,12 @@ type walSample struct { ref storage.SeriesRef } -func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { +func TestDBStartTimestampSamplesIngestion(t *testing.T) { t.Parallel() type appendableSample struct { t int64 - ct int64 + st int64 v float64 lbls labels.Labels h *histogram.Histogram @@ -1169,8 +1169,8 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { { name: "in order ct+normal sample/floatSamples", inputSamples: []appendableSample{ - {t: 100, ct: 1, v: 10, lbls: defLbls}, - {t: 101, ct: 1, v: 10, lbls: defLbls}, + {t: 100, st: 1, v: 10, lbls: defLbls}, + {t: 101, st: 1, v: 10, lbls: defLbls}, }, expectedSamples: []*walSample{ {t: 1, f: 0, lbls: defLbls}, @@ -1179,17 +1179,17 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { }, }, { - name: "CT+float && CT+histogram samples", + name: "ST+float && ST+histogram samples", inputSamples: []appendableSample{ { t: 100, - ct: 30, + st: 30, v: 20, lbls: defLbls, }, { t: 300, - ct: 230, + st: 230, h: testHistogram, lbls: defLbls, }, @@ -1203,20 +1203,20 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { expectedSeriesCount: 1, }, { - name: "CT+float && CT+histogram samples with error", + name: "ST+float && ST+histogram samples with error", inputSamples: []appendableSample{ { - // invalid CT + // invalid ST t: 100, - ct: 100, + st: 100, v: 10, lbls: defLbls, expectsError: true, }, { - // invalid CT histogram + // invalid ST histogram t: 300, - ct: 300, + st: 300, h: testHistogram, lbls: defLbls, expectsError: true, @@ -1231,8 +1231,8 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { { name: "In order ct+normal sample/histogram", inputSamples: []appendableSample{ - {t: 100, h: testHistogram, ct: 1, lbls: defLbls}, - {t: 101, h: testHistogram, ct: 1, lbls: defLbls}, + {t: 100, h: testHistogram, st: 1, lbls: defLbls}, + {t: 101, h: testHistogram, st: 1, lbls: defLbls}, }, expectedSamples: []*walSample{ {t: 1, h: &histogram.Histogram{}}, @@ -1243,10 +1243,10 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { { name: "ct+normal then OOO sample/float", inputSamples: []appendableSample{ - {t: 60_000, ct: 40_000, v: 10, lbls: defLbls}, - {t: 120_000, ct: 40_000, v: 10, lbls: defLbls}, - {t: 180_000, ct: 40_000, v: 10, lbls: defLbls}, - {t: 50_000, ct: 40_000, v: 10, lbls: defLbls}, + {t: 60_000, st: 40_000, v: 10, lbls: defLbls}, + {t: 120_000, st: 40_000, v: 10, lbls: defLbls}, + {t: 180_000, st: 40_000, v: 10, lbls: defLbls}, + {t: 50_000, st: 40_000, v: 10, lbls: defLbls}, }, expectedSamples: []*walSample{ {t: 40_000, f: 0, lbls: defLbls}, @@ -1271,8 +1271,8 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { for _, sample := range tc.inputSamples { // We supposed to write a Histogram to the WAL if sample.h != nil { - _, err := app.AppendHistogramCTZeroSample(0, sample.lbls, sample.t, sample.ct, zeroHistogram, nil) - if !errors.Is(err, storage.ErrOutOfOrderCT) { + _, err := app.AppendHistogramSTZeroSample(0, sample.lbls, sample.t, sample.st, zeroHistogram, nil) + if !errors.Is(err, storage.ErrOutOfOrderST) { require.Equal(t, sample.expectsError, err != nil, "expected error: %v, got: %v", sample.expectsError, err) } @@ -1280,8 +1280,8 @@ func TestDBCreatedTimestampSamplesIngestion(t *testing.T) { require.NoError(t, err) } else { // We supposed to write a float sample to the WAL - _, err := app.AppendCTZeroSample(0, sample.lbls, sample.t, sample.ct) - if !errors.Is(err, storage.ErrOutOfOrderCT) { + _, err := app.AppendSTZeroSample(0, sample.lbls, sample.t, sample.st) + if !errors.Is(err, storage.ErrOutOfOrderST) { require.Equal(t, sample.expectsError, err != nil, "expected error: %v, got: %v", sample.expectsError, err) } diff --git a/tsdb/head_append.go b/tsdb/head_append.go index 8740d2f5ad..942c3ce974 100644 --- a/tsdb/head_append.go +++ b/tsdb/head_append.go @@ -83,14 +83,14 @@ func (a *initAppender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t return a.app.AppendHistogram(ref, l, t, h, fh) } -func (a *initAppender) AppendHistogramCTZeroSample(ref storage.SeriesRef, l labels.Labels, t, ct int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { +func (a *initAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { if a.app != nil { - return a.app.AppendHistogramCTZeroSample(ref, l, t, ct, h, fh) + return a.app.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) } a.head.initTime(t) a.app = a.head.appender() - return a.app.AppendHistogramCTZeroSample(ref, l, t, ct, h, fh) + return a.app.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) } func (a *initAppender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) { @@ -102,15 +102,15 @@ func (a *initAppender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m return a.app.UpdateMetadata(ref, l, m) } -func (a *initAppender) AppendCTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, ct int64) (storage.SeriesRef, error) { +func (a *initAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64) (storage.SeriesRef, error) { if a.app != nil { - return a.app.AppendCTZeroSample(ref, lset, t, ct) + return a.app.AppendSTZeroSample(ref, lset, t, st) } a.head.initTime(t) a.app = a.head.appender() - return a.app.AppendCTZeroSample(ref, lset, t, ct) + return a.app.AppendSTZeroSample(ref, lset, t, st) } // initTime initializes a head with the first timestamp. This only needs to be called @@ -483,12 +483,12 @@ func (a *headAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64 return storage.SeriesRef(s.ref), nil } -// AppendCTZeroSample appends synthetic zero sample for ct timestamp. It returns +// AppendSTZeroSample appends synthetic zero sample for st timestamp. It returns // error when sample can't be appended. See -// storage.CreatedTimestampAppender.AppendCTZeroSample for further documentation. -func (a *headAppender) AppendCTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, ct int64) (storage.SeriesRef, error) { - if ct >= t { - return 0, storage.ErrCTNewerThanSample +// storage.StartTimestampAppender.AppendSTZeroSample for further documentation. +func (a *headAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64) (storage.SeriesRef, error) { + if st >= t { + return 0, storage.ErrSTNewerThanSample } s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) @@ -500,11 +500,11 @@ func (a *headAppender) AppendCTZeroSample(ref storage.SeriesRef, lset labels.Lab } } - // Check if CT wouldn't be OOO vs samples we already might have for this series. + // Check if ST wouldn't be OOO vs samples we already might have for this series. // NOTE(bwplotka): This will be often hit as it's expected for long living - // counters to share the same CT. + // counters to share the same ST. s.Lock() - isOOO, _, err := s.appendable(ct, 0, a.headMaxt, a.minValidTime, a.oooTimeWindow) + isOOO, _, err := s.appendable(st, 0, a.headMaxt, a.minValidTime, a.oooTimeWindow) if err == nil { s.pendingCommit = true } @@ -513,14 +513,14 @@ func (a *headAppender) AppendCTZeroSample(ref storage.SeriesRef, lset labels.Lab return 0, err } if isOOO { - return storage.SeriesRef(s.ref), storage.ErrOutOfOrderCT + return storage.SeriesRef(s.ref), storage.ErrOutOfOrderST } - if ct > a.maxt { - a.maxt = ct + if st > a.maxt { + a.maxt = st } b := a.getCurrentBatch(stFloat, s.ref) - b.floats = append(b.floats, record.RefSample{Ref: s.ref, T: ct, V: 0.0}) + b.floats = append(b.floats, record.RefSample{Ref: s.ref, T: st, V: 0.0}) b.floatSeries = append(b.floatSeries, s) return storage.SeriesRef(s.ref), nil } @@ -902,9 +902,9 @@ func (a *headAppender) AppendHistogram(ref storage.SeriesRef, lset labels.Labels return storage.SeriesRef(s.ref), nil } -func (a *headAppender) AppendHistogramCTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, ct int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - if ct >= t { - return 0, storage.ErrCTNewerThanSample +func (a *headAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + if st >= t { + return 0, storage.ErrSTNewerThanSample } s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) @@ -919,7 +919,7 @@ func (a *headAppender) AppendHistogramCTZeroSample(ref storage.SeriesRef, lset l switch { case h != nil: zeroHistogram := &histogram.Histogram{ - // The CTZeroSample represents a counter reset by definition. + // The STZeroSample represents a counter reset by definition. CounterResetHint: histogram.CounterReset, // Replicate other fields to avoid needless chunk creation. Schema: h.Schema, @@ -927,41 +927,41 @@ func (a *headAppender) AppendHistogramCTZeroSample(ref storage.SeriesRef, lset l CustomValues: h.CustomValues, } s.Lock() - // For CTZeroSamples OOO is not allowed. + // For STZeroSamples OOO is not allowed. // We set it to true to make this implementation as close as possible to the float implementation. - isOOO, _, err := s.appendableHistogram(ct, zeroHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) + isOOO, _, err := s.appendableHistogram(st, zeroHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) if err != nil { s.Unlock() if errors.Is(err, storage.ErrOutOfOrderSample) { - return 0, storage.ErrOutOfOrderCT + return 0, storage.ErrOutOfOrderST } return 0, err } - // OOO is not allowed because after the first scrape, CT will be the same for most (if not all) future samples. + // OOO is not allowed because after the first scrape, ST will be the same for most (if not all) future samples. // This is to prevent the injected zero from being marked as OOO forever. if isOOO { s.Unlock() - return 0, storage.ErrOutOfOrderCT + return 0, storage.ErrOutOfOrderST } s.pendingCommit = true s.Unlock() - st := stHistogram + sTyp := stHistogram if h.UsesCustomBuckets() { - st = stCustomBucketHistogram + sTyp = stCustomBucketHistogram } - b := a.getCurrentBatch(st, s.ref) + b := a.getCurrentBatch(sTyp, s.ref) b.histograms = append(b.histograms, record.RefHistogramSample{ Ref: s.ref, - T: ct, + T: st, H: zeroHistogram, }) b.histogramSeries = append(b.histogramSeries, s) case fh != nil: zeroFloatHistogram := &histogram.FloatHistogram{ - // The CTZeroSample represents a counter reset by definition. + // The STZeroSample represents a counter reset by definition. CounterResetHint: histogram.CounterReset, // Replicate other fields to avoid needless chunk creation. Schema: fh.Schema, @@ -970,40 +970,40 @@ func (a *headAppender) AppendHistogramCTZeroSample(ref storage.SeriesRef, lset l } s.Lock() // We set it to true to make this implementation as close as possible to the float implementation. - isOOO, _, err := s.appendableFloatHistogram(ct, zeroFloatHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) // OOO is not allowed for CTZeroSamples. + isOOO, _, err := s.appendableFloatHistogram(st, zeroFloatHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) // OOO is not allowed for STZeroSamples. if err != nil { s.Unlock() if errors.Is(err, storage.ErrOutOfOrderSample) { - return 0, storage.ErrOutOfOrderCT + return 0, storage.ErrOutOfOrderST } return 0, err } - // OOO is not allowed because after the first scrape, CT will be the same for most (if not all) future samples. + // OOO is not allowed because after the first scrape, ST will be the same for most (if not all) future samples. // This is to prevent the injected zero from being marked as OOO forever. if isOOO { s.Unlock() - return 0, storage.ErrOutOfOrderCT + return 0, storage.ErrOutOfOrderST } s.pendingCommit = true s.Unlock() - st := stFloatHistogram + sTyp := stFloatHistogram if fh.UsesCustomBuckets() { - st = stCustomBucketFloatHistogram + sTyp = stCustomBucketFloatHistogram } - b := a.getCurrentBatch(st, s.ref) + b := a.getCurrentBatch(sTyp, s.ref) b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ Ref: s.ref, - T: ct, + T: st, FH: zeroFloatHistogram, }) b.floatHistogramSeries = append(b.floatHistogramSeries, s) } - if ct > a.maxt { - a.maxt = ct + if st > a.maxt { + a.maxt = st } return storage.SeriesRef(s.ref), nil diff --git a/tsdb/head_test.go b/tsdb/head_test.go index 63ec9739c4..552db13d07 100644 --- a/tsdb/head_test.go +++ b/tsdb/head_test.go @@ -6715,7 +6715,7 @@ func TestHeadAppender_AppendFloatWithSameTimestampAsPreviousHistogram(t *testing require.ErrorIs(t, err, storage.NewDuplicateHistogramToFloatErr(2_000, 10.0)) } -func TestHeadAppender_AppendCT(t *testing.T) { +func TestHeadAppender_AppendST(t *testing.T) { testHistogram := tsdbutil.GenerateTestHistogram(1) testHistogram.CounterResetHint = histogram.NotCounterReset testFloatHistogram := tsdbutil.GenerateTestFloatHistogram(1) @@ -6743,7 +6743,7 @@ func TestHeadAppender_AppendCT(t *testing.T) { fSample float64 h *histogram.Histogram fh *histogram.FloatHistogram - ct int64 + st int64 } for _, tc := range []struct { name string @@ -6753,8 +6753,8 @@ func TestHeadAppender_AppendCT(t *testing.T) { { name: "In order ct+normal sample/floatSample", appendableSamples: []appendableSamples{ - {ts: 100, fSample: 10, ct: 1}, - {ts: 101, fSample: 10, ct: 1}, + {ts: 100, fSample: 10, st: 1}, + {ts: 101, fSample: 10, st: 1}, }, expectedSamples: []chunks.Sample{ sample{t: 1, f: 0}, @@ -6765,8 +6765,8 @@ func TestHeadAppender_AppendCT(t *testing.T) { { name: "In order ct+normal sample/histogram", appendableSamples: []appendableSamples{ - {ts: 100, h: testHistogram, ct: 1}, - {ts: 101, h: testHistogram, ct: 1}, + {ts: 100, h: testHistogram, st: 1}, + {ts: 101, h: testHistogram, st: 1}, }, expectedSamples: func() []chunks.Sample { return []chunks.Sample{ @@ -6779,8 +6779,8 @@ func TestHeadAppender_AppendCT(t *testing.T) { { name: "In order ct+normal sample/floathistogram", appendableSamples: []appendableSamples{ - {ts: 100, fh: testFloatHistogram, ct: 1}, - {ts: 101, fh: testFloatHistogram, ct: 1}, + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 101, fh: testFloatHistogram, st: 1}, }, expectedSamples: func() []chunks.Sample { return []chunks.Sample{ @@ -6791,10 +6791,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }(), }, { - name: "Consecutive appends with same ct ignore ct/floatSample", + name: "Consecutive appends with same st ignore st/floatSample", appendableSamples: []appendableSamples{ - {ts: 100, fSample: 10, ct: 1}, - {ts: 101, fSample: 10, ct: 1}, + {ts: 100, fSample: 10, st: 1}, + {ts: 101, fSample: 10, st: 1}, }, expectedSamples: []chunks.Sample{ sample{t: 1, f: 0}, @@ -6803,10 +6803,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }, }, { - name: "Consecutive appends with same ct ignore ct/histogram", + name: "Consecutive appends with same st ignore st/histogram", appendableSamples: []appendableSamples{ - {ts: 100, h: testHistogram, ct: 1}, - {ts: 101, h: testHistogram, ct: 1}, + {ts: 100, h: testHistogram, st: 1}, + {ts: 101, h: testHistogram, st: 1}, }, expectedSamples: func() []chunks.Sample { return []chunks.Sample{ @@ -6817,10 +6817,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }(), }, { - name: "Consecutive appends with same ct ignore ct/floathistogram", + name: "Consecutive appends with same st ignore st/floathistogram", appendableSamples: []appendableSamples{ - {ts: 100, fh: testFloatHistogram, ct: 1}, - {ts: 101, fh: testFloatHistogram, ct: 1}, + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 101, fh: testFloatHistogram, st: 1}, }, expectedSamples: func() []chunks.Sample { return []chunks.Sample{ @@ -6831,10 +6831,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }(), }, { - name: "Consecutive appends with newer ct do not ignore ct/floatSample", + name: "Consecutive appends with newer st do not ignore st/floatSample", appendableSamples: []appendableSamples{ - {ts: 100, fSample: 10, ct: 1}, - {ts: 102, fSample: 10, ct: 101}, + {ts: 100, fSample: 10, st: 1}, + {ts: 102, fSample: 10, st: 101}, }, expectedSamples: []chunks.Sample{ sample{t: 1, f: 0}, @@ -6844,10 +6844,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }, }, { - name: "Consecutive appends with newer ct do not ignore ct/histogram", + name: "Consecutive appends with newer st do not ignore st/histogram", appendableSamples: []appendableSamples{ - {ts: 100, h: testHistogram, ct: 1}, - {ts: 102, h: testHistogram, ct: 101}, + {ts: 100, h: testHistogram, st: 1}, + {ts: 102, h: testHistogram, st: 101}, }, expectedSamples: []chunks.Sample{ sample{t: 1, h: testZeroHistogram}, @@ -6857,10 +6857,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }, }, { - name: "Consecutive appends with newer ct do not ignore ct/floathistogram", + name: "Consecutive appends with newer st do not ignore st/floathistogram", appendableSamples: []appendableSamples{ - {ts: 100, fh: testFloatHistogram, ct: 1}, - {ts: 102, fh: testFloatHistogram, ct: 101}, + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 102, fh: testFloatHistogram, st: 101}, }, expectedSamples: []chunks.Sample{ sample{t: 1, fh: testZeroFloatHistogram}, @@ -6870,10 +6870,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }, }, { - name: "CT equals to previous sample timestamp is ignored/floatSample", + name: "ST equals to previous sample timestamp is ignored/floatSample", appendableSamples: []appendableSamples{ - {ts: 100, fSample: 10, ct: 1}, - {ts: 101, fSample: 10, ct: 100}, + {ts: 100, fSample: 10, st: 1}, + {ts: 101, fSample: 10, st: 100}, }, expectedSamples: []chunks.Sample{ sample{t: 1, f: 0}, @@ -6882,10 +6882,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }, }, { - name: "CT equals to previous sample timestamp is ignored/histogram", + name: "ST equals to previous sample timestamp is ignored/histogram", appendableSamples: []appendableSamples{ - {ts: 100, h: testHistogram, ct: 1}, - {ts: 101, h: testHistogram, ct: 100}, + {ts: 100, h: testHistogram, st: 1}, + {ts: 101, h: testHistogram, st: 100}, }, expectedSamples: func() []chunks.Sample { return []chunks.Sample{ @@ -6896,10 +6896,10 @@ func TestHeadAppender_AppendCT(t *testing.T) { }(), }, { - name: "CT equals to previous sample timestamp is ignored/floathistogram", + name: "ST equals to previous sample timestamp is ignored/floathistogram", appendableSamples: []appendableSamples{ - {ts: 100, fh: testFloatHistogram, ct: 1}, - {ts: 101, fh: testFloatHistogram, ct: 100}, + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 101, fh: testFloatHistogram, st: 100}, }, expectedSamples: func() []chunks.Sample { return []chunks.Sample{ @@ -6920,7 +6920,7 @@ func TestHeadAppender_AppendCT(t *testing.T) { for _, sample := range tc.appendableSamples { // Append float if it's a float test case if sample.fSample != 0 { - _, err := a.AppendCTZeroSample(0, lbls, sample.ts, sample.ct) + _, err := a.AppendSTZeroSample(0, lbls, sample.ts, sample.st) require.NoError(t, err) _, err = a.Append(0, lbls, sample.ts, sample.fSample) require.NoError(t, err) @@ -6928,7 +6928,7 @@ func TestHeadAppender_AppendCT(t *testing.T) { // Append histograms if it's a histogram test case if sample.h != nil || sample.fh != nil { - ref, err := a.AppendHistogramCTZeroSample(0, lbls, sample.ts, sample.ct, sample.h, sample.fh) + ref, err := a.AppendHistogramSTZeroSample(0, lbls, sample.ts, sample.st, sample.h, sample.fh) require.NoError(t, err) _, err = a.AppendHistogram(ref, lbls, sample.ts, sample.h, sample.fh) require.NoError(t, err) @@ -6944,12 +6944,12 @@ func TestHeadAppender_AppendCT(t *testing.T) { } } -func TestHeadAppender_AppendHistogramCTZeroSample(t *testing.T) { +func TestHeadAppender_AppendHistogramSTZeroSample(t *testing.T) { type appendableSamples struct { ts int64 h *histogram.Histogram fh *histogram.FloatHistogram - ct int64 // 0 if no created timestamp. + st int64 // 0 if no created timestamp. } for _, tc := range []struct { name string @@ -6957,32 +6957,32 @@ func TestHeadAppender_AppendHistogramCTZeroSample(t *testing.T) { expectedError error }{ { - name: "integer histogram CT lower than minValidTime initiates ErrOutOfBounds", + name: "integer histogram ST lower than minValidTime initiates ErrOutOfBounds", appendableSamples: []appendableSamples{ - {ts: 100, h: tsdbutil.GenerateTestHistogram(1), ct: -1}, + {ts: 100, h: tsdbutil.GenerateTestHistogram(1), st: -1}, }, expectedError: storage.ErrOutOfBounds, }, { - name: "float histograms CT lower than minValidTime initiates ErrOutOfBounds", + name: "float histograms ST lower than minValidTime initiates ErrOutOfBounds", appendableSamples: []appendableSamples{ - {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1), ct: -1}, + {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1), st: -1}, }, expectedError: storage.ErrOutOfBounds, }, { - name: "integer histogram CT duplicates an existing sample", + name: "integer histogram ST duplicates an existing sample", appendableSamples: []appendableSamples{ {ts: 100, h: tsdbutil.GenerateTestHistogram(1)}, - {ts: 200, h: tsdbutil.GenerateTestHistogram(1), ct: 100}, + {ts: 200, h: tsdbutil.GenerateTestHistogram(1), st: 100}, }, expectedError: storage.ErrDuplicateSampleForTimestamp, }, { - name: "float histogram CT duplicates an existing sample", + name: "float histogram ST duplicates an existing sample", appendableSamples: []appendableSamples{ {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1)}, - {ts: 200, fh: tsdbutil.GenerateTestFloatHistogram(1), ct: 100}, + {ts: 200, fh: tsdbutil.GenerateTestFloatHistogram(1), st: 100}, }, expectedError: storage.ErrDuplicateSampleForTimestamp, }, @@ -7000,8 +7000,8 @@ func TestHeadAppender_AppendHistogramCTZeroSample(t *testing.T) { for _, sample := range tc.appendableSamples { a := h.Appender(context.Background()) var err error - if sample.ct != 0 { - ref, err = a.AppendHistogramCTZeroSample(ref, lbls, sample.ts, sample.ct, sample.h, sample.fh) + if sample.st != 0 { + ref, err = a.AppendHistogramSTZeroSample(ref, lbls, sample.ts, sample.st, sample.h, sample.fh) require.ErrorIs(t, err, tc.expectedError) } diff --git a/web/api/v1/api.go b/web/api/v1/api.go index 793e5a0075..86c0461087 100644 --- a/web/api/v1/api.go +++ b/web/api/v1/api.go @@ -290,7 +290,7 @@ func NewAPI( rwEnabled bool, acceptRemoteWriteProtoMsgs remoteapi.MessageTypes, otlpEnabled, otlpDeltaToCumulative, otlpNativeDeltaIngestion bool, - ctZeroIngestionEnabled bool, + stZeroIngestionEnabled bool, lookbackDelta time.Duration, enableTypeAndUnitLabels bool, appendMetadata bool, @@ -339,14 +339,14 @@ func NewAPI( } if rwEnabled { - a.remoteWriteHandler = remote.NewWriteHandler(logger, registerer, ap, acceptRemoteWriteProtoMsgs, ctZeroIngestionEnabled, enableTypeAndUnitLabels, appendMetadata) + a.remoteWriteHandler = remote.NewWriteHandler(logger, registerer, ap, acceptRemoteWriteProtoMsgs, stZeroIngestionEnabled, enableTypeAndUnitLabels, appendMetadata) } if otlpEnabled { a.otlpWriteHandler = remote.NewOTLPWriteHandler(logger, registerer, ap, configFunc, remote.OTLPOptions{ ConvertDelta: otlpDeltaToCumulative, NativeDelta: otlpNativeDeltaIngestion, LookbackDelta: lookbackDelta, - IngestCTZeroSample: ctZeroIngestionEnabled, + IngestSTZeroSample: stZeroIngestionEnabled, EnableTypeAndUnitLabels: enableTypeAndUnitLabels, AppendMetadata: appendMetadata, }) diff --git a/web/web.go b/web/web.go index 2d353a8af8..d7b647e3db 100644 --- a/web/web.go +++ b/web/web.go @@ -293,7 +293,7 @@ type Options struct { ConvertOTLPDelta bool NativeOTLPDeltaIngestion bool IsAgent bool - CTZeroIngestionEnabled bool + STZeroIngestionEnabled bool EnableTypeAndUnitLabels bool AppendMetadata bool AppName string @@ -394,7 +394,7 @@ func New(logger *slog.Logger, o *Options) *Handler { o.EnableOTLPWriteReceiver, o.ConvertOTLPDelta, o.NativeOTLPDeltaIngestion, - o.CTZeroIngestionEnabled, + o.STZeroIngestionEnabled, o.LookbackDelta, o.EnableTypeAndUnitLabels, o.AppendMetadata, From 743116649bb85a9e22eff1960606a944b149321b Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Fri, 7 Nov 2025 14:59:19 +0100 Subject: [PATCH 044/439] prepare release 3.8.0-rc.0 Signed-off-by: Jan Fajerski --- CHANGELOG.md | 34 +++++++++++++++++++- VERSION | 2 +- web/ui/mantine-ui/package.json | 4 +-- web/ui/module/codemirror-promql/package.json | 4 +-- web/ui/module/lezer-promql/package.json | 2 +- web/ui/package-lock.json | 14 ++++---- web/ui/package.json | 2 +- 7 files changed, 47 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 11ec004cb6..07b6e1d74b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,39 @@ ## main / unreleased -* [FEATURE] Templates: Add urlQueryEscape to template functions. #17403 +## 3.8.0-rc.0 / 2025-11-07 + +* [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #17287 +* [FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. #16483 +* [FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. #17046 +* [FEATURE] Native histograms are now a stable, but optional feature, use the `scrape_native_histogram` config setting. #17232 #17315 +* [FEATURE] UI: Support anchored and smoothed keyword in promql editor. #17239 +* [FEATURE] UI: Show detailed relabeling steps for each discovered target. #17337 +* [FEATURE] Alerting: Add urlQueryEscape to template functions. #17403 +* [FEATURE] Promtool: Add Remote-Write 2.0 support to `promtool push metrics` via the `--protobuf_message` flag. #17417 +* [ENHANCEMENT] Clarify the docs about handling negative native histograms. #17249 +* [ENHANCEMENT] Mixin: Add static UID to the remote-write dashboard. #17256 +* [ENHANCEMENT] PromQL: Reconcile mismatched NHCB bounds in `Add` and `Sub`. #17278 +* [ENHANCEMENT] Alerting: Add "unknown" state for alerting rules that haven't been evaluated yet. #17282 +* [ENHANCEMENT] Scrape: Allow simultaneous use of classic histogram → NHCB conversion and zero-timestamp ingestion. #17305 +* [ENHANCEMENT] UI: Add smoothed/anchored in explain. #17334 +* [ENHANCEMENT] OTLP: De-duplicate any `target_info` samples with the same timestamp for the same series. #17400 +* [ENHANCEMENT] Document `use_fips_sts_endpoint` in `sigv4` config sections. #17304 +* [ENHANCEMENT] Document Prometheus Agent. #14519 +* [PERF] PromQL: Speed up parsing of variadic functions. #17316 +* [PERF] UI: Speed up alerts/rules/... pages by not rendering collapsed content. #17485 +* [PERF] UI: Performance improvement when getting label name and values in promql editor. #17194 +* [PERF] UI: Speed up /alerts for many firing alerts via virtual scrolling. #17254 +* [BUGFIX] PromQL: Fix slice indexing bug in info function on churning series. #17199 +* [BUGFIX] API: Reduce lock contention on `/api/v1/targets`. #17306 +* [BUGFIX] PromQL: Consistent handling of gauge vs. counter histograms in aggregations. #17312 +* [BUGFIX] TSDB: Allow NHCB with -Inf as the first custom value. #17320 +* [BUGFIX] UI: Fix duplicate loading of data from the API speed up rendering of some pages. #17357 +* [BUGFIX] Old UI: Fix createExpressionLink to correctly build /graph URLs so links from Alerts/Rules work again. #17365 +* [BUGFIX] PromQL: Avoid panic when parsing malformed `info` call. #17379 +* [BUGFIX] PromQL: Include histograms when enforcing sample_limit. #17390 +* [BUGFIX] Config: Fix panic if TLS CA file is absent. #17418 +* [BUGFIX] PromQL: Fix `histogram_fraction` for classic histograms and NHCB if lower bound is in the first bucket. #17424 ## 3.7.3 / 2025-10-29 diff --git a/VERSION b/VERSION index c1e43e6d45..100ac3dfd6 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.7.3 +3.8.0-rc.0 diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json index 219d357f0d..1f10e4b620 100644 --- a/web/ui/mantine-ui/package.json +++ b/web/ui/mantine-ui/package.json @@ -1,7 +1,7 @@ { "name": "@prometheus-io/mantine-ui", "private": true, - "version": "0.307.3", + "version": "0.308.0-rc.0", "type": "module", "scripts": { "start": "vite", @@ -28,7 +28,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.307.3", + "@prometheus-io/codemirror-promql": "0.308.0-rc.0", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json index f850342728..b32fd59d19 100644 --- a/web/ui/module/codemirror-promql/package.json +++ b/web/ui/module/codemirror-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/codemirror-promql", - "version": "0.307.3", + "version": "0.308.0-rc.0", "description": "a CodeMirror mode for the PromQL language", "types": "dist/esm/index.d.ts", "module": "dist/esm/index.js", @@ -29,7 +29,7 @@ }, "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md", "dependencies": { - "@prometheus-io/lezer-promql": "0.307.3", + "@prometheus-io/lezer-promql": "0.308.0-rc.0", "lru-cache": "^11.2.2" }, "devDependencies": { diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json index 05511c2b89..d86f1a1e7a 100644 --- a/web/ui/module/lezer-promql/package.json +++ b/web/ui/module/lezer-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/lezer-promql", - "version": "0.307.3", + "version": "0.308.0-rc.0", "description": "lezer-based PromQL grammar", "main": "dist/index.cjs", "type": "module", diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json index 2631802e53..a9a75a131a 100644 --- a/web/ui/package-lock.json +++ b/web/ui/package-lock.json @@ -1,12 +1,12 @@ { "name": "prometheus-io", - "version": "0.307.3", + "version": "0.308.0-rc.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "prometheus-io", - "version": "0.307.3", + "version": "0.308.0-rc.0", "workspaces": [ "mantine-ui", "module/*" @@ -24,7 +24,7 @@ }, "mantine-ui": { "name": "@prometheus-io/mantine-ui", - "version": "0.307.3", + "version": "0.308.0-rc.0", "dependencies": { "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", @@ -42,7 +42,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.307.3", + "@prometheus-io/codemirror-promql": "0.308.0-rc.0", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", @@ -88,10 +88,10 @@ }, "module/codemirror-promql": { "name": "@prometheus-io/codemirror-promql", - "version": "0.307.3", + "version": "0.308.0-rc.0", "license": "Apache-2.0", "dependencies": { - "@prometheus-io/lezer-promql": "0.307.3", + "@prometheus-io/lezer-promql": "0.308.0-rc.0", "lru-cache": "^11.2.2" }, "devDependencies": { @@ -121,7 +121,7 @@ }, "module/lezer-promql": { "name": "@prometheus-io/lezer-promql", - "version": "0.307.3", + "version": "0.308.0-rc.0", "license": "Apache-2.0", "devDependencies": { "@lezer/generator": "^1.8.0", diff --git a/web/ui/package.json b/web/ui/package.json index e237294df8..d8f2c712ff 100644 --- a/web/ui/package.json +++ b/web/ui/package.json @@ -1,7 +1,7 @@ { "name": "prometheus-io", "description": "Monorepo for the Prometheus UI", - "version": "0.307.3", + "version": "0.308.0-rc.0", "private": true, "scripts": { "build": "bash build_ui.sh --all", From 987b28e26ccaba6d39590b0dc55a430ae70b3716 Mon Sep 17 00:00:00 2001 From: Julius Hinze Date: Thu, 13 Nov 2025 16:59:14 +0100 Subject: [PATCH 045/439] discovery: fix constructor arguments in aws discovery (#17526) Signed-off-by: Julius Hinze --- discovery/aws/aws.go | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/discovery/aws/aws.go b/discovery/aws/aws.go index 0fd5160b04..bfb2be183c 100644 --- a/discovery/aws/aws.go +++ b/discovery/aws/aws.go @@ -215,11 +215,13 @@ func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Di switch c.Role { case RoleEC2: - return NewEC2Discovery(c.EC2SDConfig, opts.Logger, &ec2Metrics{refreshMetrics: awsMetrics.refreshMetrics}) + opts.Metrics = &ec2Metrics{refreshMetrics: awsMetrics.refreshMetrics} + return NewEC2Discovery(c.EC2SDConfig, opts) case RoleECS: return NewECSDiscovery(c.ECSSDConfig, opts.Logger, &ecsMetrics{refreshMetrics: awsMetrics.refreshMetrics}) case RoleLightsail: - return NewLightsailDiscovery(c.LightsailSDConfig, opts.Logger, &lightsailMetrics{refreshMetrics: awsMetrics.refreshMetrics}) + opts.Metrics = &lightsailMetrics{refreshMetrics: awsMetrics.refreshMetrics} + return NewLightsailDiscovery(c.LightsailSDConfig, opts) default: return nil, fmt.Errorf("unknown AWS SD role %q", c.Role) } From 35c3232a2ee541273828982b2ce6aadd6c1c9a5f Mon Sep 17 00:00:00 2001 From: Ayoub Mrini Date: Fri, 14 Nov 2025 09:42:20 +0100 Subject: [PATCH 046/439] test: skip TestRemoteWrite_ReshardingWithoutDeadlock temporarily as flaky (#17534) Signed-off-by: machine424 --- cmd/prometheus/main_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/cmd/prometheus/main_test.go b/cmd/prometheus/main_test.go index ccc9151492..e5e3db39ae 100644 --- a/cmd/prometheus/main_test.go +++ b/cmd/prometheus/main_test.go @@ -969,6 +969,7 @@ remote_write: // TestRemoteWrite_ReshardingWithoutDeadlock ensures that resharding (scaling up) doesn't block when the shards are full. // See: https://github.com/prometheus/prometheus/issues/17384. func TestRemoteWrite_ReshardingWithoutDeadlock(t *testing.T) { + t.Skip("flaky test, see https://github.com/prometheus/prometheus/issues/17489") t.Parallel() tmpDir := t.TempDir() From e022a727a8f165c2f9f04443be5f43e659b572d3 Mon Sep 17 00:00:00 2001 From: Raul Leite Date: Fri, 14 Nov 2025 15:31:21 -0600 Subject: [PATCH 047/439] =?UTF-8?q?I=E2=80=99ve=20proposed=20a=20slight=20?= =?UTF-8?q?rewording=20of=20this=20section=20to=20improve=20clarity=20and?= =?UTF-8?q?=20readability.=20(On-Disk=20Layout=20Paragraph)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Raul Leite --- docs/storage.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/storage.md b/docs/storage.md index 7b6e3bffe8..de8a8ed065 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -11,13 +11,13 @@ Prometheus's local time series database stores data in a custom, highly efficien ### On-disk layout -Ingested samples are grouped into blocks of two hours. Each two-hour block consists -of a directory containing a chunks subdirectory containing all the time series samples -for that window of time, a metadata file, and an index file (which indexes metric names -and labels to time series in the chunks directory). The samples in the chunks directory -are grouped together into one or more segment files of up to 512MB each by default. When -series are deleted via the API, deletion records are stored in separate tombstone files -(instead of deleting the data immediately from the chunk segments). +Ingested samples are grouped into two-hour blocks. Each block consists of a directory that +contains a chunks subdirectory with all the time series samples for that time window, +a metadata file, and an index file (which maps metric names and labels to the time series +in the chunks directory). By default, the samples in the chunks directory are organized +into one or more segment files, each up to 512 MB. When series are deleted via the API, +deletion records are stored in separate tombstone files rather than being immediately +removed from the chunk segments. The current block for incoming samples is kept in memory and is not fully persisted. It is secured against crashes by a write-ahead log (WAL) that can be From c9827ef983038183a99b5ee851dc50332b29392e Mon Sep 17 00:00:00 2001 From: Raul Leite Date: Sat, 15 Nov 2025 10:52:18 -0600 Subject: [PATCH 048/439] Fix formatting in storage.md extra space removed Signed-off-by: Raul Leite --- docs/storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/storage.md b/docs/storage.md index de8a8ed065..848c0d58e8 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -16,7 +16,7 @@ contains a chunks subdirectory with all the time series samples for that time wi a metadata file, and an index file (which maps metric names and labels to the time series in the chunks directory). By default, the samples in the chunks directory are organized into one or more segment files, each up to 512 MB. When series are deleted via the API, -deletion records are stored in separate tombstone files rather than being immediately +deletion records are stored in separate tombstone files rather than being immediately removed from the chunk segments. The current block for incoming samples is kept in memory and is not fully From 407b697ee2986430891a56c439167bf25113ad9b Mon Sep 17 00:00:00 2001 From: Raul Leite Date: Sat, 15 Nov 2025 10:55:26 -0600 Subject: [PATCH 049/439] structure adjusted as reccomended Corrected the structure of the explanation regarding how samples are organized in the chunks directory and the handling of deletion records. Signed-off-by: Raul Leite --- docs/storage.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/storage.md b/docs/storage.md index 848c0d58e8..1feb0a4940 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -14,9 +14,9 @@ Prometheus's local time series database stores data in a custom, highly efficien Ingested samples are grouped into two-hour blocks. Each block consists of a directory that contains a chunks subdirectory with all the time series samples for that time window, a metadata file, and an index file (which maps metric names and labels to the time series -in the chunks directory). By default, the samples in the chunks directory are organized -into one or more segment files, each up to 512 MB. When series are deleted via the API, -deletion records are stored in separate tombstone files rather than being immediately +in the chunks directory). The samples in the chunks directory are organized into one +or more segment files, each up to 512 MB by default. When series are deleted via the API, +the deletion records are stored in separate tombstone files rather than being immediately removed from the chunk segments. The current block for incoming samples is kept in memory and is not fully From c64dd612efadf8ccafc4bf985bd866ade719a727 Mon Sep 17 00:00:00 2001 From: zenador Date: Sun, 16 Nov 2025 04:07:36 +0800 Subject: [PATCH 050/439] PromQL: Fix bug with inconsistent results for queries with OR expression and EnableDelayedNameRemoval (#17161) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Jeanette Tan Signed-off-by: zenador Co-authored-by: Björn Rabenstein --- docs/feature_flags.md | 2 ++ promql/engine.go | 8 ++++- .../testdata/name_label_dropping.test | 35 +++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/docs/feature_flags.md b/docs/feature_flags.md index 384b124c6a..f9f27bfb7f 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -150,6 +150,8 @@ These queries are rare to occur and easy to fix. (In the above example, removing `by (__name__)` doesn't change anything without the feature flag and fixes the possible problem with the feature flag.) +It is possible to craft a query that aggregates by `__name__` and puts samples with and without delayed name removal into the same group. In that case, the name is removed from the affected group. Note that this case hardly occurs in queries that fulfill a practical purpose. + ## Auto Reload Config `--enable-feature=auto-reload-config` diff --git a/promql/engine.go b/promql/engine.go index 74864bdcae..67f9b9e3ba 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -3185,6 +3185,7 @@ type groupedAggregation struct { incrementalMean bool // True after reverting to incremental calculation of the mean value. counterResetSeen bool // Counter reset hint CounterReset seen. Currently only used for histogram samples. notCounterResetSeen bool // Counter reset hint NotCounterReset seen. Currently only used for histogram samples. + dropName bool // True if any sample in this group has DropName set. } // aggregation evaluates sum, avg, count, stdvar, stddev or quantile at one timestep on inputMatrix. @@ -3213,6 +3214,7 @@ func (ev *evaluator) aggregation(e *parser.AggregateExpr, q float64, inputMatrix floatMean: f, incompatibleHistograms: false, groupCount: 1, + dropName: inputMatrix[si].DropName, } switch op { case parser.AVG, parser.SUM: @@ -3269,6 +3271,10 @@ func (ev *evaluator) aggregation(e *parser.AggregateExpr, q float64, inputMatrix continue } + if inputMatrix[si].DropName { + group.dropName = true + } + switch op { case parser.SUM: if h != nil { @@ -3507,7 +3513,7 @@ func (ev *evaluator) aggregation(e *parser.AggregateExpr, q float64, inputMatrix ss := &outputMatrix[ri] addToSeries(ss, enh.Ts, aggr.floatValue, aggr.histogramValue, numSteps) - ss.DropName = inputMatrix[ri].DropName + ss.DropName = aggr.dropName } return annos diff --git a/promql/promqltest/testdata/name_label_dropping.test b/promql/promqltest/testdata/name_label_dropping.test index 3682021ba9..3a6f4098df 100644 --- a/promql/promqltest/testdata/name_label_dropping.test +++ b/promql/promqltest/testdata/name_label_dropping.test @@ -91,3 +91,38 @@ eval instant at 10m topk(10, sum by (__name__, env) (metric_total{env="1"})) eval instant at 10m topk(10, sum by (__name__, env) (rate(metric_total{env="1"}[10m]))) {env="1"} 0.2 + +clear + +# More testing for __name__ label drop with different input series. +load 1m + metric_total{env="1"} 0+1x10 + metric_total{env="2"} 0+3x10 + +# Metric name is preserved as there is no function that drops it. +eval instant at 10m sum by (__name__) (metric_total{env="1"}) + metric_total 10 + +# Metric name is dropped at the end because of rate and because there is no label function to preserve it. +eval instant at 10m sum by (__name__) (rate(metric_total{env="2"}[5m])) + {} 0.05 + +# Metric name is preserved with label_replace even though it would have been dropped with rate. +eval instant at 10m label_replace(sum by (__name__) (rate(metric_total{env="2"}[5m])), "__name__", "$1", "__name__", "(.+)") + metric_total 0.05 + +# Combining the above cases in an OR expression, we drop the name if any of the series drops it. +eval instant at 10m sum by (__name__) (metric_total{env="1"} or rate(metric_total{env="2"}[5m])) + {} 10.05 + +# Changing the order of the OR expression should not change the result. +eval instant at 10m sum by (__name__) (rate(metric_total{env="2"}[5m]) or metric_total{env="1"}) + {} 10.05 + +# With non-matching first selector, we use the second to determine if __name__ is dropped. +eval instant at 10m sum by (__name__) (metric_total{env="3"} or rate(metric_total{env="2"}[5m])) + {} 0.05 + +# Same as above, but with reversed order. +eval instant at 10m sum by (__name__) (rate(metric_total{env="3"}[5m]) or metric_total{env="1"}) + metric_total 10 From ae00fd45abb8e2e3f0a75cef8f4c8c43f33832cd Mon Sep 17 00:00:00 2001 From: 0xkato <106168398+0xkato@users.noreply.github.com> Date: Sat, 15 Nov 2025 21:09:00 +0100 Subject: [PATCH 051/439] tsdb: guard chunk length overflow in head chunk reader (#17533) Signed-off-by: 0xkato <0xkkato@gmail.com> --- tsdb/chunks/head_chunks.go | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/tsdb/chunks/head_chunks.go b/tsdb/chunks/head_chunks.go index 41fce69c72..5e143b8b32 100644 --- a/tsdb/chunks/head_chunks.go +++ b/tsdb/chunks/head_chunks.go @@ -20,6 +20,7 @@ import ( "fmt" "hash" "io" + "math" "os" "path/filepath" "slices" @@ -768,8 +769,25 @@ func (cdm *ChunkDiskMapper) Chunk(ref ChunkDiskMapperRef) (chunkenc.Chunk, error } } + if chkDataLen > uint64(math.MaxInt) { + return nil, &CorruptionErr{ + Dir: cdm.dir.Name(), + FileIndex: sgmIndex, + Err: fmt.Errorf("chunk length %d exceeds supported size", chkDataLen), + } + } + + chkDataLenInt := int(chkDataLen) + if chkDataLenStart > math.MaxInt-n-chkDataLenInt { + return nil, &CorruptionErr{ + Dir: cdm.dir.Name(), + FileIndex: sgmIndex, + Err: fmt.Errorf("chunk data end overflows supported size (start=%d, len=%d, n=%d)", chkDataLenStart, chkDataLenInt, n), + } + } + // Verify the chunk data end. - chkDataEnd := chkDataLenStart + n + int(chkDataLen) + chkDataEnd := chkDataLenStart + n + chkDataLenInt if chkDataEnd > mmapFile.byteSlice.Len() { return nil, &CorruptionErr{ Dir: cdm.dir.Name(), From 4aa8941eb186264e570ee76eda8b23ff8989abc5 Mon Sep 17 00:00:00 2001 From: Will Bollock Date: Sun, 16 Nov 2025 05:28:50 -0500 Subject: [PATCH 052/439] fix(discovery): aws discovery test fix (#17527) * fix: aws discovery test fix Fixes a problem introduced after the merge of this https://github.com/prometheus/prometheus/pull/17138 PR didn't take into account another merged PR! ``` discovery/aws/aws.go:218:54: too many arguments in call to NewEC2Discovery have (*EC2SDConfig, *slog.Logger, *ec2Metrics) want (*EC2SDConfig, discovery.DiscovererOptions) discovery/aws/aws.go:222:66: too many arguments in call to NewLightsailDiscovery have (*LightsailSDConfig, *slog.Logger, *lightsailMetrics) want (*LightsailSDConfig, discovery.DiscovererOptions) ``` Signed-off-by: Will Bollock * fix: align ecs style ECS was a new service discovery tool added after this PR was merged: https://github.com/prometheus/prometheus/pull/17138 Aligns the style of passing a single "opts" to it like almost all the other service discovery engines now use Signed-off-by: Will Bollock --------- Signed-off-by: Will Bollock --- discovery/aws/aws.go | 3 ++- discovery/aws/ecs.go | 14 +++++++------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/discovery/aws/aws.go b/discovery/aws/aws.go index bfb2be183c..1ac97b3c9e 100644 --- a/discovery/aws/aws.go +++ b/discovery/aws/aws.go @@ -218,7 +218,8 @@ func (c *SDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Di opts.Metrics = &ec2Metrics{refreshMetrics: awsMetrics.refreshMetrics} return NewEC2Discovery(c.EC2SDConfig, opts) case RoleECS: - return NewECSDiscovery(c.ECSSDConfig, opts.Logger, &ecsMetrics{refreshMetrics: awsMetrics.refreshMetrics}) + opts.Metrics = &ecsMetrics{refreshMetrics: awsMetrics.refreshMetrics} + return NewECSDiscovery(c.ECSSDConfig, opts) case RoleLightsail: opts.Metrics = &lightsailMetrics{refreshMetrics: awsMetrics.refreshMetrics} return NewLightsailDiscovery(c.LightsailSDConfig, opts) diff --git a/discovery/aws/ecs.go b/discovery/aws/ecs.go index 286c002a71..3794ad178d 100644 --- a/discovery/aws/ecs.go +++ b/discovery/aws/ecs.go @@ -118,7 +118,7 @@ func (*ECSSDConfig) Name() string { return "ecs" } // NewDiscoverer returns a Discoverer for the EC2 Config. func (c *ECSSDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery.Discoverer, error) { - return NewECSDiscovery(c, opts.Logger, opts.Metrics) + return NewECSDiscovery(c, opts) } // UnmarshalYAML implements the yaml.Unmarshaler interface for the ECS Config. @@ -165,22 +165,22 @@ type ECSDiscovery struct { } // NewECSDiscovery returns a new ECSDiscovery which periodically refreshes its targets. -func NewECSDiscovery(conf *ECSSDConfig, logger *slog.Logger, metrics discovery.DiscovererMetrics) (*ECSDiscovery, error) { - m, ok := metrics.(*ecsMetrics) +func NewECSDiscovery(conf *ECSSDConfig, opts discovery.DiscovererOptions) (*ECSDiscovery, error) { + m, ok := opts.Metrics.(*ecsMetrics) if !ok { return nil, errors.New("invalid discovery metrics type") } - if logger == nil { - logger = promslog.NewNopLogger() + if opts.Logger == nil { + opts.Logger = promslog.NewNopLogger() } d := &ECSDiscovery{ - logger: logger, + logger: opts.Logger, cfg: conf, } d.Discovery = refresh.NewDiscovery( refresh.Options{ - Logger: logger, + Logger: opts.Logger, Mech: "ecs", Interval: time.Duration(d.cfg.RefreshInterval), RefreshF: d.refresh, From be4efd740c5833bb34d03b8b2f2e549f236fb71c Mon Sep 17 00:00:00 2001 From: beorn7 Date: Thu, 13 Nov 2025 16:07:21 +0100 Subject: [PATCH 053/439] cmd: Make feature flag `native-histograms` a no-op. Signed-off-by: beorn7 --- cmd/prometheus/main.go | 8 ++------ docs/command-line/prometheus.md | 2 +- docs/feature_flags.md | 14 -------------- docs/migration.md | 19 ++++++++++--------- 4 files changed, 13 insertions(+), 30 deletions(-) diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go index 75b268322a..6ea65c879a 100644 --- a/cmd/prometheus/main.go +++ b/cmd/prometheus/main.go @@ -255,11 +255,7 @@ func (c *flagConfig) setFeatureListOptions(logger *slog.Logger) error { parser.ExperimentalDurationExpr = true logger.Info("Experimental duration expression parsing enabled.") case "native-histograms": - // Change relevant global variables. Hacky, but it's hard to pass a new option or default to unmarshallers. - t := true - config.DefaultConfig.GlobalConfig.ScrapeNativeHistograms = &t - config.DefaultGlobalConfig.ScrapeNativeHistograms = &t - logger.Warn("This option for --enable-feature is being phased out. It currently changes the default for the scrape_native_histograms scrape config setting to true, but will become a no-op in v3.9+. Stop using this option and set scrape_native_histograms in the scrape config instead.", "option", o) + logger.Warn("This option for --enable-feature is a no-op. To scrape native histograms, set the scrape_native_histograms scrape config setting to true.", "option", o) case "ooo-native-histograms": logger.Warn("This option for --enable-feature is now permanently enabled and therefore a no-op.", "option", o) case "created-timestamp-zero-ingestion": @@ -564,7 +560,7 @@ func main() { a.Flag("scrape.discovery-reload-interval", "Interval used by scrape manager to throttle target groups updates."). Hidden().Default("5s").SetValue(&cfg.scrape.DiscoveryReloadInterval) - a.Flag("enable-feature", "Comma separated feature names to enable. Valid options: exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-per-step-stats, promql-experimental-functions, extra-scrape-metrics, auto-gomaxprocs, native-histograms, created-timestamp-zero-ingestion, concurrent-rule-eval, delayed-compaction, old-ui, otlp-deltatocumulative, promql-duration-expr, use-uncached-io, promql-extended-range-selectors. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details."). + a.Flag("enable-feature", "Comma separated feature names to enable. Valid options: exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-per-step-stats, promql-experimental-functions, extra-scrape-metrics, auto-gomaxprocs, created-timestamp-zero-ingestion, concurrent-rule-eval, delayed-compaction, old-ui, otlp-deltatocumulative, promql-duration-expr, use-uncached-io, promql-extended-range-selectors. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details."). Default("").StringsVar(&cfg.featureList) a.Flag("agent", "Run Prometheus in 'Agent mode'.").BoolVar(&agentMode) diff --git a/docs/command-line/prometheus.md b/docs/command-line/prometheus.md index 0396f90bee..c79dad40a2 100644 --- a/docs/command-line/prometheus.md +++ b/docs/command-line/prometheus.md @@ -58,7 +58,7 @@ The Prometheus monitoring server | --query.timeout | Maximum time a query may take before being aborted. Use with server mode only. | `2m` | | --query.max-concurrency | Maximum number of queries executed concurrently. Use with server mode only. | `20` | | --query.max-samples | Maximum number of samples a single query can load into memory. Note that queries will fail if they try to load more samples than this into memory, so this also limits the number of samples a query can return. Use with server mode only. | `50000000` | -| --enable-feature ... | Comma separated feature names to enable. Valid options: exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-per-step-stats, promql-experimental-functions, extra-scrape-metrics, auto-gomaxprocs, native-histograms, created-timestamp-zero-ingestion, concurrent-rule-eval, delayed-compaction, old-ui, otlp-deltatocumulative, promql-duration-expr, use-uncached-io, promql-extended-range-selectors. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details. | | +| --enable-feature ... | Comma separated feature names to enable. Valid options: exemplar-storage, expand-external-labels, memory-snapshot-on-shutdown, promql-per-step-stats, promql-experimental-functions, extra-scrape-metrics, auto-gomaxprocs, created-timestamp-zero-ingestion, concurrent-rule-eval, delayed-compaction, old-ui, otlp-deltatocumulative, promql-duration-expr, use-uncached-io, promql-extended-range-selectors. See https://prometheus.io/docs/prometheus/latest/feature_flags/ for more details. | | | --agent | Run Prometheus in 'Agent mode'. | | | --log.level | Only log messages with the given severity or above. One of: [debug, info, warn, error] | `info` | | --log.format | Output format of log messages. One of: [logfmt, json] | `logfmt` | diff --git a/docs/feature_flags.md b/docs/feature_flags.md index f9f27bfb7f..0051859d66 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -45,20 +45,6 @@ statistics. Currently this is limited to totalQueryableSamples. When disabled in either the engine or the query, per-step statistics are not computed at all. -## Native Histograms - -`--enable-feature=native-histograms` - -_This feature flag is being phased out. You should not use it anymore._ - -Native histograms are a stable feature by now. However, to scrape native -histograms, a scrape config setting `scrape_native_histograms` is required. To -ease the transition, this feature flag sets the default value of -`scrape_native_histograms` to `true`. From v3.9 on, this feature flag will be a -true no-op, and the default value of `scrape_native_histograms` will be always -`false`. If you are still using this feature flag while running v3.8, update -your scrape configs and stop using the feature flag before upgrading to v3.9. - ## Experimental PromQL functions `--enable-feature=promql-experimental-functions` diff --git a/docs/migration.md b/docs/migration.md index 78db5b7d0f..6a08373f5c 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -43,18 +43,19 @@ This document offers guidance on migrating from Prometheus 2.x to Prometheus 3.0 Prometheus v3 will log a warning if you continue to pass these to `--enable-feature`. -- Starting from Prometheus version v3.8, the feature flag `native-histograms` is - deprecated. Use the new `scrape_native_histograms` global and per-scrape - configuration option instead. +- Starting from v3.9, the feature flag `native-histograms` is a no-op. Native + histograms are a stable feature now, but scraping them has to be enabled via + the `scrape_native_histograms` global or per-scrape configuration option + (added in v3.8). ## Configuration -- The scrape job level configuration option `scrape_classic_histograms` has been - renamed to `always_scrape_classic_histograms`. If you use the - `--enable-feature=native-histograms` feature flag to ingest native histograms - and you also want to ingest classic histograms that an endpoint might expose - along with native histograms, be sure to add this configuration or change your - configuration from the old name. +- The scrape job level configuration option `scrape_classic_histograms` has + been renamed to `always_scrape_classic_histograms`. If you use the + `scrape_native_histograms` scrape configuration option to ingest native + histograms and you also want to ingest classic histograms that an endpoint + might expose along with native histograms, be sure to add this configuration + or change your configuration from the old name. - The `http_config.enable_http2` in `remote_write` items default has been changed to `false`. In Prometheus v2 the remote write http client would default to use http2. In order to parallelize multiple remote write queues From d99f8dacc4c13368583759ea260327af30c50f62 Mon Sep 17 00:00:00 2001 From: Laurent Dufresne Date: Mon, 17 Nov 2025 10:37:55 +0100 Subject: [PATCH 054/439] chore: remove dead code (#17542) Signed-off-by: Laurent Dufresne --- .../prometheusremotewrite/metrics_to_prw.go | 8 -------- 1 file changed, 8 deletions(-) diff --git a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go index 5e575e6174..f43e4964b1 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go @@ -140,14 +140,6 @@ func (c *PrometheusConverter) FromMetrics(ctx context.Context, md pmetric.Metric c.seenTargetInfo = make(map[targetInfoKey]struct{}) resourceMetricsSlice := md.ResourceMetrics() - numMetrics := 0 - for i := 0; i < resourceMetricsSlice.Len(); i++ { - scopeMetricsSlice := resourceMetricsSlice.At(i).ScopeMetrics() - for j := 0; j < scopeMetricsSlice.Len(); j++ { - numMetrics += scopeMetricsSlice.At(j).Metrics().Len() - } - } - for i := 0; i < resourceMetricsSlice.Len(); i++ { resourceMetrics := resourceMetricsSlice.At(i) resource := resourceMetrics.Resource() From cefefc689766827a8c933e3181e9dd548656e71a Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Mon, 17 Nov 2025 15:59:40 +0100 Subject: [PATCH 055/439] prw2: Move Remote Write 2.0 CT to be per Sample; Rename to ST (start timestamp) (#17411) Relates to https://github.com/prometheus/prometheus/issues/16944#issuecomment-3164760343 Signed-off-by: bwplotka --- prompb/io/prometheus/write/v2/custom.go | 5 - prompb/io/prometheus/write/v2/types.pb.go | 302 +++++++++++------- prompb/io/prometheus/write/v2/types.proto | 74 +++-- storage/interface.go | 2 +- storage/remote/codec_test.go | 6 +- .../prometheusremotewrite/histograms.go | 4 +- storage/remote/write_handler.go | 41 ++- storage/remote/write_handler_test.go | 20 +- 8 files changed, 260 insertions(+), 194 deletions(-) diff --git a/prompb/io/prometheus/write/v2/custom.go b/prompb/io/prometheus/write/v2/custom.go index 3aa778eb60..5721aec532 100644 --- a/prompb/io/prometheus/write/v2/custom.go +++ b/prompb/io/prometheus/write/v2/custom.go @@ -80,11 +80,6 @@ func (m *TimeSeries) OptimizedMarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } - if m.CreatedTimestamp != 0 { - i = encodeVarintTypes(dAtA, i, uint64(m.CreatedTimestamp)) - i-- - dAtA[i] = 0x30 - } { size, err := m.Metadata.MarshalToSizedBuffer(dAtA[:i]) if err != nil { diff --git a/prompb/io/prometheus/write/v2/types.pb.go b/prompb/io/prometheus/write/v2/types.pb.go index 1419de217e..a726efb5b5 100644 --- a/prompb/io/prometheus/write/v2/types.pb.go +++ b/prompb/io/prometheus/write/v2/types.pb.go @@ -106,6 +106,8 @@ func (Histogram_ResetHint) EnumDescriptor() ([]byte, []int) { // The canonical Content-Type request header value for this message is // "application/x-protobuf;proto=io.prometheus.write.v2.Request" // +// Version: v2.0-rc.4 +// // NOTE: gogoproto options might change in future for this file, they // are not part of the spec proto (they only modify the generated Go code, not // the serialized message). See: https://github.com/prometheus/prometheus/issues/11908 @@ -181,7 +183,7 @@ type TimeSeries struct { // // Note that there might be multiple TimeSeries objects in the same // Requests with the same labels e.g. for different exemplars, metadata - // or created timestamp. + // or start timestamp. LabelsRefs []uint32 `protobuf:"varint,1,rep,packed,name=labels_refs,json=labelsRefs,proto3" json:"labels_refs,omitempty"` // Timeseries messages can either specify samples or (native) histogram samples // (histogram field), but not both. For a typical sender (real-time metric @@ -193,24 +195,7 @@ type TimeSeries struct { // exemplars represents an optional set of exemplars attached to this series' samples. Exemplars []Exemplar `protobuf:"bytes,4,rep,name=exemplars,proto3" json:"exemplars"` // metadata represents the metadata associated with the given series' samples. - Metadata Metadata `protobuf:"bytes,5,opt,name=metadata,proto3" json:"metadata"` - // created_timestamp represents an optional created timestamp associated with - // this series' samples in ms format, typically for counter or histogram type - // metrics. Created timestamp represents the time when the counter started - // counting (sometimes referred to as start timestamp), which can increase - // the accuracy of query results. - // - // Note that some receivers might require this and in return fail to - // ingest such samples within the Request. - // - // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go - // for conversion from/to time.Time to Prometheus timestamp. - // - // Note that the "optional" keyword is omitted due to - // https://cloud.google.com/apis/design/design_patterns.md#optional_primitive_fields - // Zero value means value not set. If you need to use exactly zero value for - // the timestamp, use 1 millisecond before or after. - CreatedTimestamp int64 `protobuf:"varint,6,opt,name=created_timestamp,json=createdTimestamp,proto3" json:"created_timestamp,omitempty"` + Metadata Metadata `protobuf:"bytes,5,opt,name=metadata,proto3" json:"metadata"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` @@ -284,13 +269,6 @@ func (m *TimeSeries) GetMetadata() Metadata { return Metadata{} } -func (m *TimeSeries) GetCreatedTimestamp() int64 { - if m != nil { - return m.CreatedTimestamp - } - return 0 -} - // Exemplar is an additional information attached to some series' samples. // It is typically used to attach an example trace or request ID associated with // the metric changes. @@ -375,7 +353,27 @@ type Sample struct { // // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go // for conversion from/to time.Time to Prometheus timestamp. - Timestamp int64 `protobuf:"varint,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"` + Timestamp int64 `protobuf:"varint,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"` + // start_timestamp represents an optional start timestamp for the sample, + // in ms format. This information is typically used for counter, histogram (cumulative) + // or delta type metrics. + // + // For cumulative metrics, the start timestamp represents the time when the + // counter started counting (sometimes referred to as start timestamp), which + // can increase the accuracy of certain processing and query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + StartTimestamp int64 `protobuf:"varint,3,opt,name=start_timestamp,json=startTimestamp,proto3" json:"start_timestamp,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` @@ -428,6 +426,13 @@ func (m *Sample) GetTimestamp() int64 { return 0 } +func (m *Sample) GetStartTimestamp() int64 { + if m != nil { + return m.StartTimestamp + } + return 0 +} + // Metadata represents the metadata associated with the given series' samples. type Metadata struct { Type Metadata_MetricType `protobuf:"varint,1,opt,name=type,proto3,enum=io.prometheus.write.v2.Metadata_MetricType" json:"type,omitempty"` @@ -498,12 +503,11 @@ func (m *Metadata) GetUnitRef() uint32 { return 0 } -// A native histogram, also known as a sparse histogram. -// Original design doc: -// https://docs.google.com/document/d/1cLNv3aufPZb3fNfaJgdaRBZsInZKKIHo9E6HinJVbpM/edit -// The appendix of this design doc also explains the concept of float -// histograms. This Histogram message can represent both, the usual -// integer histogram as well as a float histogram. +// A native histogram message, supporting +// * sparse exponential bucketing, custom bucketing. +// * float or integer histograms. +// +// See the full spec: https://prometheus.io/docs/specs/native_histograms/ type Histogram struct { // Types that are valid to be assigned to Count: // @@ -581,10 +585,27 @@ type Histogram struct { // // The last element is not only the upper inclusive bound of the last regular // bucket, but implicitly the lower exclusive bound of the +Inf bucket. - CustomValues []float64 `protobuf:"fixed64,16,rep,packed,name=custom_values,json=customValues,proto3" json:"custom_values,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + CustomValues []float64 `protobuf:"fixed64,16,rep,packed,name=custom_values,json=customValues,proto3" json:"custom_values,omitempty"` + // start_timestamp represents an optional start timestamp for the histogram sample, + // in ms format. The start timestamp represents the time when the histogram + // started counting, which can increase the accuracy of certain processing and + // query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + StartTimestamp int64 `protobuf:"varint,17,opt,name=start_timestamp,json=startTimestamp,proto3" json:"start_timestamp,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *Histogram) Reset() { *m = Histogram{} } @@ -774,6 +795,13 @@ func (m *Histogram) GetCustomValues() []float64 { return nil } +func (m *Histogram) GetStartTimestamp() int64 { + if m != nil { + return m.StartTimestamp + } + return 0 +} + // XXX_OneofWrappers is for the internal use of the proto package. func (*Histogram) XXX_OneofWrappers() []interface{} { return []interface{}{ @@ -861,65 +889,66 @@ func init() { } var fileDescriptor_f139519efd9fa8d7 = []byte{ - // 926 bytes of a gzipped FileDescriptorProto + // 931 bytes of a gzipped FileDescriptorProto 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x94, 0x55, 0x5d, 0x6f, 0xe3, 0x44, - 0x14, 0xed, 0xc4, 0x69, 0x3e, 0x6e, 0x9a, 0xac, 0x33, 0xb4, 0x5d, 0x6f, 0x81, 0x6c, 0xd6, 0x08, - 0x88, 0x58, 0x29, 0x91, 0xc2, 0xeb, 0x0a, 0xd4, 0xb4, 0x6e, 0x93, 0x95, 0x92, 0xac, 0x26, 0x2e, - 0x52, 0x79, 0xb1, 0xdc, 0x64, 0x92, 0x58, 0xd8, 0xb1, 0xf1, 0x4c, 0x02, 0xe5, 0xf7, 0xf1, 0xb0, - 0x8f, 0xfc, 0x01, 0x10, 0xf4, 0x9d, 0xff, 0x80, 0x66, 0xfc, 0xd9, 0x42, 0xbb, 0xe2, 0x6d, 0xe6, - 0xdc, 0x73, 0xee, 0x3d, 0xb9, 0xbe, 0x77, 0x02, 0xba, 0xe3, 0xf7, 0x82, 0xd0, 0xf7, 0x28, 0x5f, - 0xd3, 0x2d, 0xeb, 0xfd, 0x14, 0x3a, 0x9c, 0xf6, 0x76, 0xfd, 0x1e, 0xbf, 0x0d, 0x28, 0xeb, 0x06, - 0xa1, 0xcf, 0x7d, 0x7c, 0xec, 0xf8, 0xdd, 0x8c, 0xd3, 0x95, 0x9c, 0xee, 0xae, 0x7f, 0x72, 0xb8, - 0xf2, 0x57, 0xbe, 0xa4, 0xf4, 0xc4, 0x29, 0x62, 0xeb, 0x0c, 0xca, 0x84, 0xfe, 0xb8, 0xa5, 0x8c, - 0x63, 0x0d, 0xca, 0xec, 0xd6, 0xbb, 0xf1, 0x5d, 0xa6, 0x15, 0xdb, 0x4a, 0xa7, 0x4a, 0x92, 0x2b, - 0x1e, 0x02, 0x70, 0xc7, 0xa3, 0x8c, 0x86, 0x0e, 0x65, 0xda, 0x7e, 0x5b, 0xe9, 0xd4, 0xfa, 0x7a, - 0xf7, 0xbf, 0xeb, 0x74, 0x4d, 0xc7, 0xa3, 0x33, 0xc9, 0x1c, 0x14, 0xdf, 0xff, 0xf1, 0x72, 0x8f, - 0xe4, 0xb4, 0x6f, 0x8b, 0x15, 0xa4, 0x16, 0xf5, 0xbf, 0x0b, 0x00, 0x19, 0x0d, 0xbf, 0x84, 0x9a, - 0x6b, 0xdf, 0x50, 0x97, 0x59, 0x21, 0x5d, 0x32, 0x0d, 0xb5, 0x95, 0x4e, 0x9d, 0x40, 0x04, 0x11, - 0xba, 0x64, 0xf8, 0x1b, 0x28, 0x33, 0xdb, 0x0b, 0x5c, 0xca, 0xb4, 0x82, 0x2c, 0xde, 0x7a, 0xac, - 0xf8, 0x4c, 0xd2, 0xe2, 0xc2, 0x89, 0x08, 0x5f, 0x02, 0xac, 0x1d, 0xc6, 0xfd, 0x55, 0x68, 0x7b, - 0x4c, 0x53, 0x64, 0x8a, 0x57, 0x8f, 0xa5, 0x18, 0x26, 0xcc, 0xc4, 0x7e, 0x26, 0xc5, 0xe7, 0x50, - 0xa5, 0x3f, 0x53, 0x2f, 0x70, 0xed, 0x30, 0x6a, 0x52, 0xad, 0xdf, 0x7e, 0x2c, 0x8f, 0x11, 0x13, - 0xe3, 0x34, 0x99, 0x10, 0x0f, 0xa0, 0xe2, 0x51, 0x6e, 0x2f, 0x6c, 0x6e, 0x6b, 0xfb, 0x6d, 0xf4, - 0x54, 0x92, 0x71, 0xcc, 0x8b, 0x93, 0xa4, 0x3a, 0xfc, 0x1a, 0x9a, 0xf3, 0x90, 0xda, 0x9c, 0x2e, - 0x2c, 0xd9, 0x5e, 0x6e, 0x7b, 0x81, 0x56, 0x6a, 0xa3, 0x8e, 0x42, 0xd4, 0x38, 0x60, 0x26, 0xb8, - 0x6e, 0x41, 0x25, 0x71, 0xf3, 0xe1, 0x66, 0x1f, 0xc2, 0xfe, 0xce, 0x76, 0xb7, 0x54, 0x2b, 0xb4, - 0x51, 0x07, 0x91, 0xe8, 0x82, 0x3f, 0x81, 0x6a, 0x56, 0x47, 0x91, 0x75, 0x32, 0x40, 0x7f, 0x03, - 0xa5, 0xa8, 0xf3, 0x99, 0x1a, 0x3d, 0xaa, 0x2e, 0x3c, 0x54, 0xff, 0x55, 0x80, 0x4a, 0xf2, 0x43, - 0xf1, 0xb7, 0x50, 0x14, 0xd3, 0x2c, 0xf5, 0x8d, 0xfe, 0xeb, 0x0f, 0x35, 0x46, 0x1c, 0x42, 0x67, - 0x6e, 0xde, 0x06, 0x94, 0x48, 0x21, 0x7e, 0x01, 0x95, 0x35, 0x75, 0x03, 0xf1, 0xf3, 0xa4, 0xd1, - 0x3a, 0x29, 0x8b, 0x3b, 0xa1, 0x4b, 0x11, 0xda, 0x6e, 0x1c, 0x2e, 0x43, 0xc5, 0x28, 0x24, 0xee, - 0x84, 0x2e, 0xf5, 0xdf, 0x11, 0x40, 0x96, 0x0a, 0x7f, 0x0c, 0xcf, 0xc7, 0x86, 0x49, 0x46, 0x67, - 0x96, 0x79, 0xfd, 0xce, 0xb0, 0xae, 0x26, 0xb3, 0x77, 0xc6, 0xd9, 0xe8, 0x62, 0x64, 0x9c, 0xab, - 0x7b, 0xf8, 0x39, 0x7c, 0x94, 0x0f, 0x9e, 0x4d, 0xaf, 0x26, 0xa6, 0x41, 0x54, 0x84, 0x8f, 0xa0, - 0x99, 0x0f, 0x5c, 0x9e, 0x5e, 0x5d, 0x1a, 0x6a, 0x01, 0xbf, 0x80, 0xa3, 0x3c, 0x3c, 0x1c, 0xcd, - 0xcc, 0xe9, 0x25, 0x39, 0x1d, 0xab, 0x0a, 0x6e, 0xc1, 0xc9, 0xbf, 0x14, 0x59, 0xbc, 0xf8, 0xb0, - 0xd4, 0xec, 0x6a, 0x3c, 0x3e, 0x25, 0xd7, 0xea, 0x3e, 0x3e, 0x04, 0x35, 0x1f, 0x18, 0x4d, 0x2e, - 0xa6, 0x6a, 0x09, 0x6b, 0x70, 0x78, 0x8f, 0x6e, 0x9e, 0x9a, 0xc6, 0xcc, 0x30, 0xd5, 0xb2, 0xfe, - 0x6b, 0x09, 0xaa, 0xe9, 0x64, 0xe3, 0x4f, 0xa1, 0x3a, 0xf7, 0xb7, 0x1b, 0x6e, 0x39, 0x1b, 0x2e, - 0x3b, 0x5d, 0x1c, 0xee, 0x91, 0x8a, 0x84, 0x46, 0x1b, 0x8e, 0x5f, 0x41, 0x2d, 0x0a, 0x2f, 0x5d, - 0xdf, 0xe6, 0xd1, 0x20, 0x0c, 0xf7, 0x08, 0x48, 0xf0, 0x42, 0x60, 0x58, 0x05, 0x85, 0x6d, 0x3d, - 0xd9, 0x60, 0x44, 0xc4, 0x11, 0x1f, 0x43, 0x89, 0xcd, 0xd7, 0xd4, 0xb3, 0x65, 0x6b, 0x9b, 0x24, - 0xbe, 0xe1, 0xcf, 0xa1, 0xf1, 0x0b, 0x0d, 0x7d, 0x8b, 0xaf, 0x43, 0xca, 0xd6, 0xbe, 0xbb, 0x90, - 0x33, 0x8f, 0x48, 0x5d, 0xa0, 0x66, 0x02, 0xe2, 0x2f, 0x62, 0x5a, 0xe6, 0xab, 0x24, 0x7d, 0x21, - 0x72, 0x20, 0xf0, 0xb3, 0xc4, 0xdb, 0x57, 0xa0, 0xe6, 0x78, 0x91, 0xc1, 0xb2, 0x34, 0x88, 0x48, - 0x23, 0x65, 0x46, 0x26, 0xa7, 0xd0, 0xd8, 0xd0, 0x95, 0xcd, 0x9d, 0x1d, 0xb5, 0x58, 0x60, 0x6f, - 0x98, 0x56, 0x79, 0xfa, 0xed, 0x1a, 0x6c, 0xe7, 0x3f, 0x50, 0x3e, 0x0b, 0xec, 0x4d, 0xbc, 0x70, - 0xf5, 0x44, 0x2f, 0x30, 0x86, 0xbf, 0x84, 0x67, 0x69, 0xc2, 0x05, 0x75, 0xb9, 0xcd, 0xb4, 0x6a, - 0x5b, 0xe9, 0x60, 0x92, 0xd6, 0x39, 0x97, 0xe8, 0x3d, 0xa2, 0x74, 0xca, 0x34, 0x68, 0x2b, 0x1d, - 0x94, 0x11, 0xa5, 0x4d, 0x26, 0x2c, 0x06, 0x3e, 0x73, 0x72, 0x16, 0x6b, 0xff, 0xd7, 0x62, 0xa2, - 0x4f, 0x2d, 0xa6, 0x09, 0x63, 0x8b, 0x07, 0x91, 0xc5, 0x04, 0xce, 0x2c, 0xa6, 0xc4, 0xd8, 0x62, - 0x3d, 0xb2, 0x98, 0xc0, 0xb1, 0xc5, 0xb7, 0x00, 0x21, 0x65, 0x94, 0x5b, 0x6b, 0xf1, 0x55, 0x1a, - 0x4f, 0xef, 0x65, 0x3a, 0x63, 0x5d, 0x22, 0x34, 0x43, 0x67, 0xc3, 0x49, 0x35, 0x4c, 0x8e, 0xf7, - 0x1f, 0x82, 0x67, 0x0f, 0x1e, 0x02, 0xfc, 0x19, 0xd4, 0xe7, 0x5b, 0xc6, 0x7d, 0xcf, 0x92, 0xcf, - 0x06, 0xd3, 0x54, 0x69, 0xe8, 0x20, 0x02, 0xbf, 0x93, 0x98, 0xbe, 0x80, 0x6a, 0x9a, 0x1a, 0x9f, - 0xc0, 0x31, 0x11, 0x13, 0x6e, 0x0d, 0x47, 0x13, 0xf3, 0xc1, 0x9a, 0x62, 0x68, 0xe4, 0x62, 0xd7, - 0xc6, 0x4c, 0x45, 0xb8, 0x09, 0xf5, 0x1c, 0x36, 0x99, 0xaa, 0x05, 0xb1, 0x49, 0x39, 0x28, 0xda, - 0x59, 0x65, 0x50, 0x86, 0x7d, 0xd9, 0x94, 0xc1, 0x01, 0x40, 0x36, 0x6f, 0xfa, 0x1b, 0x80, 0xec, - 0x03, 0x88, 0x91, 0xf7, 0x97, 0x4b, 0x46, 0xa3, 0x1d, 0x6a, 0x92, 0xf8, 0x26, 0x70, 0x97, 0x6e, - 0x56, 0x7c, 0x2d, 0x57, 0xa7, 0x4e, 0xe2, 0xdb, 0xe0, 0xe8, 0xfd, 0x5d, 0x0b, 0xfd, 0x76, 0xd7, - 0x42, 0x7f, 0xde, 0xb5, 0xd0, 0xf7, 0x65, 0xd9, 0xb4, 0x5d, 0xff, 0xa6, 0x24, 0xff, 0x8a, 0xbf, - 0xfe, 0x27, 0x00, 0x00, 0xff, 0xff, 0x3e, 0xfc, 0x93, 0x1c, 0xde, 0x07, 0x00, 0x00, + 0x14, 0xed, 0xc4, 0xf9, 0xbc, 0x69, 0xb2, 0xce, 0xd0, 0x76, 0xbd, 0x05, 0xb2, 0xd9, 0x20, 0x20, + 0x02, 0x29, 0x91, 0xc2, 0x2b, 0x02, 0x35, 0xad, 0xdb, 0xa4, 0x52, 0x92, 0xd5, 0xc4, 0x45, 0x2a, + 0x2f, 0x96, 0x9b, 0x4e, 0x12, 0x0b, 0x3b, 0x36, 0x9e, 0x49, 0xa0, 0xfc, 0x40, 0xb4, 0x8f, 0xfc, + 0x01, 0x10, 0xf4, 0x9d, 0xff, 0x80, 0x66, 0xfc, 0xd9, 0xd0, 0x76, 0xb5, 0x6f, 0x33, 0xe7, 0x9e, + 0x73, 0xef, 0xc9, 0xf5, 0xbd, 0x13, 0x68, 0xdb, 0x5e, 0xcf, 0x0f, 0x3c, 0x97, 0xf2, 0x15, 0xdd, + 0xb0, 0xde, 0x2f, 0x81, 0xcd, 0x69, 0x6f, 0xdb, 0xef, 0xf1, 0x3b, 0x9f, 0xb2, 0xae, 0x1f, 0x78, + 0xdc, 0xc3, 0x47, 0xb6, 0xd7, 0x4d, 0x39, 0x5d, 0xc9, 0xe9, 0x6e, 0xfb, 0xc7, 0x07, 0x4b, 0x6f, + 0xe9, 0x49, 0x4a, 0x4f, 0x9c, 0x42, 0x76, 0x9b, 0x41, 0x89, 0xd0, 0x9f, 0x37, 0x94, 0x71, 0xac, + 0x41, 0x89, 0xdd, 0xb9, 0x37, 0x9e, 0xc3, 0xb4, 0x7c, 0x4b, 0xe9, 0x54, 0x48, 0x7c, 0xc5, 0x43, + 0x00, 0x6e, 0xbb, 0x94, 0xd1, 0xc0, 0xa6, 0x4c, 0x2b, 0xb4, 0x94, 0x4e, 0xb5, 0xdf, 0xee, 0x3e, + 0x5e, 0xa7, 0x6b, 0xd8, 0x2e, 0x9d, 0x49, 0xe6, 0x20, 0xff, 0xee, 0xaf, 0xd7, 0x7b, 0x24, 0xa3, + 0xbd, 0xcc, 0x97, 0x91, 0x9a, 0x6f, 0xff, 0x9e, 0x03, 0x48, 0x69, 0xf8, 0x35, 0x54, 0x1d, 0xeb, + 0x86, 0x3a, 0xcc, 0x0c, 0xe8, 0x82, 0x69, 0xa8, 0xa5, 0x74, 0x6a, 0x04, 0x42, 0x88, 0xd0, 0x05, + 0xc3, 0xdf, 0x41, 0x89, 0x59, 0xae, 0xef, 0x50, 0xa6, 0xe5, 0x64, 0xf1, 0xe6, 0x53, 0xc5, 0x67, + 0x92, 0x16, 0x15, 0x8e, 0x45, 0xf8, 0x02, 0x60, 0x65, 0x33, 0xee, 0x2d, 0x03, 0xcb, 0x65, 0x9a, + 0x22, 0x53, 0xbc, 0x79, 0x2a, 0xc5, 0x30, 0x66, 0xc6, 0xf6, 0x53, 0x29, 0x3e, 0x83, 0x0a, 0xfd, + 0x95, 0xba, 0xbe, 0x63, 0x05, 0x61, 0x93, 0xaa, 0xfd, 0xd6, 0x53, 0x79, 0xf4, 0x88, 0x18, 0xa5, + 0x49, 0x85, 0x78, 0x00, 0x65, 0x97, 0x72, 0xeb, 0xd6, 0xe2, 0x96, 0x56, 0x68, 0xa1, 0xe7, 0x92, + 0x8c, 0x23, 0x5e, 0x94, 0x24, 0xd1, 0x5d, 0xe6, 0xcb, 0x45, 0xb5, 0xd4, 0x36, 0xa1, 0x1c, 0x97, + 0x79, 0x7f, 0x17, 0x0f, 0xa0, 0xb0, 0xb5, 0x9c, 0x0d, 0xd5, 0x72, 0x2d, 0xd4, 0x41, 0x24, 0xbc, + 0xe0, 0x4f, 0xa0, 0x22, 0xbf, 0x0f, 0xb7, 0x5c, 0x5f, 0x53, 0x5a, 0xa8, 0xa3, 0x90, 0x14, 0x68, + 0x53, 0x28, 0x86, 0x2d, 0x4d, 0xd5, 0xe8, 0x49, 0x75, 0x6e, 0x47, 0x8d, 0xbf, 0x84, 0x17, 0x8c, + 0x5b, 0x01, 0x37, 0x77, 0x2b, 0xd4, 0x25, 0x6c, 0x24, 0x65, 0xfe, 0xc9, 0x41, 0x39, 0xfe, 0xa9, + 0xf8, 0x7b, 0xc8, 0x8b, 0x79, 0x96, 0x85, 0xea, 0xfd, 0xaf, 0xdf, 0xd7, 0x1a, 0x71, 0x08, 0xec, + 0xb9, 0x71, 0xe7, 0x53, 0x22, 0x85, 0xf8, 0x15, 0x94, 0x57, 0xd4, 0xf1, 0x45, 0x1f, 0x64, 0xbd, + 0x1a, 0x29, 0x89, 0x3b, 0xa1, 0x0b, 0x11, 0xda, 0xac, 0x6d, 0x2e, 0x43, 0xf9, 0x30, 0x24, 0xee, + 0x84, 0x2e, 0xda, 0x7f, 0x22, 0x80, 0x34, 0x15, 0xfe, 0x18, 0x5e, 0x8e, 0x75, 0x83, 0x8c, 0x4e, + 0x4d, 0xe3, 0xfa, 0xad, 0x6e, 0x5e, 0x4d, 0x66, 0x6f, 0xf5, 0xd3, 0xd1, 0xf9, 0x48, 0x3f, 0x53, + 0xf7, 0xf0, 0x4b, 0xf8, 0x28, 0x1b, 0x3c, 0x9d, 0x5e, 0x4d, 0x0c, 0x9d, 0xa8, 0x08, 0x1f, 0x42, + 0x23, 0x1b, 0xb8, 0x38, 0xb9, 0xba, 0xd0, 0xd5, 0x1c, 0x7e, 0x05, 0x87, 0x59, 0x78, 0x38, 0x9a, + 0x19, 0xd3, 0x0b, 0x72, 0x32, 0x56, 0x15, 0xdc, 0x84, 0xe3, 0xff, 0x29, 0xd2, 0x78, 0x7e, 0xb7, + 0xd4, 0xec, 0x6a, 0x3c, 0x3e, 0x21, 0xd7, 0x6a, 0x01, 0x1f, 0x80, 0x9a, 0x0d, 0x8c, 0x26, 0xe7, + 0x53, 0xb5, 0x88, 0x35, 0x38, 0x78, 0x40, 0x37, 0x4e, 0x0c, 0x7d, 0xa6, 0x1b, 0x6a, 0xa9, 0xfd, + 0x6f, 0x11, 0x2a, 0xc9, 0x6c, 0xe3, 0x4f, 0xa1, 0x32, 0xf7, 0x36, 0x6b, 0x6e, 0xda, 0x6b, 0x2e, + 0x3b, 0x9d, 0x1f, 0xee, 0x91, 0xb2, 0x84, 0x46, 0x6b, 0x8e, 0xdf, 0x40, 0x35, 0x0c, 0x2f, 0x1c, + 0xcf, 0xe2, 0xe1, 0xc4, 0x0c, 0xf7, 0x08, 0x48, 0xf0, 0x5c, 0x60, 0x58, 0x05, 0x85, 0x6d, 0x5c, + 0xd9, 0x60, 0x44, 0xc4, 0x11, 0x1f, 0x41, 0x91, 0xcd, 0x57, 0xd4, 0xb5, 0x64, 0x6b, 0x1b, 0x24, + 0xba, 0xe1, 0xcf, 0xa1, 0xfe, 0x1b, 0x0d, 0x3c, 0x93, 0xaf, 0x02, 0xca, 0x56, 0x9e, 0x73, 0x2b, + 0xa7, 0x1e, 0x91, 0x9a, 0x40, 0x8d, 0x18, 0xc4, 0x5f, 0x44, 0xb4, 0xd4, 0x57, 0x51, 0xfa, 0x42, + 0x64, 0x5f, 0xe0, 0xa7, 0xb1, 0xb7, 0xaf, 0x40, 0xcd, 0xf0, 0x42, 0x83, 0x25, 0x69, 0x10, 0x91, + 0x7a, 0xc2, 0x0c, 0x4d, 0x4e, 0xa1, 0xbe, 0xa6, 0x4b, 0x8b, 0xdb, 0x5b, 0x6a, 0x32, 0xdf, 0x5a, + 0x33, 0xad, 0xfc, 0xfc, 0xeb, 0x35, 0xd8, 0xcc, 0x7f, 0xa2, 0x7c, 0xe6, 0x5b, 0xeb, 0x68, 0xe5, + 0x6a, 0xb1, 0x5e, 0x60, 0x4c, 0x8c, 0x74, 0x92, 0xf0, 0x96, 0x3a, 0xdc, 0x62, 0x5a, 0xa5, 0xa5, + 0x74, 0x30, 0x49, 0xea, 0x9c, 0x49, 0xf4, 0x01, 0x51, 0x3a, 0x65, 0x1a, 0xb4, 0x94, 0x0e, 0x4a, + 0x89, 0xd2, 0x26, 0x13, 0x16, 0x7d, 0x8f, 0xd9, 0x19, 0x8b, 0xd5, 0x0f, 0xb5, 0x18, 0xeb, 0x13, + 0x8b, 0x49, 0xc2, 0xc8, 0xe2, 0x7e, 0x68, 0x31, 0x86, 0x53, 0x8b, 0x09, 0x31, 0xb2, 0x58, 0x0b, + 0x2d, 0xc6, 0x70, 0x64, 0xf1, 0x12, 0x20, 0xa0, 0x8c, 0x72, 0x73, 0x25, 0xbe, 0x4a, 0xfd, 0xf9, + 0xbd, 0x4c, 0x66, 0xac, 0x4b, 0x84, 0x66, 0x68, 0xaf, 0x39, 0xa9, 0x04, 0xf1, 0xf1, 0xe1, 0x8b, + 0xf1, 0x62, 0xf7, 0xc5, 0xf8, 0x0c, 0x6a, 0xf3, 0x0d, 0xe3, 0x9e, 0x6b, 0xca, 0xf7, 0x85, 0x69, + 0xaa, 0x34, 0xb4, 0x1f, 0x82, 0x3f, 0x48, 0xec, 0xb1, 0x67, 0xa5, 0xf1, 0xe8, 0xb3, 0x72, 0x0b, + 0x95, 0xc4, 0x03, 0x3e, 0x86, 0x23, 0x22, 0x56, 0xc1, 0x1c, 0x8e, 0x26, 0xc6, 0xce, 0x3e, 0x63, + 0xa8, 0x67, 0x62, 0xd7, 0xfa, 0x4c, 0x45, 0xb8, 0x01, 0xb5, 0x0c, 0x36, 0x99, 0xaa, 0x39, 0xb1, + 0x72, 0x19, 0x28, 0x5c, 0x6e, 0x65, 0x50, 0x82, 0x82, 0xec, 0xde, 0x60, 0x1f, 0x20, 0x1d, 0xcc, + 0xf6, 0xb7, 0x00, 0xe9, 0x97, 0x12, 0xbb, 0xe1, 0x2d, 0x16, 0x8c, 0x86, 0xcb, 0xd6, 0x20, 0xd1, + 0x4d, 0xe0, 0x0e, 0x5d, 0x2f, 0xf9, 0x4a, 0xee, 0x58, 0x8d, 0x44, 0xb7, 0xc1, 0xe1, 0xbb, 0xfb, + 0x26, 0xfa, 0xe3, 0xbe, 0x89, 0xfe, 0xbe, 0x6f, 0xa2, 0x1f, 0x4b, 0xb2, 0xbb, 0xdb, 0xfe, 0x4d, + 0x51, 0xfe, 0x6b, 0x7f, 0xf3, 0x5f, 0x00, 0x00, 0x00, 0xff, 0xff, 0x62, 0x8f, 0x36, 0x4b, 0x09, + 0x08, 0x00, 0x00, } func (m *Request) Marshal() (dAtA []byte, err error) { @@ -996,11 +1025,6 @@ func (m *TimeSeries) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } - if m.CreatedTimestamp != 0 { - i = encodeVarintTypes(dAtA, i, uint64(m.CreatedTimestamp)) - i-- - dAtA[i] = 0x30 - } { size, err := m.Metadata.MarshalToSizedBuffer(dAtA[:i]) if err != nil { @@ -1154,6 +1178,11 @@ func (m *Sample) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.StartTimestamp != 0 { + i = encodeVarintTypes(dAtA, i, uint64(m.StartTimestamp)) + i-- + dAtA[i] = 0x18 + } if m.Timestamp != 0 { i = encodeVarintTypes(dAtA, i, uint64(m.Timestamp)) i-- @@ -1234,6 +1263,13 @@ func (m *Histogram) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.StartTimestamp != 0 { + i = encodeVarintTypes(dAtA, i, uint64(m.StartTimestamp)) + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0x88 + } if len(m.CustomValues) > 0 { for iNdEx := len(m.CustomValues) - 1; iNdEx >= 0; iNdEx-- { f6 := math.Float64bits(float64(m.CustomValues[iNdEx])) @@ -1535,9 +1571,6 @@ func (m *TimeSeries) Size() (n int) { } l = m.Metadata.Size() n += 1 + l + sovTypes(uint64(l)) - if m.CreatedTimestamp != 0 { - n += 1 + sovTypes(uint64(m.CreatedTimestamp)) - } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -1581,6 +1614,9 @@ func (m *Sample) Size() (n int) { if m.Timestamp != 0 { n += 1 + sovTypes(uint64(m.Timestamp)) } + if m.StartTimestamp != 0 { + n += 1 + sovTypes(uint64(m.StartTimestamp)) + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -1670,6 +1706,9 @@ func (m *Histogram) Size() (n int) { if len(m.CustomValues) > 0 { n += 2 + sovTypes(uint64(len(m.CustomValues)*8)) + len(m.CustomValues)*8 } + if m.StartTimestamp != 0 { + n += 2 + sovTypes(uint64(m.StartTimestamp)) + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -2093,25 +2132,6 @@ func (m *TimeSeries) Unmarshal(dAtA []byte) error { return err } iNdEx = postIndex - case 6: - if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field CreatedTimestamp", wireType) - } - m.CreatedTimestamp = 0 - for shift := uint(0); ; shift += 7 { - if shift >= 64 { - return ErrIntOverflowTypes - } - if iNdEx >= l { - return io.ErrUnexpectedEOF - } - b := dAtA[iNdEx] - iNdEx++ - m.CreatedTimestamp |= int64(b&0x7F) << shift - if b < 0x80 { - break - } - } default: iNdEx = preIndex skippy, err := skipTypes(dAtA[iNdEx:]) @@ -2350,6 +2370,25 @@ func (m *Sample) Unmarshal(dAtA []byte) error { break } } + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field StartTimestamp", wireType) + } + m.StartTimestamp = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowTypes + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.StartTimestamp |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipTypes(dAtA[iNdEx:]) @@ -3038,6 +3077,25 @@ func (m *Histogram) Unmarshal(dAtA []byte) error { } else { return fmt.Errorf("proto: wrong wireType = %d for field CustomValues", wireType) } + case 17: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field StartTimestamp", wireType) + } + m.StartTimestamp = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowTypes + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.StartTimestamp |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipTypes(dAtA[iNdEx:]) diff --git a/prompb/io/prometheus/write/v2/types.proto b/prompb/io/prometheus/write/v2/types.proto index ff6c4936bb..c1ae04d206 100644 --- a/prompb/io/prometheus/write/v2/types.proto +++ b/prompb/io/prometheus/write/v2/types.proto @@ -14,6 +14,7 @@ // NOTE: This file is also available on https://buf.build/prometheus/prometheus/docs/main:io.prometheus.write.v2 syntax = "proto3"; + package io.prometheus.write.v2; option go_package = "writev2"; @@ -27,6 +28,8 @@ import "gogoproto/gogo.proto"; // The canonical Content-Type request header value for this message is // "application/x-protobuf;proto=io.prometheus.write.v2.Request" // +// Version: v2.0-rc.4 +// // NOTE: gogoproto options might change in future for this file, they // are not part of the spec proto (they only modify the generated Go code, not // the serialized message). See: https://github.com/prometheus/prometheus/issues/11908 @@ -59,7 +62,7 @@ message TimeSeries { // // Note that there might be multiple TimeSeries objects in the same // Requests with the same labels e.g. for different exemplars, metadata - // or created timestamp. + // or start timestamp. repeated uint32 labels_refs = 1; // Timeseries messages can either specify samples or (native) histogram samples @@ -76,23 +79,9 @@ message TimeSeries { // metadata represents the metadata associated with the given series' samples. Metadata metadata = 5 [(gogoproto.nullable) = false]; - // created_timestamp represents an optional created timestamp associated with - // this series' samples in ms format, typically for counter or histogram type - // metrics. Created timestamp represents the time when the counter started - // counting (sometimes referred to as start timestamp), which can increase - // the accuracy of query results. - // - // Note that some receivers might require this and in return fail to - // ingest such samples within the Request. - // - // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go - // for conversion from/to time.Time to Prometheus timestamp. - // - // Note that the "optional" keyword is omitted due to - // https://cloud.google.com/apis/design/design_patterns.md#optional_primitive_fields - // Zero value means value not set. If you need to use exactly zero value for - // the timestamp, use 1 millisecond before or after. - int64 created_timestamp = 6; + // This field is reserved for backward compatibility with the deprecated fields; + // previously present in the experimental remote write period. + reserved 6; } // Exemplar is an additional information attached to some series' samples. @@ -123,6 +112,26 @@ message Sample { // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go // for conversion from/to time.Time to Prometheus timestamp. int64 timestamp = 2; + // start_timestamp represents an optional start timestamp for the sample, + // in ms format. This information is typically used for counter, histogram (cumulative) + // or delta type metrics. + // + // For cumulative metrics, the start timestamp represents the time when the + // counter started counting (sometimes referred to as start timestamp), which + // can increase the accuracy of certain processing and query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + int64 start_timestamp = 3; } // Metadata represents the metadata associated with the given series' samples. @@ -148,12 +157,11 @@ message Metadata { uint32 unit_ref = 4; } -// A native histogram, also known as a sparse histogram. -// Original design doc: -// https://docs.google.com/document/d/1cLNv3aufPZb3fNfaJgdaRBZsInZKKIHo9E6HinJVbpM/edit -// The appendix of this design doc also explains the concept of float -// histograms. This Histogram message can represent both, the usual -// integer histogram as well as a float histogram. +// A native histogram message, supporting +// * sparse exponential bucketing, custom bucketing. +// * float or integer histograms. +// +// See the full spec: https://prometheus.io/docs/specs/native_histograms/ message Histogram { enum ResetHint { RESET_HINT_UNSPECIFIED = 0; // Need to test for a counter reset explicitly. @@ -242,6 +250,24 @@ message Histogram { // The last element is not only the upper inclusive bound of the last regular // bucket, but implicitly the lower exclusive bound of the +Inf bucket. repeated double custom_values = 16; + + // start_timestamp represents an optional start timestamp for the histogram sample, + // in ms format. The start timestamp represents the time when the histogram + // started counting, which can increase the accuracy of certain processing and + // query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + int64 start_timestamp = 17; } // A BucketSpan defines a number of consecutive buckets with their diff --git a/storage/interface.go b/storage/interface.go index d4c98e0710..6139a49511 100644 --- a/storage/interface.go +++ b/storage/interface.go @@ -49,7 +49,7 @@ var ( // NOTE(bwplotka): This can be both an instrumentation failure or commonly expected // behaviour, and we currently don't have a way to determine this. As a result // it's recommended to ignore this error for now. - ErrOutOfOrderST = errors.New("created timestamp out of order, ignoring") + ErrOutOfOrderST = errors.New("start timestamp out of order, ignoring") ErrSTNewerThanSample = errors.New("ST is newer or the same as sample's timestamp, ignoring") ) diff --git a/storage/remote/codec_test.go b/storage/remote/codec_test.go index 8bbe3c2813..ce3a09b878 100644 --- a/storage/remote/codec_test.go +++ b/storage/remote/codec_test.go @@ -114,7 +114,7 @@ var ( HelpRef: 15, // Symbolized writeV2RequestSeries1Metadata.Help. UnitRef: 16, // Symbolized writeV2RequestSeries1Metadata.Unit. }, - Samples: []writev2.Sample{{Value: 1, Timestamp: 10}}, + Samples: []writev2.Sample{{Value: 1, Timestamp: 10, StartTimestamp: 1}}, // ST needs to be lower than the sample's timestamp. Exemplars: []writev2.Exemplar{{LabelsRefs: []uint32{11, 12}, Value: 1, Timestamp: 10}}, Histograms: []writev2.Histogram{ writev2.FromIntHistogram(10, &testHistogram), @@ -122,7 +122,6 @@ var ( writev2.FromIntHistogram(30, &testHistogramCustomBuckets), writev2.FromFloatHistogram(40, testHistogramCustomBuckets.ToFloat(nil)), }, - CreatedTimestamp: 1, // ST needs to be lower than the sample's timestamp. }, { LabelsRefs: []uint32{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, // Same series as first. @@ -182,7 +181,7 @@ func TestWriteV2RequestFixture(t *testing.T) { HelpRef: st.Symbolize(writeV2RequestSeries1Metadata.Help), UnitRef: st.Symbolize(writeV2RequestSeries1Metadata.Unit), }, - Samples: []writev2.Sample{{Value: 1, Timestamp: 10}}, + Samples: []writev2.Sample{{Value: 1, Timestamp: 10, StartTimestamp: 1}}, Exemplars: []writev2.Exemplar{{LabelsRefs: exemplar1LabelRefs, Value: 1, Timestamp: 10}}, Histograms: []writev2.Histogram{ writev2.FromIntHistogram(10, &testHistogram), @@ -190,7 +189,6 @@ func TestWriteV2RequestFixture(t *testing.T) { writev2.FromIntHistogram(30, &testHistogramCustomBuckets), writev2.FromFloatHistogram(40, testHistogramCustomBuckets.ToFloat(nil)), }, - CreatedTimestamp: 1, }, { LabelsRefs: labelRefs, diff --git a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go index ecf7338c96..c93a00db76 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go @@ -106,7 +106,7 @@ func exponentialToNativeHistogram(p pmetric.ExponentialHistogramDataPoint, tempo // Sending a sample that triggers counter reset but with ResetHint==NO // would lead to Prometheus panic as it does not double check the hint. // Thus we're explicitly saying UNKNOWN here, which is always safe. - // TODO: using created time stamp should be accurate, but we + // TODO: using start timestamp should be accurate, but we // need to know here if it was used for the detection. // Ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28663#issuecomment-1810577303 // Counter reset detection in Prometheus: https://github.com/prometheus/prometheus/blob/f997c72f294c0f18ca13fa06d51889af04135195/tsdb/chunkenc/histogram.go#L232 @@ -312,7 +312,7 @@ func explicitHistogramToCustomBucketsHistogram(p pmetric.HistogramDataPoint, tem // Sending a sample that triggers counter reset but with ResetHint==NO // would lead to Prometheus panic as it does not double check the hint. // Thus we're explicitly saying UNKNOWN here, which is always safe. - // TODO: using created time stamp should be accurate, but we + // TODO: using start timestamp should be accurate, but we // need to know here if it was used for the detection. // Ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28663#issuecomment-1810577303 // Counter reset detection in Prometheus: https://github.com/prometheus/prometheus/blob/f997c72f294c0f18ca13fa06d51889af04135195/tsdb/chunkenc/histogram.go#L232 diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index 579c7a794f..f4672d971e 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -353,20 +353,18 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * allSamplesSoFar := rs.AllSamples() var ref storage.SeriesRef - - // Samples. - if h.ingestSTZeroSample && len(ts.Samples) > 0 && ts.Samples[0].Timestamp != 0 && ts.CreatedTimestamp != 0 { - // ST only needs to be ingested for the first sample, it will be considered - // out of order for the rest. - ref, err = app.AppendSTZeroSample(ref, ls, ts.Samples[0].Timestamp, ts.CreatedTimestamp) - if err != nil && !errors.Is(err, storage.ErrOutOfOrderST) { - // Even for the first sample OOO is a common scenario because - // we can't tell if a ST was already ingested in a previous request. - // We ignore the error. - h.logger.Debug("Error when appending ST in remote write request", "err", err, "series", ls.String(), "start_timestamp", ts.CreatedTimestamp, "timestamp", ts.Samples[0].Timestamp) - } - } for _, s := range ts.Samples { + if h.ingestSTZeroSample && s.StartTimestamp != 0 && s.Timestamp != 0 { + ref, err = app.AppendSTZeroSample(ref, ls, s.Timestamp, s.StartTimestamp) + // We treat OOO errors specially as it's a common scenario given: + // * We can't tell if ST was already ingested in a previous request. + // * We don't check if ST changed for stream of samples (we typically have one though), + // as it's checked in the AppendSTZeroSample reliably. + if err != nil && !errors.Is(err, storage.ErrOutOfOrderST) { + h.logger.Debug("Error when appending ST from remote write request", "err", err, "series", ls.String(), "start_timestamp", s.StartTimestamp, "timestamp", s.Timestamp) + } + } + ref, err = app.Append(ref, ls, s.GetTimestamp(), s.GetValue()) if err == nil { rs.Samples++ @@ -387,15 +385,14 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * // Native Histograms. for _, hp := range ts.Histograms { - if h.ingestSTZeroSample && hp.Timestamp != 0 && ts.CreatedTimestamp != 0 { - // Differently from samples, we need to handle ST for each histogram instead of just the first one. - // This is because histograms and float histograms are stored separately, even if they have the same labels. - ref, err = h.handleHistogramZeroSample(app, ref, ls, hp, ts.CreatedTimestamp) + if h.ingestSTZeroSample && hp.StartTimestamp != 0 && hp.Timestamp != 0 { + ref, err = h.handleHistogramZeroSample(app, ref, ls, hp, hp.StartTimestamp) + // We treat OOO errors specially as it's a common scenario given: + // * We can't tell if ST was already ingested in a previous request. + // * We don't check if ST changed for stream of samples (we typically have one though), + // as it's checked in the ingestSTZeroSample reliably. if err != nil && !errors.Is(err, storage.ErrOutOfOrderST) { - // Even for the first sample OOO is a common scenario because - // we can't tell if a ST was already ingested in a previous request. - // We ignore the error. - h.logger.Debug("Error when appending ST in remote write request", "err", err, "series", ls.String(), "start_timestamp", ts.CreatedTimestamp, "timestamp", hp.Timestamp) + h.logger.Debug("Error when appending ST from remote write request", "err", err, "series", ls.String(), "start_timestamp", hp.StartTimestamp, "timestamp", hp.Timestamp) } } if hp.IsFloatHistogram() { @@ -474,7 +471,7 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * return samplesWithoutMetadata, http.StatusBadRequest, errors.Join(badRequestErrs...) } -// handleHistogramZeroSample appends ST as a zero-value sample with ST value as the sample timestamp. +// handleHistogramZeroSample appends ST as a zero-value sample with st value as the sample timestamp. // It doesn't return errors in case of out of order ST. func (*writeHandler) handleHistogramZeroSample(app storage.Appender, ref storage.SeriesRef, l labels.Labels, hist writev2.Histogram, st int64) (storage.SeriesRef, error) { var err error diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go index 40f1bdff0f..92fda192c6 100644 --- a/storage/remote/write_handler_test.go +++ b/storage/remote/write_handler_test.go @@ -752,14 +752,12 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { i, j, k, m int ) for _, ts := range writeV2RequestFixture.Timeseries { - zeroHistogramIngested := false - zeroFloatHistogramIngested := false ls, err := ts.ToLabels(&b, writeV2RequestFixture.Symbols) require.NoError(t, err) for _, s := range ts.Samples { - if ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { - requireEqual(t, mockSample{ls, ts.CreatedTimestamp, 0}, appendable.samples[i]) + if s.StartTimestamp != 0 && tc.ingestSTZeroSample { + requireEqual(t, mockSample{ls, s.StartTimestamp, 0}, appendable.samples[i]) i++ } requireEqual(t, mockSample{ls, s.Timestamp, s.Value}, appendable.samples[i]) @@ -768,27 +766,21 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { for _, hp := range ts.Histograms { if hp.IsFloatHistogram() { fh := hp.ToFloatHistogram() - if !zeroFloatHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { - requireEqual(t, mockHistogram{ls, ts.CreatedTimestamp, nil, &histogram.FloatHistogram{}}, appendable.histograms[k]) + if hp.StartTimestamp != 0 && tc.ingestSTZeroSample { + requireEqual(t, mockHistogram{ls, hp.StartTimestamp, nil, &histogram.FloatHistogram{}}, appendable.histograms[k]) k++ - zeroFloatHistogramIngested = true } requireEqual(t, mockHistogram{ls, hp.Timestamp, nil, fh}, appendable.histograms[k]) } else { h := hp.ToIntHistogram() - if !zeroHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { - requireEqual(t, mockHistogram{ls, ts.CreatedTimestamp, &histogram.Histogram{}, nil}, appendable.histograms[k]) + if hp.StartTimestamp != 0 && tc.ingestSTZeroSample { + requireEqual(t, mockHistogram{ls, hp.StartTimestamp, &histogram.Histogram{}, nil}, appendable.histograms[k]) k++ - zeroHistogramIngested = true } requireEqual(t, mockHistogram{ls, hp.Timestamp, h, nil}, appendable.histograms[k]) } k++ } - if ts.CreatedTimestamp != 0 && tc.ingestSTZeroSample { - require.True(t, zeroHistogramIngested) - require.True(t, zeroFloatHistogramIngested) - } if tc.appendExemplarErr == nil { for _, e := range ts.Exemplars { ex, err := e.ToExemplar(&b, writeV2RequestFixture.Symbols) From 26f8c92de8b5ddec530a53c74a6f50ac28f65388 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Mon, 17 Nov 2025 16:00:12 +0100 Subject: [PATCH 056/439] test: skip TestRemoteWrite_ReshardingWithoutDeadlock temporarily as flaky (#17534) (#17543) (cherry picked from commit 35c3232a2ee541273828982b2ce6aadd6c1c9a5f) Signed-off-by: machine424 Signed-off-by: Jan Fajerski Co-authored-by: Ayoub Mrini --- cmd/prometheus/main_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/cmd/prometheus/main_test.go b/cmd/prometheus/main_test.go index ccc9151492..e5e3db39ae 100644 --- a/cmd/prometheus/main_test.go +++ b/cmd/prometheus/main_test.go @@ -969,6 +969,7 @@ remote_write: // TestRemoteWrite_ReshardingWithoutDeadlock ensures that resharding (scaling up) doesn't block when the shards are full. // See: https://github.com/prometheus/prometheus/issues/17384. func TestRemoteWrite_ReshardingWithoutDeadlock(t *testing.T) { + t.Skip("flaky test, see https://github.com/prometheus/prometheus/issues/17489") t.Parallel() tmpDir := t.TempDir() From 5087a258481215c468de2027862ddbbaa35e4d5a Mon Sep 17 00:00:00 2001 From: Minh Nguyen <148210689+pipiland2612@users.noreply.github.com> Date: Tue, 18 Nov 2025 11:37:09 +0200 Subject: [PATCH 057/439] Remote Write Receive Fix: Remove duplicate labels when type-and-unit-label feature is on (#17546) * drop extra label from receiver Signed-off-by: pipiland2612 * used constant Signed-off-by: pipiland2612 --------- Signed-off-by: pipiland2612 --- storage/remote/write_handler.go | 6 +- storage/remote/write_handler_test.go | 98 ++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+), 1 deletion(-) diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index f4672d971e..67c244167b 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -325,7 +325,11 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * if h.enableTypeAndUnitLabels && (m.Type != model.MetricTypeUnknown || m.Unit != "") { slb := labels.NewScratchBuilder(ls.Len() + 2) // +2 for __type__ and __unit__ ls.Range(func(l labels.Label) { - slb.Add(l.Name, l.Value) + // Skip __type__ and __unit__ labels if they exist in the incoming labels. + // They will be added from metadata to avoid duplicates. + if l.Name != model.MetricTypeLabel && l.Name != model.MetricUnitLabel { + slb.Add(l.Name, l.Value) + } }) schema.Metadata{Type: m.Type, Unit: m.Unit}.AddToLabels(&slb) slb.Sort() diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go index 92fda192c6..afc0d985ff 100644 --- a/storage/remote/write_handler_test.go +++ b/storage/remote/write_handler_test.go @@ -805,6 +805,104 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { } } +// TestRemoteWriteHandler_V2Message_NoDuplicateTypeAndUnitLabels verifies that when +// type-and-unit-labels feature is enabled, the receiver correctly handles cases where +// __type__ and __unit__ labels are already present in the incoming labels. +// Regression test for https://github.com/prometheus/prometheus/issues/17480. +func TestRemoteWriteHandler_V2Message_NoDuplicateTypeAndUnitLabels(t *testing.T) { + for _, tc := range []struct { + desc string + labelsToSend labels.Labels + metadataToSend writev2.Metadata + expectedLabels labels.Labels + }{ + { + desc: "Labels with __type__ and __unit__ should not be duplicated", + labelsToSend: labels.FromStrings("__name__", "node_cpu_seconds_total", "__type__", "counter", "__unit__", "seconds", "cpu", "0", "mode", "idle"), + metadataToSend: writev2.Metadata{ + Type: writev2.Metadata_METRIC_TYPE_COUNTER, + }, + expectedLabels: labels.FromStrings("__name__", "node_cpu_seconds_total", "__type__", "counter", "__unit__", "seconds", "cpu", "0", "mode", "idle"), + }, + { + desc: "Labels with __type__ only should not be duplicated", + labelsToSend: labels.FromStrings("__name__", "test_gauge", "__type__", "gauge", "instance", "localhost"), + metadataToSend: writev2.Metadata{ + Type: writev2.Metadata_METRIC_TYPE_GAUGE, + }, + expectedLabels: labels.FromStrings("__name__", "test_gauge", "__type__", "gauge", "instance", "localhost"), + }, + { + desc: "Labels with __unit__ only should not be duplicated when metadata has unit", + labelsToSend: labels.FromStrings("__name__", "test_metric", "__unit__", "bytes", "job", "test"), + metadataToSend: writev2.Metadata{ + Type: writev2.Metadata_METRIC_TYPE_GAUGE, + }, + expectedLabels: labels.FromStrings("__name__", "test_metric", "__type__", "gauge", "__unit__", "bytes", "job", "test"), + }, + { + desc: "Metadata type and unit override labels", + labelsToSend: labels.FromStrings("__name__", "test_metric", "__type__", "counter", "__unit__", "seconds", "job", "test"), + metadataToSend: writev2.Metadata{ + Type: writev2.Metadata_METRIC_TYPE_GAUGE, + }, + expectedLabels: labels.FromStrings("__name__", "test_metric", "__type__", "gauge", "__unit__", "seconds", "job", "test"), + }, + } { + t.Run(tc.desc, func(t *testing.T) { + symbolTable := writev2.NewSymbolTable() + labelRefs := symbolTable.SymbolizeLabels(tc.labelsToSend, nil) + + var unitRef uint32 + if unit := tc.labelsToSend.Get("__unit__"); unit != "" { + unitRef = symbolTable.Symbolize(unit) + } + + ts := []writev2.TimeSeries{ + { + LabelsRefs: labelRefs, + Metadata: writev2.Metadata{ + Type: tc.metadataToSend.Type, + UnitRef: unitRef, + }, + Samples: []writev2.Sample{{Value: 42.0, Timestamp: 1000}}, + }, + } + + payload, _, _, err := buildV2WriteRequest(promslog.NewNopLogger(), ts, symbolTable.Symbols(), nil, nil, nil, "snappy") + require.NoError(t, err) + + req, err := http.NewRequest(http.MethodPost, "", bytes.NewReader(payload)) + require.NoError(t, err) + + req.Header.Set("Content-Type", remoteWriteContentTypeHeaders[remoteapi.WriteV2MessageType]) + req.Header.Set("Content-Encoding", compression.Snappy) + req.Header.Set(RemoteWriteVersionHeader, RemoteWriteVersion20HeaderValue) + + appendable := &mockAppendable{} + handler := NewWriteHandler(promslog.NewNopLogger(), nil, appendable, []remoteapi.WriteMessageType{remoteapi.WriteV2MessageType}, false, true, false) + + recorder := httptest.NewRecorder() + handler.ServeHTTP(recorder, req) + + resp := recorder.Result() + require.Equal(t, http.StatusNoContent, resp.StatusCode) + + require.Len(t, appendable.samples, 1) + receivedLabels := appendable.samples[0].l + + duplicateLabel, hasDuplicate := receivedLabels.HasDuplicateLabelNames() + require.False(t, hasDuplicate, "Labels should NOT contain duplicates, but found duplicate label: %s\nReceived labels: %s", duplicateLabel, receivedLabels.String()) + + require.Equal(t, tc.expectedLabels.String(), receivedLabels.String(), "Labels should match expected") + + if tc.expectedLabels.Get("__type__") != "" { + require.NotEmpty(t, receivedLabels.Get("__type__"), "__type__ should be present in labels") + } + }) + } +} + // NOTE: V2 Message is tested in TestRemoteWriteHandler_V2Message. func TestOutOfOrderSample_V1Message(t *testing.T) { for _, tc := range []struct { From de084ae0e7adeafbcc8fcac30b9bf3fdf10481ff Mon Sep 17 00:00:00 2001 From: Linas Medziunas Date: Tue, 18 Nov 2025 12:06:32 +0200 Subject: [PATCH 058/439] [PromQL] Improve BenchmarkJoinQuery Signed-off-by: Linas Medziunas --- promql/bench_test.go | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/promql/bench_test.go b/promql/bench_test.go index 37c8311305..a15f19e17e 100644 --- a/promql/bench_test.go +++ b/promql/bench_test.go @@ -393,40 +393,44 @@ func BenchmarkJoinQuery(b *testing.B) { } engine := promqltest.NewTestEngineWithOpts(b, opts) - const interval = 10000 // 10s interval. + const ( + interval = 10000 // 10s interval. + steps = 5000 + numInstances = 1000 + ) - // A day of data plus 10k steps. - numIntervals := 8640 + 10000 + // A day of data plus steps. + numIntervals := 8640 + steps - require.NoError(b, setupJoinQueryTestData(stor, engine, interval, numIntervals, 1000)) + require.NoError(b, setupJoinQueryTestData(stor, engine, interval, numIntervals, numInstances)) for _, c := range []benchCase{ { expr: `rpc_request_success_total + rpc_request_error_total`, - steps: 10000, + steps: steps, }, { expr: `rpc_request_success_total + ON (job, instance) GROUP_LEFT rpc_request_error_total`, - steps: 10000, + steps: steps, }, { expr: `rpc_request_success_total AND rpc_request_error_total{instance=~"0.*"}`, // 0.* keeps 1/16 of UUID values - steps: 10000, + steps: steps, }, { expr: `rpc_request_success_total OR rpc_request_error_total{instance=~"0.*"}`, // 0.* keeps 1/16 of UUID values - steps: 10000, + steps: steps, }, { expr: `rpc_request_success_total UNLESS rpc_request_error_total{instance=~"0.*"}`, // 0.* keeps 1/16 of UUID values - steps: 10000, + steps: steps, }, } { name := fmt.Sprintf("expr=%s/steps=%d", c.expr, c.steps) b.Run(name, func(b *testing.B) { ctx := context.Background() - b.ReportAllocs() - for b.Loop() { + + queryFn := func() { qry, err := engine.NewRangeQuery( ctx, stor, nil, c.expr, timestamp.Time(int64((numIntervals-c.steps)*10_000)), @@ -439,6 +443,14 @@ func BenchmarkJoinQuery(b *testing.B) { qry.Close() } + + queryFn() // Warm up run. + + b.ResetTimer() + b.ReportAllocs() + for b.Loop() { + queryFn() + } }) } } From 4bee2c754eac940a9943f65560af816211010350 Mon Sep 17 00:00:00 2001 From: SuperQ Date: Tue, 18 Nov 2025 11:18:02 +0100 Subject: [PATCH 059/439] Improve repo sync script logging Improve the repo sync logging output and add some additional logging. This should help debugging some failed updates. Signed-off-by: SuperQ --- scripts/sync_repo_files.sh | 44 +++++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 12 deletions(-) diff --git a/scripts/sync_repo_files.sh b/scripts/sync_repo_files.sh index 09b0e4d93a..04735475da 100755 --- a/scripts/sync_repo_files.sh +++ b/scripts/sync_repo_files.sh @@ -30,6 +30,22 @@ echo_yellow() { echo -e "${color_yellow}$@${color_none}" 1>&2 } +repo_log_red() { + echo_red "${org_repo}: $@" +} + +repo_log_green() { + echo_green "${org_repo}: $@" +} + +repo_log_yellow() { + echo_yellow "${org_repo}: $@" +} + +repo_log() { + echo "${org_repo}: $@" 1>&2 +} + GITHUB_TOKEN="${GITHUB_TOKEN:-}" if [ -z "${GITHUB_TOKEN}" ]; then echo_red 'GitHub token (GITHUB_TOKEN) not set. Terminating.' @@ -112,28 +128,28 @@ process_repo() { local org_repo local default_branch org_repo="$1" - echo_green "Analyzing '${org_repo}'" + repo_log_green "Analyzing '${org_repo}'" default_branch="$(get_default_branch "${org_repo}")" if [[ -z "${default_branch}" ]]; then - echo "Can't get the default branch." + repo_log_red "Can't get the default branch." return fi - echo "Default branch: ${default_branch}" + repo_log "Default branch: ${default_branch}" local needs_update=() for source_file in ${SYNC_FILES}; do source_checksum="$(sha256sum "${source_dir}/${source_file}" | cut -d' ' -f1)" if [[ "${source_file}" == 'scripts/golangci-lint.yml' ]] && ! check_go "${org_repo}" "${default_branch}" ; then - echo "${org_repo} is not Go, skipping golangci-lint.yml." + repo_log "${org_repo} is not Go, skipping golangci-lint.yml." continue fi if [[ "${source_file}" == '.github/workflows/container_description.yml' ]] && ! check_docker "${org_repo}" "${default_branch}" ; then - echo "${org_repo} has no Dockerfile, skipping container_description.yml." + repo_log "${org_repo} has no Dockerfile, skipping container_description.yml." continue fi if [[ "${source_file}" == 'LICENSE' ]] && ! check_license "${target_file}" ; then - echo "LICENSE in ${org_repo} is not apache, skipping." + repo_log "LICENSE in ${org_repo} is not apache, skipping." continue fi target_filename="${source_file}" @@ -142,10 +158,10 @@ process_repo() { fi target_file="$(curl -sL --fail "https://raw.githubusercontent.com/${org_repo}/${default_branch}/${target_filename}")" if [[ -z "${target_file}" ]]; then - echo "${target_filename} doesn't exist in ${org_repo}" + repo_log "${target_filename} doesn't exist in ${org_repo}" case "${source_file}" in CODE_OF_CONDUCT.md | SECURITY.md | .github/workflows/container_description.yml) - echo "${source_file} missing in ${org_repo}, force updating." + repo_log_yellow "${source_file} missing in ${org_repo}, force updating." needs_update+=("${source_file}") ;; esac @@ -153,15 +169,15 @@ process_repo() { fi target_checksum="$(echo "${target_file}" | sha256sum | cut -d' ' -f1)" if [ "${source_checksum}" == "${target_checksum}" ]; then - echo "${source_file} is already in sync." + repo_log_green "${source_file} is already in sync." continue fi - echo "${source_file} needs updating." + repo_log_yellow "${source_file} needs updating." needs_update+=("${source_file}") done if [[ "${#needs_update[@]}" -eq 0 ]] ; then - echo "No files need sync." + repo_log_green "No files need sync." return fi @@ -184,17 +200,21 @@ process_repo() { esac done + repo_log "File sync complete" + if [[ -n "$(git status --porcelain)" ]]; then git config user.email "${git_mail}" git config user.name "${git_user}" git add . git commit -s -m "${commit_msg}" + repo_log "Commit created" if push_branch "${org_repo}"; then if ! post_pull_request "${org_repo}" "${default_branch}"; then + repo_log_red "Posting PR failed" return 1 fi else - echo "Pushing ${branch} to ${org_repo} failed" + repo_log_red "Pushing ${branch} to ${org_repo} failed" return 1 fi fi From 3a7ee37d9d1917af5758983ff6d170e7057426b9 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Tue, 18 Nov 2025 11:55:36 +0100 Subject: [PATCH 060/439] chore(deps): bump prometheus/promci from 0.4.7 to 0.5.0 Signed-off-by: Jan Fajerski --- .github/workflows/ci.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ed4cfbf356..1e9118bc3e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,7 +19,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/setup_environment with: enable_npm: true @@ -37,7 +37,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/setup_environment - run: go test --tags=dedupelabels ./... - run: go test --tags=slicelabels -race ./cmd/prometheus ./model/textparse ./prompb/... @@ -81,7 +81,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/setup_environment with: enable_go: false @@ -146,7 +146,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/build with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" @@ -173,7 +173,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/build with: parallelism: 12 @@ -268,7 +268,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/publish_main with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -287,7 +287,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - uses: ./.github/promci/actions/publish_release with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -304,7 +304,7 @@ jobs: uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@443c7fc2397e946bc9f5029e313a9c3441b9b86d # v0.4.7 + - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 - name: Install nodejs uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0 with: From 1193e6389694b7e4c2c3e613d927e9a092ddfe00 Mon Sep 17 00:00:00 2001 From: Andrew Hall Date: Tue, 18 Nov 2025 22:44:40 +0800 Subject: [PATCH 061/439] PromQL: Modify RatioSampler to expose more methods for the benefit of downstream projects (#17516) Methods added: - `SampleOffset(metric *labels.Labels) float64` to calculate the sample offset for a given label set. - `AddRatioSampleWithOffset(ratioLimit, sampleOffset float64) bool` to find out whether a given sample offset falls within a given ratio limit. The already existing method `AddRatioSample(ratioLimit float64, sample *Sample) bool` is now implemented as a simple combination of the two other methods. Exposing these methods helps downstream projects to re-use the implementations including easier testing. Signed-off-by: Andrew Hall --- promql/engine.go | 43 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 37 insertions(+), 6 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 67f9b9e3ba..eac3b64093 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -4151,13 +4151,17 @@ func makeInt64Pointer(val int64) *int64 { // RatioSampler allows unit-testing (previously: Randomizer). type RatioSampler interface { - // Return this sample "offset" between [0.0, 1.0] - sampleOffset(ts int64, sample *Sample) float64 - AddRatioSample(r float64, sample *Sample) bool + // SampleOffset returns this sample "offset" between [0.0, 1.0]. + SampleOffset(metric *labels.Labels) float64 + // AddRatioSample reports whether the sampling offset for the given sample falls within the specified ratio limit. + AddRatioSample(ratioLimit float64, sample *Sample) bool + // AddRatioSampleWithOffset reports whether the given sampling offset falls within the specified ratio limit. + AddRatioSampleWithOffset(ratioLimit, sampleOffset float64) bool } // HashRatioSampler uses Hash(labels.String()) / maxUint64 as a "deterministic" // value in [0.0, 1.0]. +// It is a utility used for limit_ratio aggregations. type HashRatioSampler struct{} var ratiosampler RatioSampler = NewHashRatioSampler() @@ -4166,14 +4170,42 @@ func NewHashRatioSampler() *HashRatioSampler { return &HashRatioSampler{} } -func (*HashRatioSampler) sampleOffset(_ int64, sample *Sample) float64 { +// SampleOffset returns a deterministic sampling offset in the range [0, 1) +// derived from the hash of the provided metric labels. +// +// The offset is computed by normalizing the 64-bit hash value of the label set +// to a float64 fraction of math.MaxUint64. This ensures that metrics with the +// same label set always produce the same offset, while different label sets +// produce uniformly distributed offsets suitable for sampling decisions. +func (*HashRatioSampler) SampleOffset(metric *labels.Labels) float64 { const ( float64MaxUint64 = float64(math.MaxUint64) ) - return float64(sample.Metric.Hash()) / float64MaxUint64 + return float64(metric.Hash()) / float64MaxUint64 } +// AddRatioSample returns a bool indicating if the sampling offset for the given sample is +// within the given ratio limit. +// +// See SampleOffset() for further details on the sample offset. +// See AddRatioSampleWithOffset() for further details on the ratioLimit and sampling offset comparison. func (s *HashRatioSampler) AddRatioSample(ratioLimit float64, sample *Sample) bool { + sampleOffset := s.SampleOffset(&sample.Metric) + return s.AddRatioSampleWithOffset(ratioLimit, sampleOffset) +} + +// AddRatioSampleWithOffset reports whether the given sampling offset falls within +// the specified ratio limit. +// +// The ratioLimit must be in the range [-1, 1]. The sampleOffset should be derived +// using SampleOffset(). +// +// When ratioLimit >= 0, the function returns true if sampleOffset < ratioLimit. +// When ratioLimit < 0, the function returns true if sampleOffset >= 1 + ratioLimit. +// +// Note that this method could be moved into AddRatioSample and removed from the Prometheus codebase, +// but it is useful for downstream projects using this code as a library. +func (*HashRatioSampler) AddRatioSampleWithOffset(ratioLimit, sampleOffset float64) bool { // If ratioLimit >= 0: add sample if sampleOffset is lesser than ratioLimit // // 0.0 ratioLimit 1.0 @@ -4194,7 +4226,6 @@ func (s *HashRatioSampler) AddRatioSample(ratioLimit float64, sample *Sample) bo // e.g.: // sampleOffset==0.3 && ratioLimit==-0.6 // 0.3 >= 0.4 ? --> don't add sample - sampleOffset := s.sampleOffset(sample.T, sample) return (ratioLimit >= 0 && sampleOffset < ratioLimit) || (ratioLimit < 0 && sampleOffset >= (1.0+ratioLimit)) } From 34b382099df790d3fdf0fd00b2501adfb82c4d07 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Tue, 18 Nov 2025 17:30:44 +0100 Subject: [PATCH 062/439] chore(deps): bump prometheus/promci from 0.5.0 to 0.5.1 Signed-off-by: Jan Fajerski --- .github/workflows/ci.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1e9118bc3e..1f1e86028a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,7 +19,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/setup_environment with: enable_npm: true @@ -37,7 +37,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/setup_environment - run: go test --tags=dedupelabels ./... - run: go test --tags=slicelabels -race ./cmd/prometheus ./model/textparse ./prompb/... @@ -81,7 +81,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/setup_environment with: enable_go: false @@ -146,7 +146,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/build with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" @@ -173,7 +173,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/build with: parallelism: 12 @@ -268,7 +268,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/publish_main with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -287,7 +287,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - uses: ./.github/promci/actions/publish_release with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -304,7 +304,7 @@ jobs: uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@0de5887ada1e8a8a665d8f619d4dd3afec8ba7e5 # v0.5.0 + - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 - name: Install nodejs uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0 with: From b17349bd56f80101b2cddec5ef6e52941a11e103 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Wed, 19 Nov 2025 12:05:20 +0100 Subject: [PATCH 063/439] chore(deps): bump prometheus/promci from 0.5.1 to 0.5.2 Signed-off-by: Jan Fajerski --- .github/workflows/ci.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1f1e86028a..82f4416c14 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,7 +19,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/setup_environment with: enable_npm: true @@ -37,7 +37,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/setup_environment - run: go test --tags=dedupelabels ./... - run: go test --tags=slicelabels -race ./cmd/prometheus ./model/textparse ./prompb/... @@ -81,7 +81,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/setup_environment with: enable_go: false @@ -146,7 +146,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/build with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" @@ -173,7 +173,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/build with: parallelism: 12 @@ -268,7 +268,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/publish_main with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -287,7 +287,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - uses: ./.github/promci/actions/publish_release with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -304,7 +304,7 @@ jobs: uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@c6d50d8e6149e4079661f68b565de32a2053ec5b # v0.5.1 + - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 - name: Install nodejs uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0 with: From 1174b0ce4f1f302b96bc0dcc1112e142801b5eb1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gr=C3=A9goire?= Date: Wed, 19 Nov 2025 14:03:32 +0100 Subject: [PATCH 064/439] model/textparse: Remove unit validation in protobuf parsing (#16834) Signed-off-by: Gregoire Verdier --- model/textparse/protobufparse.go | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/model/textparse/protobufparse.go b/model/textparse/protobufparse.go index 8b517f4c49..a48aa4af69 100644 --- a/model/textparse/protobufparse.go +++ b/model/textparse/protobufparse.go @@ -19,7 +19,6 @@ import ( "fmt" "io" "math" - "strings" "unicode/utf8" "github.com/gogo/protobuf/types" @@ -466,16 +465,6 @@ func (p *ProtobufParser) Next() (Entry, error) { default: return EntryInvalid, fmt.Errorf("unknown metric type for metric %q: %s", name, p.dec.GetType()) } - unit := p.dec.GetUnit() - if len(unit) > 0 { - if p.dec.GetType() == dto.MetricType_COUNTER && strings.HasSuffix(name, "_total") { - if !strings.HasSuffix(name[:len(name)-6], unit) || len(name)-6 < len(unit)+1 || name[len(name)-6-len(unit)-1] != '_' { - return EntryInvalid, fmt.Errorf("unit %q not a suffix of counter %q", unit, name) - } - } else if !strings.HasSuffix(name, unit) || len(name) < len(unit)+1 || name[len(name)-len(unit)-1] != '_' { - return EntryInvalid, fmt.Errorf("unit %q not a suffix of metric %q", unit, name) - } - } p.entryBytes.Reset() p.entryBytes.WriteString(name) p.state = EntryHelp From 2dfc3248218f547fa6408b492b199cd74f6fd97b Mon Sep 17 00:00:00 2001 From: beorn7 Date: Tue, 18 Nov 2025 22:59:46 +0100 Subject: [PATCH 065/439] model/histogram: Make histogram bucket iterators more robust Currently, iterating over histogram buckets can panic if the spans are not consistent with the buckets. We aim for validating histograms upon ingestion, but there might still be data corruptions on disk that could trigger the panic. While data corruption on disk is really bad and will lead to all kind of weirdness, we should still avoid panic'ing. Note, though, that chunks are secured by checksums, so the corruptions won't realistically happen because of disk faults, but more likely because a chunk was generated in a faulty way in the first place, by a software bug or even maliciously. This commit prevents panics in the situation where there are fewer buckets than described by the spans. Note that the missing buckets will simply not be iterated over. There is no signalling of this problem. We might still consider this separately, but for now, I would say that this kind of corruption is exceedingly rare and doesn't deserve special treatment (which will add a whole lot of complexity to the code). Signed-off-by: beorn7 --- model/histogram/float_histogram.go | 21 +- model/histogram/float_histogram_test.go | 323 +++++++++++++++++++++--- model/histogram/histogram.go | 10 + model/histogram/histogram_test.go | 80 ++++++ 4 files changed, 393 insertions(+), 41 deletions(-) diff --git a/model/histogram/float_histogram.go b/model/histogram/float_histogram.go index c607448f38..28f35572c2 100644 --- a/model/histogram/float_histogram.go +++ b/model/histogram/float_histogram.go @@ -1050,16 +1050,21 @@ func (i *floatBucketIterator) Next() bool { if i.spansIdx >= len(i.spans) { return false } + span := i.spans[i.spansIdx] if i.schema == i.targetSchema { // Fast path for the common case. - span := i.spans[i.spansIdx] if i.bucketsIdx == 0 { // Seed origIdx for the first bucket. i.currIdx = span.Offset } else { i.currIdx++ } + if i.bucketsIdx >= len(i.buckets) { + // This protects against index out of range panic, which + // can only happen with an invalid histogram. + return false + } for i.idxInSpan >= span.Length { // We have exhausted the current span and have to find a new @@ -1080,7 +1085,6 @@ func (i *floatBucketIterator) Next() bool { // Copy all of these into local variables so that we can forward to the // next bucket and then roll back if needed. origIdx, spansIdx, idxInSpan := i.origIdx, i.spansIdx, i.idxInSpan - span := i.spans[spansIdx] firstPass := true i.currCount = 0 @@ -1092,6 +1096,14 @@ func (i *floatBucketIterator) Next() bool { } else { origIdx++ } + if i.bucketsIdx >= len(i.buckets) { + // This protects against index out of range panic, which + // can only happen with an invalid histogram. + if firstPass { + return false + } + break mergeLoop + } for idxInSpan >= span.Length { // We have exhausted the current span and have to find a new // one. We even handle pathologic spans of length 0 here. @@ -1152,6 +1164,11 @@ func (i *reverseFloatBucketIterator) Next() bool { // We have exhausted the current span and have to find a new // one. We'll even handle pathologic spans of length 0. i.spansIdx-- + if i.spansIdx < 0 { + // This protects against index out of range panic, which + // can only happen with an invalid histogram. + return false + } i.idxInSpan = int32(i.spans[i.spansIdx].Length) - 1 i.currIdx -= i.spans[i.spansIdx+1].Offset } diff --git a/model/histogram/float_histogram_test.go b/model/histogram/float_histogram_test.go index 7454e9c77c..ac339f152e 100644 --- a/model/histogram/float_histogram_test.go +++ b/model/histogram/float_histogram_test.go @@ -3344,6 +3344,84 @@ func TestAllReverseFloatBucketIterator(t *testing.T) { includeZero: true, includePos: true, }, + { + h: FloatHistogram{ + Count: 405, + ZeroCount: 102, + ZeroThreshold: 0.001, + Sum: 1008.4, + Schema: 1, + PositiveSpans: []Span{ + {Offset: 0, Length: 4}, + {Offset: 1, Length: 0}, + {Offset: 3, Length: 3}, + {Offset: 3, Length: 0}, + {Offset: 2, Length: 0}, + {Offset: 5, Length: 3}, + }, + // Spans expect one more bucket than listed + // here. We mostly want to make sure here that + // no panic happens. Data is invalid anyway, so + // there is no real "correct" data anymore. + PositiveBuckets: []float64{100, 344, 123, 55, 3, 63, 2, 54, 235}, + NegativeSpans: []Span{ + {Offset: 0, Length: 3}, + {Offset: 1, Length: 0}, + {Offset: 3, Length: 0}, + {Offset: 3, Length: 4}, + {Offset: 2, Length: 0}, + {Offset: 5, Length: 3}, + }, + // Spans expect one more bucket than listed + // here. We mostly want to make sure here that + // no panic happens. Data is invalid anyway, so + // there is no real "correct" data anymore. + NegativeBuckets: []float64{10, 34, 1230, 54, 67, 63, 2, 554, 235}, + }, + includeNeg: true, + includeZero: true, + includePos: true, + }, + { + h: FloatHistogram{ + Count: 447, + ZeroCount: 42, + ZeroThreshold: 0.6, // Within the bucket closest to zero. + Sum: 1008.4, + Schema: 0, + PositiveSpans: []Span{ + {Offset: 0, Length: 4}, + {Offset: 1, Length: 0}, + {Offset: 3, Length: 3}, + {Offset: 3, Length: 0}, + {Offset: 2, Length: 0}, + {Offset: 5, Length: 3}, + }, + // One more bucket listed here than expected by + // the spans. We mostly want to make sure here + // that no panic happens. Data is invalid + // anyway, so there is no real "correct" data + // anymore. + PositiveBuckets: []float64{100, 344, 123, 55, 3, 63, 2, 54, 235, 33, 42}, + NegativeSpans: []Span{ + {Offset: 0, Length: 3}, + {Offset: 1, Length: 0}, + {Offset: 3, Length: 0}, + {Offset: 3, Length: 4}, + {Offset: 2, Length: 0}, + {Offset: 5, Length: 3}, + }, + // One more bucket listed here than expected by + // the spans. We mostly want to make sure here + // that no panic happens. Data is invalid + // anyway, so there is no real "correct" data + // anymore. + NegativeBuckets: []float64{10, 34, 1230, 54, 67, 63, 2, 554, 235, 33, 42}, + }, + includeNeg: true, + includeZero: true, + includePos: true, + }, } for i, c := range cases { @@ -3391,50 +3469,217 @@ func TestAllReverseFloatBucketIterator(t *testing.T) { } func TestFloatBucketIteratorTargetSchema(t *testing.T) { - h := FloatHistogram{ - Count: 405, - Sum: 1008.4, - Schema: 1, - PositiveSpans: []Span{ - {Offset: 0, Length: 4}, - {Offset: 1, Length: 3}, - {Offset: 2, Length: 3}, + cases := map[string]struct { + h FloatHistogram + expPositiveBuckets []Bucket[float64] + expNegativeBuckets []Bucket[float64] + }{ + "regular": { + h: FloatHistogram{ + Count: 405, + Sum: 1008.4, + Schema: 1, + PositiveSpans: []Span{ + {Offset: 0, Length: 4}, + {Offset: 1, Length: 3}, + {Offset: 2, Length: 3}, + }, + PositiveBuckets: []float64{100, 344, 123, 55, 3, 63, 2, 54, 235, 33}, + NegativeSpans: []Span{ + {Offset: 0, Length: 3}, + {Offset: 7, Length: 4}, + {Offset: 1, Length: 3}, + }, + NegativeBuckets: []float64{10, 34, 1230, 54, 67, 63, 2, 554, 235, 33}, + }, + expPositiveBuckets: []Bucket[float64]{ + {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, + {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, + {Lower: 4, Upper: 16, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 2}, + {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 322, Index: 3}, + }, + expNegativeBuckets: []Bucket[float64]{ + {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 10, Index: 0}, + {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 1264, Index: 1}, + {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 184, Index: 3}, + {Lower: -256, Upper: -64, LowerInclusive: true, UpperInclusive: false, Count: 791, Index: 4}, + {Lower: -1024, Upper: -256, LowerInclusive: true, UpperInclusive: false, Count: 33, Index: 5}, + }, }, - PositiveBuckets: []float64{100, 344, 123, 55, 3, 63, 2, 54, 235, 33}, - NegativeSpans: []Span{ - {Offset: 0, Length: 3}, - {Offset: 7, Length: 4}, - {Offset: 1, Length: 3}, + "missing buckets": { + // One fewer bucket than expected based on spans. This + // can only happen with invalid histograms. We still + // want to handle it gracefully, essentially by + // considering the missing bucket as empty. + h: FloatHistogram{ + Count: 405, + Sum: 1008.4, + Schema: 1, + PositiveSpans: []Span{ + {Offset: 0, Length: 4}, + {Offset: 1, Length: 3}, + {Offset: 2, Length: 3}, + }, + PositiveBuckets: []float64{100, 344, 123, 55, 3, 63, 2, 54, 235}, + NegativeSpans: []Span{ + {Offset: 0, Length: 3}, + {Offset: 7, Length: 4}, + {Offset: 1, Length: 3}, + }, + NegativeBuckets: []float64{10, 34, 1230, 54, 67, 63, 2, 554, 235}, + }, + expPositiveBuckets: []Bucket[float64]{ + {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, + {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, + {Lower: 4, Upper: 16, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 2}, + {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 289, Index: 3}, + }, + expNegativeBuckets: []Bucket[float64]{ + {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 10, Index: 0}, + {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 1264, Index: 1}, + {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 184, Index: 3}, + {Lower: -256, Upper: -64, LowerInclusive: true, UpperInclusive: false, Count: 791, Index: 4}, + }, + }, + "spurious bucket": { + // One more bucket than expected based on spans. This + // can only happen with invalid histograms. We still + // want to handle it gracefully, essentially by ignoring + // the spurious bucket. + h: FloatHistogram{ + Count: 405, + Sum: 1008.4, + Schema: 1, + PositiveSpans: []Span{ + {Offset: 0, Length: 4}, + {Offset: 1, Length: 3}, + {Offset: 2, Length: 3}, + }, + PositiveBuckets: []float64{100, 344, 123, 55, 3, 63, 2, 54, 235, 33, 42}, + NegativeSpans: []Span{ + {Offset: 0, Length: 3}, + {Offset: 7, Length: 4}, + {Offset: 1, Length: 3}, + }, + NegativeBuckets: []float64{10, 34, 1230, 54, 67, 63, 2, 554, 235, 33, 42}, + }, + expPositiveBuckets: []Bucket[float64]{ + {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, + {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, + {Lower: 4, Upper: 16, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 2}, + {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 322, Index: 3}, + }, + expNegativeBuckets: []Bucket[float64]{ + {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 10, Index: 0}, + {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 1264, Index: 1}, + {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 184, Index: 3}, + {Lower: -256, Upper: -64, LowerInclusive: true, UpperInclusive: false, Count: 791, Index: 4}, + {Lower: -1024, Upper: -256, LowerInclusive: true, UpperInclusive: false, Count: 33, Index: 5}, + }, + }, + "no schema change": { + h: FloatHistogram{ + Count: 405, + Sum: 1008.4, + Schema: -1, + PositiveSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + PositiveBuckets: []float64{100, 522, 68, 322}, + NegativeSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + NegativeBuckets: []float64{100, 522, 68, 322}, + }, + expPositiveBuckets: []Bucket[float64]{ + {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, + {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, + {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 3}, + {Lower: 64, Upper: 256, LowerInclusive: false, UpperInclusive: true, Count: 322, Index: 4}, + }, + expNegativeBuckets: []Bucket[float64]{ + {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 100, Index: 0}, + {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 522, Index: 1}, + {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 68, Index: 3}, + {Lower: -256, Upper: -64, LowerInclusive: true, UpperInclusive: false, Count: 322, Index: 4}, + }, + }, + "no schema change, missing bucket": { + h: FloatHistogram{ + Count: 405, + Sum: 1008.4, + Schema: -1, + PositiveSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + PositiveBuckets: []float64{100, 522, 68}, + NegativeSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + NegativeBuckets: []float64{100, 522, 68}, + }, + expPositiveBuckets: []Bucket[float64]{ + {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, + {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, + {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 3}, + }, + expNegativeBuckets: []Bucket[float64]{ + {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 100, Index: 0}, + {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 522, Index: 1}, + {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 68, Index: 3}, + }, + }, + "no schema change, spurious bucket": { + h: FloatHistogram{ + Count: 405, + Sum: 1008.4, + Schema: -1, + PositiveSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + PositiveBuckets: []float64{100, 522, 68, 322, 42}, + NegativeSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + NegativeBuckets: []float64{100, 522, 68, 322, 42}, + }, + expPositiveBuckets: []Bucket[float64]{ + {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, + {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, + {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 3}, + {Lower: 64, Upper: 256, LowerInclusive: false, UpperInclusive: true, Count: 322, Index: 4}, + }, + expNegativeBuckets: []Bucket[float64]{ + {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 100, Index: 0}, + {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 522, Index: 1}, + {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 68, Index: 3}, + {Lower: -256, Upper: -64, LowerInclusive: true, UpperInclusive: false, Count: 322, Index: 4}, + }, }, - NegativeBuckets: []float64{10, 34, 1230, 54, 67, 63, 2, 554, 235, 33}, - } - expPositiveBuckets := []Bucket[float64]{ - {Lower: 0.25, Upper: 1, LowerInclusive: false, UpperInclusive: true, Count: 100, Index: 0}, - {Lower: 1, Upper: 4, LowerInclusive: false, UpperInclusive: true, Count: 522, Index: 1}, - {Lower: 4, Upper: 16, LowerInclusive: false, UpperInclusive: true, Count: 68, Index: 2}, - {Lower: 16, Upper: 64, LowerInclusive: false, UpperInclusive: true, Count: 322, Index: 3}, - } - expNegativeBuckets := []Bucket[float64]{ - {Lower: -1, Upper: -0.25, LowerInclusive: true, UpperInclusive: false, Count: 10, Index: 0}, - {Lower: -4, Upper: -1, LowerInclusive: true, UpperInclusive: false, Count: 1264, Index: 1}, - {Lower: -64, Upper: -16, LowerInclusive: true, UpperInclusive: false, Count: 184, Index: 3}, - {Lower: -256, Upper: -64, LowerInclusive: true, UpperInclusive: false, Count: 791, Index: 4}, - {Lower: -1024, Upper: -256, LowerInclusive: true, UpperInclusive: false, Count: 33, Index: 5}, } + for tn, tc := range cases { + t.Run(tn, func(t *testing.T) { + it := tc.h.floatBucketIterator(true, 0, -1) + for i, b := range tc.expPositiveBuckets { + require.True(t, it.Next(), "positive iterator exhausted too early") + require.Equal(t, b, it.At(), "bucket %d", i) + } + require.False(t, it.Next(), "positive iterator not exhausted") - it := h.floatBucketIterator(true, 0, -1) - for i, b := range expPositiveBuckets { - require.True(t, it.Next(), "positive iterator exhausted too early") - require.Equal(t, b, it.At(), "bucket %d", i) + it = tc.h.floatBucketIterator(false, 0, -1) + for i, b := range tc.expNegativeBuckets { + require.True(t, it.Next(), "negative iterator exhausted too early") + require.Equal(t, b, it.At(), "bucket %d", i) + } + require.False(t, it.Next(), "negative iterator not exhausted") + }) } - require.False(t, it.Next(), "positive iterator not exhausted") - - it = h.floatBucketIterator(false, 0, -1) - for i, b := range expNegativeBuckets { - require.True(t, it.Next(), "negative iterator exhausted too early") - require.Equal(t, b, it.At(), "bucket %d", i) - } - require.False(t, it.Next(), "negative iterator not exhausted") } func TestFloatCustomBucketsIterators(t *testing.T) { diff --git a/model/histogram/histogram.go b/model/histogram/histogram.go index a7d9ce80f0..959df4c87a 100644 --- a/model/histogram/histogram.go +++ b/model/histogram/histogram.go @@ -515,6 +515,11 @@ func (r *regularBucketIterator) Next() bool { r.currIdx += span.Offset } + // This protects against index out of range panic, which + // can only happen with an invalid histogram. + if r.bucketsIdx >= len(r.buckets) { + return false + } r.currCount += r.buckets[r.bucketsIdx] r.idxInSpan++ r.bucketsIdx++ @@ -576,6 +581,11 @@ func (c *cumulativeBucketIterator) Next() bool { c.initialized = true } + // This protects against index out of range panic, which + // can only happen with an invalid histogram. + if c.posBucketsIdx >= len(c.h.PositiveBuckets) { + return false + } c.currCount += c.h.PositiveBuckets[c.posBucketsIdx] c.currCumulativeCount += uint64(c.currCount) c.currUpper = getBound(c.currIdx, c.h.Schema, c.h.CustomValues) diff --git a/model/histogram/histogram_test.go b/model/histogram/histogram_test.go index d65049c68c..e4c6ce683b 100644 --- a/model/histogram/histogram_test.go +++ b/model/histogram/histogram_test.go @@ -243,6 +243,46 @@ func TestCumulativeBucketIterator(t *testing.T) { {Lower: math.Inf(-1), Upper: math.Inf(1), Count: 5, LowerInclusive: true, UpperInclusive: true, Index: 4}, }, }, + { + histogram: Histogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + // One spurious bucket, which we expect to be ignored. + PositiveBuckets: []int64{1, 1, -1, 0, 2}, + }, + expectedBuckets: []Bucket[uint64]{ + {Lower: math.Inf(-1), Upper: 1, Count: 1, LowerInclusive: true, UpperInclusive: true, Index: 0}, + {Lower: math.Inf(-1), Upper: 2, Count: 3, LowerInclusive: true, UpperInclusive: true, Index: 1}, + + {Lower: math.Inf(-1), Upper: 4, Count: 3, LowerInclusive: true, UpperInclusive: true, Index: 2}, + + {Lower: math.Inf(-1), Upper: 8, Count: 4, LowerInclusive: true, UpperInclusive: true, Index: 3}, + {Lower: math.Inf(-1), Upper: 16, Count: 5, LowerInclusive: true, UpperInclusive: true, Index: 4}, + }, + }, + { + histogram: Histogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 3}, + }, + // One bucket is missing. We expect the iteration to end with the last bucket. + PositiveBuckets: []int64{1, 1, -1, 0}, + }, + expectedBuckets: []Bucket[uint64]{ + {Lower: math.Inf(-1), Upper: 1, Count: 1, LowerInclusive: true, UpperInclusive: true, Index: 0}, + {Lower: math.Inf(-1), Upper: 2, Count: 3, LowerInclusive: true, UpperInclusive: true, Index: 1}, + + {Lower: math.Inf(-1), Upper: 4, Count: 3, LowerInclusive: true, UpperInclusive: true, Index: 2}, + + {Lower: math.Inf(-1), Upper: 8, Count: 4, LowerInclusive: true, UpperInclusive: true, Index: 3}, + {Lower: math.Inf(-1), Upper: 16, Count: 5, LowerInclusive: true, UpperInclusive: true, Index: 4}, + }, + }, } for i, c := range cases { @@ -459,6 +499,46 @@ func TestRegularBucketIterator(t *testing.T) { }, expectedNegativeBuckets: []Bucket[uint64]{}, }, + { + histogram: Histogram{ + Schema: 0, + NegativeSpans: []Span{ + {Offset: 0, Length: 5}, + {Offset: 1, Length: 1}, + }, + // One spurious bucket, which we expect to be ignored. + NegativeBuckets: []int64{1, 2, -2, 1, -1, 0, 3}, + }, + expectedPositiveBuckets: []Bucket[uint64]{}, + expectedNegativeBuckets: []Bucket[uint64]{ + {Lower: -1, Upper: -0.5, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 0}, + {Lower: -2, Upper: -1, Count: 3, LowerInclusive: true, UpperInclusive: false, Index: 1}, + {Lower: -4, Upper: -2, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 2}, + {Lower: -8, Upper: -4, Count: 2, LowerInclusive: true, UpperInclusive: false, Index: 3}, + {Lower: -16, Upper: -8, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 4}, + + {Lower: -64, Upper: -32, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 6}, + }, + }, + { + histogram: Histogram{ + Schema: 0, + NegativeSpans: []Span{ + {Offset: 0, Length: 5}, + {Offset: 1, Length: 1}, + }, + // One bucket is missing. We expect the iteration to end with the last bucket. + NegativeBuckets: []int64{1, 2, -2, 1, -1}, + }, + expectedPositiveBuckets: []Bucket[uint64]{}, + expectedNegativeBuckets: []Bucket[uint64]{ + {Lower: -1, Upper: -0.5, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 0}, + {Lower: -2, Upper: -1, Count: 3, LowerInclusive: true, UpperInclusive: false, Index: 1}, + {Lower: -4, Upper: -2, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 2}, + {Lower: -8, Upper: -4, Count: 2, LowerInclusive: true, UpperInclusive: false, Index: 3}, + {Lower: -16, Upper: -8, Count: 1, LowerInclusive: true, UpperInclusive: false, Index: 4}, + }, + }, } for i, c := range cases { From 93cde0f9147045d949d51065fbe1c2b80b476e42 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Wed, 19 Nov 2025 20:16:54 +0100 Subject: [PATCH 066/439] chore(deps): bump prometheus/promci from 0.5.2 to 0.5.3 Signed-off-by: Jan Fajerski --- .github/workflows/ci.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 82f4416c14..e4c2fbce18 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,7 +19,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/setup_environment with: enable_npm: true @@ -37,7 +37,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/setup_environment - run: go test --tags=dedupelabels ./... - run: go test --tags=slicelabels -race ./cmd/prometheus ./model/textparse ./prompb/... @@ -81,7 +81,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/setup_environment with: enable_go: false @@ -146,7 +146,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/build with: promu_opts: "-p linux/amd64 -p windows/amd64 -p linux/arm64 -p darwin/amd64 -p darwin/arm64 -p linux/386" @@ -173,7 +173,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/build with: parallelism: 12 @@ -268,7 +268,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/publish_main with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -287,7 +287,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - uses: ./.github/promci/actions/publish_release with: docker_hub_login: ${{ secrets.docker_hub_login }} @@ -304,7 +304,7 @@ jobs: uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - uses: prometheus/promci@bb1909986b91b37c4104dd742c4684f43cf1b260 # v0.5.2 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 - name: Install nodejs uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0 with: From 61f64a4cb1dc3bafb5b4227831e6254b038cd43d Mon Sep 17 00:00:00 2001 From: Julien <291750+roidelapluie@users.noreply.github.com> Date: Thu, 20 Nov 2025 08:56:39 +0100 Subject: [PATCH 067/439] Makefile.common: Use git ls-files instead of find for license check and style check (#17557) Also improve find fallback to use -prune for better performance. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- Makefile.common | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile.common b/Makefile.common index 143bf03fbc..3ed717b460 100644 --- a/Makefile.common +++ b/Makefile.common @@ -112,7 +112,7 @@ common-all: precheck style check_license lint yamllint unused build test .PHONY: common-style common-style: @echo ">> checking code style" - @fmtRes=$$($(GOFMT) -d $$(find . -path ./vendor -prune -o -name '*.go' -print)); \ + @fmtRes=$$($(GOFMT) -d $$(git ls-files '*.go' ':!:vendor/*' || find . -path ./vendor -prune -o -name '*.go' -print)); \ if [ -n "$${fmtRes}" ]; then \ echo "gofmt checking failed!"; echo "$${fmtRes}"; echo; \ echo "Please ensure you are using $$($(GO) version) for formatting code."; \ @@ -122,7 +122,7 @@ common-style: .PHONY: common-check_license common-check_license: @echo ">> checking license header" - @licRes=$$(for file in $$(find . -type f -iname '*.go' ! -path './vendor/*') ; do \ + @licRes=$$(for file in $$(git ls-files '*.go' ':!:vendor/*' || find . -path ./vendor -prune -o -type f -iname '*.go' -print) ; do \ awk 'NR<=3' $$file | grep -Eq "(Copyright|generated|GENERATED)" || echo $$file; \ done); \ if [ -n "$${licRes}" ]; then \ From 3bcc88b053e7d772b8c8d7e38745c10ceb42a985 Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Mon, 17 Nov 2025 15:59:40 +0100 Subject: [PATCH 068/439] prw2: Move Remote Write 2.0 CT to be per Sample; Rename to ST (start timestamp) (#17411) Relates to https://github.com/prometheus/prometheus/issues/16944#issuecomment-3164760343 Signed-off-by: bwplotka (cherry picked from commit cefefc689766827a8c933e3181e9dd548656e71a) --- prompb/io/prometheus/write/v2/custom.go | 5 - prompb/io/prometheus/write/v2/types.pb.go | 302 +++++++++++------- prompb/io/prometheus/write/v2/types.proto | 74 +++-- storage/remote/codec_test.go | 6 +- .../prometheusremotewrite/histograms.go | 4 +- storage/remote/write_handler.go | 49 ++- storage/remote/write_handler_test.go | 20 +- 7 files changed, 263 insertions(+), 197 deletions(-) diff --git a/prompb/io/prometheus/write/v2/custom.go b/prompb/io/prometheus/write/v2/custom.go index 3aa778eb60..5721aec532 100644 --- a/prompb/io/prometheus/write/v2/custom.go +++ b/prompb/io/prometheus/write/v2/custom.go @@ -80,11 +80,6 @@ func (m *TimeSeries) OptimizedMarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } - if m.CreatedTimestamp != 0 { - i = encodeVarintTypes(dAtA, i, uint64(m.CreatedTimestamp)) - i-- - dAtA[i] = 0x30 - } { size, err := m.Metadata.MarshalToSizedBuffer(dAtA[:i]) if err != nil { diff --git a/prompb/io/prometheus/write/v2/types.pb.go b/prompb/io/prometheus/write/v2/types.pb.go index 1419de217e..a726efb5b5 100644 --- a/prompb/io/prometheus/write/v2/types.pb.go +++ b/prompb/io/prometheus/write/v2/types.pb.go @@ -106,6 +106,8 @@ func (Histogram_ResetHint) EnumDescriptor() ([]byte, []int) { // The canonical Content-Type request header value for this message is // "application/x-protobuf;proto=io.prometheus.write.v2.Request" // +// Version: v2.0-rc.4 +// // NOTE: gogoproto options might change in future for this file, they // are not part of the spec proto (they only modify the generated Go code, not // the serialized message). See: https://github.com/prometheus/prometheus/issues/11908 @@ -181,7 +183,7 @@ type TimeSeries struct { // // Note that there might be multiple TimeSeries objects in the same // Requests with the same labels e.g. for different exemplars, metadata - // or created timestamp. + // or start timestamp. LabelsRefs []uint32 `protobuf:"varint,1,rep,packed,name=labels_refs,json=labelsRefs,proto3" json:"labels_refs,omitempty"` // Timeseries messages can either specify samples or (native) histogram samples // (histogram field), but not both. For a typical sender (real-time metric @@ -193,24 +195,7 @@ type TimeSeries struct { // exemplars represents an optional set of exemplars attached to this series' samples. Exemplars []Exemplar `protobuf:"bytes,4,rep,name=exemplars,proto3" json:"exemplars"` // metadata represents the metadata associated with the given series' samples. - Metadata Metadata `protobuf:"bytes,5,opt,name=metadata,proto3" json:"metadata"` - // created_timestamp represents an optional created timestamp associated with - // this series' samples in ms format, typically for counter or histogram type - // metrics. Created timestamp represents the time when the counter started - // counting (sometimes referred to as start timestamp), which can increase - // the accuracy of query results. - // - // Note that some receivers might require this and in return fail to - // ingest such samples within the Request. - // - // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go - // for conversion from/to time.Time to Prometheus timestamp. - // - // Note that the "optional" keyword is omitted due to - // https://cloud.google.com/apis/design/design_patterns.md#optional_primitive_fields - // Zero value means value not set. If you need to use exactly zero value for - // the timestamp, use 1 millisecond before or after. - CreatedTimestamp int64 `protobuf:"varint,6,opt,name=created_timestamp,json=createdTimestamp,proto3" json:"created_timestamp,omitempty"` + Metadata Metadata `protobuf:"bytes,5,opt,name=metadata,proto3" json:"metadata"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` @@ -284,13 +269,6 @@ func (m *TimeSeries) GetMetadata() Metadata { return Metadata{} } -func (m *TimeSeries) GetCreatedTimestamp() int64 { - if m != nil { - return m.CreatedTimestamp - } - return 0 -} - // Exemplar is an additional information attached to some series' samples. // It is typically used to attach an example trace or request ID associated with // the metric changes. @@ -375,7 +353,27 @@ type Sample struct { // // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go // for conversion from/to time.Time to Prometheus timestamp. - Timestamp int64 `protobuf:"varint,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"` + Timestamp int64 `protobuf:"varint,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"` + // start_timestamp represents an optional start timestamp for the sample, + // in ms format. This information is typically used for counter, histogram (cumulative) + // or delta type metrics. + // + // For cumulative metrics, the start timestamp represents the time when the + // counter started counting (sometimes referred to as start timestamp), which + // can increase the accuracy of certain processing and query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + StartTimestamp int64 `protobuf:"varint,3,opt,name=start_timestamp,json=startTimestamp,proto3" json:"start_timestamp,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` @@ -428,6 +426,13 @@ func (m *Sample) GetTimestamp() int64 { return 0 } +func (m *Sample) GetStartTimestamp() int64 { + if m != nil { + return m.StartTimestamp + } + return 0 +} + // Metadata represents the metadata associated with the given series' samples. type Metadata struct { Type Metadata_MetricType `protobuf:"varint,1,opt,name=type,proto3,enum=io.prometheus.write.v2.Metadata_MetricType" json:"type,omitempty"` @@ -498,12 +503,11 @@ func (m *Metadata) GetUnitRef() uint32 { return 0 } -// A native histogram, also known as a sparse histogram. -// Original design doc: -// https://docs.google.com/document/d/1cLNv3aufPZb3fNfaJgdaRBZsInZKKIHo9E6HinJVbpM/edit -// The appendix of this design doc also explains the concept of float -// histograms. This Histogram message can represent both, the usual -// integer histogram as well as a float histogram. +// A native histogram message, supporting +// * sparse exponential bucketing, custom bucketing. +// * float or integer histograms. +// +// See the full spec: https://prometheus.io/docs/specs/native_histograms/ type Histogram struct { // Types that are valid to be assigned to Count: // @@ -581,10 +585,27 @@ type Histogram struct { // // The last element is not only the upper inclusive bound of the last regular // bucket, but implicitly the lower exclusive bound of the +Inf bucket. - CustomValues []float64 `protobuf:"fixed64,16,rep,packed,name=custom_values,json=customValues,proto3" json:"custom_values,omitempty"` - XXX_NoUnkeyedLiteral struct{} `json:"-"` - XXX_unrecognized []byte `json:"-"` - XXX_sizecache int32 `json:"-"` + CustomValues []float64 `protobuf:"fixed64,16,rep,packed,name=custom_values,json=customValues,proto3" json:"custom_values,omitempty"` + // start_timestamp represents an optional start timestamp for the histogram sample, + // in ms format. The start timestamp represents the time when the histogram + // started counting, which can increase the accuracy of certain processing and + // query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + StartTimestamp int64 `protobuf:"varint,17,opt,name=start_timestamp,json=startTimestamp,proto3" json:"start_timestamp,omitempty"` + XXX_NoUnkeyedLiteral struct{} `json:"-"` + XXX_unrecognized []byte `json:"-"` + XXX_sizecache int32 `json:"-"` } func (m *Histogram) Reset() { *m = Histogram{} } @@ -774,6 +795,13 @@ func (m *Histogram) GetCustomValues() []float64 { return nil } +func (m *Histogram) GetStartTimestamp() int64 { + if m != nil { + return m.StartTimestamp + } + return 0 +} + // XXX_OneofWrappers is for the internal use of the proto package. func (*Histogram) XXX_OneofWrappers() []interface{} { return []interface{}{ @@ -861,65 +889,66 @@ func init() { } var fileDescriptor_f139519efd9fa8d7 = []byte{ - // 926 bytes of a gzipped FileDescriptorProto + // 931 bytes of a gzipped FileDescriptorProto 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x94, 0x55, 0x5d, 0x6f, 0xe3, 0x44, - 0x14, 0xed, 0xc4, 0x69, 0x3e, 0x6e, 0x9a, 0xac, 0x33, 0xb4, 0x5d, 0x6f, 0x81, 0x6c, 0xd6, 0x08, - 0x88, 0x58, 0x29, 0x91, 0xc2, 0xeb, 0x0a, 0xd4, 0xb4, 0x6e, 0x93, 0x95, 0x92, 0xac, 0x26, 0x2e, - 0x52, 0x79, 0xb1, 0xdc, 0x64, 0x92, 0x58, 0xd8, 0xb1, 0xf1, 0x4c, 0x02, 0xe5, 0xf7, 0xf1, 0xb0, - 0x8f, 0xfc, 0x01, 0x10, 0xf4, 0x9d, 0xff, 0x80, 0x66, 0xfc, 0xd9, 0x42, 0xbb, 0xe2, 0x6d, 0xe6, - 0xdc, 0x73, 0xee, 0x3d, 0xb9, 0xbe, 0x77, 0x02, 0xba, 0xe3, 0xf7, 0x82, 0xd0, 0xf7, 0x28, 0x5f, - 0xd3, 0x2d, 0xeb, 0xfd, 0x14, 0x3a, 0x9c, 0xf6, 0x76, 0xfd, 0x1e, 0xbf, 0x0d, 0x28, 0xeb, 0x06, - 0xa1, 0xcf, 0x7d, 0x7c, 0xec, 0xf8, 0xdd, 0x8c, 0xd3, 0x95, 0x9c, 0xee, 0xae, 0x7f, 0x72, 0xb8, - 0xf2, 0x57, 0xbe, 0xa4, 0xf4, 0xc4, 0x29, 0x62, 0xeb, 0x0c, 0xca, 0x84, 0xfe, 0xb8, 0xa5, 0x8c, - 0x63, 0x0d, 0xca, 0xec, 0xd6, 0xbb, 0xf1, 0x5d, 0xa6, 0x15, 0xdb, 0x4a, 0xa7, 0x4a, 0x92, 0x2b, - 0x1e, 0x02, 0x70, 0xc7, 0xa3, 0x8c, 0x86, 0x0e, 0x65, 0xda, 0x7e, 0x5b, 0xe9, 0xd4, 0xfa, 0x7a, - 0xf7, 0xbf, 0xeb, 0x74, 0x4d, 0xc7, 0xa3, 0x33, 0xc9, 0x1c, 0x14, 0xdf, 0xff, 0xf1, 0x72, 0x8f, - 0xe4, 0xb4, 0x6f, 0x8b, 0x15, 0xa4, 0x16, 0xf5, 0xbf, 0x0b, 0x00, 0x19, 0x0d, 0xbf, 0x84, 0x9a, - 0x6b, 0xdf, 0x50, 0x97, 0x59, 0x21, 0x5d, 0x32, 0x0d, 0xb5, 0x95, 0x4e, 0x9d, 0x40, 0x04, 0x11, - 0xba, 0x64, 0xf8, 0x1b, 0x28, 0x33, 0xdb, 0x0b, 0x5c, 0xca, 0xb4, 0x82, 0x2c, 0xde, 0x7a, 0xac, - 0xf8, 0x4c, 0xd2, 0xe2, 0xc2, 0x89, 0x08, 0x5f, 0x02, 0xac, 0x1d, 0xc6, 0xfd, 0x55, 0x68, 0x7b, - 0x4c, 0x53, 0x64, 0x8a, 0x57, 0x8f, 0xa5, 0x18, 0x26, 0xcc, 0xc4, 0x7e, 0x26, 0xc5, 0xe7, 0x50, - 0xa5, 0x3f, 0x53, 0x2f, 0x70, 0xed, 0x30, 0x6a, 0x52, 0xad, 0xdf, 0x7e, 0x2c, 0x8f, 0x11, 0x13, - 0xe3, 0x34, 0x99, 0x10, 0x0f, 0xa0, 0xe2, 0x51, 0x6e, 0x2f, 0x6c, 0x6e, 0x6b, 0xfb, 0x6d, 0xf4, - 0x54, 0x92, 0x71, 0xcc, 0x8b, 0x93, 0xa4, 0x3a, 0xfc, 0x1a, 0x9a, 0xf3, 0x90, 0xda, 0x9c, 0x2e, - 0x2c, 0xd9, 0x5e, 0x6e, 0x7b, 0x81, 0x56, 0x6a, 0xa3, 0x8e, 0x42, 0xd4, 0x38, 0x60, 0x26, 0xb8, - 0x6e, 0x41, 0x25, 0x71, 0xf3, 0xe1, 0x66, 0x1f, 0xc2, 0xfe, 0xce, 0x76, 0xb7, 0x54, 0x2b, 0xb4, - 0x51, 0x07, 0x91, 0xe8, 0x82, 0x3f, 0x81, 0x6a, 0x56, 0x47, 0x91, 0x75, 0x32, 0x40, 0x7f, 0x03, - 0xa5, 0xa8, 0xf3, 0x99, 0x1a, 0x3d, 0xaa, 0x2e, 0x3c, 0x54, 0xff, 0x55, 0x80, 0x4a, 0xf2, 0x43, - 0xf1, 0xb7, 0x50, 0x14, 0xd3, 0x2c, 0xf5, 0x8d, 0xfe, 0xeb, 0x0f, 0x35, 0x46, 0x1c, 0x42, 0x67, - 0x6e, 0xde, 0x06, 0x94, 0x48, 0x21, 0x7e, 0x01, 0x95, 0x35, 0x75, 0x03, 0xf1, 0xf3, 0xa4, 0xd1, - 0x3a, 0x29, 0x8b, 0x3b, 0xa1, 0x4b, 0x11, 0xda, 0x6e, 0x1c, 0x2e, 0x43, 0xc5, 0x28, 0x24, 0xee, - 0x84, 0x2e, 0xf5, 0xdf, 0x11, 0x40, 0x96, 0x0a, 0x7f, 0x0c, 0xcf, 0xc7, 0x86, 0x49, 0x46, 0x67, - 0x96, 0x79, 0xfd, 0xce, 0xb0, 0xae, 0x26, 0xb3, 0x77, 0xc6, 0xd9, 0xe8, 0x62, 0x64, 0x9c, 0xab, - 0x7b, 0xf8, 0x39, 0x7c, 0x94, 0x0f, 0x9e, 0x4d, 0xaf, 0x26, 0xa6, 0x41, 0x54, 0x84, 0x8f, 0xa0, - 0x99, 0x0f, 0x5c, 0x9e, 0x5e, 0x5d, 0x1a, 0x6a, 0x01, 0xbf, 0x80, 0xa3, 0x3c, 0x3c, 0x1c, 0xcd, - 0xcc, 0xe9, 0x25, 0x39, 0x1d, 0xab, 0x0a, 0x6e, 0xc1, 0xc9, 0xbf, 0x14, 0x59, 0xbc, 0xf8, 0xb0, - 0xd4, 0xec, 0x6a, 0x3c, 0x3e, 0x25, 0xd7, 0xea, 0x3e, 0x3e, 0x04, 0x35, 0x1f, 0x18, 0x4d, 0x2e, - 0xa6, 0x6a, 0x09, 0x6b, 0x70, 0x78, 0x8f, 0x6e, 0x9e, 0x9a, 0xc6, 0xcc, 0x30, 0xd5, 0xb2, 0xfe, - 0x6b, 0x09, 0xaa, 0xe9, 0x64, 0xe3, 0x4f, 0xa1, 0x3a, 0xf7, 0xb7, 0x1b, 0x6e, 0x39, 0x1b, 0x2e, - 0x3b, 0x5d, 0x1c, 0xee, 0x91, 0x8a, 0x84, 0x46, 0x1b, 0x8e, 0x5f, 0x41, 0x2d, 0x0a, 0x2f, 0x5d, - 0xdf, 0xe6, 0xd1, 0x20, 0x0c, 0xf7, 0x08, 0x48, 0xf0, 0x42, 0x60, 0x58, 0x05, 0x85, 0x6d, 0x3d, - 0xd9, 0x60, 0x44, 0xc4, 0x11, 0x1f, 0x43, 0x89, 0xcd, 0xd7, 0xd4, 0xb3, 0x65, 0x6b, 0x9b, 0x24, - 0xbe, 0xe1, 0xcf, 0xa1, 0xf1, 0x0b, 0x0d, 0x7d, 0x8b, 0xaf, 0x43, 0xca, 0xd6, 0xbe, 0xbb, 0x90, - 0x33, 0x8f, 0x48, 0x5d, 0xa0, 0x66, 0x02, 0xe2, 0x2f, 0x62, 0x5a, 0xe6, 0xab, 0x24, 0x7d, 0x21, - 0x72, 0x20, 0xf0, 0xb3, 0xc4, 0xdb, 0x57, 0xa0, 0xe6, 0x78, 0x91, 0xc1, 0xb2, 0x34, 0x88, 0x48, - 0x23, 0x65, 0x46, 0x26, 0xa7, 0xd0, 0xd8, 0xd0, 0x95, 0xcd, 0x9d, 0x1d, 0xb5, 0x58, 0x60, 0x6f, - 0x98, 0x56, 0x79, 0xfa, 0xed, 0x1a, 0x6c, 0xe7, 0x3f, 0x50, 0x3e, 0x0b, 0xec, 0x4d, 0xbc, 0x70, - 0xf5, 0x44, 0x2f, 0x30, 0x86, 0xbf, 0x84, 0x67, 0x69, 0xc2, 0x05, 0x75, 0xb9, 0xcd, 0xb4, 0x6a, - 0x5b, 0xe9, 0x60, 0x92, 0xd6, 0x39, 0x97, 0xe8, 0x3d, 0xa2, 0x74, 0xca, 0x34, 0x68, 0x2b, 0x1d, - 0x94, 0x11, 0xa5, 0x4d, 0x26, 0x2c, 0x06, 0x3e, 0x73, 0x72, 0x16, 0x6b, 0xff, 0xd7, 0x62, 0xa2, - 0x4f, 0x2d, 0xa6, 0x09, 0x63, 0x8b, 0x07, 0x91, 0xc5, 0x04, 0xce, 0x2c, 0xa6, 0xc4, 0xd8, 0x62, - 0x3d, 0xb2, 0x98, 0xc0, 0xb1, 0xc5, 0xb7, 0x00, 0x21, 0x65, 0x94, 0x5b, 0x6b, 0xf1, 0x55, 0x1a, - 0x4f, 0xef, 0x65, 0x3a, 0x63, 0x5d, 0x22, 0x34, 0x43, 0x67, 0xc3, 0x49, 0x35, 0x4c, 0x8e, 0xf7, - 0x1f, 0x82, 0x67, 0x0f, 0x1e, 0x02, 0xfc, 0x19, 0xd4, 0xe7, 0x5b, 0xc6, 0x7d, 0xcf, 0x92, 0xcf, - 0x06, 0xd3, 0x54, 0x69, 0xe8, 0x20, 0x02, 0xbf, 0x93, 0x98, 0xbe, 0x80, 0x6a, 0x9a, 0x1a, 0x9f, - 0xc0, 0x31, 0x11, 0x13, 0x6e, 0x0d, 0x47, 0x13, 0xf3, 0xc1, 0x9a, 0x62, 0x68, 0xe4, 0x62, 0xd7, - 0xc6, 0x4c, 0x45, 0xb8, 0x09, 0xf5, 0x1c, 0x36, 0x99, 0xaa, 0x05, 0xb1, 0x49, 0x39, 0x28, 0xda, - 0x59, 0x65, 0x50, 0x86, 0x7d, 0xd9, 0x94, 0xc1, 0x01, 0x40, 0x36, 0x6f, 0xfa, 0x1b, 0x80, 0xec, - 0x03, 0x88, 0x91, 0xf7, 0x97, 0x4b, 0x46, 0xa3, 0x1d, 0x6a, 0x92, 0xf8, 0x26, 0x70, 0x97, 0x6e, - 0x56, 0x7c, 0x2d, 0x57, 0xa7, 0x4e, 0xe2, 0xdb, 0xe0, 0xe8, 0xfd, 0x5d, 0x0b, 0xfd, 0x76, 0xd7, - 0x42, 0x7f, 0xde, 0xb5, 0xd0, 0xf7, 0x65, 0xd9, 0xb4, 0x5d, 0xff, 0xa6, 0x24, 0xff, 0x8a, 0xbf, - 0xfe, 0x27, 0x00, 0x00, 0xff, 0xff, 0x3e, 0xfc, 0x93, 0x1c, 0xde, 0x07, 0x00, 0x00, + 0x14, 0xed, 0xc4, 0xf9, 0xbc, 0x69, 0xb2, 0xce, 0xd0, 0x76, 0xbd, 0x05, 0xb2, 0xd9, 0x20, 0x20, + 0x02, 0x29, 0x91, 0xc2, 0x2b, 0x02, 0x35, 0xad, 0xdb, 0xa4, 0x52, 0x92, 0xd5, 0xc4, 0x45, 0x2a, + 0x2f, 0x96, 0x9b, 0x4e, 0x12, 0x0b, 0x3b, 0x36, 0x9e, 0x49, 0xa0, 0xfc, 0x40, 0xb4, 0x8f, 0xfc, + 0x01, 0x10, 0xf4, 0x9d, 0xff, 0x80, 0x66, 0xfc, 0xd9, 0xd0, 0x76, 0xb5, 0x6f, 0x33, 0xe7, 0x9e, + 0x73, 0xef, 0xc9, 0xf5, 0xbd, 0x13, 0x68, 0xdb, 0x5e, 0xcf, 0x0f, 0x3c, 0x97, 0xf2, 0x15, 0xdd, + 0xb0, 0xde, 0x2f, 0x81, 0xcd, 0x69, 0x6f, 0xdb, 0xef, 0xf1, 0x3b, 0x9f, 0xb2, 0xae, 0x1f, 0x78, + 0xdc, 0xc3, 0x47, 0xb6, 0xd7, 0x4d, 0x39, 0x5d, 0xc9, 0xe9, 0x6e, 0xfb, 0xc7, 0x07, 0x4b, 0x6f, + 0xe9, 0x49, 0x4a, 0x4f, 0x9c, 0x42, 0x76, 0x9b, 0x41, 0x89, 0xd0, 0x9f, 0x37, 0x94, 0x71, 0xac, + 0x41, 0x89, 0xdd, 0xb9, 0x37, 0x9e, 0xc3, 0xb4, 0x7c, 0x4b, 0xe9, 0x54, 0x48, 0x7c, 0xc5, 0x43, + 0x00, 0x6e, 0xbb, 0x94, 0xd1, 0xc0, 0xa6, 0x4c, 0x2b, 0xb4, 0x94, 0x4e, 0xb5, 0xdf, 0xee, 0x3e, + 0x5e, 0xa7, 0x6b, 0xd8, 0x2e, 0x9d, 0x49, 0xe6, 0x20, 0xff, 0xee, 0xaf, 0xd7, 0x7b, 0x24, 0xa3, + 0xbd, 0xcc, 0x97, 0x91, 0x9a, 0x6f, 0xff, 0x9e, 0x03, 0x48, 0x69, 0xf8, 0x35, 0x54, 0x1d, 0xeb, + 0x86, 0x3a, 0xcc, 0x0c, 0xe8, 0x82, 0x69, 0xa8, 0xa5, 0x74, 0x6a, 0x04, 0x42, 0x88, 0xd0, 0x05, + 0xc3, 0xdf, 0x41, 0x89, 0x59, 0xae, 0xef, 0x50, 0xa6, 0xe5, 0x64, 0xf1, 0xe6, 0x53, 0xc5, 0x67, + 0x92, 0x16, 0x15, 0x8e, 0x45, 0xf8, 0x02, 0x60, 0x65, 0x33, 0xee, 0x2d, 0x03, 0xcb, 0x65, 0x9a, + 0x22, 0x53, 0xbc, 0x79, 0x2a, 0xc5, 0x30, 0x66, 0xc6, 0xf6, 0x53, 0x29, 0x3e, 0x83, 0x0a, 0xfd, + 0x95, 0xba, 0xbe, 0x63, 0x05, 0x61, 0x93, 0xaa, 0xfd, 0xd6, 0x53, 0x79, 0xf4, 0x88, 0x18, 0xa5, + 0x49, 0x85, 0x78, 0x00, 0x65, 0x97, 0x72, 0xeb, 0xd6, 0xe2, 0x96, 0x56, 0x68, 0xa1, 0xe7, 0x92, + 0x8c, 0x23, 0x5e, 0x94, 0x24, 0xd1, 0x5d, 0xe6, 0xcb, 0x45, 0xb5, 0xd4, 0x36, 0xa1, 0x1c, 0x97, + 0x79, 0x7f, 0x17, 0x0f, 0xa0, 0xb0, 0xb5, 0x9c, 0x0d, 0xd5, 0x72, 0x2d, 0xd4, 0x41, 0x24, 0xbc, + 0xe0, 0x4f, 0xa0, 0x22, 0xbf, 0x0f, 0xb7, 0x5c, 0x5f, 0x53, 0x5a, 0xa8, 0xa3, 0x90, 0x14, 0x68, + 0x53, 0x28, 0x86, 0x2d, 0x4d, 0xd5, 0xe8, 0x49, 0x75, 0x6e, 0x47, 0x8d, 0xbf, 0x84, 0x17, 0x8c, + 0x5b, 0x01, 0x37, 0x77, 0x2b, 0xd4, 0x25, 0x6c, 0x24, 0x65, 0xfe, 0xc9, 0x41, 0x39, 0xfe, 0xa9, + 0xf8, 0x7b, 0xc8, 0x8b, 0x79, 0x96, 0x85, 0xea, 0xfd, 0xaf, 0xdf, 0xd7, 0x1a, 0x71, 0x08, 0xec, + 0xb9, 0x71, 0xe7, 0x53, 0x22, 0x85, 0xf8, 0x15, 0x94, 0x57, 0xd4, 0xf1, 0x45, 0x1f, 0x64, 0xbd, + 0x1a, 0x29, 0x89, 0x3b, 0xa1, 0x0b, 0x11, 0xda, 0xac, 0x6d, 0x2e, 0x43, 0xf9, 0x30, 0x24, 0xee, + 0x84, 0x2e, 0xda, 0x7f, 0x22, 0x80, 0x34, 0x15, 0xfe, 0x18, 0x5e, 0x8e, 0x75, 0x83, 0x8c, 0x4e, + 0x4d, 0xe3, 0xfa, 0xad, 0x6e, 0x5e, 0x4d, 0x66, 0x6f, 0xf5, 0xd3, 0xd1, 0xf9, 0x48, 0x3f, 0x53, + 0xf7, 0xf0, 0x4b, 0xf8, 0x28, 0x1b, 0x3c, 0x9d, 0x5e, 0x4d, 0x0c, 0x9d, 0xa8, 0x08, 0x1f, 0x42, + 0x23, 0x1b, 0xb8, 0x38, 0xb9, 0xba, 0xd0, 0xd5, 0x1c, 0x7e, 0x05, 0x87, 0x59, 0x78, 0x38, 0x9a, + 0x19, 0xd3, 0x0b, 0x72, 0x32, 0x56, 0x15, 0xdc, 0x84, 0xe3, 0xff, 0x29, 0xd2, 0x78, 0x7e, 0xb7, + 0xd4, 0xec, 0x6a, 0x3c, 0x3e, 0x21, 0xd7, 0x6a, 0x01, 0x1f, 0x80, 0x9a, 0x0d, 0x8c, 0x26, 0xe7, + 0x53, 0xb5, 0x88, 0x35, 0x38, 0x78, 0x40, 0x37, 0x4e, 0x0c, 0x7d, 0xa6, 0x1b, 0x6a, 0xa9, 0xfd, + 0x6f, 0x11, 0x2a, 0xc9, 0x6c, 0xe3, 0x4f, 0xa1, 0x32, 0xf7, 0x36, 0x6b, 0x6e, 0xda, 0x6b, 0x2e, + 0x3b, 0x9d, 0x1f, 0xee, 0x91, 0xb2, 0x84, 0x46, 0x6b, 0x8e, 0xdf, 0x40, 0x35, 0x0c, 0x2f, 0x1c, + 0xcf, 0xe2, 0xe1, 0xc4, 0x0c, 0xf7, 0x08, 0x48, 0xf0, 0x5c, 0x60, 0x58, 0x05, 0x85, 0x6d, 0x5c, + 0xd9, 0x60, 0x44, 0xc4, 0x11, 0x1f, 0x41, 0x91, 0xcd, 0x57, 0xd4, 0xb5, 0x64, 0x6b, 0x1b, 0x24, + 0xba, 0xe1, 0xcf, 0xa1, 0xfe, 0x1b, 0x0d, 0x3c, 0x93, 0xaf, 0x02, 0xca, 0x56, 0x9e, 0x73, 0x2b, + 0xa7, 0x1e, 0x91, 0x9a, 0x40, 0x8d, 0x18, 0xc4, 0x5f, 0x44, 0xb4, 0xd4, 0x57, 0x51, 0xfa, 0x42, + 0x64, 0x5f, 0xe0, 0xa7, 0xb1, 0xb7, 0xaf, 0x40, 0xcd, 0xf0, 0x42, 0x83, 0x25, 0x69, 0x10, 0x91, + 0x7a, 0xc2, 0x0c, 0x4d, 0x4e, 0xa1, 0xbe, 0xa6, 0x4b, 0x8b, 0xdb, 0x5b, 0x6a, 0x32, 0xdf, 0x5a, + 0x33, 0xad, 0xfc, 0xfc, 0xeb, 0x35, 0xd8, 0xcc, 0x7f, 0xa2, 0x7c, 0xe6, 0x5b, 0xeb, 0x68, 0xe5, + 0x6a, 0xb1, 0x5e, 0x60, 0x4c, 0x8c, 0x74, 0x92, 0xf0, 0x96, 0x3a, 0xdc, 0x62, 0x5a, 0xa5, 0xa5, + 0x74, 0x30, 0x49, 0xea, 0x9c, 0x49, 0xf4, 0x01, 0x51, 0x3a, 0x65, 0x1a, 0xb4, 0x94, 0x0e, 0x4a, + 0x89, 0xd2, 0x26, 0x13, 0x16, 0x7d, 0x8f, 0xd9, 0x19, 0x8b, 0xd5, 0x0f, 0xb5, 0x18, 0xeb, 0x13, + 0x8b, 0x49, 0xc2, 0xc8, 0xe2, 0x7e, 0x68, 0x31, 0x86, 0x53, 0x8b, 0x09, 0x31, 0xb2, 0x58, 0x0b, + 0x2d, 0xc6, 0x70, 0x64, 0xf1, 0x12, 0x20, 0xa0, 0x8c, 0x72, 0x73, 0x25, 0xbe, 0x4a, 0xfd, 0xf9, + 0xbd, 0x4c, 0x66, 0xac, 0x4b, 0x84, 0x66, 0x68, 0xaf, 0x39, 0xa9, 0x04, 0xf1, 0xf1, 0xe1, 0x8b, + 0xf1, 0x62, 0xf7, 0xc5, 0xf8, 0x0c, 0x6a, 0xf3, 0x0d, 0xe3, 0x9e, 0x6b, 0xca, 0xf7, 0x85, 0x69, + 0xaa, 0x34, 0xb4, 0x1f, 0x82, 0x3f, 0x48, 0xec, 0xb1, 0x67, 0xa5, 0xf1, 0xe8, 0xb3, 0x72, 0x0b, + 0x95, 0xc4, 0x03, 0x3e, 0x86, 0x23, 0x22, 0x56, 0xc1, 0x1c, 0x8e, 0x26, 0xc6, 0xce, 0x3e, 0x63, + 0xa8, 0x67, 0x62, 0xd7, 0xfa, 0x4c, 0x45, 0xb8, 0x01, 0xb5, 0x0c, 0x36, 0x99, 0xaa, 0x39, 0xb1, + 0x72, 0x19, 0x28, 0x5c, 0x6e, 0x65, 0x50, 0x82, 0x82, 0xec, 0xde, 0x60, 0x1f, 0x20, 0x1d, 0xcc, + 0xf6, 0xb7, 0x00, 0xe9, 0x97, 0x12, 0xbb, 0xe1, 0x2d, 0x16, 0x8c, 0x86, 0xcb, 0xd6, 0x20, 0xd1, + 0x4d, 0xe0, 0x0e, 0x5d, 0x2f, 0xf9, 0x4a, 0xee, 0x58, 0x8d, 0x44, 0xb7, 0xc1, 0xe1, 0xbb, 0xfb, + 0x26, 0xfa, 0xe3, 0xbe, 0x89, 0xfe, 0xbe, 0x6f, 0xa2, 0x1f, 0x4b, 0xb2, 0xbb, 0xdb, 0xfe, 0x4d, + 0x51, 0xfe, 0x6b, 0x7f, 0xf3, 0x5f, 0x00, 0x00, 0x00, 0xff, 0xff, 0x62, 0x8f, 0x36, 0x4b, 0x09, + 0x08, 0x00, 0x00, } func (m *Request) Marshal() (dAtA []byte, err error) { @@ -996,11 +1025,6 @@ func (m *TimeSeries) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } - if m.CreatedTimestamp != 0 { - i = encodeVarintTypes(dAtA, i, uint64(m.CreatedTimestamp)) - i-- - dAtA[i] = 0x30 - } { size, err := m.Metadata.MarshalToSizedBuffer(dAtA[:i]) if err != nil { @@ -1154,6 +1178,11 @@ func (m *Sample) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.StartTimestamp != 0 { + i = encodeVarintTypes(dAtA, i, uint64(m.StartTimestamp)) + i-- + dAtA[i] = 0x18 + } if m.Timestamp != 0 { i = encodeVarintTypes(dAtA, i, uint64(m.Timestamp)) i-- @@ -1234,6 +1263,13 @@ func (m *Histogram) MarshalToSizedBuffer(dAtA []byte) (int, error) { i -= len(m.XXX_unrecognized) copy(dAtA[i:], m.XXX_unrecognized) } + if m.StartTimestamp != 0 { + i = encodeVarintTypes(dAtA, i, uint64(m.StartTimestamp)) + i-- + dAtA[i] = 0x1 + i-- + dAtA[i] = 0x88 + } if len(m.CustomValues) > 0 { for iNdEx := len(m.CustomValues) - 1; iNdEx >= 0; iNdEx-- { f6 := math.Float64bits(float64(m.CustomValues[iNdEx])) @@ -1535,9 +1571,6 @@ func (m *TimeSeries) Size() (n int) { } l = m.Metadata.Size() n += 1 + l + sovTypes(uint64(l)) - if m.CreatedTimestamp != 0 { - n += 1 + sovTypes(uint64(m.CreatedTimestamp)) - } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -1581,6 +1614,9 @@ func (m *Sample) Size() (n int) { if m.Timestamp != 0 { n += 1 + sovTypes(uint64(m.Timestamp)) } + if m.StartTimestamp != 0 { + n += 1 + sovTypes(uint64(m.StartTimestamp)) + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -1670,6 +1706,9 @@ func (m *Histogram) Size() (n int) { if len(m.CustomValues) > 0 { n += 2 + sovTypes(uint64(len(m.CustomValues)*8)) + len(m.CustomValues)*8 } + if m.StartTimestamp != 0 { + n += 2 + sovTypes(uint64(m.StartTimestamp)) + } if m.XXX_unrecognized != nil { n += len(m.XXX_unrecognized) } @@ -2093,25 +2132,6 @@ func (m *TimeSeries) Unmarshal(dAtA []byte) error { return err } iNdEx = postIndex - case 6: - if wireType != 0 { - return fmt.Errorf("proto: wrong wireType = %d for field CreatedTimestamp", wireType) - } - m.CreatedTimestamp = 0 - for shift := uint(0); ; shift += 7 { - if shift >= 64 { - return ErrIntOverflowTypes - } - if iNdEx >= l { - return io.ErrUnexpectedEOF - } - b := dAtA[iNdEx] - iNdEx++ - m.CreatedTimestamp |= int64(b&0x7F) << shift - if b < 0x80 { - break - } - } default: iNdEx = preIndex skippy, err := skipTypes(dAtA[iNdEx:]) @@ -2350,6 +2370,25 @@ func (m *Sample) Unmarshal(dAtA []byte) error { break } } + case 3: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field StartTimestamp", wireType) + } + m.StartTimestamp = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowTypes + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.StartTimestamp |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipTypes(dAtA[iNdEx:]) @@ -3038,6 +3077,25 @@ func (m *Histogram) Unmarshal(dAtA []byte) error { } else { return fmt.Errorf("proto: wrong wireType = %d for field CustomValues", wireType) } + case 17: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field StartTimestamp", wireType) + } + m.StartTimestamp = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowTypes + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.StartTimestamp |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipTypes(dAtA[iNdEx:]) diff --git a/prompb/io/prometheus/write/v2/types.proto b/prompb/io/prometheus/write/v2/types.proto index ff6c4936bb..c1ae04d206 100644 --- a/prompb/io/prometheus/write/v2/types.proto +++ b/prompb/io/prometheus/write/v2/types.proto @@ -14,6 +14,7 @@ // NOTE: This file is also available on https://buf.build/prometheus/prometheus/docs/main:io.prometheus.write.v2 syntax = "proto3"; + package io.prometheus.write.v2; option go_package = "writev2"; @@ -27,6 +28,8 @@ import "gogoproto/gogo.proto"; // The canonical Content-Type request header value for this message is // "application/x-protobuf;proto=io.prometheus.write.v2.Request" // +// Version: v2.0-rc.4 +// // NOTE: gogoproto options might change in future for this file, they // are not part of the spec proto (they only modify the generated Go code, not // the serialized message). See: https://github.com/prometheus/prometheus/issues/11908 @@ -59,7 +62,7 @@ message TimeSeries { // // Note that there might be multiple TimeSeries objects in the same // Requests with the same labels e.g. for different exemplars, metadata - // or created timestamp. + // or start timestamp. repeated uint32 labels_refs = 1; // Timeseries messages can either specify samples or (native) histogram samples @@ -76,23 +79,9 @@ message TimeSeries { // metadata represents the metadata associated with the given series' samples. Metadata metadata = 5 [(gogoproto.nullable) = false]; - // created_timestamp represents an optional created timestamp associated with - // this series' samples in ms format, typically for counter or histogram type - // metrics. Created timestamp represents the time when the counter started - // counting (sometimes referred to as start timestamp), which can increase - // the accuracy of query results. - // - // Note that some receivers might require this and in return fail to - // ingest such samples within the Request. - // - // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go - // for conversion from/to time.Time to Prometheus timestamp. - // - // Note that the "optional" keyword is omitted due to - // https://cloud.google.com/apis/design/design_patterns.md#optional_primitive_fields - // Zero value means value not set. If you need to use exactly zero value for - // the timestamp, use 1 millisecond before or after. - int64 created_timestamp = 6; + // This field is reserved for backward compatibility with the deprecated fields; + // previously present in the experimental remote write period. + reserved 6; } // Exemplar is an additional information attached to some series' samples. @@ -123,6 +112,26 @@ message Sample { // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go // for conversion from/to time.Time to Prometheus timestamp. int64 timestamp = 2; + // start_timestamp represents an optional start timestamp for the sample, + // in ms format. This information is typically used for counter, histogram (cumulative) + // or delta type metrics. + // + // For cumulative metrics, the start timestamp represents the time when the + // counter started counting (sometimes referred to as start timestamp), which + // can increase the accuracy of certain processing and query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + int64 start_timestamp = 3; } // Metadata represents the metadata associated with the given series' samples. @@ -148,12 +157,11 @@ message Metadata { uint32 unit_ref = 4; } -// A native histogram, also known as a sparse histogram. -// Original design doc: -// https://docs.google.com/document/d/1cLNv3aufPZb3fNfaJgdaRBZsInZKKIHo9E6HinJVbpM/edit -// The appendix of this design doc also explains the concept of float -// histograms. This Histogram message can represent both, the usual -// integer histogram as well as a float histogram. +// A native histogram message, supporting +// * sparse exponential bucketing, custom bucketing. +// * float or integer histograms. +// +// See the full spec: https://prometheus.io/docs/specs/native_histograms/ message Histogram { enum ResetHint { RESET_HINT_UNSPECIFIED = 0; // Need to test for a counter reset explicitly. @@ -242,6 +250,24 @@ message Histogram { // The last element is not only the upper inclusive bound of the last regular // bucket, but implicitly the lower exclusive bound of the +Inf bucket. repeated double custom_values = 16; + + // start_timestamp represents an optional start timestamp for the histogram sample, + // in ms format. The start timestamp represents the time when the histogram + // started counting, which can increase the accuracy of certain processing and + // query semantics (e.g. rates). + // + // Note: + // * That some receivers might require start timestamps for certain metric + // types; rejecting such samples within the Request as a result. + // * start timestamp is the same as "created timestamp" name Prometheus used in the past. + // + // For Go, see github.com/prometheus/prometheus/model/timestamp/timestamp.go + // for conversion from/to time.Time to Prometheus timestamp. + // + // Note that the "optional" keyword is omitted due to efficiency and consistency. + // Zero value means value not set. If you need to use exactly zero value for + // the timestamp, use 1 millisecond before or after. + int64 start_timestamp = 17; } // A BucketSpan defines a number of consecutive buckets with their diff --git a/storage/remote/codec_test.go b/storage/remote/codec_test.go index ddf8f76cf6..ce3a09b878 100644 --- a/storage/remote/codec_test.go +++ b/storage/remote/codec_test.go @@ -114,7 +114,7 @@ var ( HelpRef: 15, // Symbolized writeV2RequestSeries1Metadata.Help. UnitRef: 16, // Symbolized writeV2RequestSeries1Metadata.Unit. }, - Samples: []writev2.Sample{{Value: 1, Timestamp: 10}}, + Samples: []writev2.Sample{{Value: 1, Timestamp: 10, StartTimestamp: 1}}, // ST needs to be lower than the sample's timestamp. Exemplars: []writev2.Exemplar{{LabelsRefs: []uint32{11, 12}, Value: 1, Timestamp: 10}}, Histograms: []writev2.Histogram{ writev2.FromIntHistogram(10, &testHistogram), @@ -122,7 +122,6 @@ var ( writev2.FromIntHistogram(30, &testHistogramCustomBuckets), writev2.FromFloatHistogram(40, testHistogramCustomBuckets.ToFloat(nil)), }, - CreatedTimestamp: 1, // CT needs to be lower than the sample's timestamp. }, { LabelsRefs: []uint32{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, // Same series as first. @@ -182,7 +181,7 @@ func TestWriteV2RequestFixture(t *testing.T) { HelpRef: st.Symbolize(writeV2RequestSeries1Metadata.Help), UnitRef: st.Symbolize(writeV2RequestSeries1Metadata.Unit), }, - Samples: []writev2.Sample{{Value: 1, Timestamp: 10}}, + Samples: []writev2.Sample{{Value: 1, Timestamp: 10, StartTimestamp: 1}}, Exemplars: []writev2.Exemplar{{LabelsRefs: exemplar1LabelRefs, Value: 1, Timestamp: 10}}, Histograms: []writev2.Histogram{ writev2.FromIntHistogram(10, &testHistogram), @@ -190,7 +189,6 @@ func TestWriteV2RequestFixture(t *testing.T) { writev2.FromIntHistogram(30, &testHistogramCustomBuckets), writev2.FromFloatHistogram(40, testHistogramCustomBuckets.ToFloat(nil)), }, - CreatedTimestamp: 1, }, { LabelsRefs: labelRefs, diff --git a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go index 0bc8a876e4..5606fa4d91 100644 --- a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go +++ b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go @@ -106,7 +106,7 @@ func exponentialToNativeHistogram(p pmetric.ExponentialHistogramDataPoint, tempo // Sending a sample that triggers counter reset but with ResetHint==NO // would lead to Prometheus panic as it does not double check the hint. // Thus we're explicitly saying UNKNOWN here, which is always safe. - // TODO: using created time stamp should be accurate, but we + // TODO: using start timestamp should be accurate, but we // need to know here if it was used for the detection. // Ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28663#issuecomment-1810577303 // Counter reset detection in Prometheus: https://github.com/prometheus/prometheus/blob/f997c72f294c0f18ca13fa06d51889af04135195/tsdb/chunkenc/histogram.go#L232 @@ -312,7 +312,7 @@ func explicitHistogramToCustomBucketsHistogram(p pmetric.HistogramDataPoint, tem // Sending a sample that triggers counter reset but with ResetHint==NO // would lead to Prometheus panic as it does not double check the hint. // Thus we're explicitly saying UNKNOWN here, which is always safe. - // TODO: using created time stamp should be accurate, but we + // TODO: using start timestamp should be accurate, but we // need to know here if it was used for the detection. // Ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28663#issuecomment-1810577303 // Counter reset detection in Prometheus: https://github.com/prometheus/prometheus/blob/f997c72f294c0f18ca13fa06d51889af04135195/tsdb/chunkenc/histogram.go#L232 diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index e8559dd00e..f8296b4a80 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -353,20 +353,18 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * allSamplesSoFar := rs.AllSamples() var ref storage.SeriesRef - - // Samples. - if h.ingestCTZeroSample && len(ts.Samples) > 0 && ts.Samples[0].Timestamp != 0 && ts.CreatedTimestamp != 0 { - // CT only needs to be ingested for the first sample, it will be considered - // out of order for the rest. - ref, err = app.AppendCTZeroSample(ref, ls, ts.Samples[0].Timestamp, ts.CreatedTimestamp) - if err != nil && !errors.Is(err, storage.ErrOutOfOrderCT) { - // Even for the first sample OOO is a common scenario because - // we can't tell if a CT was already ingested in a previous request. - // We ignore the error. - h.logger.Debug("Error when appending CT in remote write request", "err", err, "series", ls.String(), "created_timestamp", ts.CreatedTimestamp, "timestamp", ts.Samples[0].Timestamp) - } - } for _, s := range ts.Samples { + if h.ingestCTZeroSample && s.StartTimestamp != 0 && s.Timestamp != 0 { + ref, err = app.AppendCTZeroSample(ref, ls, s.Timestamp, s.StartTimestamp) + // We treat OOO errors specially as it's a common scenario given: + // * We can't tell if ST was already ingested in a previous request. + // * We don't check if ST changed for stream of samples (we typically have one though), + // as it's checked in the AppendSTZeroSample reliably. + if err != nil && !errors.Is(err, storage.ErrOutOfOrderCT) { + h.logger.Debug("Error when appending ST from remote write request", "err", err, "series", ls.String(), "start_timestamp", s.StartTimestamp, "timestamp", s.Timestamp) + } + } + ref, err = app.Append(ref, ls, s.GetTimestamp(), s.GetValue()) if err == nil { rs.Samples++ @@ -387,15 +385,14 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * // Native Histograms. for _, hp := range ts.Histograms { - if h.ingestCTZeroSample && hp.Timestamp != 0 && ts.CreatedTimestamp != 0 { - // Differently from samples, we need to handle CT for each histogram instead of just the first one. - // This is because histograms and float histograms are stored separately, even if they have the same labels. - ref, err = h.handleHistogramZeroSample(app, ref, ls, hp, ts.CreatedTimestamp) + if h.ingestCTZeroSample && hp.StartTimestamp != 0 && hp.Timestamp != 0 { + ref, err = h.handleHistogramZeroSample(app, ref, ls, hp, hp.StartTimestamp) + // We treat OOO errors specially as it's a common scenario given: + // * We can't tell if ST was already ingested in a previous request. + // * We don't check if ST changed for stream of samples (we typically have one though), + // as it's checked in the ingestSTZeroSample reliably. if err != nil && !errors.Is(err, storage.ErrOutOfOrderCT) { - // Even for the first sample OOO is a common scenario because - // we can't tell if a CT was already ingested in a previous request. - // We ignore the error. - h.logger.Debug("Error when appending CT in remote write request", "err", err, "series", ls.String(), "created_timestamp", ts.CreatedTimestamp, "timestamp", hp.Timestamp) + h.logger.Debug("Error when appending ST from remote write request", "err", err, "series", ls.String(), "start_timestamp", hp.StartTimestamp, "timestamp", hp.Timestamp) } } if hp.IsFloatHistogram() { @@ -474,14 +471,14 @@ func (h *writeHandler) appendV2(app storage.Appender, req *writev2.Request, rs * return samplesWithoutMetadata, http.StatusBadRequest, errors.Join(badRequestErrs...) } -// handleHistogramZeroSample appends CT as a zero-value sample with CT value as the sample timestamp. -// It doesn't return errors in case of out of order CT. -func (*writeHandler) handleHistogramZeroSample(app storage.Appender, ref storage.SeriesRef, l labels.Labels, hist writev2.Histogram, ct int64) (storage.SeriesRef, error) { +// handleHistogramZeroSample appends ST as a zero-value sample with st value as the sample timestamp. +// It doesn't return errors in case of out of order ST. +func (*writeHandler) handleHistogramZeroSample(app storage.Appender, ref storage.SeriesRef, l labels.Labels, hist writev2.Histogram, st int64) (storage.SeriesRef, error) { var err error if hist.IsFloatHistogram() { - ref, err = app.AppendHistogramCTZeroSample(ref, l, hist.Timestamp, ct, nil, hist.ToFloatHistogram()) + ref, err = app.AppendHistogramCTZeroSample(ref, l, hist.Timestamp, st, nil, hist.ToFloatHistogram()) } else { - ref, err = app.AppendHistogramCTZeroSample(ref, l, hist.Timestamp, ct, hist.ToIntHistogram(), nil) + ref, err = app.AppendHistogramCTZeroSample(ref, l, hist.Timestamp, st, hist.ToIntHistogram(), nil) } return ref, err } diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go index 536fba63cd..f1c064c64d 100644 --- a/storage/remote/write_handler_test.go +++ b/storage/remote/write_handler_test.go @@ -752,14 +752,12 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { i, j, k, m int ) for _, ts := range writeV2RequestFixture.Timeseries { - zeroHistogramIngested := false - zeroFloatHistogramIngested := false ls, err := ts.ToLabels(&b, writeV2RequestFixture.Symbols) require.NoError(t, err) for _, s := range ts.Samples { - if ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { - requireEqual(t, mockSample{ls, ts.CreatedTimestamp, 0}, appendable.samples[i]) + if s.StartTimestamp != 0 && tc.ingestCTZeroSample { + requireEqual(t, mockSample{ls, s.StartTimestamp, 0}, appendable.samples[i]) i++ } requireEqual(t, mockSample{ls, s.Timestamp, s.Value}, appendable.samples[i]) @@ -768,27 +766,21 @@ func TestRemoteWriteHandler_V2Message(t *testing.T) { for _, hp := range ts.Histograms { if hp.IsFloatHistogram() { fh := hp.ToFloatHistogram() - if !zeroFloatHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { - requireEqual(t, mockHistogram{ls, ts.CreatedTimestamp, nil, &histogram.FloatHistogram{}}, appendable.histograms[k]) + if hp.StartTimestamp != 0 && tc.ingestCTZeroSample { + requireEqual(t, mockHistogram{ls, hp.StartTimestamp, nil, &histogram.FloatHistogram{}}, appendable.histograms[k]) k++ - zeroFloatHistogramIngested = true } requireEqual(t, mockHistogram{ls, hp.Timestamp, nil, fh}, appendable.histograms[k]) } else { h := hp.ToIntHistogram() - if !zeroHistogramIngested && ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { - requireEqual(t, mockHistogram{ls, ts.CreatedTimestamp, &histogram.Histogram{}, nil}, appendable.histograms[k]) + if hp.StartTimestamp != 0 && tc.ingestCTZeroSample { + requireEqual(t, mockHistogram{ls, hp.StartTimestamp, &histogram.Histogram{}, nil}, appendable.histograms[k]) k++ - zeroHistogramIngested = true } requireEqual(t, mockHistogram{ls, hp.Timestamp, h, nil}, appendable.histograms[k]) } k++ } - if ts.CreatedTimestamp != 0 && tc.ingestCTZeroSample { - require.True(t, zeroHistogramIngested) - require.True(t, zeroFloatHistogramIngested) - } if tc.appendExemplarErr == nil { for _, e := range ts.Exemplars { ex, err := e.ToExemplar(&b, writeV2RequestFixture.Symbols) From 48645e1d697befff82ac2dc883084b99ec6b123a Mon Sep 17 00:00:00 2001 From: bwplotka Date: Thu, 20 Nov 2025 08:39:29 +0000 Subject: [PATCH 069/439] chore: prepare 3.8.0-rc.1 entry Signed-off-by: bwplotka --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 07b6e1d74b..b35e0d4e50 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## main / unreleased +## 3.8.0-rc.1 / 2025-11-20 + +* [CHANGE] Remote-write (receiving): Updated experimental Remote Write implementation to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md); notably "created timestamp" (CT) is now called "start timestamp" (ST) and it's moved from TimeSeries message to Sample message. #17411 +* ## 3.8.0-rc.0 / 2025-11-07 * [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #17287 From 36d054cb2e76b474798d74efd6c1ffb93a47b855 Mon Sep 17 00:00:00 2001 From: Julius Volz Date: Thu, 20 Nov 2025 14:28:18 +0100 Subject: [PATCH 070/439] UI: Add graph option to start the chart's Y axis at zero (#17565) To reduce main UI clutter, I added a new settings submenu above the chart itself for the new setting. So far it only has the one new axis setting, but it could accommodate further settings in the future. For now I'm only adding a boolean on/off setting to the UI to set the Y axis to 0 or not. However, the underlying stored URL field is already named y_axis_min={number} and would support other Y axis minima, in case we want to support custom values in the UI in the future - but then we'd probably also want to add an axis maximum and possibly other settings. Fixes https://github.com/prometheus/prometheus/issues/520 Signed-off-by: Julius Volz --- web/ui/mantine-ui/src/pages/query/Graph.tsx | 3 ++ .../mantine-ui/src/pages/query/QueryPanel.tsx | 39 +++++++++++++++++++ .../mantine-ui/src/pages/query/UPlotChart.tsx | 4 ++ .../src/pages/query/uPlotChartHelpers.ts | 12 ++++++ .../src/pages/query/urlStateEncoding.ts | 8 ++++ web/ui/mantine-ui/src/state/queryPageSlice.ts | 2 + 6 files changed, 68 insertions(+) diff --git a/web/ui/mantine-ui/src/pages/query/Graph.tsx b/web/ui/mantine-ui/src/pages/query/Graph.tsx index a30d52c822..fda14210b9 100644 --- a/web/ui/mantine-ui/src/pages/query/Graph.tsx +++ b/web/ui/mantine-ui/src/pages/query/Graph.tsx @@ -25,6 +25,7 @@ export interface GraphProps { resolution: GraphResolution; showExemplars: boolean; displayMode: GraphDisplayMode; + yAxisMin: number | null; retriggerIdx: number; onSelectRange: (start: number, end: number) => void; } @@ -37,6 +38,7 @@ const Graph: FC = ({ resolution, showExemplars, displayMode, + yAxisMin, retriggerIdx, onSelectRange, }) => { @@ -222,6 +224,7 @@ const Graph: FC = ({ width={width} showExemplars={showExemplars} displayMode={displayMode} + yAxisMin={yAxisMin} onSelectRange={onSelectRange} /> diff --git a/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx b/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx index 24eb8c59cb..5e41be7bb3 100644 --- a/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx +++ b/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx @@ -7,8 +7,12 @@ import { SegmentedControl, Stack, Skeleton, + ActionIcon, + Popover, + Checkbox, } from "@mantine/core"; import { + IconAdjustmentsHorizontal, IconChartAreaFilled, IconChartLine, IconGraph, @@ -37,6 +41,7 @@ import ErrorBoundary from "../../components/ErrorBoundary"; import ASTNode from "../../promql/ast"; import serializeNode from "../../promql/serialize"; import ExplainView from "./ExplainViews/ExplainView"; +import { actionIconStyle } from "../../styles"; export interface PanelProps { idx: number; @@ -290,6 +295,39 @@ const QueryPanel: FC = ({ idx, metricNames }) => { // }, ]} /> + + + + + + + + + dispatch( + setVisualizer({ + idx, + visualizer: { + ...panel.visualizer, + yAxisMin: event.currentTarget.checked ? 0 : null, + }, + }) + ) + } + /> + + @@ -301,6 +339,7 @@ const QueryPanel: FC = ({ idx, metricNames }) => { resolution={panel.visualizer.resolution} showExemplars={panel.visualizer.showExemplars} displayMode={panel.visualizer.displayMode} + yAxisMin={panel.visualizer.yAxisMin} retriggerIdx={retriggerIdx} onSelectRange={onSelectRange} /> diff --git a/web/ui/mantine-ui/src/pages/query/UPlotChart.tsx b/web/ui/mantine-ui/src/pages/query/UPlotChart.tsx index b3b2d75578..11e57f40f9 100644 --- a/web/ui/mantine-ui/src/pages/query/UPlotChart.tsx +++ b/web/ui/mantine-ui/src/pages/query/UPlotChart.tsx @@ -24,6 +24,7 @@ export interface UPlotChartProps { width: number; showExemplars: boolean; displayMode: GraphDisplayMode; + yAxisMin: number | null; onSelectRange: (start: number, end: number) => void; } @@ -34,6 +35,7 @@ const UPlotChart: FC = ({ range: { startTime, endTime, resolution }, width, displayMode, + yAxisMin, onSelectRange, }) => { const [options, setOptions] = useState(null); @@ -60,6 +62,7 @@ const UPlotChart: FC = ({ width, data, useLocalTime, + yAxisMin, theme === "light", onSelectRange ); @@ -81,6 +84,7 @@ const UPlotChart: FC = ({ useLocalTime, theme, onSelectRange, + yAxisMin, ]); if (options === null || processedData === null) { diff --git a/web/ui/mantine-ui/src/pages/query/uPlotChartHelpers.ts b/web/ui/mantine-ui/src/pages/query/uPlotChartHelpers.ts index 3249afd454..ba6cdbae41 100644 --- a/web/ui/mantine-ui/src/pages/query/uPlotChartHelpers.ts +++ b/web/ui/mantine-ui/src/pages/query/uPlotChartHelpers.ts @@ -289,6 +289,7 @@ export const getUPlotOptions = ( width: number, result: RangeSamples[], useLocalTime: boolean, + yAxisMin: number | null, light: boolean, onSelectRange: (_start: number, _end: number) => void ): uPlot.Options => ({ @@ -330,6 +331,17 @@ export const getUPlotOptions = ( focus: { alpha: 1, }, + scales: + yAxisMin !== null + ? { + y: { + range: (_u, _min, max) => { + const minMax = uPlot.rangeNum(yAxisMin, max, 0.1, true); + return [yAxisMin, minMax[1]]; + }, + }, + } + : undefined, axes: [ // X axis (time). { diff --git a/web/ui/mantine-ui/src/pages/query/urlStateEncoding.ts b/web/ui/mantine-ui/src/pages/query/urlStateEncoding.ts index ca9988e60d..18b63d9ed4 100644 --- a/web/ui/mantine-ui/src/pages/query/urlStateEncoding.ts +++ b/web/ui/mantine-ui/src/pages/query/urlStateEncoding.ts @@ -63,6 +63,9 @@ export const decodePanelOptionsFromURLParams = (query: string): Panel[] => { panel.visualizer.displayMode = value === "1" ? GraphDisplayMode.Stacked : GraphDisplayMode.Lines; }); + decodeSetting("y_axis_min", (value) => { + panel.visualizer.yAxisMin = value === null ? null : parseFloat(value); + }); decodeSetting("show_exemplars", (value) => { panel.visualizer.showExemplars = value === "1"; }); @@ -171,6 +174,11 @@ export const encodePanelOptionsToURLParams = ( } addParam(idx, "display_mode", p.visualizer.displayMode); + addParam( + idx, + "y_axis_min", + p.visualizer.yAxisMin === null ? "" : p.visualizer.yAxisMin.toString() + ); addParam(idx, "show_exemplars", p.visualizer.showExemplars ? "1" : "0"); }); diff --git a/web/ui/mantine-ui/src/state/queryPageSlice.ts b/web/ui/mantine-ui/src/state/queryPageSlice.ts index 253b3ee9c6..4cf483e2b6 100644 --- a/web/ui/mantine-ui/src/state/queryPageSlice.ts +++ b/web/ui/mantine-ui/src/state/queryPageSlice.ts @@ -58,6 +58,7 @@ export interface Visualizer { resolution: GraphResolution; displayMode: GraphDisplayMode; showExemplars: boolean; + yAxisMin: number | null; } export type Panel = { @@ -86,6 +87,7 @@ export const newDefaultPanel = (): Panel => ({ resolution: { type: "auto", density: "medium" }, displayMode: GraphDisplayMode.Lines, showExemplars: false, + yAxisMin: null, }, }); From fc27eef43f2032ceee2e9af11e2e0739409929a0 Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Thu, 20 Nov 2025 15:36:21 +0100 Subject: [PATCH 071/439] Group more dependabot updates (#17563) Reduce the number of dependabot PRs for related udpdates. Signed-off-by: SuperQ --- .github/dependabot.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 191e07ffac..99a9ce05a4 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -18,6 +18,12 @@ updates: schedule: interval: "monthly" groups: + aws: + patterns: + - "github.com/aws/*" + azure: + patterns: + - "github.com/Azure/*" k8s.io: patterns: - "k8s.io/*" From b8d19543b8ddaac0c528f73db308a5771d423304 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Rabenstein?= Date: Fri, 21 Nov 2025 00:22:24 +0100 Subject: [PATCH 072/439] Add histogram validation in remote-read and during reducing resolution (#17561) ReduceResolution is currently called before validation during ingestion. This will cause a panic if there are not enough buckets in the histogram. If there are too many buckets, the spurious buckets are ignored, and therefore the error in the input histogram is masked. Furthermore, invalid negative offsets might cause problems, too. Therefore, we need to do some minimal validation in reduceResolution. Fortunately, it is easy and shouldn't slow things down. Sadly, it requires to return errors, which triggers a bunch of code changes. Even here is a bright side, we can get rud of a few panics. (Remember: Don't panic!) In different news, we haven't done a full validation of histograms read via remote-read. This is not so much a security concern (as you can throw off Prometheus easily by feeding it bogus data via remote-read) but more that remote-read sources might be makeshift and could accidentally create invalid histograms. We really don't want to panic in that case. So this commit does not only add a check of the spans and buckets as needed for resolution reduction but also a full validation during remote-read. Signed-off-by: beorn7 --- model/histogram/float_histogram.go | 55 ++++++++----- model/histogram/float_histogram_test.go | 67 +++++++++++++--- model/histogram/generic.go | 37 ++++++++- model/histogram/generic_test.go | 8 +- model/histogram/histogram.go | 36 ++++++--- model/histogram/histogram_test.go | 67 +++++++++++++--- model/textparse/nhcbparse.go | 2 +- scrape/target.go | 24 ++++-- storage/remote/codec.go | 100 +++++++++++++++--------- storage/remote/codec_test.go | 43 +++++++++- storage/remote/write_handler.go | 12 ++- tsdb/chunkenc/float_histogram.go | 17 +++- tsdb/chunkenc/histogram.go | 34 +++++++- tsdb/record/record.go | 8 +- 14 files changed, 395 insertions(+), 115 deletions(-) diff --git a/model/histogram/float_histogram.go b/model/histogram/float_histogram.go index 28f35572c2..91fcac1cfb 100644 --- a/model/histogram/float_histogram.go +++ b/model/histogram/float_histogram.go @@ -164,8 +164,8 @@ func (h *FloatHistogram) CopyToSchema(targetSchema int32) *FloatHistogram { Sum: h.Sum, } - c.PositiveSpans, c.PositiveBuckets = reduceResolution(h.PositiveSpans, h.PositiveBuckets, h.Schema, targetSchema, false, false) - c.NegativeSpans, c.NegativeBuckets = reduceResolution(h.NegativeSpans, h.NegativeBuckets, h.Schema, targetSchema, false, false) + c.PositiveSpans, c.PositiveBuckets = mustReduceResolution(h.PositiveSpans, h.PositiveBuckets, h.Schema, targetSchema, false, false) + c.NegativeSpans, c.NegativeBuckets = mustReduceResolution(h.NegativeSpans, h.NegativeBuckets, h.Schema, targetSchema, false, false) return &c } @@ -393,13 +393,13 @@ func (h *FloatHistogram) Add(other *FloatHistogram) (res *FloatHistogram, counte switch { case other.Schema < h.Schema: - hPositiveSpans, hPositiveBuckets = reduceResolution(hPositiveSpans, hPositiveBuckets, h.Schema, other.Schema, false, true) - hNegativeSpans, hNegativeBuckets = reduceResolution(hNegativeSpans, hNegativeBuckets, h.Schema, other.Schema, false, true) + hPositiveSpans, hPositiveBuckets = mustReduceResolution(hPositiveSpans, hPositiveBuckets, h.Schema, other.Schema, false, true) + hNegativeSpans, hNegativeBuckets = mustReduceResolution(hNegativeSpans, hNegativeBuckets, h.Schema, other.Schema, false, true) h.Schema = other.Schema case other.Schema > h.Schema: - otherPositiveSpans, otherPositiveBuckets = reduceResolution(otherPositiveSpans, otherPositiveBuckets, other.Schema, h.Schema, false, false) - otherNegativeSpans, otherNegativeBuckets = reduceResolution(otherNegativeSpans, otherNegativeBuckets, other.Schema, h.Schema, false, false) + otherPositiveSpans, otherPositiveBuckets = mustReduceResolution(otherPositiveSpans, otherPositiveBuckets, other.Schema, h.Schema, false, false) + otherNegativeSpans, otherNegativeBuckets = mustReduceResolution(otherNegativeSpans, otherNegativeBuckets, other.Schema, h.Schema, false, false) } h.PositiveSpans, h.PositiveBuckets = addBuckets(h.Schema, h.ZeroThreshold, false, hPositiveSpans, hPositiveBuckets, otherPositiveSpans, otherPositiveBuckets) @@ -459,12 +459,12 @@ func (h *FloatHistogram) Sub(other *FloatHistogram) (res *FloatHistogram, counte switch { case other.Schema < h.Schema: - hPositiveSpans, hPositiveBuckets = reduceResolution(hPositiveSpans, hPositiveBuckets, h.Schema, other.Schema, false, true) - hNegativeSpans, hNegativeBuckets = reduceResolution(hNegativeSpans, hNegativeBuckets, h.Schema, other.Schema, false, true) + hPositiveSpans, hPositiveBuckets = mustReduceResolution(hPositiveSpans, hPositiveBuckets, h.Schema, other.Schema, false, true) + hNegativeSpans, hNegativeBuckets = mustReduceResolution(hNegativeSpans, hNegativeBuckets, h.Schema, other.Schema, false, true) h.Schema = other.Schema case other.Schema > h.Schema: - otherPositiveSpans, otherPositiveBuckets = reduceResolution(otherPositiveSpans, otherPositiveBuckets, other.Schema, h.Schema, false, false) - otherNegativeSpans, otherNegativeBuckets = reduceResolution(otherNegativeSpans, otherNegativeBuckets, other.Schema, h.Schema, false, false) + otherPositiveSpans, otherPositiveBuckets = mustReduceResolution(otherPositiveSpans, otherPositiveBuckets, other.Schema, h.Schema, false, false) + otherNegativeSpans, otherNegativeBuckets = mustReduceResolution(otherNegativeSpans, otherNegativeBuckets, other.Schema, h.Schema, false, false) } h.PositiveSpans, h.PositiveBuckets = addBuckets(h.Schema, h.ZeroThreshold, true, hPositiveSpans, hPositiveBuckets, otherPositiveSpans, otherPositiveBuckets) @@ -1582,25 +1582,40 @@ func addCustomBucketsWithMismatches( } // ReduceResolution reduces the float histogram's spans, buckets into target schema. -// The target schema must be smaller than the current float histogram's schema. -// This will panic if the histogram has custom buckets or if the target schema is -// a custom buckets schema. -func (h *FloatHistogram) ReduceResolution(targetSchema int32) *FloatHistogram { +// An error is returned in the following cases: +// - The target schema is not smaller than the current histogram's schema. +// - The histogram has custom buckets. +// - The target schema is a custom buckets schema. +// - Any spans have an invalid offset. +// - The spans are inconsistent with the number of buckets. +func (h *FloatHistogram) ReduceResolution(targetSchema int32) error { + // Note that the follow three returns are not returning a + // histogram.Error because they are programming errors. if h.UsesCustomBuckets() { - panic("cannot reduce resolution when there are custom buckets") + return errors.New("cannot reduce resolution when there are custom buckets") } if IsCustomBucketsSchema(targetSchema) { - panic("cannot reduce resolution to custom buckets schema") + return errors.New("cannot reduce resolution to custom buckets schema") } if targetSchema >= h.Schema { - panic(fmt.Errorf("cannot reduce resolution from schema %d to %d", h.Schema, targetSchema)) + return fmt.Errorf("cannot reduce resolution from schema %d to %d", h.Schema, targetSchema) } - h.PositiveSpans, h.PositiveBuckets = reduceResolution(h.PositiveSpans, h.PositiveBuckets, h.Schema, targetSchema, false, true) - h.NegativeSpans, h.NegativeBuckets = reduceResolution(h.NegativeSpans, h.NegativeBuckets, h.Schema, targetSchema, false, true) + var err error + + if h.PositiveSpans, h.PositiveBuckets, err = reduceResolution( + h.PositiveSpans, h.PositiveBuckets, h.Schema, targetSchema, false, true, + ); err != nil { + return err + } + if h.NegativeSpans, h.NegativeBuckets, err = reduceResolution( + h.NegativeSpans, h.NegativeBuckets, h.Schema, targetSchema, false, true, + ); err != nil { + return err + } h.Schema = targetSchema - return h + return nil } // checkSchemaAndBounds checks if two histograms are compatible because they diff --git a/model/histogram/float_histogram_test.go b/model/histogram/float_histogram_test.go index ac339f152e..e79f5a0f49 100644 --- a/model/histogram/float_histogram_test.go +++ b/model/histogram/float_histogram_test.go @@ -4141,14 +4141,16 @@ func createRandomSpans(rng *rand.Rand, spanNum int32) ([]Span, []float64) { func TestFloatHistogramReduceResolution(t *testing.T) { tcs := map[string]struct { - origin *FloatHistogram - target *FloatHistogram + origin *FloatHistogram + targetSchema int32 + target *FloatHistogram + errorMsg string }{ "valid float histogram": { origin: &FloatHistogram{ Schema: 0, PositiveSpans: []Span{ - {Offset: 0, Length: 4}, + {Offset: -2, Length: 4}, {Offset: 0, Length: 0}, {Offset: 3, Length: 2}, }, @@ -4160,10 +4162,11 @@ func TestFloatHistogramReduceResolution(t *testing.T) { }, NegativeBuckets: []float64{1, 3, 1, 2, 1, 1}, }, + targetSchema: -1, target: &FloatHistogram{ Schema: -1, PositiveSpans: []Span{ - {Offset: 0, Length: 3}, + {Offset: -1, Length: 3}, {Offset: 1, Length: 1}, }, PositiveBuckets: []float64{1, 4, 2, 2}, @@ -4174,12 +4177,58 @@ func TestFloatHistogramReduceResolution(t *testing.T) { NegativeBuckets: []float64{1, 4, 2, 2}, }, }, + "not enough buckets": { + origin: &FloatHistogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: -2, Length: 4}, + {Offset: 0, Length: 0}, + {Offset: 3, Length: 2}, + }, + PositiveBuckets: []float64{1, 3, 1, 2, 1}, + }, + targetSchema: -1, + errorMsg: "have 5 buckets but spans need more: histogram spans specify different number of buckets than provided", + }, + "too many buckets": { + origin: &FloatHistogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: -2, Length: 4}, + {Offset: 0, Length: 0}, + {Offset: 3, Length: 2}, + }, + PositiveBuckets: []float64{1, 3, 1, 2, 1, 1, 5}, + }, + targetSchema: -1, + errorMsg: "spans need 6 buckets, have 7 buckets: histogram spans specify different number of buckets than provided", + }, + "negative offset": { + origin: &FloatHistogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: -2, Length: 4}, + {Offset: -1, Length: 0}, + {Offset: 3, Length: 2}, + }, + PositiveBuckets: []float64{1, 3, 1, 2, 1, 1}, + }, + targetSchema: -1, + errorMsg: "span number 2 with offset -1: histogram has a span whose offset is negative", + }, } - for _, tc := range tcs { - target := tc.origin.ReduceResolution(tc.target.Schema) - require.Equal(t, tc.target, target) - // Check that receiver histogram was mutated: - require.Equal(t, tc.target, tc.origin) + for tn, tc := range tcs { + t.Run(tn, func(t *testing.T) { + err := tc.origin.ReduceResolution(tc.targetSchema) + if tc.errorMsg != "" { + require.Equal(t, tc.errorMsg, err.Error()) + // The returned error should be a histogram.Error. + require.ErrorAs(t, err, &Error{}) + return + } + require.NoError(t, err) + require.Equal(t, tc.target, tc.origin) + }) } } diff --git a/model/histogram/generic.go b/model/histogram/generic.go index cd385407d5..649db769c7 100644 --- a/model/histogram/generic.go +++ b/model/histogram/generic.go @@ -738,6 +738,8 @@ var exponentialBounds = [][]float64{ // deltas. Set it to false if the buckets contain absolute counts. // Set inplace to true to reuse input slices and avoid allocations (otherwise // new slices will be allocated for result). +// The functions returns an error if there are too many or too few buckets for the spans +// or if any span except the first has a negative offset. func reduceResolution[IBC InternalBucketCount]( originSpans []Span, originBuckets []IBC, @@ -745,7 +747,7 @@ func reduceResolution[IBC InternalBucketCount]( targetSchema int32, deltaBuckets bool, inplace bool, -) ([]Span, []IBC) { +) ([]Span, []IBC, error) { var ( targetSpans []Span // The spans in the target schema. targetBuckets []IBC // The bucket counts in the target schema. @@ -764,10 +766,18 @@ func reduceResolution[IBC InternalBucketCount]( targetBuckets = originBuckets[:0] } - for _, span := range originSpans { + for n, span := range originSpans { + if n > 0 && span.Offset < 0 { + return nil, nil, fmt.Errorf("span number %d with offset %d: %w", n+1, span.Offset, ErrHistogramSpanNegativeOffset) + } // Determine the index of the first bucket in this span. bucketIdx += span.Offset for j := 0; j < int(span.Length); j++ { + // Protect against too few buckets in the origin. + if bucketCountIdx >= len(originBuckets) { + return nil, nil, fmt.Errorf("have %d buckets but spans need more: %w", len(originBuckets), ErrHistogramSpansBucketsMismatch) + } + // Determine the index of the bucket in the target schema from the index in the original schema. targetBucketIdx = targetIdx(bucketIdx, originSchema, targetSchema) @@ -826,12 +836,33 @@ func reduceResolution[IBC InternalBucketCount]( targetBuckets = append(targetBuckets, originBuckets[bucketCountIdx]) } } - bucketIdx++ bucketCountIdx++ } } + if bucketCountIdx != len(originBuckets) { + return nil, nil, fmt.Errorf("spans need %d buckets, have %d buckets: %w", bucketCountIdx, len(originBuckets), ErrHistogramSpansBucketsMismatch) + } + return targetSpans, targetBuckets, nil +} +// mustReduceResolution works like reduceResolution, but panics instead of +// returning an error. Use mustReduceResolution if you are sure that the spans +// and buckets are valid. +func mustReduceResolution[IBC InternalBucketCount]( + originSpans []Span, + originBuckets []IBC, + originSchema, + targetSchema int32, + deltaBuckets bool, + inplace bool, +) ([]Span, []IBC) { + targetSpans, targetBuckets, err := reduceResolution( + originSpans, originBuckets, originSchema, targetSchema, deltaBuckets, inplace, + ) + if err != nil { + panic(err) + } return targetSpans, targetBuckets } diff --git a/model/histogram/generic_test.go b/model/histogram/generic_test.go index 1651830e9d..54324beaff 100644 --- a/model/histogram/generic_test.go +++ b/model/histogram/generic_test.go @@ -142,7 +142,7 @@ func TestReduceResolutionHistogram(t *testing.T) { for _, tc := range cases { spansCopy, bucketsCopy := slices.Clone(tc.spans), slices.Clone(tc.buckets) - spans, buckets := reduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, true, false) + spans, buckets := mustReduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, true, false) require.Equal(t, tc.expectedSpans, spans) require.Equal(t, tc.expectedBuckets, buckets) // Verify inputs were not mutated: @@ -151,7 +151,7 @@ func TestReduceResolutionHistogram(t *testing.T) { // Output slices reuse input slices: const inplace = true - spans, buckets = reduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, true, inplace) + spans, buckets = mustReduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, true, inplace) require.Equal(t, tc.expectedSpans, spans) require.Equal(t, tc.expectedBuckets, buckets) // Verify inputs were mutated which is now expected: @@ -190,7 +190,7 @@ func TestReduceResolutionFloatHistogram(t *testing.T) { for _, tc := range cases { spansCopy, bucketsCopy := slices.Clone(tc.spans), slices.Clone(tc.buckets) - spans, buckets := reduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, false, false) + spans, buckets := mustReduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, false, false) require.Equal(t, tc.expectedSpans, spans) require.Equal(t, tc.expectedBuckets, buckets) // Verify inputs were not mutated: @@ -199,7 +199,7 @@ func TestReduceResolutionFloatHistogram(t *testing.T) { // Output slices reuse input slices: const inplace = true - spans, buckets = reduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, false, inplace) + spans, buckets = mustReduceResolution(tc.spans, tc.buckets, tc.schema, tc.targetSchema, false, inplace) require.Equal(t, tc.expectedSpans, spans) require.Equal(t, tc.expectedBuckets, buckets) // Verify inputs were mutated which is now expected: diff --git a/model/histogram/histogram.go b/model/histogram/histogram.go index 959df4c87a..5fc68ef9d0 100644 --- a/model/histogram/histogram.go +++ b/model/histogram/histogram.go @@ -14,6 +14,7 @@ package histogram import ( + "errors" "fmt" "math" "slices" @@ -617,26 +618,37 @@ func (c *cumulativeBucketIterator) At() Bucket[uint64] { } // ReduceResolution reduces the histogram's spans, buckets into target schema. -// The target schema must be smaller than the current histogram's schema. -// This will panic if the histogram has custom buckets or if the target schema is -// a custom buckets schema. -func (h *Histogram) ReduceResolution(targetSchema int32) *Histogram { +// An error is returned in the following cases: +// - The target schema is not smaller than the current histogram's schema. +// - The histogram has custom buckets. +// - The target schema is a custom buckets schema. +// - Any spans have an invalid offset. +// - The spans are inconsistent with the number of buckets. +func (h *Histogram) ReduceResolution(targetSchema int32) error { + // Note that the follow three returns are not returning a + // histogram.Error because they are programming errors. if h.UsesCustomBuckets() { - panic("cannot reduce resolution when there are custom buckets") + return errors.New("cannot reduce resolution when there are custom buckets") } if IsCustomBucketsSchema(targetSchema) { - panic("cannot reduce resolution to custom buckets schema") + return errors.New("cannot reduce resolution to custom buckets schema") } if targetSchema >= h.Schema { - panic(fmt.Errorf("cannot reduce resolution from schema %d to %d", h.Schema, targetSchema)) + return fmt.Errorf("cannot reduce resolution from schema %d to %d", h.Schema, targetSchema) } - h.PositiveSpans, h.PositiveBuckets = reduceResolution( + var err error + + if h.PositiveSpans, h.PositiveBuckets, err = reduceResolution( h.PositiveSpans, h.PositiveBuckets, h.Schema, targetSchema, true, true, - ) - h.NegativeSpans, h.NegativeBuckets = reduceResolution( + ); err != nil { + return err + } + if h.NegativeSpans, h.NegativeBuckets, err = reduceResolution( h.NegativeSpans, h.NegativeBuckets, h.Schema, targetSchema, true, true, - ) + ); err != nil { + return err + } h.Schema = targetSchema - return h + return nil } diff --git a/model/histogram/histogram_test.go b/model/histogram/histogram_test.go index e4c6ce683b..ae17f9be37 100644 --- a/model/histogram/histogram_test.go +++ b/model/histogram/histogram_test.go @@ -1719,14 +1719,16 @@ func BenchmarkHistogramValidation(b *testing.B) { func TestHistogramReduceResolution(t *testing.T) { tcs := map[string]struct { - origin *Histogram - target *Histogram + origin *Histogram + targetSchema int32 + target *Histogram + errorMsg string }{ "valid histogram": { origin: &Histogram{ Schema: 0, PositiveSpans: []Span{ - {Offset: 0, Length: 4}, + {Offset: -2, Length: 4}, {Offset: 0, Length: 0}, {Offset: 3, Length: 2}, }, @@ -1738,10 +1740,11 @@ func TestHistogramReduceResolution(t *testing.T) { }, NegativeBuckets: []int64{1, 2, -2, 1, -1, 0}, }, + targetSchema: -1, target: &Histogram{ Schema: -1, PositiveSpans: []Span{ - {Offset: 0, Length: 3}, + {Offset: -1, Length: 3}, {Offset: 1, Length: 1}, }, PositiveBuckets: []int64{1, 3, -2, 0}, @@ -1752,12 +1755,58 @@ func TestHistogramReduceResolution(t *testing.T) { NegativeBuckets: []int64{1, 3, -2, 0}, }, }, + "not enough buckets": { + origin: &Histogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: -2, Length: 4}, + {Offset: 0, Length: 0}, + {Offset: 3, Length: 2}, + }, + PositiveBuckets: []int64{1, 2, -2, 1, -1}, + }, + targetSchema: -1, + errorMsg: "have 5 buckets but spans need more: histogram spans specify different number of buckets than provided", + }, + "too many buckets": { + origin: &Histogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: -2, Length: 4}, + {Offset: 0, Length: 0}, + {Offset: 3, Length: 2}, + }, + PositiveBuckets: []int64{1, 2, -2, 1, -1, 0, 3}, + }, + targetSchema: -1, + errorMsg: "spans need 6 buckets, have 7 buckets: histogram spans specify different number of buckets than provided", + }, + "negative offset": { + origin: &Histogram{ + Schema: 0, + PositiveSpans: []Span{ + {Offset: -2, Length: 4}, + {Offset: -1, Length: 0}, + {Offset: 3, Length: 2}, + }, + PositiveBuckets: []int64{1, 2, -2, 1, -1, 0}, + }, + targetSchema: -1, + errorMsg: "span number 2 with offset -1: histogram has a span whose offset is negative", + }, } - for _, tc := range tcs { - target := tc.origin.ReduceResolution(tc.target.Schema) - require.Equal(t, tc.target, target) - // Check that receiver histogram was mutated: - require.Equal(t, tc.target, tc.origin) + for tn, tc := range tcs { + t.Run(tn, func(t *testing.T) { + err := tc.origin.ReduceResolution(tc.targetSchema) + if tc.errorMsg != "" { + require.Equal(t, tc.errorMsg, err.Error()) + // The returned error should be a histogram.Error. + require.ErrorAs(t, err, &Error{}) + return + } + require.NoError(t, err) + require.Equal(t, tc.target, tc.origin) + }) } } diff --git a/model/textparse/nhcbparse.go b/model/textparse/nhcbparse.go index ab821f0e63..79441e1f75 100644 --- a/model/textparse/nhcbparse.go +++ b/model/textparse/nhcbparse.go @@ -352,7 +352,7 @@ func (p *NHCBParser) swapExemplars() { } // processNHCB converts the collated classic histogram series to NHCB and caches the info -// to be returned to callers. Retruns true if the conversion was successful. +// to be returned to callers. Returns true if the conversion was successful. func (p *NHCBParser) processNHCB() bool { if p.state != stateCollecting { return false diff --git a/scrape/target.go b/scrape/target.go index 563fe33f82..2aabff20e2 100644 --- a/scrape/target.go +++ b/scrape/target.go @@ -389,6 +389,7 @@ type bucketLimitAppender struct { } func (app *bucketLimitAppender) AppendHistogram(ref storage.SeriesRef, lset labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + var err error if h != nil { // Return with an early error if the histogram has too many buckets and the // schema is not exponential, in which case we can't reduce the resolution. @@ -399,7 +400,9 @@ func (app *bucketLimitAppender) AppendHistogram(ref storage.SeriesRef, lset labe if h.Schema <= histogram.ExponentialSchemaMin { return 0, errBucketLimit } - h = h.ReduceResolution(h.Schema - 1) + if err = h.ReduceResolution(h.Schema - 1); err != nil { + return 0, err + } } } if fh != nil { @@ -412,11 +415,12 @@ func (app *bucketLimitAppender) AppendHistogram(ref storage.SeriesRef, lset labe if fh.Schema <= histogram.ExponentialSchemaMin { return 0, errBucketLimit } - fh = fh.ReduceResolution(fh.Schema - 1) + if err = fh.ReduceResolution(fh.Schema - 1); err != nil { + return 0, err + } } } - ref, err := app.Appender.AppendHistogram(ref, lset, t, h, fh) - if err != nil { + if ref, err = app.Appender.AppendHistogram(ref, lset, t, h, fh); err != nil { return 0, err } return ref, nil @@ -429,18 +433,22 @@ type maxSchemaAppender struct { } func (app *maxSchemaAppender) AppendHistogram(ref storage.SeriesRef, lset labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + var err error if h != nil { if histogram.IsExponentialSchemaReserved(h.Schema) && h.Schema > app.maxSchema { - h = h.ReduceResolution(app.maxSchema) + if err = h.ReduceResolution(app.maxSchema); err != nil { + return 0, err + } } } if fh != nil { if histogram.IsExponentialSchemaReserved(fh.Schema) && fh.Schema > app.maxSchema { - fh = fh.ReduceResolution(app.maxSchema) + if err = fh.ReduceResolution(app.maxSchema); err != nil { + return 0, err + } } } - ref, err := app.Appender.AppendHistogram(ref, lset, t, h, fh) - if err != nil { + if ref, err = app.Appender.AppendHistogram(ref, lset, t, h, fh); err != nil { return 0, err } return ref, nil diff --git a/storage/remote/codec.go b/storage/remote/codec.go index 7e21909354..059d5e66ce 100644 --- a/storage/remote/codec.go +++ b/storage/remote/codec.go @@ -389,6 +389,11 @@ type concreteSeriesIterator struct { curValType chunkenc.ValueType series *concreteSeries err error + + // These are pre-filled with the current model histogram if curValType + // is ValHistogram or ValFloatHistogram, respectively. + curH *histogram.Histogram + curFH *histogram.FloatHistogram } func newConcreteSeriesIterator(series *concreteSeries) chunkenc.Iterator { @@ -461,9 +466,7 @@ func (c *concreteSeriesIterator) Seek(t int64) chunkenc.ValueType { c.curValType = chunkenc.ValHistogram } if c.curValType == chunkenc.ValHistogram { - h := &c.series.histograms[c.histogramsCur] - c.curValType = getHistogramValType(h) - c.err = validateHistogramSchema(h) + c.setCurrentHistogram() } if c.err != nil { c.curValType = chunkenc.ValNone @@ -471,18 +474,57 @@ func (c *concreteSeriesIterator) Seek(t int64) chunkenc.ValueType { return c.curValType } -func validateHistogramSchema(h *prompb.Histogram) error { - if histogram.IsKnownSchema(h.Schema) { - return nil - } - return histogram.UnknownSchemaError(h.Schema) -} +// setCurrentHistogram pre-fills either the curH or the curFH field with a +// converted model histogram and sets c.curValType accordingly. It validates the +// histogram and sets c.err accordingly. This all has to be done in Seek() and +// Next() already so that we know if the histogram we got from the remote-read +// source is valid or not before we allow the AtHistogram()/AtFloatHistogram() +// call. +func (c *concreteSeriesIterator) setCurrentHistogram() { + pbH := c.series.histograms[c.histogramsCur] -func getHistogramValType(h *prompb.Histogram) chunkenc.ValueType { - if h.IsFloatHistogram() { - return chunkenc.ValFloatHistogram + // Basic schema check first. + schema := pbH.Schema + if !histogram.IsKnownSchema(schema) { + c.err = histogram.UnknownSchemaError(schema) + return } - return chunkenc.ValHistogram + + if pbH.IsFloatHistogram() { + c.curValType = chunkenc.ValFloatHistogram + mFH := pbH.ToFloatHistogram() + if mFH.Schema > histogram.ExponentialSchemaMax && mFH.Schema <= histogram.ExponentialSchemaMaxReserved { + // This is a very slow path, but it should only happen if the + // sample is from a newer Prometheus version that supports higher + // resolution. + if err := mFH.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + c.err = err + return + } + } + if err := mFH.Validate(); err != nil { + c.err = err + return + } + c.curFH = mFH + return + } + c.curValType = chunkenc.ValHistogram + mH := pbH.ToIntHistogram() + if mH.Schema > histogram.ExponentialSchemaMax && mH.Schema <= histogram.ExponentialSchemaMaxReserved { + // This is a very slow path, but it should only happen if the + // sample is from a newer Prometheus version that supports higher + // resolution. + if err := mH.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + c.err = err + return + } + } + if err := mH.Validate(); err != nil { + c.err = err + return + } + c.curH = mH } // At implements chunkenc.Iterator. @@ -499,31 +541,19 @@ func (c *concreteSeriesIterator) AtHistogram(*histogram.Histogram) (int64, *hist if c.curValType != chunkenc.ValHistogram { panic("iterator is not on an integer histogram sample") } - h := c.series.histograms[c.histogramsCur] - mh := h.ToIntHistogram() - if mh.Schema > histogram.ExponentialSchemaMax && mh.Schema <= histogram.ExponentialSchemaMaxReserved { - // This is a very slow path, but it should only happen if the - // sample is from a newer Prometheus version that supports higher - // resolution. - mh.ReduceResolution(histogram.ExponentialSchemaMax) - } - return h.Timestamp, mh + return c.series.histograms[c.histogramsCur].Timestamp, c.curH } // AtFloatHistogram implements chunkenc.Iterator. func (c *concreteSeriesIterator) AtFloatHistogram(*histogram.FloatHistogram) (int64, *histogram.FloatHistogram) { - if c.curValType == chunkenc.ValHistogram || c.curValType == chunkenc.ValFloatHistogram { - fh := c.series.histograms[c.histogramsCur] - mfh := fh.ToFloatHistogram() // integer will be auto-converted. - if mfh.Schema > histogram.ExponentialSchemaMax && mfh.Schema <= histogram.ExponentialSchemaMaxReserved { - // This is a very slow path, but it should only happen if the - // sample is from a newer Prometheus version that supports higher - // resolution. - mfh.ReduceResolution(histogram.ExponentialSchemaMax) - } - return fh.Timestamp, mfh + switch c.curValType { + case chunkenc.ValFloatHistogram: + return c.series.histograms[c.histogramsCur].Timestamp, c.curFH + case chunkenc.ValHistogram: + return c.series.histograms[c.histogramsCur].Timestamp, c.curH.ToFloat(nil) + default: + panic("iterator is not on a histogram sample") } - panic("iterator is not on a histogram sample") } // AtT implements chunkenc.Iterator. @@ -571,9 +601,7 @@ func (c *concreteSeriesIterator) Next() chunkenc.ValueType { } if c.curValType == chunkenc.ValHistogram { - h := &c.series.histograms[c.histogramsCur] - c.curValType = getHistogramValType(h) - c.err = validateHistogramSchema(h) + c.setCurrentHistogram() } if c.err != nil { c.curValType = chunkenc.ValNone diff --git a/storage/remote/codec_test.go b/storage/remote/codec_test.go index ce3a09b878..ba67ff33d9 100644 --- a/storage/remote/codec_test.go +++ b/storage/remote/codec_test.go @@ -546,7 +546,7 @@ func TestConcreteSeriesIterator_FloatAndHistogramSamples(t *testing.T) { require.Equal(t, chunkenc.ValNone, it.Seek(1)) } -func TestConcreteSeriesIterator_InvalidHistogramSamples(t *testing.T) { +func TestConcreteSeriesIterator_HistogramSamplesWithInvalidSchema(t *testing.T) { for _, schema := range []int32{-100, 100} { t.Run(fmt.Sprintf("schema=%d", schema), func(t *testing.T) { h := prompb.FromIntHistogram(2, &testHistogram) @@ -591,6 +591,47 @@ func TestConcreteSeriesIterator_InvalidHistogramSamples(t *testing.T) { } } +func TestConcreteSeriesIterator_HistogramSamplesWithMissingBucket(t *testing.T) { + mh := testHistogram.Copy() + mh.PositiveSpans = []histogram.Span{{Offset: 0, Length: 2}} + h := prompb.FromIntHistogram(2, mh) + fh := prompb.FromFloatHistogram(4, mh.ToFloat(nil)) + series := &concreteSeries{ + labels: labels.FromStrings("foo", "bar"), + floats: []prompb.Sample{ + {Value: 1, Timestamp: 0}, + {Value: 2, Timestamp: 3}, + }, + histograms: []prompb.Histogram{ + h, + fh, + }, + } + it := series.Iterator(nil) + require.Equal(t, chunkenc.ValFloat, it.Next()) + require.Equal(t, chunkenc.ValNone, it.Next()) + require.Error(t, it.Err()) + require.ErrorIs(t, it.Err(), histogram.ErrHistogramSpansBucketsMismatch) + + it = series.Iterator(it) + require.Equal(t, chunkenc.ValFloat, it.Next()) + require.Equal(t, chunkenc.ValNone, it.Next()) + require.ErrorIs(t, it.Err(), histogram.ErrHistogramSpansBucketsMismatch) + + it = series.Iterator(it) + require.Equal(t, chunkenc.ValNone, it.Seek(1)) + require.ErrorIs(t, it.Err(), histogram.ErrHistogramSpansBucketsMismatch) + + it = series.Iterator(it) + require.Equal(t, chunkenc.ValFloat, it.Seek(3)) + require.Equal(t, chunkenc.ValNone, it.Next()) + require.ErrorIs(t, it.Err(), histogram.ErrHistogramSpansBucketsMismatch) + + it = series.Iterator(it) + require.Equal(t, chunkenc.ValNone, it.Seek(4)) + require.ErrorIs(t, it.Err(), histogram.ErrHistogramSpansBucketsMismatch) +} + func TestConcreteSeriesIterator_ReducesHighResolutionHistograms(t *testing.T) { for _, schema := range []int32{9, 52} { t.Run(fmt.Sprintf("schema=%d", schema), func(t *testing.T) { diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index 67c244167b..b95c85b6c4 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -697,19 +697,23 @@ func (app *remoteWriteAppender) Append(ref storage.SeriesRef, lset labels.Labels } func (app *remoteWriteAppender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + var err error if t > app.maxTime { return 0, fmt.Errorf("%w: timestamp is too far in the future", storage.ErrOutOfBounds) } if h != nil && histogram.IsExponentialSchemaReserved(h.Schema) && h.Schema > histogram.ExponentialSchemaMax { - h = h.ReduceResolution(histogram.ExponentialSchemaMax) + if err = h.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + return 0, err + } } if fh != nil && histogram.IsExponentialSchemaReserved(fh.Schema) && fh.Schema > histogram.ExponentialSchemaMax { - fh = fh.ReduceResolution(histogram.ExponentialSchemaMax) + if err = fh.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + return 0, err + } } - ref, err := app.Appender.AppendHistogram(ref, l, t, h, fh) - if err != nil { + if ref, err = app.Appender.AppendHistogram(ref, l, t, h, fh); err != nil { return 0, err } return ref, nil diff --git a/tsdb/chunkenc/float_histogram.go b/tsdb/chunkenc/float_histogram.go index 8002dd0d4e..d960e835f2 100644 --- a/tsdb/chunkenc/float_histogram.go +++ b/tsdb/chunkenc/float_histogram.go @@ -884,7 +884,14 @@ func (it *floatHistogramIterator) AtFloatHistogram(fh *histogram.FloatHistogram) // chunk is from a newer Prometheus version that supports higher // resolution. fh = fh.Copy() - fh.ReduceResolution(histogram.ExponentialSchemaMax) + if err := fh.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + // With the checks above, this can only happen + // with invalid data in a chunk. As this is a + // rare edge case of a rare edge case, we'd + // rather not create all the plumbing to handle + // this error gracefully. + panic(err) + } } return it.t, fh } @@ -915,7 +922,13 @@ func (it *floatHistogramIterator) AtFloatHistogram(fh *histogram.FloatHistogram) // This is a very slow path, but it should only happen if the // chunk is from a newer Prometheus version that supports higher // resolution. - fh.ReduceResolution(histogram.ExponentialSchemaMax) + if err := fh.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + // With the checks above, this can only happen with + // invalid data in a chunk. As this is a rare edge case + // of a rare edge case, we'd rather not create all the + // plumbing to handle this error gracefully. + panic(err) + } } return it.t, fh diff --git a/tsdb/chunkenc/histogram.go b/tsdb/chunkenc/histogram.go index cc1d771235..be1c31ae76 100644 --- a/tsdb/chunkenc/histogram.go +++ b/tsdb/chunkenc/histogram.go @@ -939,7 +939,14 @@ func (it *histogramIterator) AtHistogram(h *histogram.Histogram) (int64, *histog // chunk is from a newer Prometheus version that supports higher // resolution. h = h.Copy() - h.ReduceResolution(histogram.ExponentialSchemaMax) + if err := h.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + // With the checks above, this can only happen + // with invalid data in a chunk. As this is a + // rare edge case of a rare edge case, we'd + // rather not create all the plumbing to handle + // this error gracefully. + panic(err) + } } return it.t, h } @@ -970,7 +977,13 @@ func (it *histogramIterator) AtHistogram(h *histogram.Histogram) (int64, *histog // This is a very slow path, but it should only happen if the // chunk is from a newer Prometheus version that supports higher // resolution. - h.ReduceResolution(histogram.ExponentialSchemaMax) + if err := h.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + // With the checks above, this can only happen with + // invalid data in a chunk. As this is a rare edge case + // of a rare edge case, we'd rather not create all the + // plumbing to handle this error gracefully. + panic(err) + } } return it.t, h @@ -1000,7 +1013,14 @@ func (it *histogramIterator) AtFloatHistogram(fh *histogram.FloatHistogram) (int // chunk is from a newer Prometheus version that supports higher // resolution. fh = fh.Copy() - fh.ReduceResolution(histogram.ExponentialSchemaMax) + if err := fh.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + // With the checks above, this can only happen + // with invalid data in a chunk. As this is a + // rare edge case of a rare edge case, we'd + // rather not create all the plumbing to handle + // this error gracefully. + panic(err) + } } return it.t, fh } @@ -1039,7 +1059,13 @@ func (it *histogramIterator) AtFloatHistogram(fh *histogram.FloatHistogram) (int // This is a very slow path, but it should only happen if the // chunk is from a newer Prometheus version that supports higher // resolution. - fh.ReduceResolution(histogram.ExponentialSchemaMax) + if err := fh.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + // With the checks above, this can only happen with + // invalid data in a chunk. As this is a rare edge case + // of a rare edge case, we'd rather not create all the + // plumbing to handle this error gracefully. + panic(err) + } } return it.t, fh diff --git a/tsdb/record/record.go b/tsdb/record/record.go index 561810a3a5..5791f60df4 100644 --- a/tsdb/record/record.go +++ b/tsdb/record/record.go @@ -475,7 +475,9 @@ func (d *Decoder) HistogramSamples(rec []byte, histograms []RefHistogramSample) // This is a very slow path, but it should only happen if the // record is from a newer Prometheus version that supports higher // resolution. - rh.H.ReduceResolution(histogram.ExponentialSchemaMax) + if err := rh.H.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + return nil, fmt.Errorf("error reducing resolution of histogram #%d: %w", len(histograms)+1, err) + } } histograms = append(histograms, rh) @@ -579,7 +581,9 @@ func (d *Decoder) FloatHistogramSamples(rec []byte, histograms []RefFloatHistogr // This is a very slow path, but it should only happen if the // record is from a newer Prometheus version that supports higher // resolution. - rh.FH.ReduceResolution(histogram.ExponentialSchemaMax) + if err := rh.FH.ReduceResolution(histogram.ExponentialSchemaMax); err != nil { + return nil, fmt.Errorf("error reducing resolution of histogram #%d: %w", len(histograms)+1, err) + } } histograms = append(histograms, rh) From 5cc29813c260184db966f9f042cd6c04b0c06574 Mon Sep 17 00:00:00 2001 From: Jorge Turrado Date: Wed, 19 Nov 2025 13:10:10 +0100 Subject: [PATCH 073/439] [chore]: bump common dep to support RFC7523 3.1 Signed-off-by: Jorge Turrado --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 55b8d2ce1f..864388bfab 100644 --- a/go.mod +++ b/go.mod @@ -58,7 +58,7 @@ require ( github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.67.2 + github.com/prometheus/common v0.67.3 github.com/prometheus/common/assets v0.2.0 github.com/prometheus/exporter-toolkit v0.15.0 github.com/prometheus/sigv4 v0.3.0 diff --git a/go.sum b/go.sum index 2c0042edbb..244268ae75 100644 --- a/go.sum +++ b/go.sum @@ -447,8 +447,8 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4= -github.com/prometheus/common v0.67.2 h1:PcBAckGFTIHt2+L3I33uNRTlKTplNzFctXcWhPyAEN8= -github.com/prometheus/common v0.67.2/go.mod h1:63W3KZb1JOKgcjlIr64WW/LvFGAqKPj0atm+knVGEko= +github.com/prometheus/common v0.67.3 h1:shd26MlnwTw5jksTDhC7rTQIteBxy+ZZDr3t7F2xN2Q= +github.com/prometheus/common v0.67.3/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= github.com/prometheus/common/assets v0.2.0 h1:0P5OrzoHrYBOSM1OigWL3mY8ZvV2N4zIE/5AahrSrfM= github.com/prometheus/common/assets v0.2.0/go.mod h1:D17UVUE12bHbim7HzwUvtqm6gwBEaDQ0F+hIGbFbccI= github.com/prometheus/exporter-toolkit v0.15.0 h1:Pcle5sSViwR1x0gdPd0wtYrPQENBieQAM7TmT0qtb2U= From d0d2699dc5f454ebca53e56247281447f8dd8c5a Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Fri, 21 Nov 2025 11:34:19 +0100 Subject: [PATCH 074/439] Update Prometheus Agent doc (#17591) * Add a nav title to fix docs website generator. * Make it more clear that "Prometheus Agent" is a mode, not a seaparate service. * Add to index. * Cleanup some wording. * Add a downsides section. Signed-off-by: SuperQ --- docs/index.md | 1 + docs/prometheus_agent.md | 23 +++++++++++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/docs/index.md b/docs/index.md index d9d4d2b152..fff28fa54a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,6 +13,7 @@ The documentation is available alongside all the project documentation at - [Getting started](getting_started.md) - [Installation](installation.md) +- [Agent Mode](prometheus_agent.md) - [Configuration](configuration/configuration.md) - [Querying](querying/basics.md) - [Storage](storage.md) diff --git a/docs/prometheus_agent.md b/docs/prometheus_agent.md index 9e2d922b10..468b5565d1 100644 --- a/docs/prometheus_agent.md +++ b/docs/prometheus_agent.md @@ -1,8 +1,11 @@ --- -# todo: internal +title: Prometheus Agent Mode +nav_title: Agent Mode +sort_rank: 4 --- -## Prometheus Agent +## Prometheus Agent Mode + The Prometheus Agent is an operational mode built into the Prometheus binary with the same scraping APIs, semantics, configuration, and discovery mechanism; this agent mode disables some of Prometheus' usual features(TSDB, alerting, and rule evaluations) and optimizes the binary for scraping and remote writing to remote locations. The Prometheus Remote Write protocol forwards(streams) all or a subset of metrics collected by Prometheus to a remote location; you can configure Prometheus to forward some metrics (if you want, with all metadata and exemplars!) to one or more locations that support the Remote Write API. @@ -15,11 +18,19 @@ The Agent mode optimizes Prometheus for the remote write use case. It disables q In essence, it looks like this: ![Prometheus Agent Remote Write](./images/prometheus_agent.png) -### Benefits of The Prometheus Agent -- First of all, efficiency; The customized Agent TSDB WAL removes the data immediately after successful writes. If it cannot reach the remote endpoint, it persists the data temporarily on the disk until the remote endpoint is back online. This is currently limited to a two-hour buffer only, similar to non-agent Prometheus. This means that we don't need to build chunks of data in memory. We don't need to maintain a full index for querying purposes. Essentially the Agent mode uses a fraction of the resources that a normal Prometheus server would use in a similar situation. -- Secondly, the benefit of the Agent mode is that it enables easier [horizontal scalability for ingestion](https://prometheus.io/blog/2021/11/16/agent/#the-dream-auto-scalable-metric-ingestion). +### Benefits of agent mode + +- Improved efficency. The customized Agent TSDB WAL removes the data immediately after successful writes. If it cannot reach the remote endpoint, it persists the data temporarily on the disk until the remote endpoint is back online. This is currently limited to a two-hour buffer only, similar to non-agent Prometheus. This means that there is no need to build chunks of data in memory or maintain a full index for querying purposes. Essentially the Agent mode uses a fraction of the resources that a normal Prometheus server would use in a similar situation. +- Agent mode eables easier [horizontal scalability for ingestion](https://prometheus.io/blog/2021/11/16/agent/#the-dream-auto-scalable-metric-ingestion). + +### Downsides of agent mode + +- No local queries. You can not query the local Prometheus instance. +- Recording rules are not possible. You can not pre-summarize data for sending to remote write. Rules must be done remotely. +- No alerting. All alerting must be done by the remote system. ### How to Use Agent Mode in Detail + If you show the help output of Prometheus (--help flag), you should see more or less the following: ``` @@ -40,4 +51,4 @@ Flags: Use the `--agent` flag to run Prometheus in the Agent mode. The rest of the flags are either for both server and Agent or only for a specific mode. You can see which flag is for which mode by checking the last sentence of a flag's help string. "Use with server mode only" means it's only for server mode. If you don't see any mention like this, it means the flag is shared. -The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. \ No newline at end of file +The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. From b415208a9089f6da4ffc140929eaabea0cad9e18 Mon Sep 17 00:00:00 2001 From: Ben Kochie Date: Fri, 21 Nov 2025 11:34:19 +0100 Subject: [PATCH 075/439] Update Prometheus Agent doc (#17591) * Add a nav title to fix docs website generator. * Make it more clear that "Prometheus Agent" is a mode, not a seaparate service. * Add to index. * Cleanup some wording. * Add a downsides section. Signed-off-by: SuperQ (cherry picked from commit d0d2699dc5f454ebca53e56247281447f8dd8c5a) --- docs/index.md | 1 + docs/prometheus_agent.md | 23 +++++++++++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/docs/index.md b/docs/index.md index d9d4d2b152..fff28fa54a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,6 +13,7 @@ The documentation is available alongside all the project documentation at - [Getting started](getting_started.md) - [Installation](installation.md) +- [Agent Mode](prometheus_agent.md) - [Configuration](configuration/configuration.md) - [Querying](querying/basics.md) - [Storage](storage.md) diff --git a/docs/prometheus_agent.md b/docs/prometheus_agent.md index 9e2d922b10..468b5565d1 100644 --- a/docs/prometheus_agent.md +++ b/docs/prometheus_agent.md @@ -1,8 +1,11 @@ --- -# todo: internal +title: Prometheus Agent Mode +nav_title: Agent Mode +sort_rank: 4 --- -## Prometheus Agent +## Prometheus Agent Mode + The Prometheus Agent is an operational mode built into the Prometheus binary with the same scraping APIs, semantics, configuration, and discovery mechanism; this agent mode disables some of Prometheus' usual features(TSDB, alerting, and rule evaluations) and optimizes the binary for scraping and remote writing to remote locations. The Prometheus Remote Write protocol forwards(streams) all or a subset of metrics collected by Prometheus to a remote location; you can configure Prometheus to forward some metrics (if you want, with all metadata and exemplars!) to one or more locations that support the Remote Write API. @@ -15,11 +18,19 @@ The Agent mode optimizes Prometheus for the remote write use case. It disables q In essence, it looks like this: ![Prometheus Agent Remote Write](./images/prometheus_agent.png) -### Benefits of The Prometheus Agent -- First of all, efficiency; The customized Agent TSDB WAL removes the data immediately after successful writes. If it cannot reach the remote endpoint, it persists the data temporarily on the disk until the remote endpoint is back online. This is currently limited to a two-hour buffer only, similar to non-agent Prometheus. This means that we don't need to build chunks of data in memory. We don't need to maintain a full index for querying purposes. Essentially the Agent mode uses a fraction of the resources that a normal Prometheus server would use in a similar situation. -- Secondly, the benefit of the Agent mode is that it enables easier [horizontal scalability for ingestion](https://prometheus.io/blog/2021/11/16/agent/#the-dream-auto-scalable-metric-ingestion). +### Benefits of agent mode + +- Improved efficency. The customized Agent TSDB WAL removes the data immediately after successful writes. If it cannot reach the remote endpoint, it persists the data temporarily on the disk until the remote endpoint is back online. This is currently limited to a two-hour buffer only, similar to non-agent Prometheus. This means that there is no need to build chunks of data in memory or maintain a full index for querying purposes. Essentially the Agent mode uses a fraction of the resources that a normal Prometheus server would use in a similar situation. +- Agent mode eables easier [horizontal scalability for ingestion](https://prometheus.io/blog/2021/11/16/agent/#the-dream-auto-scalable-metric-ingestion). + +### Downsides of agent mode + +- No local queries. You can not query the local Prometheus instance. +- Recording rules are not possible. You can not pre-summarize data for sending to remote write. Rules must be done remotely. +- No alerting. All alerting must be done by the remote system. ### How to Use Agent Mode in Detail + If you show the help output of Prometheus (--help flag), you should see more or less the following: ``` @@ -40,4 +51,4 @@ Flags: Use the `--agent` flag to run Prometheus in the Agent mode. The rest of the flags are either for both server and Agent or only for a specific mode. You can see which flag is for which mode by checking the last sentence of a flag's help string. "Use with server mode only" means it's only for server mode. If you don't see any mention like this, it means the flag is shared. -The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. \ No newline at end of file +The Agent mode accepts the same scrape configuration with the same discovery options and remote write options. It also exposes a web UI on port 9095 with disabled query capabilities but shows build info, configuration, targets, and service discovery information as in a normal Prometheus server. From 591b484a6ae6b42b09145399a6104aed46fdd05b Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Fri, 21 Nov 2025 16:10:48 +0100 Subject: [PATCH 076/439] chore(deps): bump github.com/prometheus/common from 0.67.3 to 0.67.4 (#17594) Signed-off-by: Jan Fajerski --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 864388bfab..9cf136eb39 100644 --- a/go.mod +++ b/go.mod @@ -58,7 +58,7 @@ require ( github.com/prometheus/client_golang v1.23.2 github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a github.com/prometheus/client_model v0.6.2 - github.com/prometheus/common v0.67.3 + github.com/prometheus/common v0.67.4 github.com/prometheus/common/assets v0.2.0 github.com/prometheus/exporter-toolkit v0.15.0 github.com/prometheus/sigv4 v0.3.0 diff --git a/go.sum b/go.sum index 244268ae75..579e86ca58 100644 --- a/go.sum +++ b/go.sum @@ -447,8 +447,8 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4= -github.com/prometheus/common v0.67.3 h1:shd26MlnwTw5jksTDhC7rTQIteBxy+ZZDr3t7F2xN2Q= -github.com/prometheus/common v0.67.3/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= +github.com/prometheus/common v0.67.4 h1:yR3NqWO1/UyO1w2PhUvXlGQs/PtFmoveVO0KZ4+Lvsc= +github.com/prometheus/common v0.67.4/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= github.com/prometheus/common/assets v0.2.0 h1:0P5OrzoHrYBOSM1OigWL3mY8ZvV2N4zIE/5AahrSrfM= github.com/prometheus/common/assets v0.2.0/go.mod h1:D17UVUE12bHbim7HzwUvtqm6gwBEaDQ0F+hIGbFbccI= github.com/prometheus/exporter-toolkit v0.15.0 h1:Pcle5sSViwR1x0gdPd0wtYrPQENBieQAM7TmT0qtb2U= From f1b0dd2cdd7786d465c6f8683de017b14a94d9c4 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Fri, 21 Nov 2025 11:32:01 +0100 Subject: [PATCH 077/439] prepare release v3.8.0-rc.1 Signed-off-by: Jan Fajerski --- CHANGELOG.md | 7 ++++--- VERSION | 2 +- web/ui/mantine-ui/package.json | 4 ++-- web/ui/module/codemirror-promql/package.json | 4 ++-- web/ui/module/lezer-promql/package.json | 2 +- web/ui/package-lock.json | 14 +++++++------- web/ui/package.json | 2 +- 7 files changed, 18 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b35e0d4e50..37db23b1a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,10 +2,11 @@ ## main / unreleased -## 3.8.0-rc.1 / 2025-11-20 +## 3.8.0-rc.1 / 2025-11-21 + +* [CHANGE] Remote-write 2 (receiving): Update to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md). "created timestamp" (CT) is now called "start timestamp" (ST). #17411 +* [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #17592 -* [CHANGE] Remote-write (receiving): Updated experimental Remote Write implementation to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md); notably "created timestamp" (CT) is now called "start timestamp" (ST) and it's moved from TimeSeries message to Sample message. #17411 -* ## 3.8.0-rc.0 / 2025-11-07 * [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #17287 diff --git a/VERSION b/VERSION index 100ac3dfd6..493bf7002c 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.8.0-rc.0 +3.8.0-rc.1 diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json index 1f10e4b620..b35ebf8f92 100644 --- a/web/ui/mantine-ui/package.json +++ b/web/ui/mantine-ui/package.json @@ -1,7 +1,7 @@ { "name": "@prometheus-io/mantine-ui", "private": true, - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "type": "module", "scripts": { "start": "vite", @@ -28,7 +28,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.308.0-rc.0", + "@prometheus-io/codemirror-promql": "0.308.0-rc.1", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json index b32fd59d19..5a23ead1f0 100644 --- a/web/ui/module/codemirror-promql/package.json +++ b/web/ui/module/codemirror-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/codemirror-promql", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "description": "a CodeMirror mode for the PromQL language", "types": "dist/esm/index.d.ts", "module": "dist/esm/index.js", @@ -29,7 +29,7 @@ }, "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md", "dependencies": { - "@prometheus-io/lezer-promql": "0.308.0-rc.0", + "@prometheus-io/lezer-promql": "0.308.0-rc.1", "lru-cache": "^11.2.2" }, "devDependencies": { diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json index d86f1a1e7a..f6152a35b7 100644 --- a/web/ui/module/lezer-promql/package.json +++ b/web/ui/module/lezer-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/lezer-promql", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "description": "lezer-based PromQL grammar", "main": "dist/index.cjs", "type": "module", diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json index a9a75a131a..e2271f2977 100644 --- a/web/ui/package-lock.json +++ b/web/ui/package-lock.json @@ -1,12 +1,12 @@ { "name": "prometheus-io", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "prometheus-io", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "workspaces": [ "mantine-ui", "module/*" @@ -24,7 +24,7 @@ }, "mantine-ui": { "name": "@prometheus-io/mantine-ui", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "dependencies": { "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", @@ -42,7 +42,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.308.0-rc.0", + "@prometheus-io/codemirror-promql": "0.308.0-rc.1", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", @@ -88,10 +88,10 @@ }, "module/codemirror-promql": { "name": "@prometheus-io/codemirror-promql", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "license": "Apache-2.0", "dependencies": { - "@prometheus-io/lezer-promql": "0.308.0-rc.0", + "@prometheus-io/lezer-promql": "0.308.0-rc.1", "lru-cache": "^11.2.2" }, "devDependencies": { @@ -121,7 +121,7 @@ }, "module/lezer-promql": { "name": "@prometheus-io/lezer-promql", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "license": "Apache-2.0", "devDependencies": { "@lezer/generator": "^1.8.0", diff --git a/web/ui/package.json b/web/ui/package.json index d8f2c712ff..2cf4a6819f 100644 --- a/web/ui/package.json +++ b/web/ui/package.json @@ -1,7 +1,7 @@ { "name": "prometheus-io", "description": "Monorepo for the Prometheus UI", - "version": "0.308.0-rc.0", + "version": "0.308.0-rc.1", "private": true, "scripts": { "build": "bash build_ui.sh --all", From b2eb2bc98930080822d62bf2f44003bb77dd31e3 Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Mon, 24 Nov 2025 09:31:28 +0100 Subject: [PATCH 078/439] chore(labels): add more context to labels.MetricName deprecation. (#17590) Signed-off-by: bwplotka --- model/labels/labels_common.go | 4 +++- schema/labels.go | 38 +++++++++++++---------------------- schema/labels_test.go | 22 ++++++++++---------- 3 files changed, 28 insertions(+), 36 deletions(-) diff --git a/model/labels/labels_common.go b/model/labels/labels_common.go index 5a3979784c..ab82ae6a8f 100644 --- a/model/labels/labels_common.go +++ b/model/labels/labels_common.go @@ -26,7 +26,9 @@ import ( const ( // MetricName is a special label name that represent a metric name. // - // Deprecated: Use schema.Metadata structure and its methods. + // Deprecated: Instead, consider using schema.Metadata structure and its methods for consistent metadata behaviour with the newly added __type__ and __unit__ labels. Alternatively use github.com/prometheus/common/model.MetricNameLabel for the direct replacement. + // + // labels package is providing label transport, agnostic to semantic meaning of each label. MetricName = "__name__" AlertName = "alertname" diff --git a/schema/labels.go b/schema/labels.go index 6df7445171..05329af7f6 100644 --- a/schema/labels.go +++ b/schema/labels.go @@ -19,20 +19,10 @@ import ( "github.com/prometheus/prometheus/model/labels" ) -const ( - // Special label names and selectors for schema.Metadata fields. - // They are currently private to ensure __name__, __type__ and __unit__ are used - // together and remain extensible in Prometheus. See NewMetadataFromLabels and Metadata - // methods for the interactions with the labels package structs. - metricName = "__name__" - metricType = "__type__" - metricUnit = "__unit__" -) - // IsMetadataLabel returns true if the given label name is a special // schema Metadata label. func IsMetadataLabel(name string) bool { - return name == metricName || name == metricType || name == metricUnit + return name == model.MetricNameLabel || name == model.MetricTypeLabel || name == model.MetricUnitLabel } // Metadata represents the core metric schema/metadata elements that: @@ -79,13 +69,13 @@ type Metadata struct { // NewMetadataFromLabels returns the schema metadata from the labels. func NewMetadataFromLabels(ls labels.Labels) Metadata { typ := model.MetricTypeUnknown - if got := ls.Get(metricType); got != "" { + if got := ls.Get(model.MetricTypeLabel); got != "" { typ = model.MetricType(got) } return Metadata{ - Name: ls.Get(metricName), + Name: ls.Get(model.MetricNameLabel), Type: typ, - Unit: ls.Get(metricUnit), + Unit: ls.Get(model.MetricUnitLabel), } } @@ -99,11 +89,11 @@ func (m Metadata) IsTypeEmpty() bool { // IsEmptyFor returns true. func (m Metadata) IsEmptyFor(labelName string) bool { switch labelName { - case metricName: + case model.MetricNameLabel: return m.Name == "" - case metricType: + case model.MetricTypeLabel: return m.IsTypeEmpty() - case metricUnit: + case model.MetricUnitLabel: return m.Unit == "" default: return true @@ -114,13 +104,13 @@ func (m Metadata) IsEmptyFor(labelName string) bool { // Empty Metadata fields will be ignored (not added). func (m Metadata) AddToLabels(b *labels.ScratchBuilder) { if m.Name != "" { - b.Add(metricName, m.Name) + b.Add(model.MetricNameLabel, m.Name) } if !m.IsTypeEmpty() { - b.Add(metricType, string(m.Type)) + b.Add(model.MetricTypeLabel, string(m.Type)) } if m.Unit != "" { - b.Add(metricUnit, m.Unit) + b.Add(model.MetricUnitLabel, m.Unit) } } @@ -128,15 +118,15 @@ func (m Metadata) AddToLabels(b *labels.ScratchBuilder) { // It follows the labels.Builder.Set semantics, so empty Metadata fields will // remove the corresponding existing labels if they were previously set. func (m Metadata) SetToLabels(b *labels.Builder) { - b.Set(metricName, m.Name) + b.Set(model.MetricNameLabel, m.Name) if m.Type == model.MetricTypeUnknown { // Unknown equals empty semantically, so remove the label on unknown too as per // method signature comment. - b.Set(metricType, "") + b.Set(model.MetricTypeLabel, "") } else { - b.Set(metricType, string(m.Type)) + b.Set(model.MetricTypeLabel, string(m.Type)) } - b.Set(metricUnit, m.Unit) + b.Set(model.MetricUnitLabel, m.Unit) } // NewIgnoreOverriddenMetadataLabelScratchBuilder creates IgnoreOverriddenMetadataLabelScratchBuilder. diff --git a/schema/labels_test.go b/schema/labels_test.go index 57b0401157..ae1ec9e90b 100644 --- a/schema/labels_test.go +++ b/schema/labels_test.go @@ -50,17 +50,17 @@ func TestMetadata(t *testing.T) { lb.Add("foo", "bar") if !tcase.emptyName { - lb.Add(metricName, testMeta.Name) + lb.Add(model.MetricNameLabel, testMeta.Name) expectedMeta.Name = testMeta.Name } if !tcase.emptyType { - lb.Add(metricType, string(testMeta.Type)) + lb.Add(model.MetricTypeLabel, string(testMeta.Type)) expectedMeta.Type = testMeta.Type } else { expectedMeta.Type = model.MetricTypeUnknown } if !tcase.emptyUnit { - lb.Add(metricUnit, testMeta.Unit) + lb.Add(model.MetricUnitLabel, testMeta.Unit) expectedMeta.Unit = testMeta.Unit } lb.Sort() @@ -75,10 +75,10 @@ func TestMetadata(t *testing.T) { } { // Empty methods. - require.Equal(t, tcase.emptyName, expectedMeta.IsEmptyFor(metricName)) - require.Equal(t, tcase.emptyType, expectedMeta.IsEmptyFor(metricType)) + require.Equal(t, tcase.emptyName, expectedMeta.IsEmptyFor(model.MetricNameLabel)) + require.Equal(t, tcase.emptyType, expectedMeta.IsEmptyFor(model.MetricTypeLabel)) require.Equal(t, tcase.emptyType, expectedMeta.IsTypeEmpty()) - require.Equal(t, tcase.emptyUnit, expectedMeta.IsEmptyFor(metricUnit)) + require.Equal(t, tcase.emptyUnit, expectedMeta.IsEmptyFor(model.MetricUnitLabel)) } { // From Metadata to labels for various builders. @@ -100,7 +100,7 @@ func TestIgnoreOverriddenMetadataLabelsScratchBuilder(t *testing.T) { // PROM-39 specifies that metadata labels should be sourced primarily from the metadata structures. // However, the original labels should be preserved IF the metadata structure does not set or support certain information. // Test those cases with common label interactions. - incomingLabels := labels.FromStrings(metricName, "different_name", metricType, string(model.MetricTypeSummary), metricUnit, "MB", "foo", "bar") + incomingLabels := labels.FromStrings(model.MetricNameLabel, "different_name", model.MetricTypeLabel, string(model.MetricTypeSummary), model.MetricUnitLabel, "MB", "foo", "bar") for _, tcase := range []struct { highPrioMeta Metadata expectedLabels labels.Labels @@ -114,21 +114,21 @@ func TestIgnoreOverriddenMetadataLabelsScratchBuilder(t *testing.T) { Type: model.MetricTypeCounter, Unit: "seconds", }, - expectedLabels: labels.FromStrings(metricName, "metric_total", metricType, string(model.MetricTypeCounter), metricUnit, "seconds", "foo", "bar"), + expectedLabels: labels.FromStrings(model.MetricNameLabel, "metric_total", model.MetricTypeLabel, string(model.MetricTypeCounter), model.MetricUnitLabel, "seconds", "foo", "bar"), }, { highPrioMeta: Metadata{ Name: "metric_total", Type: model.MetricTypeCounter, }, - expectedLabels: labels.FromStrings(metricName, "metric_total", metricType, string(model.MetricTypeCounter), metricUnit, "MB", "foo", "bar"), + expectedLabels: labels.FromStrings(model.MetricNameLabel, "metric_total", model.MetricTypeLabel, string(model.MetricTypeCounter), model.MetricUnitLabel, "MB", "foo", "bar"), }, { highPrioMeta: Metadata{ Type: model.MetricTypeCounter, Unit: "seconds", }, - expectedLabels: labels.FromStrings(metricName, "different_name", metricType, string(model.MetricTypeCounter), metricUnit, "seconds", "foo", "bar"), + expectedLabels: labels.FromStrings(model.MetricNameLabel, "different_name", model.MetricTypeLabel, string(model.MetricTypeCounter), model.MetricUnitLabel, "seconds", "foo", "bar"), }, { highPrioMeta: Metadata{ @@ -136,7 +136,7 @@ func TestIgnoreOverriddenMetadataLabelsScratchBuilder(t *testing.T) { Type: model.MetricTypeUnknown, Unit: "seconds", }, - expectedLabels: labels.FromStrings(metricName, "metric_total", metricType, string(model.MetricTypeSummary), metricUnit, "seconds", "foo", "bar"), + expectedLabels: labels.FromStrings(model.MetricNameLabel, "metric_total", model.MetricTypeLabel, string(model.MetricTypeSummary), model.MetricUnitLabel, "seconds", "foo", "bar"), }, } { t.Run(fmt.Sprintf("meta=%#v", tcase.highPrioMeta), func(t *testing.T) { From 02f405692e6f66d50fb98f6ed7c0c513fa359a97 Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Mon, 24 Nov 2025 09:05:48 +0000 Subject: [PATCH 079/439] fix: autocomplete suggestions for using cursor position Signed-off-by: ADITYA TIWARI --- .../module/codemirror-promql/src/complete/hybrid.test.ts | 6 ++++++ web/ui/module/codemirror-promql/src/complete/hybrid.ts | 8 +++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts index 587b31e743..526a5ce4f8 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts @@ -299,6 +299,12 @@ describe('analyzeCompletion test', () => { pos: 33, // cursor is between the bracket after the comma expectedContext: [{ kind: ContextKind.LabelName, metricName: 'metric_name' }], }, + { + title: 'no label suggestions after closing matcher', + expr: 'up{job="prometheus"}', + pos: 20, // cursor is right after the closing curly bracket + expectedContext: [], + }, { title: 'continue autocomplete labelName that defined a metric', expr: '{myL}', diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index b2d439d2fe..76efc34442 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -400,12 +400,18 @@ export function analyzeCompletion(state: EditorState, node: SyntaxNode, pos: num // so we have or to autocomplete any kind of labelName or to autocomplete only the labelName associated to the metric result.push({ kind: ContextKind.LabelName, metricName: getMetricNameInGroupBy(node, state) }); break; - case LabelMatchers: + case LabelMatchers: { + if (pos >= node.to) { + // Cursor is outside of the label matcher block (e.g. right after `}`), + // so don't offer label-related completions anymore. + break; + } // In that case we are in the given situation: // metric_name{} or {} // so we have or to autocomplete any kind of labelName or to autocomplete only the labelName associated to the metric result.push({ kind: ContextKind.LabelName, metricName: getMetricNameInVectorSelector(node, state) }); break; + } case LabelName: if (node.parent?.type.id === GroupingLabels) { // In this case we are in the given situation: From 04a5a488b89ce6ba05b6657b509785d64c328b59 Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Mon, 24 Nov 2025 17:51:14 +0000 Subject: [PATCH 080/439] fix: Suppress autocomplete for duration units when unit already present - No duration unit suggestions shown if a valid unit follows the digit (e.g. , ) - Adds related test cases Signed-off-by: ADITYA TIWARI --- .../src/complete/hybrid.test.ts | 34 +++++++++++++++++++ .../codemirror-promql/src/complete/hybrid.ts | 27 ++++++++++++++- 2 files changed, 60 insertions(+), 1 deletion(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts index 526a5ce4f8..25a2e8fb78 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts @@ -559,6 +559,18 @@ describe('analyzeCompletion test', () => { pos: 28, expectedContext: [{ kind: ContextKind.Duration }], }, + { + title: 'do not autocomplete duration when unit already present in matrixSelector', + expr: 'rate(foo[5m])', + pos: 10, + expectedContext: [], + }, + { + title: 'do not autocomplete duration when multi char unit already present in matrixSelector', + expr: 'rate(foo[5ms])', + pos: 10, + expectedContext: [], + }, { title: 'autocomplete duration for a subQuery', expr: 'go[5d:5]', @@ -1229,6 +1241,28 @@ describe('autocomplete promQL test', () => { validFor: undefined, }, }, + { + title: 'offline do not autocomplete duration when unit already present in matrixSelector', + expr: 'rate(foo[5m])', + pos: 10, + expectedResult: { + options: [], + from: 10, + to: 10, + validFor: /^[a-zA-Z0-9_:]+$/, + }, + }, + { + title: 'offline do not autocomplete duration when multi char unit already present in matrixSelector', + expr: 'rate(foo[5ms])', + pos: 10, + expectedResult: { + options: [], + from: 10, + to: 10, + validFor: /^[a-zA-Z0-9_:]+$/, + }, + }, { title: 'offline autocomplete duration for a subQuery', expr: 'go[5d:5]', diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 76efc34442..429ac468dd 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -166,6 +166,25 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i } as CompletionResult; } +const durationUnitLabels = durationTerms + .map((term) => term.label) + .filter((label): label is string => typeof label === 'string') + .sort((a, b) => b.length - a.length); + +const durationWithUnitRegexp = new RegExp(`^\\d+(?:${durationUnitLabels.map((label) => escapeRegExp(label)).join('|')})$`); + +function escapeRegExp(value: string): string { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +function hasCompleteDurationUnit(state: EditorState, node: SyntaxNode): boolean { + if (node.from >= node.to) { + return false; + } + const nodeContent = state.sliceDoc(node.from, node.to); + return durationWithUnitRegexp.test(nodeContent); +} + // computeStartCompleteLabelPositionInLabelMatcherOrInGroupingLabel calculates the start position only when the node is a LabelMatchers or a GroupingLabels function computeStartCompleteLabelPositionInLabelMatcherOrInGroupingLabel(node: SyntaxNode, pos: number): number { // Here we can have two different situations: @@ -477,12 +496,18 @@ export function analyzeCompletion(state: EditorState, node: SyntaxNode, pos: num // Duration, Duration, ⚠(NumberLiteral) // ) // So we should continue to autocomplete a duration - result.push({ kind: ContextKind.Duration }); + if (!hasCompleteDurationUnit(state, node)) { + result.push({ kind: ContextKind.Duration }); + } } else { result.push({ kind: ContextKind.Number }); } break; case NumberDurationLiteralInDurationContext: + if (!hasCompleteDurationUnit(state, node)) { + result.push({ kind: ContextKind.Duration }); + } + break; case OffsetExpr: result.push({ kind: ContextKind.Duration }); break; From e43f1bafca5b6cc0b0451a52fd56321f1c1350b0 Mon Sep 17 00:00:00 2001 From: Faustas Butkus Date: Tue, 25 Nov 2025 11:06:30 +0200 Subject: [PATCH 081/439] chore: fix rangeEval comment (#17607) Signed-off-by: Faustas Butkus --- promql/engine.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/promql/engine.go b/promql/engine.go index eac3b64093..a5b66052f3 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -1259,7 +1259,7 @@ func (enh *EvalNodeHelper) resetHistograms(inVec Vector, arg parser.Expr) annota // the given funcCall with the values computed for each expression at that // step. The return value is the combination into time series of all the // function call results. -// The prepSeries function (if provided) can be used to prepare the helper +// The matching (if provided) can be used to prepare the helper // for each series, then passed to each call funcCall. func (ev *evaluator) rangeEval(ctx context.Context, matching *parser.VectorMatching, funcCall func([]Vector, Matrix, [][]EvalSeriesHelper, *EvalNodeHelper) (Vector, annotations.Annotations), exprs ...parser.Expr) (Matrix, annotations.Annotations) { numSteps := int((ev.endTimestamp-ev.startTimestamp)/ev.interval) + 1 From bf76fde0c8104de155f64022bf96c957bc5eb4ee Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Tue, 25 Nov 2025 17:56:35 +0530 Subject: [PATCH 082/439] Update duration regex for complete duration matching Refactor duration regex to match complete durations with units. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> --- web/ui/module/codemirror-promql/src/complete/hybrid.ts | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 429ac468dd..32d76956d8 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -166,17 +166,14 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i } as CompletionResult; } -const durationUnitLabels = durationTerms - .map((term) => term.label) - .filter((label): label is string => typeof label === 'string') - .sort((a, b) => b.length - a.length); - -const durationWithUnitRegexp = new RegExp(`^\\d+(?:${durationUnitLabels.map((label) => escapeRegExp(label)).join('|')})$`); +// Matches complete duration with units (e.g., 5m, 30s, 1h, 500ms) +const durationWithUnitRegexp = new RegExp(`^\\d+(?:${durationTerms.map((term) => escapeRegExp(term.label)).join('|')})$`); function escapeRegExp(value: string): string { return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); } +// Determines if a duration already has a complete time unit to prevent autocomplete insertion (issue #15452) function hasCompleteDurationUnit(state: EditorState, node: SyntaxNode): boolean { if (node.from >= node.to) { return false; From 137f8465272432b771c33acee0fb208e33ff142b Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Tue, 25 Nov 2025 18:30:30 +0530 Subject: [PATCH 083/439] Add tests for durationWithUnitRegexp functionality Added tests for durationWithUnitRegexp to validate matching of complete durations with units and ensure non-matching cases are correctly identified. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> --- .../src/complete/hybrid.test.ts | 28 ++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts index 25a2e8fb78..cc73161dce 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts @@ -11,7 +11,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -import { analyzeCompletion, computeStartCompletePosition, ContextKind } from './hybrid'; +import { analyzeCompletion, computeStartCompletePosition, ContextKind, durationWithUnitRegexp } from './hybrid'; import { createEditorState, mockedMetricsTerms, mockPrometheusServer } from '../test/utils-test'; import { Completion, CompletionContext } from '@codemirror/autocomplete'; import { @@ -642,6 +642,32 @@ describe('analyzeCompletion test', () => { }); }); +describe('durationWithUnitRegexp test', () => { + it('should match complete durations with units', () => { + const testCases = [ + { input: '5m', expected: true }, + { input: '30s', expected: true }, + { input: '1h', expected: true }, + { input: '500ms', expected: true }, + { input: '2d', expected: true }, + { input: '1w', expected: true }, + { input: '1y', expected: true }, + ]; + + testCases.forEach(({ input, expected }) => { + expect(durationWithUnitRegexp.test(input)).toBe(expected); + }); + }); + + it('should not match durations without units or partial units', () => { + const testCases = ['5', '30', '100', '5m5', 'm', 'd']; + + testCases.forEach((input) => { + expect(durationWithUnitRegexp.test(input)).toBe(false); + }); + }); +}); + describe('computeStartCompletePosition test', () => { const testCases = [ { From 3b098799d4729c79048e23532d5a2c75d84b586b Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Tue, 25 Nov 2025 18:31:10 +0530 Subject: [PATCH 084/439] Export durationWithUnitRegexp for external use Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> --- web/ui/module/codemirror-promql/src/complete/hybrid.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 32d76956d8..8a2d575552 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -167,7 +167,7 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i } // Matches complete duration with units (e.g., 5m, 30s, 1h, 500ms) -const durationWithUnitRegexp = new RegExp(`^\\d+(?:${durationTerms.map((term) => escapeRegExp(term.label)).join('|')})$`); +export const durationWithUnitRegexp = new RegExp(`^\\d+(?:${durationTerms.map((term) => escapeRegExp(term.label)).join('|')})$`); function escapeRegExp(value: string): string { return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); From a1a3114a27e4ffab9f9c450395c6dd8a964530d3 Mon Sep 17 00:00:00 2001 From: Julius Volz Date: Tue, 25 Nov 2025 14:44:31 +0100 Subject: [PATCH 085/439] Hide alert annotations by default See https://github.com/prometheus/prometheus/issues/16911 This will create a denser layout by default, enabling people to see more information on the page without having to discover the global settings menu. Signed-off-by: Julius Volz --- web/ui/mantine-ui/src/state/settingsSlice.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/ui/mantine-ui/src/state/settingsSlice.ts b/web/ui/mantine-ui/src/state/settingsSlice.ts index 8b4a33bf76..a3e133380a 100644 --- a/web/ui/mantine-ui/src/state/settingsSlice.ts +++ b/web/ui/mantine-ui/src/state/settingsSlice.ts @@ -102,7 +102,7 @@ export const initialState: Settings = { ), showAnnotations: initializeFromLocalStorage( localStorageKeyShowAnnotations, - true + false ), showQueryWarnings: initializeFromLocalStorage( localStorageKeyShowQueryWarnings, From a66c696530e2305566c74871d0509b9fa9c502ac Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Tue, 25 Nov 2025 15:33:35 +0100 Subject: [PATCH 086/439] chore(storage): update docstring (#17609) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The original implementation in #9705 for native histograms included a technical dept #15177 where samples were committed ordered by type not by their append order. This was fixed in #17071, but this docstring was not updated. I've also took the liberty to mention that we do not order by timestamp either, thus it is possible to append out of order samples. Signed-off-by: György Krajcsovits --- storage/interface.go | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/storage/interface.go b/storage/interface.go index 6139a49511..19b4db4210 100644 --- a/storage/interface.go +++ b/storage/interface.go @@ -264,8 +264,9 @@ type AppendOptions struct { // // Operations on the Appender interface are not goroutine-safe. // -// The type of samples (float64, histogram, etc) appended for a given series must remain same within an Appender. -// The behaviour is undefined if samples of different types are appended to the same series in a single Commit(). +// The order of samples appended via the Appender is preserved within each +// series. I.e. samples are not reordered per timestamp, or by float/histogram +// type. type Appender interface { // Append adds a sample pair for the given series. // An optional series reference can be provided to accelerate calls. From 4fa435fad25598e355c85a3af1dcb3750085d4b8 Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Tue, 25 Nov 2025 16:13:52 +0000 Subject: [PATCH 087/439] feat: use RegExp.escape polyfill for robust PromQL duration regex; add compound duration test cases Signed-off-by: ADITYA TIWARI --- .../src/complete/hybrid.test.ts | 15 +++++++++++-- .../codemirror-promql/src/complete/hybrid.ts | 22 ++++++++++++++----- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts index cc73161dce..e958a8113b 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts @@ -652,11 +652,22 @@ describe('durationWithUnitRegexp test', () => { { input: '2d', expected: true }, { input: '1w', expected: true }, { input: '1y', expected: true }, + { input: '1d2h', expected: true }, + { input: '2h30m', expected: true }, + { input: '1h2m3s', expected: true }, + { input: '250ms2s', expected: true }, + { input: '2h3m4s5ms', expected: true }, + { input: '5', expected: false }, + { input: '5m5', expected: false }, + { input: 'm', expected: false }, + { input: 'd', expected: false }, + { input: '', expected: false }, + { input: '1hms', expected: false }, + { input: '2x', expected: false }, ]; - testCases.forEach(({ input, expected }) => { expect(durationWithUnitRegexp.test(input)).toBe(expected); - }); + }); }); it('should not match durations without units or partial units', () => { diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 8a2d575552..36fb59be5b 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -166,13 +166,25 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i } as CompletionResult; } -// Matches complete duration with units (e.g., 5m, 30s, 1h, 500ms) -export const durationWithUnitRegexp = new RegExp(`^\\d+(?:${durationTerms.map((term) => escapeRegExp(term.label)).join('|')})$`); - -function escapeRegExp(value: string): string { - return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +// Polyfill RegExp.escape for compatibility with ES2024 and TypeScript. +// Ensures safe, standards-based regex escaping in all environments. +declare global { + interface RegExpConstructor { + escape?: (s: string) => string; + } } +if (!RegExp.escape) { + RegExp.escape = function(s: string): string { + return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + }; +} + +// Matches complete PromQL durations, including compound units (e.g., 5m, 1d2h, 1h30m, etc.) +export const durationWithUnitRegexp = new RegExp( + `^(\\d+(${durationTerms.map(term => RegExp.escape!(term.label)).join('|')}))+$` +); + // Determines if a duration already has a complete time unit to prevent autocomplete insertion (issue #15452) function hasCompleteDurationUnit(state: EditorState, node: SyntaxNode): boolean { if (node.from >= node.to) { From 42418660d36e46bb835e957ce08826946257f127 Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Tue, 25 Nov 2025 16:30:27 +0000 Subject: [PATCH 088/439] fix: lint errors in the files; move regex to one-line only Signed-off-by: ADITYA TIWARI --- web/ui/module/codemirror-promql/src/complete/hybrid.test.ts | 3 +-- web/ui/module/codemirror-promql/src/complete/hybrid.ts | 6 ++---- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts index e958a8113b..587e9c5304 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts @@ -667,12 +667,11 @@ describe('durationWithUnitRegexp test', () => { ]; testCases.forEach(({ input, expected }) => { expect(durationWithUnitRegexp.test(input)).toBe(expected); - }); + }); }); it('should not match durations without units or partial units', () => { const testCases = ['5', '30', '100', '5m5', 'm', 'd']; - testCases.forEach((input) => { expect(durationWithUnitRegexp.test(input)).toBe(false); }); diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 36fb59be5b..11d18adcef 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -175,15 +175,13 @@ declare global { } if (!RegExp.escape) { - RegExp.escape = function(s: string): string { + RegExp.escape = function (s: string): string { return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); }; } // Matches complete PromQL durations, including compound units (e.g., 5m, 1d2h, 1h30m, etc.) -export const durationWithUnitRegexp = new RegExp( - `^(\\d+(${durationTerms.map(term => RegExp.escape!(term.label)).join('|')}))+$` -); +export const durationWithUnitRegexp = new RegExp(`^(\\d+(${durationTerms.map((term) => RegExp.escape!(term.label)).join('|')}))+$`); // Determines if a duration already has a complete time unit to prevent autocomplete insertion (issue #15452) function hasCompleteDurationUnit(state: EditorState, node: SyntaxNode): boolean { From e7999528fab9a2668bfb2b6cb1394dc11c06c54c Mon Sep 17 00:00:00 2001 From: Ayoub Mrini Date: Tue, 25 Nov 2025 22:30:42 +0100 Subject: [PATCH 089/439] fix(test): make TestRemoteWrite_ReshardingWithoutDeadlock more reliable and re-enable it (#17490) Improve test stability by waiting for the relevant metrics to appear on /metrics before the first check on the desired shard count. Increase the scrape interval to avoid timeouts, as 100 ms may be insufficient for Prometheus to scrape itself in some environments (e.g., CI). Have Prometheus scrape itself multiple times to increase the volume of data sent and help fill the queue more quickly. Signed-off-by: machine424 --- cmd/prometheus/main_test.go | 58 +++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/cmd/prometheus/main_test.go b/cmd/prometheus/main_test.go index e5e3db39ae..607e422868 100644 --- a/cmd/prometheus/main_test.go +++ b/cmd/prometheus/main_test.go @@ -968,8 +968,17 @@ remote_write: // TestRemoteWrite_ReshardingWithoutDeadlock ensures that resharding (scaling up) doesn't block when the shards are full. // See: https://github.com/prometheus/prometheus/issues/17384. +// +// The following shows key resharding metrics before and after the fix. +// In v3.7.0, the deadlock prevented the resharding logic from observing the true incoming data rate. +// +// | Metric | v3.7.0 | after the fix | +// |---------------------|---------------|---------------------| +// | dataInRate | 0.6 | 307.2 | +// | dataPendingRate | 0.2 | 306.8 | +// | dataPending | 0 | 1228.8 | +// | desiredShards | 0.6 | 369.2 |. func TestRemoteWrite_ReshardingWithoutDeadlock(t *testing.T) { - t.Skip("flaky test, see https://github.com/prometheus/prometheus/issues/17489") t.Parallel() tmpDir := t.TempDir() @@ -984,7 +993,8 @@ func TestRemoteWrite_ReshardingWithoutDeadlock(t *testing.T) { config := fmt.Sprintf(` global: - scrape_interval: 100ms + # Using a smaller interval may cause the scrape to time out. + scrape_interval: 1s scrape_configs: - job_name: 'self' static_configs: @@ -995,6 +1005,8 @@ remote_write: queue_config: # Speed up the queue being full. capacity: 1 + # Helps keep the “time to send one sample” low so it doesn’t influence the resharding logic. + max_samples_per_send: 1 `, port, server.URL) require.NoError(t, os.WriteFile(configFile, []byte(config), 0o777)) @@ -1003,36 +1015,52 @@ remote_write: configFile, port, fmt.Sprintf("--storage.tsdb.path=%s", tmpDir), + "--log.level=debug", ) require.NoError(t, prom.Start()) - var checkInitialDesiredShardsOnce sync.Once - require.Eventually(t, func() bool { + const desiredShardsMetric = "prometheus_remote_storage_shards_desired" + getMetrics := func() ([]byte, error) { r, err := http.Get(fmt.Sprintf("http://127.0.0.1:%d/metrics", port)) if err != nil { - return false + return nil, err } defer r.Body.Close() if r.StatusCode != http.StatusOK { - return false + return nil, fmt.Errorf("unexpected status code: %d", r.StatusCode) } metrics, err := io.ReadAll(r.Body) + if err != nil { + return nil, err + } + return metrics, nil + } + + // Ensure the initial desired shards is 1. + require.Eventually(t, func() bool { + metrics, err := getMetrics() if err != nil { return false } + initialDesiredShards, err := getMetricValue(t, bytes.NewReader(metrics), model.MetricTypeGauge, desiredShardsMetric) + if err != nil { + return false + } + return initialDesiredShards == 1.0 + }, 10*time.Second, 100*time.Millisecond) - checkInitialDesiredShardsOnce.Do(func() { - s, err := getMetricValue(t, bytes.NewReader(metrics), model.MetricTypeGauge, "prometheus_remote_storage_shards_desired") - require.NoError(t, err) - require.Equal(t, 1.0, s) - }) - - desiredShards, err := getMetricValue(t, bytes.NewReader(metrics), model.MetricTypeGauge, "prometheus_remote_storage_shards_desired") - if err != nil || desiredShards <= 1 { + // Ensure scaling up is triggered after some time. + require.Eventually(t, func() bool { + metrics, err := getMetrics() + if err != nil { + return false + } + desiredShards, err := getMetricValue(t, bytes.NewReader(metrics), model.MetricTypeGauge, desiredShardsMetric) + if err != nil || desiredShards <= 1.0 { return false } return true // 3*shardUpdateDuration to allow for the resharding logic to run. - }, 30*time.Second, 1*time.Second) + }, 30*time.Second, time.Second) } From 7bb95d548cf7f0aa6fa5b76facfc8155f2ac5a4b Mon Sep 17 00:00:00 2001 From: Andrew Hall Date: Thu, 27 Nov 2025 01:18:01 +0800 Subject: [PATCH 090/439] promql: Ensure that `rate`/`increase`/`delta` of histograms results in a gauge histogram. (#17608) Signed-off-by: Andrew Hall --- promql/functions.go | 1 + promql/functions_internal_test.go | 15 +++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/promql/functions.go b/promql/functions.go index 3d85719895..925ae83ae5 100644 --- a/promql/functions.go +++ b/promql/functions.go @@ -400,6 +400,7 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra annos.Add(annotations.NewNativeHistogramNotGaugeWarning(metricName, pos)) } + h.CounterResetHint = histogram.GaugeType return h.Compact(0), annos } diff --git a/promql/functions_internal_test.go b/promql/functions_internal_test.go index 658eb7550d..24d9a44e04 100644 --- a/promql/functions_internal_test.go +++ b/promql/functions_internal_test.go @@ -19,8 +19,23 @@ import ( "testing" "github.com/stretchr/testify/require" + + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/promql/parser/posrange" ) +func TestHistogramRateCounterResetHint(t *testing.T) { + points := []HPoint{ + {T: 0, H: &histogram.FloatHistogram{CounterResetHint: histogram.CounterReset, Count: 5, Sum: 5}}, + {T: 1, H: &histogram.FloatHistogram{CounterResetHint: histogram.UnknownCounterReset, Count: 10, Sum: 10}}, + } + fh, _ := histogramRate(points, false, "foo", posrange.PositionRange{}) + require.Equal(t, histogram.GaugeType, fh.CounterResetHint) + + fh, _ = histogramRate(points, true, "foo", posrange.PositionRange{}) + require.Equal(t, histogram.GaugeType, fh.CounterResetHint) +} + func TestKahanSumInc(t *testing.T) { testCases := map[string]struct { first float64 From 49427cfcd2cf13366b2a27bd0069d58566b9c8fc Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Thu, 27 Nov 2025 16:40:33 +0530 Subject: [PATCH 091/439] Refactor duration regex and remove RegExp.escape polyfill Removed polyfill for RegExp.escape and updated duration regex. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> --- .../codemirror-promql/src/complete/hybrid.ts | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 11d18adcef..814147e532 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -166,22 +166,9 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i } as CompletionResult; } -// Polyfill RegExp.escape for compatibility with ES2024 and TypeScript. -// Ensures safe, standards-based regex escaping in all environments. -declare global { - interface RegExpConstructor { - escape?: (s: string) => string; - } -} - -if (!RegExp.escape) { - RegExp.escape = function (s: string): string { - return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - }; -} - -// Matches complete PromQL durations, including compound units (e.g., 5m, 1d2h, 1h30m, etc.) -export const durationWithUnitRegexp = new RegExp(`^(\\d+(${durationTerms.map((term) => RegExp.escape!(term.label)).join('|')}))+$`); +// Matches complete PromQL durations, including compound units (e.g., 5m, 1d2h, 1h30m, etc.). +// Duration units are a fixed, safe set (no regex metacharacters), so no escaping is needed. +export const durationWithUnitRegexp = new RegExp(`^(\\d+(${durationTerms.map((term) => term.label).join('|')}))+$`); // Determines if a duration already has a complete time unit to prevent autocomplete insertion (issue #15452) function hasCompleteDurationUnit(state: EditorState, node: SyntaxNode): boolean { From 30be1483d147970ec1abd0cb0818767e2e46e20b Mon Sep 17 00:00:00 2001 From: harsh kumar <135993950+hxrshxz@users.noreply.github.com> Date: Thu, 27 Nov 2025 23:15:35 +0530 Subject: [PATCH 092/439] instrumentation: add native histograms to complement high-traffic summaries (#17374) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This adds the following native histograms (with a few classic buckets for backwards compatibility), while keeping the corresponding summaries (same name, just without `_histogram`): - `prometheus_sd_refresh_duration_histogram_seconds` - `prometheus_rule_evaluation_duration_histogram_seconds` - `prometheus_rule_group_duration_histogram_seconds` - `prometheus_target_sync_length_histogram_seconds` - `prometheus_target_interval_length_histogram_seconds` - `prometheus_engine_query_duration_histogram_seconds` Signed-off-by: Harsh Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com> Co-authored-by: Björn Rabenstein --- discovery/discovery.go | 5 ++-- discovery/metrics_refresh.go | 23 ++++++++++++--- discovery/refresh/refresh.go | 1 + promql/engine.go | 56 +++++++++++++++++++++++++----------- rules/group.go | 49 ++++++++++++++++++++++--------- rules/manager.go | 1 + scrape/metrics.go | 29 +++++++++++++++++++ scrape/scrape.go | 7 +++++ 8 files changed, 135 insertions(+), 36 deletions(-) diff --git a/discovery/discovery.go b/discovery/discovery.go index 70cd856bb2..e643cb10af 100644 --- a/discovery/discovery.go +++ b/discovery/discovery.go @@ -63,8 +63,9 @@ type DiscovererOptions struct { // We define them here in the "discovery" package in order to avoid a cyclic dependency between // "discovery" and "refresh". type RefreshMetrics struct { - Failures prometheus.Counter - Duration prometheus.Observer + Failures prometheus.Counter + Duration prometheus.Observer + DurationHistogram prometheus.Observer } // RefreshMetricsInstantiator instantiates the metrics used by the "refresh" package. diff --git a/discovery/metrics_refresh.go b/discovery/metrics_refresh.go index 8a8bf221b8..9f3eb27b49 100644 --- a/discovery/metrics_refresh.go +++ b/discovery/metrics_refresh.go @@ -14,6 +14,8 @@ package discovery import ( + "time" + "github.com/prometheus/client_golang/prometheus" ) @@ -21,8 +23,9 @@ import ( // We define them here in the "discovery" package in order to avoid a cyclic dependency between // "discovery" and "refresh". type RefreshMetricsVecs struct { - failuresVec *prometheus.CounterVec - durationVec *prometheus.SummaryVec + failuresVec *prometheus.CounterVec + durationVec *prometheus.SummaryVec + durationHistVec *prometheus.HistogramVec metricRegisterer MetricRegisterer } @@ -44,6 +47,16 @@ func NewRefreshMetrics(reg prometheus.Registerer) RefreshMetricsManager { Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001}, }, []string{"mechanism", "config"}), + durationHistVec: prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "prometheus_sd_refresh_duration_histogram_seconds", + Help: "The duration of a refresh for the given SD mechanism.", + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }, + []string{"mechanism"}), } // The reason we register metric vectors instead of metrics is so that @@ -51,6 +64,7 @@ func NewRefreshMetrics(reg prometheus.Registerer) RefreshMetricsManager { m.metricRegisterer = NewMetricRegisterer(reg, []prometheus.Collector{ m.failuresVec, m.durationVec, + m.durationHistVec, }) return m @@ -59,8 +73,9 @@ func NewRefreshMetrics(reg prometheus.Registerer) RefreshMetricsManager { // Instantiate returns metrics out of metric vectors for a given mechanism and config. func (m *RefreshMetricsVecs) Instantiate(mech, config string) *RefreshMetrics { return &RefreshMetrics{ - Failures: m.failuresVec.WithLabelValues(mech, config), - Duration: m.durationVec.WithLabelValues(mech, config), + Failures: m.failuresVec.WithLabelValues(mech, config), + Duration: m.durationVec.WithLabelValues(mech, config), + DurationHistogram: m.durationHistVec.WithLabelValues(mech), } } diff --git a/discovery/refresh/refresh.go b/discovery/refresh/refresh.go index e0bac2af5e..0613fd6c6d 100644 --- a/discovery/refresh/refresh.go +++ b/discovery/refresh/refresh.go @@ -108,6 +108,7 @@ func (d *Discovery) refresh(ctx context.Context) ([]*targetgroup.Group, error) { now := time.Now() defer func() { d.metrics.Duration.Observe(time.Since(now).Seconds()) + d.metrics.DurationHistogram.Observe(time.Since(now).Seconds()) }() tgs, err := d.refreshf(ctx) diff --git a/promql/engine.go b/promql/engine.go index a5b66052f3..d3b67e3d81 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -76,15 +76,19 @@ const ( ) type engineMetrics struct { - currentQueries prometheus.Gauge - maxConcurrentQueries prometheus.Gauge - queryLogEnabled prometheus.Gauge - queryLogFailures prometheus.Counter - queryQueueTime prometheus.Observer - queryPrepareTime prometheus.Observer - queryInnerEval prometheus.Observer - queryResultSort prometheus.Observer - querySamples prometheus.Counter + currentQueries prometheus.Gauge + maxConcurrentQueries prometheus.Gauge + queryLogEnabled prometheus.Gauge + queryLogFailures prometheus.Counter + queryQueueTime prometheus.Observer + queryQueueTimeHistogram prometheus.Observer + queryPrepareTime prometheus.Observer + queryPrepareTimeHistogram prometheus.Observer + queryInnerEval prometheus.Observer + queryInnerEvalHistogram prometheus.Observer + queryResultSort prometheus.Observer + queryResultSortHistogram prometheus.Observer + querySamples prometheus.Counter } type ( @@ -363,6 +367,19 @@ func NewEngine(opts EngineOpts) *Engine { []string{"slice"}, ) + queryResultHistogram := prometheus.NewHistogramVec(prometheus.HistogramOpts{ + Namespace: namespace, + Subsystem: subsystem, + Name: "query_duration_histogram_seconds", + Help: "The duration of various parts of PromQL query execution.", + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }, + []string{"slice"}, + ) + metrics := &engineMetrics{ currentQueries: prometheus.NewGauge(prometheus.GaugeOpts{ Namespace: namespace, @@ -394,10 +411,14 @@ func NewEngine(opts EngineOpts) *Engine { Name: "query_samples_total", Help: "The total number of samples loaded by all queries.", }), - queryQueueTime: queryResultSummary.WithLabelValues("queue_time"), - queryPrepareTime: queryResultSummary.WithLabelValues("prepare_time"), - queryInnerEval: queryResultSummary.WithLabelValues("inner_eval"), - queryResultSort: queryResultSummary.WithLabelValues("result_sort"), + queryQueueTime: queryResultSummary.WithLabelValues("queue_time"), + queryQueueTimeHistogram: queryResultHistogram.WithLabelValues("queue_time"), + queryPrepareTime: queryResultSummary.WithLabelValues("prepare_time"), + queryPrepareTimeHistogram: queryResultHistogram.WithLabelValues("prepare_time"), + queryInnerEval: queryResultSummary.WithLabelValues("inner_eval"), + queryInnerEvalHistogram: queryResultHistogram.WithLabelValues("inner_eval"), + queryResultSort: queryResultSummary.WithLabelValues("result_sort"), + queryResultSortHistogram: queryResultHistogram.WithLabelValues("result_sort"), } if t := opts.ActiveQueryTracker; t != nil { @@ -421,6 +442,7 @@ func NewEngine(opts EngineOpts) *Engine { metrics.queryLogFailures, metrics.querySamples, queryResultSummary, + queryResultHistogram, ) } @@ -701,7 +723,7 @@ func (ng *Engine) queueActive(ctx context.Context, q *query) (func(), error) { if ng.activeQueryTracker == nil { return func() {}, nil } - queueSpanTimer, _ := q.stats.GetSpanTimer(ctx, stats.ExecQueueTime, ng.metrics.queryQueueTime) + queueSpanTimer, _ := q.stats.GetSpanTimer(ctx, stats.ExecQueueTime, ng.metrics.queryQueueTime, ng.metrics.queryQueueTimeHistogram) queryIndex, err := ng.activeQueryTracker.Insert(ctx, q.q) queueSpanTimer.Finish() return func() { ng.activeQueryTracker.Delete(queryIndex) }, err @@ -717,7 +739,7 @@ func durationMilliseconds(d time.Duration) int64 { // execEvalStmt evaluates the expression of an evaluation statement for the given time range. func (ng *Engine) execEvalStmt(ctx context.Context, query *query, s *parser.EvalStmt) (parser.Value, annotations.Annotations, error) { - prepareSpanTimer, ctxPrepare := query.stats.GetSpanTimer(ctx, stats.QueryPreparationTime, ng.metrics.queryPrepareTime) + prepareSpanTimer, ctxPrepare := query.stats.GetSpanTimer(ctx, stats.QueryPreparationTime, ng.metrics.queryPrepareTime, ng.metrics.queryPrepareTimeHistogram) mint, maxt := FindMinMaxTime(s) querier, err := query.queryable.Querier(mint, maxt) if err != nil { @@ -732,7 +754,7 @@ func (ng *Engine) execEvalStmt(ctx context.Context, query *query, s *parser.Eval // Modify the offset of vector and matrix selectors for the @ modifier // w.r.t. the start time since only 1 evaluation will be done on them. setOffsetForAtModifier(timeMilliseconds(s.Start), s.Expr) - evalSpanTimer, ctxInnerEval := query.stats.GetSpanTimer(ctx, stats.InnerEvalTime, ng.metrics.queryInnerEval) + evalSpanTimer, ctxInnerEval := query.stats.GetSpanTimer(ctx, stats.InnerEvalTime, ng.metrics.queryInnerEval, ng.metrics.queryInnerEvalHistogram) // Instant evaluation. This is executed as a range evaluation with one step. if s.Start.Equal(s.End) && s.Interval == 0 { start := timeMilliseconds(s.Start) @@ -835,7 +857,7 @@ func (ng *Engine) execEvalStmt(ctx context.Context, query *query, s *parser.Eval } func (ng *Engine) sortMatrixResult(ctx context.Context, query *query, mat Matrix) { - sortSpanTimer, _ := query.stats.GetSpanTimer(ctx, stats.ResultSortTime, ng.metrics.queryResultSort) + sortSpanTimer, _ := query.stats.GetSpanTimer(ctx, stats.ResultSortTime, ng.metrics.queryResultSort, ng.metrics.queryResultSortHistogram) sort.Sort(mat) sortSpanTimer.Finish() } diff --git a/rules/group.go b/rules/group.go index 8cedcd40d1..47afe6f715 100644 --- a/rules/group.go +++ b/rules/group.go @@ -519,6 +519,7 @@ func (g *Group) Eval(ctx context.Context, ts time.Time) { since := time.Since(t) g.metrics.EvalDuration.Observe(since.Seconds()) + g.metrics.EvalDurationHistogram.Observe(since.Seconds()) rule.SetEvaluationDuration(since) rule.SetEvaluationTimestamp(t) }(time.Now()) @@ -910,19 +911,21 @@ const namespace = "prometheus" // Metrics for rule evaluation. type Metrics struct { - EvalDuration prometheus.Summary - IterationDuration prometheus.Summary - IterationsMissed *prometheus.CounterVec - IterationsScheduled *prometheus.CounterVec - EvalTotal *prometheus.CounterVec - EvalFailures *prometheus.CounterVec - GroupInterval *prometheus.GaugeVec - GroupLastEvalTime *prometheus.GaugeVec - GroupLastDuration *prometheus.GaugeVec - GroupLastRuleDurationSum *prometheus.GaugeVec - GroupLastRestoreDuration *prometheus.GaugeVec - GroupRules *prometheus.GaugeVec - GroupSamples *prometheus.GaugeVec + EvalDuration prometheus.Summary + EvalDurationHistogram prometheus.Histogram + IterationDuration prometheus.Summary + IterationDurationHistogram prometheus.Histogram + IterationsMissed *prometheus.CounterVec + IterationsScheduled *prometheus.CounterVec + EvalTotal *prometheus.CounterVec + EvalFailures *prometheus.CounterVec + GroupInterval *prometheus.GaugeVec + GroupLastEvalTime *prometheus.GaugeVec + GroupLastDuration *prometheus.GaugeVec + GroupLastRuleDurationSum *prometheus.GaugeVec + GroupLastRestoreDuration *prometheus.GaugeVec + GroupRules *prometheus.GaugeVec + GroupSamples *prometheus.GaugeVec } // NewGroupMetrics creates a new instance of Metrics and registers it with the provided registerer, @@ -936,12 +939,30 @@ func NewGroupMetrics(reg prometheus.Registerer) *Metrics { Help: "The duration for a rule to execute.", Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001}, }), + EvalDurationHistogram: prometheus.NewHistogram(prometheus.HistogramOpts{ + Namespace: namespace, + Name: "rule_evaluation_duration_histogram_seconds", + Help: "The duration for a rule to execute.", + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }), IterationDuration: prometheus.NewSummary(prometheus.SummaryOpts{ Namespace: namespace, Name: "rule_group_duration_seconds", Help: "The duration of rule group evaluations.", Objectives: map[float64]float64{0.01: 0.001, 0.05: 0.005, 0.5: 0.05, 0.90: 0.01, 0.99: 0.001}, }), + IterationDurationHistogram: prometheus.NewHistogram(prometheus.HistogramOpts{ + Namespace: namespace, + Name: "rule_group_duration_histogram_seconds", + Help: "The duration of rule group evaluations.", + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }), IterationsMissed: prometheus.NewCounterVec( prometheus.CounterOpts{ Namespace: namespace, @@ -1035,7 +1056,9 @@ func NewGroupMetrics(reg prometheus.Registerer) *Metrics { if reg != nil { reg.MustRegister( m.EvalDuration, + m.EvalDurationHistogram, m.IterationDuration, + m.IterationDurationHistogram, m.IterationsMissed, m.IterationsScheduled, m.EvalTotal, diff --git a/rules/manager.go b/rules/manager.go index d2fb0a7797..7d07217336 100644 --- a/rules/manager.go +++ b/rules/manager.go @@ -85,6 +85,7 @@ func DefaultEvalIterationFunc(ctx context.Context, g *Group, evalTimestamp time. timeSinceStart := time.Since(start) g.metrics.IterationDuration.Observe(timeSinceStart.Seconds()) + g.metrics.IterationDurationHistogram.Observe(timeSinceStart.Seconds()) g.updateRuleEvaluationTimeSum() g.setEvaluationTime(timeSinceStart) g.setLastEvaluation(start) diff --git a/scrape/metrics.go b/scrape/metrics.go index e7395c6191..634c52fb2d 100644 --- a/scrape/metrics.go +++ b/scrape/metrics.go @@ -15,6 +15,7 @@ package scrape import ( "fmt" + "time" "github.com/prometheus/client_golang/prometheus" ) @@ -36,6 +37,7 @@ type scrapeMetrics struct { targetScrapePoolTargetsAdded *prometheus.GaugeVec targetScrapePoolSymbolTableItems *prometheus.GaugeVec targetSyncIntervalLength *prometheus.SummaryVec + targetSyncIntervalLengthHistogram *prometheus.HistogramVec targetSyncFailed *prometheus.CounterVec // Used by targetScraper. @@ -46,6 +48,7 @@ type scrapeMetrics struct { // Used by scrapeLoop. targetIntervalLength *prometheus.SummaryVec + targetIntervalLengthHistogram *prometheus.HistogramVec targetScrapeSampleLimit prometheus.Counter targetScrapeSampleDuplicate prometheus.Counter targetScrapeSampleOutOfOrder prometheus.Counter @@ -152,6 +155,17 @@ func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) { }, []string{"scrape_job"}, ) + sm.targetSyncIntervalLengthHistogram = prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "prometheus_target_sync_length_histogram_seconds", + Help: "Actual interval to sync the scrape pool.", + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }, + []string{"scrape_job"}, + ) sm.targetSyncFailed = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "prometheus_target_sync_failed_total", @@ -185,6 +199,17 @@ func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) { }, []string{"interval"}, ) + sm.targetIntervalLengthHistogram = prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "prometheus_target_interval_length_histogram_seconds", + Help: "Actual intervals between scrapes.", + Buckets: []float64{.01, .1, 1, 10}, + NativeHistogramBucketFactor: 1.1, + NativeHistogramMaxBucketNumber: 100, + NativeHistogramMinResetDuration: 1 * time.Hour, + }, + []string{"interval"}, + ) sm.targetScrapeSampleLimit = prometheus.NewCounter( prometheus.CounterOpts{ Name: "prometheus_target_scrapes_exceeded_sample_limit_total", @@ -238,6 +263,7 @@ func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) { sm.targetScrapePoolReloads, sm.targetScrapePoolReloadsFailed, sm.targetSyncIntervalLength, + sm.targetSyncIntervalLengthHistogram, sm.targetScrapePoolSyncsCounter, sm.targetScrapePoolExceededTargetLimit, sm.targetScrapePoolTargetLimit, @@ -250,6 +276,7 @@ func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) { sm.targetScrapeCacheFlushForced, // Used by scrapeLoop. sm.targetIntervalLength, + sm.targetIntervalLengthHistogram, sm.targetScrapeSampleLimit, sm.targetScrapeSampleDuplicate, sm.targetScrapeSampleOutOfOrder, @@ -279,6 +306,7 @@ func (sm *scrapeMetrics) Unregister() { sm.reg.Unregister(sm.targetScrapePoolReloads) sm.reg.Unregister(sm.targetScrapePoolReloadsFailed) sm.reg.Unregister(sm.targetSyncIntervalLength) + sm.reg.Unregister(sm.targetSyncIntervalLengthHistogram) sm.reg.Unregister(sm.targetScrapePoolSyncsCounter) sm.reg.Unregister(sm.targetScrapePoolExceededTargetLimit) sm.reg.Unregister(sm.targetScrapePoolTargetLimit) @@ -288,6 +316,7 @@ func (sm *scrapeMetrics) Unregister() { sm.reg.Unregister(sm.targetScrapeExceededBodySizeLimit) sm.reg.Unregister(sm.targetScrapeCacheFlushForced) sm.reg.Unregister(sm.targetIntervalLength) + sm.reg.Unregister(sm.targetIntervalLengthHistogram) sm.reg.Unregister(sm.targetScrapeSampleLimit) sm.reg.Unregister(sm.targetScrapeSampleDuplicate) sm.reg.Unregister(sm.targetScrapeSampleOutOfOrder) diff --git a/scrape/scrape.go b/scrape/scrape.go index db662cb089..bbb93c8801 100644 --- a/scrape/scrape.go +++ b/scrape/scrape.go @@ -309,6 +309,7 @@ func (sp *scrapePool) stop() { sp.metrics.targetScrapePoolTargetsAdded.DeleteLabelValues(sp.config.JobName) sp.metrics.targetScrapePoolSymbolTableItems.DeleteLabelValues(sp.config.JobName) sp.metrics.targetSyncIntervalLength.DeleteLabelValues(sp.config.JobName) + sp.metrics.targetSyncIntervalLengthHistogram.DeleteLabelValues(sp.config.JobName) sp.metrics.targetSyncFailed.DeleteLabelValues(sp.config.JobName) } } @@ -505,6 +506,9 @@ func (sp *scrapePool) Sync(tgs []*targetgroup.Group) { sp.metrics.targetSyncIntervalLength.WithLabelValues(sp.config.JobName).Observe( time.Since(start).Seconds(), ) + sp.metrics.targetSyncIntervalLengthHistogram.WithLabelValues(sp.config.JobName).Observe( + time.Since(start).Seconds(), + ) sp.metrics.targetScrapePoolSyncsCounter.WithLabelValues(sp.config.JobName).Inc() } @@ -1420,6 +1424,9 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er sl.metrics.targetIntervalLength.WithLabelValues(sl.interval.String()).Observe( time.Since(last).Seconds(), ) + sl.metrics.targetIntervalLengthHistogram.WithLabelValues(sl.interval.String()).Observe( + time.Since(last).Seconds(), + ) } var total, added, seriesAdded, bytesRead int From abffb9284740208169cb0d226d9edecc68fb3e45 Mon Sep 17 00:00:00 2001 From: Solomon Jacobs Date: Thu, 27 Nov 2025 19:30:19 +0100 Subject: [PATCH 093/439] drop GO111MODULE=on (#17520) This is ignored as of go 1.17, see e.g., for reference https://go.dev/blog/go116-module-changes Signed-off-by: Solomon Jacobs --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1743c5a4b8..08355649f3 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ You can use the `go` tool to build and install the `prometheus` and `promtool` binaries into your `GOPATH`: ```bash -GO111MODULE=on go install github.com/prometheus/prometheus/cmd/... +go install github.com/prometheus/prometheus/cmd/... prometheus --config.file=your_config.yml ``` From b0649e08c4391c68009266779154adc72085d8cc Mon Sep 17 00:00:00 2001 From: Nikos Angelopoulos Date: Fri, 28 Nov 2025 10:41:00 +0100 Subject: [PATCH 094/439] rules: replace error strings with sentinel errors for duplicate labelsets (#17620) Signed-off-by: Nikos Angelopoulos --- rules/alerting.go | 6 +++++- rules/alerting_test.go | 2 +- rules/recording.go | 6 +++++- rules/recording_test.go | 2 +- 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/rules/alerting.go b/rules/alerting.go index b0151d7cb3..bb0763fbc6 100644 --- a/rules/alerting.go +++ b/rules/alerting.go @@ -46,6 +46,10 @@ const ( alertStateLabel = "alertstate" ) +// ErrDuplicateAlertLabelSet is returned when an alerting rule evaluation produces +// metrics with identical labelsets after applying alert labels. +var ErrDuplicateAlertLabelSet = errors.New("vector contains metrics with the same labelset after applying alert labels") + // AlertState denotes the state of an active alert. type AlertState int @@ -441,7 +445,7 @@ func (r *AlertingRule) Eval(ctx context.Context, queryOffset time.Duration, ts t resultFPs[h] = struct{}{} if _, ok := alerts[h]; ok { - return nil, errors.New("vector contains metrics with the same labelset after applying alert labels") + return nil, ErrDuplicateAlertLabelSet } alerts[h] = &Alert{ diff --git a/rules/alerting_test.go b/rules/alerting_test.go index dc5a6d1c43..b619d56b56 100644 --- a/rules/alerting_test.go +++ b/rules/alerting_test.go @@ -612,7 +612,7 @@ func TestAlertingRuleDuplicate(t *testing.T) { ) _, err := rule.Eval(ctx, 0, now, EngineQueryFunc(engine, storage), nil, 0) require.Error(t, err) - require.EqualError(t, err, "vector contains metrics with the same labelset after applying alert labels") + require.ErrorIs(t, err, ErrDuplicateAlertLabelSet) } func TestAlertingRuleLimit(t *testing.T) { diff --git a/rules/recording.go b/rules/recording.go index 2da6885f5b..1bc41b834a 100644 --- a/rules/recording.go +++ b/rules/recording.go @@ -30,6 +30,10 @@ import ( "github.com/prometheus/prometheus/promql/parser" ) +// ErrDuplicateRecordingLabelSet is returned when a recording rule evaluation produces +// metrics with identical labelsets after applying rule labels. +var ErrDuplicateRecordingLabelSet = errors.New("vector contains metrics with the same labelset after applying rule labels") + // A RecordingRule records its vector expression into new timeseries. type RecordingRule struct { name string @@ -104,7 +108,7 @@ func (rule *RecordingRule) Eval(ctx context.Context, queryOffset time.Duration, // Check that the rule does not produce identical metrics after applying // labels. if vector.ContainsSameLabelset() { - return nil, errors.New("vector contains metrics with the same labelset after applying rule labels") + return nil, ErrDuplicateRecordingLabelSet } numSeries := len(vector) diff --git a/rules/recording_test.go b/rules/recording_test.go index 014aa85ceb..44ef257f8f 100644 --- a/rules/recording_test.go +++ b/rules/recording_test.go @@ -176,7 +176,7 @@ func TestRuleEvalDuplicate(t *testing.T) { rule := NewRecordingRule("foo", expr, labels.FromStrings("test", "test")) _, err := rule.Eval(ctx, 0, now, EngineQueryFunc(engine, storage), nil, 0) require.Error(t, err) - require.EqualError(t, err, "vector contains metrics with the same labelset after applying rule labels") + require.ErrorIs(t, err, ErrDuplicateRecordingLabelSet) } func TestRecordingRuleLimit(t *testing.T) { From 77ba5c5fbdbf0623b630e604e862afc07115fc00 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Thu, 13 Nov 2025 18:16:29 +0000 Subject: [PATCH 095/439] [PERF] Scraping: skip an unnecessary step when there are relabel rules Before it would do Builder->Labels->Builder, now we skip the conversions. Signed-off-by: Bryan Boreham --- scrape/scrape.go | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/scrape/scrape.go b/scrape/scrape.go index bbb93c8801..b8f0efce3c 100644 --- a/scrape/scrape.go +++ b/scrape/scrape.go @@ -716,13 +716,9 @@ func mutateSampleLabels(lset labels.Labels, target *Target, honor bool, rc []*re } } - res := lb.Labels() + relabel.ProcessBuilder(lb, rc...) - if len(rc) > 0 { - res, _ = relabel.Process(res, rc...) - } - - return res + return lb.Labels() } func resolveConflictingExposedLabels(lb *labels.Builder, conflictingExposedLabels []labels.Label) { From 73b1fda131d7d4aa94160ad1f788e0f13d7e6a02 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Fri, 28 Nov 2025 09:12:04 +0100 Subject: [PATCH 096/439] prepare release v3.8.0 Signed-off-by: Jan Fajerski --- CHANGELOG.md | 7 ++----- VERSION | 2 +- web/ui/mantine-ui/package.json | 4 ++-- web/ui/module/codemirror-promql/package.json | 4 ++-- web/ui/module/lezer-promql/package.json | 2 +- web/ui/package-lock.json | 14 +++++++------- web/ui/package.json | 2 +- 7 files changed, 16 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 37db23b1a7..3304339867 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,14 +2,11 @@ ## main / unreleased -## 3.8.0-rc.1 / 2025-11-21 +## 3.8.0 / 2025-11-28 * [CHANGE] Remote-write 2 (receiving): Update to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md). "created timestamp" (CT) is now called "start timestamp" (ST). #17411 -* [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #17592 - -## 3.8.0-rc.0 / 2025-11-07 - * [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #17287 +* [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #17592 * [FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. #16483 * [FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. #17046 * [FEATURE] Native histograms are now a stable, but optional feature, use the `scrape_native_histogram` config setting. #17232 #17315 diff --git a/VERSION b/VERSION index 493bf7002c..19811903a7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.8.0-rc.1 +3.8.0 diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json index b35ebf8f92..7ec13b1b8d 100644 --- a/web/ui/mantine-ui/package.json +++ b/web/ui/mantine-ui/package.json @@ -1,7 +1,7 @@ { "name": "@prometheus-io/mantine-ui", "private": true, - "version": "0.308.0-rc.1", + "version": "0.308.0", "type": "module", "scripts": { "start": "vite", @@ -28,7 +28,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.308.0-rc.1", + "@prometheus-io/codemirror-promql": "0.308.0", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json index 5a23ead1f0..ee7bcc045f 100644 --- a/web/ui/module/codemirror-promql/package.json +++ b/web/ui/module/codemirror-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/codemirror-promql", - "version": "0.308.0-rc.1", + "version": "0.308.0", "description": "a CodeMirror mode for the PromQL language", "types": "dist/esm/index.d.ts", "module": "dist/esm/index.js", @@ -29,7 +29,7 @@ }, "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md", "dependencies": { - "@prometheus-io/lezer-promql": "0.308.0-rc.1", + "@prometheus-io/lezer-promql": "0.308.0", "lru-cache": "^11.2.2" }, "devDependencies": { diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json index f6152a35b7..034ead9741 100644 --- a/web/ui/module/lezer-promql/package.json +++ b/web/ui/module/lezer-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/lezer-promql", - "version": "0.308.0-rc.1", + "version": "0.308.0", "description": "lezer-based PromQL grammar", "main": "dist/index.cjs", "type": "module", diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json index e2271f2977..7f2961784b 100644 --- a/web/ui/package-lock.json +++ b/web/ui/package-lock.json @@ -1,12 +1,12 @@ { "name": "prometheus-io", - "version": "0.308.0-rc.1", + "version": "0.308.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "prometheus-io", - "version": "0.308.0-rc.1", + "version": "0.308.0", "workspaces": [ "mantine-ui", "module/*" @@ -24,7 +24,7 @@ }, "mantine-ui": { "name": "@prometheus-io/mantine-ui", - "version": "0.308.0-rc.1", + "version": "0.308.0", "dependencies": { "@codemirror/autocomplete": "^6.19.1", "@codemirror/language": "^6.11.3", @@ -42,7 +42,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.308.0-rc.1", + "@prometheus-io/codemirror-promql": "0.308.0", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", @@ -88,10 +88,10 @@ }, "module/codemirror-promql": { "name": "@prometheus-io/codemirror-promql", - "version": "0.308.0-rc.1", + "version": "0.308.0", "license": "Apache-2.0", "dependencies": { - "@prometheus-io/lezer-promql": "0.308.0-rc.1", + "@prometheus-io/lezer-promql": "0.308.0", "lru-cache": "^11.2.2" }, "devDependencies": { @@ -121,7 +121,7 @@ }, "module/lezer-promql": { "name": "@prometheus-io/lezer-promql", - "version": "0.308.0-rc.1", + "version": "0.308.0", "license": "Apache-2.0", "devDependencies": { "@lezer/generator": "^1.8.0", diff --git a/web/ui/package.json b/web/ui/package.json index 2cf4a6819f..5023d1d21b 100644 --- a/web/ui/package.json +++ b/web/ui/package.json @@ -1,7 +1,7 @@ { "name": "prometheus-io", "description": "Monorepo for the Prometheus UI", - "version": "0.308.0-rc.1", + "version": "0.308.0", "private": true, "scripts": { "build": "bash build_ui.sh --all", From e894d7a271f85ee0be0d7442f6dcd0b0ca208acb Mon Sep 17 00:00:00 2001 From: aviralgarg05 Date: Sat, 29 Nov 2025 17:15:59 +0530 Subject: [PATCH 097/439] promqltest: Add optional counter reset hint comparison for native histograms This commit implements counter reset hint comparison in the promqltest framework to address issue #17615. Previously, while test definitions could specify a counter_reset_hint in expected native histogram results, the framework did not actually compare this hint between expected and actual results. The implementation adds optional comparison logic to the compareNativeHistogram function: - If the expected histogram has UnknownCounterReset (the default), the hint is not compared (meaning "don't care") - If the expected histogram explicitly specifies CounterReset, NotCounterReset, or GaugeType, it is verified against the actual histogram's hint This allows tests to verify that PromQL functions correctly set or preserve counter reset hints while maintaining backward compatibility with existing tests that don't specify explicit hints. Fixes #17615 Signed-off-by: aviralgarg05 --- promql/promqltest/test.go | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go index b16433c14e..d1702ba61b 100644 --- a/promql/promqltest/test.go +++ b/promql/promqltest/test.go @@ -1163,6 +1163,14 @@ func compareNativeHistogram(exp, cur *histogram.FloatHistogram) bool { return false } + // Compare CounterResetHint only if explicitly specified in expected histogram. + // UnknownCounterReset (the default) means "don't care about the hint". + if exp.CounterResetHint != histogram.UnknownCounterReset { + if exp.CounterResetHint != cur.CounterResetHint { + return false + } + } + return true } From 488466246fccfa9b8c0c1454489726cb1f87c86a Mon Sep 17 00:00:00 2001 From: aviralgarg05 Date: Sun, 30 Nov 2025 18:01:51 +0530 Subject: [PATCH 098/439] promqltest: Fix test expectation for counter reset hint comparison The test at line 1283 for avg_over_time(nhcb_metric[13m]) incorrectly expected counter_reset_hint:gauge in the result. However, the actual avg_over_time implementation does not explicitly set the CounterResetHint to GaugeType on its output histogram. With the new counter reset hint comparison logic added to the promqltest framework (which compares hints when explicitly specified in expected results), this incorrect expectation was now being caught. This fix removes the incorrect counter_reset_hint:gauge from the expected result, allowing the test to correctly verify the avg_over_time behavior without asserting a specific hint value that the function does not set. The counter reset hint comparison logic works as designed: if the expected histogram has UnknownCounterReset (the default when not specified), no comparison is performed. Only when a hint is explicitly specified in the test expectation will it be compared against the actual result. Fixes the test failure introduced by the counter reset hint comparison feature in promqltest. Signed-off-by: Aviral Garg Signed-off-by: aviralgarg05 --- promql/promqltest/testdata/native_histograms.test | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/promql/promqltest/testdata/native_histograms.test b/promql/promqltest/testdata/native_histograms.test index fd4b1f4178..d66400f787 100644 --- a/promql/promqltest/testdata/native_histograms.test +++ b/promql/promqltest/testdata/native_histograms.test @@ -1283,7 +1283,7 @@ eval instant at 12m sum_over_time(nhcb_metric[13m]) eval instant at 12m avg_over_time(nhcb_metric[13m]) expect no_warn expect info msg: PromQL info: mismatched custom buckets were reconciled during aggregation - {} {{schema:-53 count:1 sum:1 custom_values:[5] counter_reset_hint:gauge buckets:[1]}} + {} {{schema:-53 count:1 sum:1 custom_values:[5] buckets:[1]}} eval instant at 12m last_over_time(nhcb_metric[13m]) expect no_warn From 0ac2221a205f06743d36d684843a568001ccbec7 Mon Sep 17 00:00:00 2001 From: zjumathcode Date: Mon, 1 Dec 2025 19:14:19 +0800 Subject: [PATCH 099/439] chore: Fix function name typo in createBatchSpan comment Signed-off-by: zjumathcode --- storage/remote/queue_manager.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/storage/remote/queue_manager.go b/storage/remote/queue_manager.go index 73a4896f19..5fc5f5564b 100644 --- a/storage/remote/queue_manager.go +++ b/storage/remote/queue_manager.go @@ -2299,7 +2299,7 @@ func (b *batchMetricsUpdater) recordRetry(sc sendBatchContext) { b.metrics.retriedHistogramsTotal.Add(float64(sc.histogramCount)) } -// createSpan creates and configures an OpenTelemetry span for batch sending. +// createBatchSpan creates and configures an OpenTelemetry span for batch sending. func createBatchSpan(ctx context.Context, sc sendBatchContext, remoteName, remoteURL string, try int) (context.Context, trace.Span) { ctx, span := otel.Tracer("").Start(ctx, "Remote Send Batch") span.SetAttributes( From 8a1086a128f5b02143c2cea4a3301242b6854646 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Mierzwa?= Date: Tue, 2 Dec 2025 10:39:45 +0000 Subject: [PATCH 100/439] feat: Add flag that blocks lvl 1 compactions until upload is confirmed in an external JSON file (#17435) * Delay compactions until Thanos uploads all blocks Using Thanos sidecar with Prometheus requires us to disable TSDB compactions on Prometheus side by setting --storage.tsdb.min-block-duration and --storage.tsdb.max-block-duration to the same value. See https://thanos.io/tip/components/sidecar.md. The main problem this avoids is that Prometheus might compact given block before Thanos uploads it, creating a gap in Thanos metrics. Thanos does not upload compacted blocks because that would upload the same sample multiple times. You can tell Thanos to upload compacted blocks but that is aimed at one time migrations. This patch creates a bridge between Thanos and Prometheus by allowing Prometheus to read the shipper file Thanos creates, where it tracks which blocks were already uploaded, and using that data delays compaction of blocks until they are marked as uploaded by Thanos. Thanks to this both services can coordinate with each other (in a way) and we can stop disabling compaction on Prometheus side when Thanos uploads are enabled. The reason to have this is that disabling compactions have very dramatic performance cost. Since most time series exist for longer than a single block duration (2h by default) large chunks of block index will reference the same series, so 10 * 2h blocks will each have an index that is usually fairly big and is almost the same for all 10 blocks. Compaction de-duplicates the index so merging 10 blocks together would leave us with a single index that is around the same size as each of these 10 2h blocks would have (plus some extra for series that only exists in some blocks, but not all). Every range query that iterates over all 10 blocks would then have to read each index and so we're doing 10x more work then if we had a single compacted block. Signed-off-by: Lukasz Mierzwa * Rename structs and functions to make this more generic Signed-off-by: Lukasz Mierzwa * Address review comments Signed-off-by: Lukasz Mierzwa * Cache UploadMeta for 1 minute Signed-off-by: Lukasz Mierzwa --------- Signed-off-by: Lukasz Mierzwa --- cmd/prometheus/main.go | 62 +++++++++++++- cmd/prometheus/upload_test.go | 144 ++++++++++++++++++++++++++++++++ docs/command-line/prometheus.md | 1 + tsdb/compact.go | 21 ++++- tsdb/db.go | 5 ++ 5 files changed, 230 insertions(+), 3 deletions(-) create mode 100644 cmd/prometheus/upload_test.go diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go index 6ea65c879a..f7757968b7 100644 --- a/cmd/prometheus/main.go +++ b/cmd/prometheus/main.go @@ -16,6 +16,7 @@ package main import ( "context" + "encoding/json" "errors" "fmt" "log/slog" @@ -456,8 +457,9 @@ func main() { Default("true").Hidden().BoolVar(&cfg.tsdb.EnableOverlappingCompaction) var ( - tsdbWALCompression bool - tsdbWALCompressionType string + tsdbWALCompression bool + tsdbWALCompressionType string + tsdbDelayCompactFilePath string ) serverOnlyFlag(a, "storage.tsdb.wal-compression", "Compress the tsdb WAL. If false, the --storage.tsdb.wal-compression-type flag is ignored."). Hidden().Default("true").BoolVar(&tsdbWALCompression) @@ -474,6 +476,9 @@ func main() { serverOnlyFlag(a, "storage.tsdb.delayed-compaction.max-percent", "Sets the upper limit for the random compaction delay, specified as a percentage of the head chunk range. 100 means the compaction can be delayed by up to the entire head chunk range. Only effective when the delayed-compaction feature flag is enabled."). Default("10").Hidden().IntVar(&cfg.tsdb.CompactionDelayMaxPercent) + serverOnlyFlag(a, "storage.tsdb.delay-compact-file.path", "Path to a JSON file with uploaded TSDB blocks e.g. Thanos shipper meta file. If set TSDB will only compact 1 level blocks that are marked as uploaded in that file, improving external storage integrations e.g. with Thanos sidecar. 1+ level compactions won't be delayed."). + Default("").StringVar(&tsdbDelayCompactFilePath) + agentOnlyFlag(a, "storage.agent.path", "Base path for metrics storage."). Default("data-agent/").StringVar(&cfg.agentStoragePath) @@ -703,6 +708,12 @@ func main() { } } + if tsdbDelayCompactFilePath != "" { + logger.Info("Compactions will be delayed for blocks not marked as uploaded in the file tracking uploads", "path", tsdbDelayCompactFilePath) + cfg.tsdb.BlockCompactionExcludeFunc = exludeBlocksPendingUpload( + logger, tsdbDelayCompactFilePath) + } + // Now that the validity of the config is established, set the config // success metrics accordingly, although the config isn't really loaded // yet. This will happen later (including setting these metrics again), @@ -1883,6 +1894,7 @@ type tsdbOptions struct { CompactionDelayMaxPercent int EnableOverlappingCompaction bool UseUncachedIO bool + BlockCompactionExcludeFunc tsdb.BlockExcludeFilterFunc } func (opts tsdbOptions) ToTSDBOptions() tsdb.Options { @@ -1906,6 +1918,7 @@ func (opts tsdbOptions) ToTSDBOptions() tsdb.Options { CompactionDelayMaxPercent: opts.CompactionDelayMaxPercent, EnableOverlappingCompaction: opts.EnableOverlappingCompaction, UseUncachedIO: opts.UseUncachedIO, + BlockCompactionExcludeFunc: opts.BlockCompactionExcludeFunc, } } @@ -1970,3 +1983,48 @@ func (p *rwProtoMsgFlagParser) Set(opt string) error { *p.msgs = append(*p.msgs, t) return nil } + +type UploadMeta struct { + Uploaded []string `json:"uploaded"` +} + +// Cache the last read UploadMeta. +var ( + tsdbDelayCompactLastMeta *UploadMeta // The content of uploadMetaPath from the last time we've opened it. + tsdbDelayCompactLastMetaTime time.Time // The timestamp at which we stored tsdbDelayCompactLastMeta last time. +) + +func exludeBlocksPendingUpload(logger *slog.Logger, uploadMetaPath string) tsdb.BlockExcludeFilterFunc { + return func(meta *tsdb.BlockMeta) bool { + if meta.Compaction.Level > 1 { + // Blocks with level > 1 are assumed to be not uploaded, thus no need to delay those. + // See `storage.tsdb.delay-compact-file.path` flag for detail. + return false + } + + // If we have cached uploadMetaPath content that was stored in the last minute the use it. + if tsdbDelayCompactLastMeta != nil && + tsdbDelayCompactLastMetaTime.After(time.Now().UTC().Add(time.Minute*-1)) { + return !slices.Contains(tsdbDelayCompactLastMeta.Uploaded, meta.ULID.String()) + } + + // We don't have anything cached or it's older than a minute. Try to open and parse the uploadMetaPath path. + data, err := os.ReadFile(uploadMetaPath) + if err != nil { + logger.Warn("cannot open TSDB upload meta file", slog.String("path", uploadMetaPath), slog.Any("err", err)) + return false + } + + var uploadMeta UploadMeta + if err = json.Unmarshal(data, &uploadMeta); err != nil { + logger.Warn("cannot parse TSDB upload meta file", slog.String("path", uploadMetaPath), slog.Any("err", err)) + return false + } + + // We have parsed the uploadMetaPath file, cache it. + tsdbDelayCompactLastMeta = &uploadMeta + tsdbDelayCompactLastMetaTime = time.Now().UTC() + + return !slices.Contains(uploadMeta.Uploaded, meta.ULID.String()) + } +} diff --git a/cmd/prometheus/upload_test.go b/cmd/prometheus/upload_test.go new file mode 100644 index 0000000000..565531b016 --- /dev/null +++ b/cmd/prometheus/upload_test.go @@ -0,0 +1,144 @@ +// Copyright 2025 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "encoding/json" + "os" + "path" + "testing" + "time" + + "github.com/oklog/ulid/v2" + "github.com/prometheus/common/promslog" + "github.com/stretchr/testify/require" + + "github.com/prometheus/prometheus/tsdb" +) + +func TestBlockExcludeFilter(t *testing.T) { + for _, test := range []struct { + summary string // Description of the test case. + uploaded []ulid.ULID // List of blocks marked as uploaded inside the shipper file. + setupFn func(string) // Optional function to run before the test, takes the path to the shipper file. + meta tsdb.BlockMeta // Meta of the block we're checking. + isExcluded bool // What do we expect to be returned. + }{ + { + summary: "missing file", + setupFn: func(path string) { + // Delete shipper file to test error handling. + os.Remove(path) + }, + meta: tsdb.BlockMeta{ULID: ulid.MustNew(1, nil)}, + isExcluded: false, + }, + { + summary: "corrupt file", + setupFn: func(path string) { + // Overwrite the shipper file content with invalid JSON. + os.WriteFile(path, []byte("{["), 0o644) + }, + meta: tsdb.BlockMeta{ULID: ulid.MustNew(1, nil)}, + isExcluded: false, + }, + { + summary: "empty uploaded list", + uploaded: []ulid.ULID{}, + meta: tsdb.BlockMeta{ULID: ulid.MustNew(1, nil)}, + isExcluded: true, + }, + { + summary: "block meta not present in the uploaded list, level=1", + uploaded: []ulid.ULID{ulid.MustNew(1, nil), ulid.MustNew(3, nil)}, + meta: tsdb.BlockMeta{ + ULID: ulid.MustNew(2, nil), + Compaction: tsdb.BlockMetaCompaction{Level: 1}, + }, + isExcluded: true, + }, + { + summary: "block meta not present in the uploaded list, level=2", + uploaded: []ulid.ULID{ulid.MustNew(1, nil), ulid.MustNew(3, nil)}, + meta: tsdb.BlockMeta{ + ULID: ulid.MustNew(2, nil), + Compaction: tsdb.BlockMetaCompaction{Level: 2}, + }, + isExcluded: false, + }, + { + summary: "block meta present in the uploaded list", + uploaded: []ulid.ULID{ulid.MustNew(1, nil), ulid.MustNew(2, nil), ulid.MustNew(3, nil)}, + meta: tsdb.BlockMeta{ULID: ulid.MustNew(2, nil)}, + isExcluded: false, + }, + { + summary: "don't read the file if there's valid cache", + setupFn: func(path string) { + // Remove the shipper file, cache should be used instead. + require.NoError(t, os.Remove(path)) + // Set cached values + tsdbDelayCompactLastMeta = &UploadMeta{ + Uploaded: []string{ + ulid.MustNew(1, nil).String(), + ulid.MustNew(2, nil).String(), + ulid.MustNew(3, nil).String(), + }, + } + tsdbDelayCompactLastMetaTime = time.Now().UTC().Add(time.Second * -1) + }, + uploaded: []ulid.ULID{}, + meta: tsdb.BlockMeta{ULID: ulid.MustNew(2, nil)}, + isExcluded: false, + }, + { + summary: "read the file if there's cache but expired", + setupFn: func(_ string) { + // Set the cache but make it too old + tsdbDelayCompactLastMeta = &UploadMeta{ + Uploaded: []string{}, + } + tsdbDelayCompactLastMetaTime = time.Now().UTC().Add(time.Second * -61) + }, + uploaded: []ulid.ULID{ulid.MustNew(1, nil), ulid.MustNew(2, nil), ulid.MustNew(3, nil)}, + meta: tsdb.BlockMeta{ULID: ulid.MustNew(2, nil)}, + isExcluded: false, + }, + } { + t.Run(test.summary, func(t *testing.T) { + dir := t.TempDir() + shipperPath := path.Join(dir, "shipper.json") + + uploaded := make([]string, 0, len(test.uploaded)) + for _, ul := range test.uploaded { + uploaded = append(uploaded, ul.String()) + } + ts := UploadMeta{Uploaded: uploaded} + data, err := json.Marshal(ts) + require.NoError(t, err, "failed to marshall upload meta file") + require.NoError(t, os.WriteFile(shipperPath, data, 0o644), "failed to write upload meta file") + + tsdbDelayCompactLastMeta = nil + tsdbDelayCompactLastMetaTime = time.Time{} + + if test.setupFn != nil { + test.setupFn(shipperPath) + } + + fn := exludeBlocksPendingUpload(promslog.NewNopLogger(), shipperPath) + isExcluded := fn(&test.meta) + require.Equal(t, test.isExcluded, isExcluded) + }) + } +} diff --git a/docs/command-line/prometheus.md b/docs/command-line/prometheus.md index c79dad40a2..d4a8cd4f20 100644 --- a/docs/command-line/prometheus.md +++ b/docs/command-line/prometheus.md @@ -38,6 +38,7 @@ The Prometheus monitoring server | --storage.tsdb.retention.size | [DEPRECATED] Maximum number of bytes that can be stored for blocks. A unit is required, supported units: B, KB, MB, GB, TB, PB, EB. Ex: "512MB". Based on powers-of-2, so 1KB is 1024B. This flag has been deprecated, use the storage.tsdb.retention.size field in the config file instead. Use with server mode only. | | | --storage.tsdb.no-lockfile | Do not create lockfile in data directory. Use with server mode only. | `false` | | --storage.tsdb.head-chunks-write-queue-size | Size of the queue through which head chunks are written to the disk to be m-mapped, 0 disables the queue completely. Experimental. Use with server mode only. | `0` | +| --storage.tsdb.delay-compact-file.path | Path to a JSON file with uploaded TSDB blocks e.g. Thanos shipper meta file. If set TSDB will only compact 1 level blocks that are marked as uploaded in that file, improving external storage integrations e.g. with Thanos sidecar. 1+ level compactions won't be delayed. Use with server mode only. | | | --storage.agent.path | Base path for metrics storage. Use with agent mode only. | `data-agent/` | | --storage.agent.wal-compression | Compress the agent WAL. If false, the --storage.agent.wal-compression-type flag is ignored. Use with agent mode only. | `true` | | --storage.agent.retention.min-time | Minimum age samples may be before being considered for deletion when the WAL is truncated Use with agent mode only. | | diff --git a/tsdb/compact.go b/tsdb/compact.go index 49e88d6320..7ad6f8bb24 100644 --- a/tsdb/compact.go +++ b/tsdb/compact.go @@ -87,6 +87,7 @@ type LeveledCompactor struct { maxBlockChunkSegmentSize int64 useUncachedIO bool mergeFunc storage.VerticalChunkSeriesMergeFunc + blockExcludeFunc BlockExcludeFilterFunc postingsEncoder index.PostingsEncoder postingsDecoderFactory PostingsDecoderFactory enableOverlappingCompaction bool @@ -160,16 +161,24 @@ type LeveledCompactorOptions struct { // PE specifies the postings encoder. It is called when compactor is writing out the postings for a label name/value pair during compaction. // If it is nil then the default encoder is used. At the moment that is the "raw" encoder. See index.EncodePostingsRaw for more. PE index.PostingsEncoder + // PD specifies the postings decoder factory to return different postings decoder based on BlockMeta. It is called when opening a block or opening the index file. // If it is nil then a default decoder is used, compatible with Prometheus v2. PD PostingsDecoderFactory + // MaxBlockChunkSegmentSize is the max block chunk segment size. If it is 0 then the default chunks.DefaultChunkSegmentSize is used. MaxBlockChunkSegmentSize int64 + // MergeFunc is used for merging series together in vertical compaction. By default storage.NewCompactingChunkSeriesMerger(storage.ChainedSeriesMerge) is used. MergeFunc storage.VerticalChunkSeriesMergeFunc + + // BlockExcludeFilter is used to decide which blocks are exluded from compactions. + BlockExcludeFilter BlockExcludeFilterFunc + // EnableOverlappingCompaction enables compaction of overlapping blocks. In Prometheus it is always enabled. // It is useful for downstream projects like Mimir, Cortex, Thanos where they have a separate component that does compaction. EnableOverlappingCompaction bool + // Metrics is set of metrics for Compactor. By default, NewCompactorMetrics would be called to initialize metrics unless it is provided. Metrics *CompactorMetrics // UseUncachedIO allows bypassing the page cache when appropriate. @@ -178,7 +187,9 @@ type LeveledCompactorOptions struct { type PostingsDecoderFactory func(meta *BlockMeta) index.PostingsDecoder -func DefaultPostingsDecoderFactory(*BlockMeta) index.PostingsDecoder { +type BlockExcludeFilterFunc func(meta *BlockMeta) bool + +func DefaultPostingsDecoderFactory(_ *BlockMeta) index.PostingsDecoder { return index.DecodePostingsRaw } @@ -226,6 +237,7 @@ func NewLeveledCompactorWithOptions(ctx context.Context, r prometheus.Registerer postingsEncoder: pe, postingsDecoderFactory: opts.PD, enableOverlappingCompaction: opts.EnableOverlappingCompaction, + blockExcludeFunc: opts.BlockExcludeFilter, }, nil } @@ -250,12 +262,19 @@ func (c *LeveledCompactor) Plan(dir string) ([]string, error) { if err != nil { return nil, err } + if c.blockExcludeFunc != nil && c.blockExcludeFunc(meta) { + break + } dms = append(dms, dirMeta{dir, meta}) } return c.plan(dms) } func (c *LeveledCompactor) plan(dms []dirMeta) ([]string, error) { + if len(dms) == 0 { + return nil, nil + } + slices.SortFunc(dms, func(a, b dirMeta) int { switch { case a.meta.MinTime < b.meta.MinTime: diff --git a/tsdb/db.go b/tsdb/db.go index c57ae84c9c..f8d36c5479 100644 --- a/tsdb/db.go +++ b/tsdb/db.go @@ -219,6 +219,10 @@ type Options struct { // UseUncachedIO allows bypassing the page cache when appropriate. UseUncachedIO bool + + // BlockCompactionExcludeFunc is a function which returns true for blocks that should NOT be compacted. + // It's passed down to the TSDB compactor. + BlockCompactionExcludeFunc BlockExcludeFilterFunc } type NewCompactorFunc func(ctx context.Context, r prometheus.Registerer, l *slog.Logger, ranges []int64, pool chunkenc.Pool, opts *Options) (Compactor, error) @@ -908,6 +912,7 @@ func open(dir string, l *slog.Logger, r prometheus.Registerer, opts *Options, rn EnableOverlappingCompaction: opts.EnableOverlappingCompaction, PD: opts.PostingsDecoderFactory, UseUncachedIO: opts.UseUncachedIO, + BlockExcludeFilter: opts.BlockCompactionExcludeFunc, }) } if err != nil { From 0e682a70a6819838cbc36fe9f51f144c8bbdb674 Mon Sep 17 00:00:00 2001 From: Ben Edmunds Date: Tue, 2 Dec 2025 11:45:23 +0000 Subject: [PATCH 101/439] RW2: Allow custom scope in azuread (#17483) Signed-off-by: Ben Edmunds --- docs/configuration/configuration.md | 8 ++ storage/remote/azuread/azuread.go | 25 ++++- storage/remote/azuread/azuread_test.go | 93 +++++++++++++++++++ .../testdata/azuread_bad_scope_invalid.yaml | 6 ++ .../azuread_good_oauth_customscope.yaml | 6 ++ 5 files changed, 134 insertions(+), 4 deletions(-) create mode 100644 storage/remote/azuread/testdata/azuread_bad_scope_invalid.yaml create mode 100644 storage/remote/azuread/testdata/azuread_good_oauth_customscope.yaml diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md index c31d70389b..0b944008ef 100644 --- a/docs/configuration/configuration.md +++ b/docs/configuration/configuration.md @@ -3277,6 +3277,14 @@ azuread: [ sdk: [ tenant_id: ] ] + # Optional custom OAuth 2.0 scope to request when acquiring tokens. + # If not specified, defaults to the appropriate monitoring scope for the cloud: + # - AzurePublic: https://monitor.azure.com//.default + # - AzureGovernment: https://monitor.azure.us//.default + # - AzureChina: https://monitor.azure.cn//.default + # Use this to authenticate against custom Azure applications or non-standard endpoints. + [ scope: ] + # WARNING: Remote write is NOT SUPPORTED by Google Cloud. This configuration is reserved for future use. # Optional Google Cloud Monitoring configuration. # Cannot be used at the same time as basic_auth, authorization, oauth2, sigv4 or azuread. diff --git a/storage/remote/azuread/azuread.go b/storage/remote/azuread/azuread.go index ea2a816d94..638ba586fc 100644 --- a/storage/remote/azuread/azuread.go +++ b/storage/remote/azuread/azuread.go @@ -103,6 +103,9 @@ type AzureADConfig struct { //nolint:revive // exported. // Cloud is the Azure cloud in which the service is running. Example: AzurePublic/AzureGovernment/AzureChina. Cloud string `yaml:"cloud,omitempty"` + + // Scope is the custom OAuth 2.0 scope to request when acquiring tokens. + Scope string `yaml:"scope,omitempty"` } // azureADRoundTripper is used to store the roundtripper and the tokenprovider. @@ -211,6 +214,12 @@ func (c *AzureADConfig) Validate() error { } } + if c.Scope != "" { + if matched, err := regexp.MatchString("^[\\w\\s:/.\\-]+$", c.Scope); err != nil || !matched { + return errors.New("the provided scope contains invalid characters") + } + } + return nil } @@ -360,14 +369,22 @@ func newSDKTokenCredential(clientOpts *azcore.ClientOptions, sdkConfig *SDKConfi // newTokenProvider helps to fetch accessToken for different types of credential. This also takes care of // refreshing the accessToken before expiry. This accessToken is attached to the Authorization header while making requests. func newTokenProvider(cfg *AzureADConfig, cred azcore.TokenCredential) (*tokenProvider, error) { - audience, err := getAudience(cfg.Cloud) - if err != nil { - return nil, err + var scopes []string + + // Use custom scope if provided, otherwise fallback to cloud-specific audience + if cfg.Scope != "" { + scopes = []string{cfg.Scope} + } else { + audience, err := getAudience(cfg.Cloud) + if err != nil { + return nil, err + } + scopes = []string{audience} } tokenProvider := &tokenProvider{ credentialClient: cred, - options: &policy.TokenRequestOptions{Scopes: []string{audience}}, + options: &policy.TokenRequestOptions{Scopes: scopes}, } return tokenProvider, nil diff --git a/storage/remote/azuread/azuread_test.go b/storage/remote/azuread/azuread_test.go index d581f0218a..986a01695c 100644 --- a/storage/remote/azuread/azuread_test.go +++ b/storage/remote/azuread/azuread_test.go @@ -198,6 +198,11 @@ func TestAzureAdConfig(t *testing.T) { filename: "testdata/azuread_bad_workloadidentity_missingtenantid.yaml", err: "must provide an Azure Workload Identity tenant_id in the Azure AD config", }, + // Invalid scope validation. + { + filename: "testdata/azuread_bad_scope_invalid.yaml", + err: "the provided scope contains invalid characters", + }, // Valid config with missing optionally cloud field. { filename: "testdata/azuread_good_cloudmissing.yaml", @@ -222,6 +227,10 @@ func TestAzureAdConfig(t *testing.T) { { filename: "testdata/azuread_good_workloadidentity.yaml", }, + // Valid OAuth config with custom scope. + { + filename: "testdata/azuread_good_oauth_customscope.yaml", + }, } for _, c := range cases { _, err := loadAzureAdConfig(c.filename) @@ -387,3 +396,87 @@ func getToken() azcore.AccessToken { ExpiresOn: time.Now().Add(10 * time.Second), } } + +func TestCustomScopeSupport(t *testing.T) { + mockCredential := new(mockCredential) + testToken := &azcore.AccessToken{ + Token: testTokenString, + ExpiresOn: testTokenExpiry(), + } + + cases := []struct { + name string + cfg *AzureADConfig + expectedScope string + }{ + { + name: "Custom scope with OAuth", + cfg: &AzureADConfig{ + Cloud: "AzurePublic", + OAuth: &OAuthConfig{ + ClientID: dummyClientID, + ClientSecret: dummyClientSecret, + TenantID: dummyTenantID, + }, + Scope: "https://custom-app.com/.default", + }, + expectedScope: "https://custom-app.com/.default", + }, + { + name: "Custom scope with Managed Identity", + cfg: &AzureADConfig{ + Cloud: "AzurePublic", + ManagedIdentity: &ManagedIdentityConfig{ + ClientID: dummyClientID, + }, + Scope: "https://monitor.azure.com//.default", + }, + expectedScope: "https://monitor.azure.com//.default", + }, + { + name: "Default scope fallback with OAuth", + cfg: &AzureADConfig{ + Cloud: "AzurePublic", + OAuth: &OAuthConfig{ + ClientID: dummyClientID, + ClientSecret: dummyClientSecret, + TenantID: dummyTenantID, + }, + }, + expectedScope: IngestionPublicAudience, + }, + { + name: "Default scope fallback with China cloud", + cfg: &AzureADConfig{ + Cloud: "AzureChina", + OAuth: &OAuthConfig{ + ClientID: dummyClientID, + ClientSecret: dummyClientSecret, + TenantID: dummyTenantID, + }, + }, + expectedScope: IngestionChinaAudience, + }, + } + + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + // Set up mock to capture the actual scopes used + mockCredential.On("GetToken", mock.Anything, mock.MatchedBy(func(options policy.TokenRequestOptions) bool { + return len(options.Scopes) == 1 && options.Scopes[0] == c.expectedScope + })).Return(*testToken, nil).Once() + + tokenProvider, err := newTokenProvider(c.cfg, mockCredential) + require.NoError(t, err) + require.NotNil(t, tokenProvider) + + // Verify that the token provider uses the expected scope + token, err := tokenProvider.getAccessToken(context.Background()) + require.NoError(t, err) + require.Equal(t, testTokenString, token) + + // Reset mock for next test + mockCredential.ExpectedCalls = nil + }) + } +} diff --git a/storage/remote/azuread/testdata/azuread_bad_scope_invalid.yaml b/storage/remote/azuread/testdata/azuread_bad_scope_invalid.yaml new file mode 100644 index 0000000000..2e5678d783 --- /dev/null +++ b/storage/remote/azuread/testdata/azuread_bad_scope_invalid.yaml @@ -0,0 +1,6 @@ +cloud: AzurePublic +oauth: + client_id: 00000000-0000-0000-0000-000000000000 + client_secret: Cl1ent$ecret! + tenant_id: 00000000-a12b-3cd4-e56f-000000000000 +scope: "invalid<>scope*chars" diff --git a/storage/remote/azuread/testdata/azuread_good_oauth_customscope.yaml b/storage/remote/azuread/testdata/azuread_good_oauth_customscope.yaml new file mode 100644 index 0000000000..f7adf8b0af --- /dev/null +++ b/storage/remote/azuread/testdata/azuread_good_oauth_customscope.yaml @@ -0,0 +1,6 @@ +cloud: AzurePublic +oauth: + client_id: 00000000-0000-0000-0000-000000000000 + client_secret: Cl1ent$ecret! + tenant_id: 00000000-a12b-3cd4-e56f-000000000000 +scope: "https://custom-app.com/.default" From 9f0b52d73a683fd10d11f554ab1d68dc30b18b2c Mon Sep 17 00:00:00 2001 From: Gabriel Filion Date: Mon, 7 Oct 2024 18:57:10 -0400 Subject: [PATCH 102/439] docs: Describe how time() is set to start at 0 in unit tests The return value of functions relating to the current time, e.g. time(), is set by promtool to start at timestamp 0 at the start of a test's evaluation. This has the very nice consequence that tests can run reliably without depending on when they are run. It does, however, mean that tests will give out results that can be unexpected by users. If this behaviour is documented, then users will be empowered to write tests for their rules that use time-dependent functions. (Closes: prometheus/docs#1464) Signed-off-by: Gabriel Filion --- docs/configuration/unit_testing_rules.md | 27 ++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/configuration/unit_testing_rules.md b/docs/configuration/unit_testing_rules.md index d237c8cf88..13b0445c7c 100644 --- a/docs/configuration/unit_testing_rules.md +++ b/docs/configuration/unit_testing_rules.md @@ -275,3 +275,30 @@ groups: summary: "Instance {{ $labels.instance }} down" description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes." ``` + +### Time within tests + +It should be noted that in all tests, either in `alert_test_case` or +`promql_test_case`, the output from all functions related to the current time, +for example the `time()` and `day_of_*()` functions, will output a consistent value +for tests. + +At the start of the test evaluation, `time()` returns 0 and therefore when under test +`time()` will return a value of `0 + eval_time`. + +If you need to write tests for alerts that use functions relating to the current +time, make sure that the values given to your `input_series` are placed far +enough in the past, relative to the evaluation time described above. The values +can for example be negative timestamps so that with a very small `eval_time` the +alert can be expected to trigger. + +Another method that's known to work is to instead bump `eval_time` in the future +so that the timestamp output by `time()` will be a higher value and the values +in `input_series` will be far enough apart from that point in time so that the +alerts will trigger. This method has the downside of making promtool generate a +timeseries database that contains a value for each `input_series` for each +`interval` for the given test. This can become very slow relatively easily and +can end up consuming a lot of RAM for running your test. By instead using values +for `input_series` relative to the timestamp described above even though the +values go into negative numbers, you can keep `eval_time` fairly lower and avoid +making your tests run very slowly. From e69806289afa02756deb96e39edf1279c0ed970b Mon Sep 17 00:00:00 2001 From: Jorge Turrado Ferrero Date: Tue, 2 Dec 2025 22:45:57 +0100 Subject: [PATCH 103/439] chore: Update docs to reflect new oauth parameters Signed-off-by: Jorge Turrado --- docs/configuration/configuration.md | 42 ++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md index 0b944008ef..09f71b5d3c 100644 --- a/docs/configuration/configuration.md +++ b/docs/configuration/configuration.md @@ -761,16 +761,56 @@ A `tls_config` allows configuring TLS connections. OAuth 2.0 authentication using the client credentials or password grant type. Prometheus fetches an access token from the specified endpoint with -the given client access and secret keys. +the given client access and credentials. ```yaml client_id: + +# OAuth2 grant type to use. It can be one of +# "client_credentials" or "urn:ietf:params:oauth:grant-type:jwt-bearer" (RFC 7523). +# Default value is "client_credentials" +[ grant_type: ] + +# Client secret to provide to authorization server. Only used if +# GrantType is set empty or set to "client_credentials". [ client_secret: ] # Read the client secret from a file. # It is mutually exclusive with `client_secret`. [ client_secret_file: ] +# RSA key to sign JWT with. Only used if +# GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". +[ client_certificate_key: ] + +# Read the RSA key from a file. +# It is mutually exclusive with `client_certificate_key`. +[ client_certificate_key_file: ] + +# JWT kid value to include in the JWT header. Only used if +# GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". +[ client_certificate_key_id: ] + +# RSA algorithm used to sign JWT token. Only used if +# GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". +# Default value is RS256 and valid values RS256, RS384, RS512 +[ signature_algorithm: ] + +# OAuth client identifier used when communicating with +# the configured OAuth provider. Default value is client_id. Only used if +# GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". +[ iss: ] + +# Intended audience of the request. If empty, the value +# of TokenURL is used as the intended audience. Only used if +# GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". +[ audience: ] + +# Map of claims to be added to the JWT token. Only used if +# GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". +claims: + [ : ... ] + # Scopes for the token request. scopes: [ - ... ] From f6ca7145ca2ffe8bdd81e373657c740544abc5ac Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Wed, 3 Dec 2025 08:55:48 +0100 Subject: [PATCH 104/439] refactor(tsdb): use one test newTestDB constructor (#17638) For tests only, we had various ways of opening DB. Reduced to one instead of: * Open * newTestDB * newTestDBOpts * openTestDB This so https://github.com/prometheus/prometheus/pull/17629 is smaller and bit easier. Also for test maintainability and consistency. Signed-off-by: bwplotka --- tsdb/compact_test.go | 19 +- tsdb/db.go | 7 + tsdb/db_test.go | 743 ++++++++++++------------------------- tsdb/ooo_head_read_test.go | 19 +- tsdb/querier_test.go | 45 +-- 5 files changed, 257 insertions(+), 576 deletions(-) diff --git a/tsdb/compact_test.go b/tsdb/compact_test.go index 203a04dec8..2b7a52c169 100644 --- a/tsdb/compact_test.go +++ b/tsdb/compact_test.go @@ -1257,10 +1257,7 @@ func BenchmarkCompactionFromOOOHead(b *testing.B) { // This is needed for unit tests that rely on // checking state before and after a compaction. func TestDisableAutoCompactions(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) blockRange := db.compactor.(*LeveledCompactor).ranges[0] label := labels.FromStrings("foo", "bar") @@ -1418,10 +1415,7 @@ func TestDeleteCompactionBlockAfterFailedReload(t *testing.T) { t.Run(title, func(t *testing.T) { ctx := context.Background() - db := openTestDB(t, nil, []int64{1, 100}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withRngs(1, 100)) db.DisableCompactions() expBlocks := bootStrap(db) @@ -1993,14 +1987,11 @@ func TestDelayedCompaction(t *testing.T) { } t.Parallel() - var options *Options + var opts *Options if c.compactionDelay > 0 { - options = &Options{CompactionDelay: c.compactionDelay} + opts = &Options{CompactionDelay: c.compactionDelay} } - db := openTestDB(t, options, []int64{10}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts), withRngs(10)) label := labels.FromStrings("foo", "bar") diff --git a/tsdb/db.go b/tsdb/db.go index f8d36c5479..dac5689b09 100644 --- a/tsdb/db.go +++ b/tsdb/db.go @@ -1986,6 +1986,13 @@ func (db *DB) Head() *Head { // Close the partition. func (db *DB) Close() error { + // Allow close-after-close operation for simpler use (e.g. tests). + select { + case <-db.donec: + return nil + default: + } + close(db.stopc) if db.compactCancel != nil { db.compactCancel() diff --git a/tsdb/db_test.go b/tsdb/db_test.go index 100318c474..4e084ef0d8 100644 --- a/tsdb/db_test.go +++ b/tsdb/db_test.go @@ -80,26 +80,69 @@ func TestMain(m *testing.M) { goleak.IgnoreTopFunction("go.opencensus.io/stats/view.(*worker).start")) } -func openTestDB(t testing.TB, opts *Options, rngs []int64) (db *DB) { - tmpdir := t.TempDir() - var err error +type testDBOptions struct { + dir string + opts *Options + rngs []int64 +} +type testDBOpt func(o *testDBOptions) - if opts == nil { - opts = DefaultOptions() +func withDir(dir string) testDBOpt { + return func(o *testDBOptions) { + o.dir = dir + } +} + +func withOpts(opts *Options) testDBOpt { + return func(o *testDBOptions) { + o.opts = opts + } +} + +func withRngs(rngs ...int64) testDBOpt { + return func(o *testDBOptions) { + o.rngs = rngs + } +} + +func newTestDB(t testing.TB, opts ...testDBOpt) (db *DB) { + var o testDBOptions + for _, opt := range opts { + opt(&o) + } + if o.opts == nil { + o.opts = DefaultOptions() + } + if o.dir == "" { + o.dir = t.TempDir() } - if len(rngs) == 0 { - db, err = Open(tmpdir, nil, nil, opts, nil) + var err error + if len(o.rngs) == 0 { + db, err = Open(o.dir, nil, nil, o.opts, nil) } else { - opts, rngs = validateOpts(opts, rngs) - db, err = open(tmpdir, nil, nil, opts, rngs, nil) + o.opts, o.rngs = validateOpts(o.opts, o.rngs) + db, err = open(o.dir, nil, nil, o.opts, o.rngs, nil) } require.NoError(t, err) - - // Do not Close() the test database by default as it will deadlock on test failures. + t.Cleanup(func() { + // Always close. DB is safe for close-after-close. + require.NoError(t, db.Close()) + }) return db } +func TestDBClose_AfterClose(t *testing.T) { + db := newTestDB(t) + require.NoError(t, db.Close()) + require.NoError(t, db.Close()) + + // Double check if we are closing correct DB after reuse. + db = newTestDB(t) + require.NoError(t, db.Close()) + require.NoError(t, db.Close()) +} + // query runs a matcher query against the querier and fully expands its data. func query(t testing.TB, q storage.Querier, matchers ...*labels.Matcher) map[string][]chunks.Sample { ss := q.Select(context.Background(), false, nil, matchers...) @@ -182,10 +225,7 @@ func queryChunks(t testing.TB, q storage.ChunkQuerier, matchers ...*labels.Match // Ensure that blocks are held in memory in their time order // and not in ULID order as they are read from the directory. func TestDB_reloadOrder(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) metas := []BlockMeta{ {MinTime: 90, MaxTime: 100}, @@ -208,10 +248,7 @@ func TestDB_reloadOrder(t *testing.T) { } func TestDataAvailableOnlyAfterCommit(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -239,7 +276,7 @@ func TestDataAvailableOnlyAfterCommit(t *testing.T) { // TestNoPanicAfterWALCorruption ensures that querying the db after a WAL corruption doesn't cause a panic. // https://github.com/prometheus/prometheus/issues/7548 func TestNoPanicAfterWALCorruption(t *testing.T) { - db := openTestDB(t, &Options{WALSegmentSize: 32 * 1024}, nil) + db := newTestDB(t, withOpts(&Options{WALSegmentSize: 32 * 1024})) // Append until the first mmapped head chunk. // This is to ensure that all samples can be read from the mmapped chunks when the WAL is corrupted. @@ -278,11 +315,7 @@ func TestNoPanicAfterWALCorruption(t *testing.T) { // Query the data. { - db, err := Open(db.Dir(), nil, nil, nil, nil) - require.NoError(t, err) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withDir(db.Dir())) require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal), "WAL corruption count mismatch") querier, err := db.Querier(0, maxt) @@ -294,10 +327,7 @@ func TestNoPanicAfterWALCorruption(t *testing.T) { } func TestDataNotAvailableAfterRollback(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) app := db.Appender(context.Background()) _, err := app.Append(0, labels.FromStrings("type", "float"), 0, 0) @@ -384,10 +414,7 @@ func TestDataNotAvailableAfterRollback(t *testing.T) { } func TestDBAppenderAddRef(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app1 := db.Appender(ctx) @@ -442,10 +469,7 @@ func TestDBAppenderAddRef(t *testing.T) { } func TestAppendEmptyLabelsIgnored(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app1 := db.Appender(ctx) @@ -495,10 +519,7 @@ func TestDeleteSimple(t *testing.T) { for _, c := range cases { t.Run("", func(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -556,10 +577,7 @@ func TestDeleteSimple(t *testing.T) { } func TestAmendHistogramDatapointCausesError(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -617,10 +635,7 @@ func TestAmendHistogramDatapointCausesError(t *testing.T) { } func TestDuplicateNaNDatapointNoAmendError(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -634,10 +649,7 @@ func TestDuplicateNaNDatapointNoAmendError(t *testing.T) { } func TestNonDuplicateNaNDatapointsCausesAmendError(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -651,10 +663,7 @@ func TestNonDuplicateNaNDatapointsCausesAmendError(t *testing.T) { } func TestEmptyLabelsetCausesError(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -664,10 +673,7 @@ func TestEmptyLabelsetCausesError(t *testing.T) { } func TestSkippingInvalidValuesInSameTxn(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) // Append AmendedValue. ctx := context.Background() @@ -707,7 +713,7 @@ func TestSkippingInvalidValuesInSameTxn(t *testing.T) { } func TestDB_Snapshot(t *testing.T) { - db := openTestDB(t, nil, nil) + db := newTestDB(t) // append data ctx := context.Background() @@ -725,9 +731,7 @@ func TestDB_Snapshot(t *testing.T) { require.NoError(t, db.Close()) // reopen DB from snapshot - db, err := Open(snap, nil, nil, nil, nil) - require.NoError(t, err) - defer func() { require.NoError(t, db.Close()) }() + db = newTestDB(t, withDir(snap)) querier, err := db.Querier(mint, mint+1000) require.NoError(t, err) @@ -754,7 +758,7 @@ func TestDB_Snapshot(t *testing.T) { // that are outside the set block time range. // See https://github.com/prometheus/prometheus/issues/5105 func TestDB_Snapshot_ChunksOutsideOfCompactedRange(t *testing.T) { - db := openTestDB(t, nil, nil) + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -773,10 +777,8 @@ func TestDB_Snapshot_ChunksOutsideOfCompactedRange(t *testing.T) { require.NoError(t, db.Snapshot(snap, true)) require.NoError(t, db.Close()) - // Reopen DB from snapshot. - db, err := Open(snap, nil, nil, nil, nil) - require.NoError(t, err) - defer func() { require.NoError(t, db.Close()) }() + // reopen DB from snapshot + db = newTestDB(t, withDir(snap)) querier, err := db.Querier(mint, mint+1000) require.NoError(t, err) @@ -804,8 +806,7 @@ func TestDB_Snapshot_ChunksOutsideOfCompactedRange(t *testing.T) { func TestDB_SnapshotWithDelete(t *testing.T) { const numSamples int64 = 10 - db := openTestDB(t, nil, nil) - defer func() { require.NoError(t, db.Close()) }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -841,12 +842,10 @@ func TestDB_SnapshotWithDelete(t *testing.T) { require.NoError(t, db.Snapshot(snap, true)) // reopen DB from snapshot - newDB, err := Open(snap, nil, nil, nil, nil) - require.NoError(t, err) - defer func() { require.NoError(t, newDB.Close()) }() + db := newTestDB(t, withDir(snap)) // Compare the result. - q, err := newDB.Querier(0, numSamples) + q, err := db.Querier(0, numSamples) require.NoError(t, err) defer func() { require.NoError(t, q.Close()) }() @@ -944,10 +943,7 @@ func TestDB_e2e(t *testing.T) { seriesMap[labels.New(l...).String()] = []chunks.Sample{} } - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -1049,9 +1045,7 @@ func TestDB_e2e(t *testing.T) { } func TestWALFlushedOnDBClose(t *testing.T) { - db := openTestDB(t, nil, nil) - - dirDb := db.Dir() + db := newTestDB(t) lbls := labels.FromStrings("labelname", "labelvalue") @@ -1063,9 +1057,7 @@ func TestWALFlushedOnDBClose(t *testing.T) { require.NoError(t, db.Close()) - db, err = Open(dirDb, nil, nil, nil, nil) - require.NoError(t, err) - defer func() { require.NoError(t, db.Close()) }() + db = newTestDB(t, withDir(db.Dir())) q, err := db.Querier(0, 1) require.NoError(t, err) @@ -1131,7 +1123,7 @@ func TestWALSegmentSizeOptions(t *testing.T) { t.Run(fmt.Sprintf("WALSegmentSize %d test", segmentSize), func(t *testing.T) { opts := DefaultOptions() opts.WALSegmentSize = segmentSize - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) for i := range int64(155) { app := db.Appender(context.Background()) @@ -1144,9 +1136,8 @@ func TestWALSegmentSizeOptions(t *testing.T) { require.NoError(t, app.Commit()) } - dbDir := db.Dir() require.NoError(t, db.Close()) - testFunc(dbDir, opts.WALSegmentSize) + testFunc(db.Dir(), opts.WALSegmentSize) }) } } @@ -1173,7 +1164,7 @@ func TestWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T) { func testWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation int) { const numSeries = 1000 - db := openTestDB(t, nil, nil) + db := newTestDB(t) db.DisableCompactions() for seriesRef := 1; seriesRef <= numSeries; seriesRef++ { @@ -1206,14 +1197,10 @@ func testWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T, numSamplesBefore require.NoError(t, db.Close()) // Reopen the DB, replaying the WAL. - reopenDB, err := Open(db.Dir(), promslog.New(&promslog.Config{}), nil, nil, nil) - require.NoError(t, err) - t.Cleanup(func() { - require.NoError(t, reopenDB.Close()) - }) + db = newTestDB(t, withDir(db.Dir())) // Query back chunks for all series. - q, err := reopenDB.ChunkQuerier(math.MinInt64, math.MaxInt64) + q, err := db.ChunkQuerier(math.MinInt64, math.MaxInt64) require.NoError(t, err) set := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchRegexp, "series_id", ".+")) @@ -1242,7 +1229,7 @@ func TestTombstoneClean(t *testing.T) { t.Parallel() const numSamples int64 = 10 - db := openTestDB(t, nil, nil) + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -1273,9 +1260,7 @@ func TestTombstoneClean(t *testing.T) { require.NoError(t, db.Close()) // Reopen DB from snapshot. - db, err := Open(snap, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() + db := newTestDB(t, withDir(snap)) for _, r := range c.intervals { require.NoError(t, db.Delete(ctx, r.Mint, r.Maxt, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) @@ -1337,7 +1322,7 @@ func TestTombstoneCleanResultEmptyBlock(t *testing.T) { t.Parallel() numSamples := int64(10) - db := openTestDB(t, nil, nil) + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -1358,9 +1343,7 @@ func TestTombstoneCleanResultEmptyBlock(t *testing.T) { require.NoError(t, db.Close()) // Reopen DB from snapshot. - db, err := Open(snap, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() + db = newTestDB(t, withDir(snap)) // Create tombstones by deleting all samples. for _, r := range intervals { @@ -1370,7 +1353,7 @@ func TestTombstoneCleanResultEmptyBlock(t *testing.T) { require.NoError(t, db.CleanTombstones()) // After cleaning tombstones that covers the entire block, no blocks should be left behind. - actualBlockDirs, err := blockDirs(db.dir) + actualBlockDirs, err := blockDirs(db.Dir()) require.NoError(t, err) require.Empty(t, actualBlockDirs) } @@ -1380,10 +1363,7 @@ func TestTombstoneCleanResultEmptyBlock(t *testing.T) { // if TombstoneClean leaves any blocks behind these will overlap. func TestTombstoneCleanFail(t *testing.T) { t.Parallel() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) var oldBlockDirs []string @@ -1415,7 +1395,7 @@ func TestTombstoneCleanFail(t *testing.T) { require.Error(t, db.CleanTombstones()) // Now check that the CleanTombstones replaced the old block even after a failure. - actualBlockDirs, err := blockDirs(db.dir) + actualBlockDirs, err := blockDirs(db.Dir()) require.NoError(t, err) // Only one block should have been replaced by a new block. require.Len(t, actualBlockDirs, len(oldBlockDirs)) @@ -1516,10 +1496,7 @@ func TestTimeRetention(t *testing.T) { } for _, tc := range testCases { t.Run(tc.name, func(t *testing.T) { - db := openTestDB(t, nil, []int64{1000}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withRngs(1000)) for _, m := range tc.blocks { createBlock(t, db.Dir(), genSeries(10, 10, m.MinTime, m.MaxTime)) @@ -1545,12 +1522,9 @@ func TestTimeRetention(t *testing.T) { } func TestRetentionDurationMetric(t *testing.T) { - db := openTestDB(t, &Options{ + db := newTestDB(t, withOpts(&Options{ RetentionDuration: 1000, - }, []int64{100}) - defer func() { - require.NoError(t, db.Close()) - }() + }), withRngs(100)) expRetentionDuration := 1.0 actRetentionDuration := prom_testutil.ToFloat64(db.metrics.retentionDuration) @@ -1561,10 +1535,7 @@ func TestSizeRetention(t *testing.T) { t.Parallel() opts := DefaultOptions() opts.OutOfOrderTimeWindow = 100 - db := openTestDB(t, opts, []int64{100}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts), withRngs(100)) blocks := []*BlockMeta{ {MinTime: 100, MaxTime: 200}, // Oldest block @@ -1708,12 +1679,9 @@ func TestSizeRetentionMetric(t *testing.T) { } for _, c := range cases { - db := openTestDB(t, &Options{ + db := newTestDB(t, withOpts(&Options{ MaxBytes: c.maxBytes, - }, []int64{100}) - defer func() { - require.NoError(t, db.Close()) - }() + }), withRngs(100)) actMaxBytes := int64(prom_testutil.ToFloat64(db.metrics.maxBytes)) require.Equal(t, c.expMaxBytes, actMaxBytes, "metric retention limit bytes mismatch") @@ -1730,12 +1698,9 @@ func TestRuntimeRetentionConfigChange(t *testing.T) { shorterRetentionDuration = int64(1 * time.Hour / time.Millisecond) // 1 hour ) - db := openTestDB(t, &Options{ + db := newTestDB(t, withOpts(&Options{ RetentionDuration: initialRetentionDuration, - }, []int64{100}) - defer func() { - require.NoError(t, db.Close()) - }() + }), withRngs(100)) nineHoursMs := int64(9 * time.Hour / time.Millisecond) nineAndHalfHoursMs := int64((9*time.Hour + 30*time.Minute) / time.Millisecond) @@ -1790,10 +1755,7 @@ func TestRuntimeRetentionConfigChange(t *testing.T) { } func TestNotMatcherSelectsLabelsUnsetSeries(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) labelpairs := []labels.Labels{ labels.FromStrings("a", "abcd", "b", "abcde"), @@ -1978,10 +1940,7 @@ func TestOverlappingBlocksDetectsAllOverlaps(t *testing.T) { // Regression test for https://github.com/prometheus/tsdb/issues/347 func TestChunkAtBlockBoundary(t *testing.T) { t.Parallel() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -2035,10 +1994,7 @@ func TestChunkAtBlockBoundary(t *testing.T) { func TestQuerierWithBoundaryChunks(t *testing.T) { t.Parallel() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) @@ -2081,11 +2037,7 @@ func TestQuerierWithBoundaryChunks(t *testing.T) { func TestInitializeHeadTimestamp(t *testing.T) { t.Parallel() t.Run("clean", func(t *testing.T) { - dir := t.TempDir() - - db, err := Open(dir, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() + db := newTestDB(t) // Should be set to init values if no WAL or blocks exist so far. require.Equal(t, int64(math.MaxInt64), db.head.MinTime()) @@ -2095,7 +2047,7 @@ func TestInitializeHeadTimestamp(t *testing.T) { // First added sample initializes the writable range. ctx := context.Background() app := db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("a", "b"), 1000, 1) + _, err := app.Append(0, labels.FromStrings("a", "b"), 1000, 1) require.NoError(t, err) require.Equal(t, int64(1000), db.head.MinTime()) @@ -2123,9 +2075,7 @@ func TestInitializeHeadTimestamp(t *testing.T) { require.NoError(t, err) require.NoError(t, w.Close()) - db, err := Open(dir, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() + db := newTestDB(t, withDir(dir)) require.Equal(t, int64(5000), db.head.MinTime()) require.Equal(t, int64(15000), db.head.MaxTime()) @@ -2136,9 +2086,7 @@ func TestInitializeHeadTimestamp(t *testing.T) { createBlock(t, dir, genSeries(1, 1, 1000, 2000)) - db, err := Open(dir, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() + db := newTestDB(t, withDir(dir)) require.Equal(t, int64(2000), db.head.MinTime()) require.Equal(t, int64(2000), db.head.MaxTime()) @@ -2167,11 +2115,7 @@ func TestInitializeHeadTimestamp(t *testing.T) { require.NoError(t, err) require.NoError(t, w.Close()) - r := prometheus.NewRegistry() - - db, err := Open(dir, nil, r, nil, nil) - require.NoError(t, err) - defer db.Close() + db := newTestDB(t, withDir(dir)) require.Equal(t, int64(6000), db.head.MinTime()) require.Equal(t, int64(15000), db.head.MaxTime()) @@ -2183,11 +2127,9 @@ func TestInitializeHeadTimestamp(t *testing.T) { func TestNoEmptyBlocks(t *testing.T) { t.Parallel() - db := openTestDB(t, nil, []int64{100}) + db := newTestDB(t, withRngs(100)) ctx := context.Background() - defer func() { - require.NoError(t, db.Close()) - }() + db.DisableCompactions() rangeToTriggerCompaction := db.compactor.(*LeveledCompactor).ranges[0]/2*3 - 1 @@ -2344,10 +2286,7 @@ func TestDB_LabelNames(t *testing.T) { for _, tst := range tests { t.Run("", func(t *testing.T) { ctx := context.Background() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) appendSamples(db, 0, 4, tst.sampleLabels1) @@ -2392,10 +2331,7 @@ func TestDB_LabelNames(t *testing.T) { func TestCorrectNumTombstones(t *testing.T) { t.Parallel() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) blockRange := db.compactor.(*LeveledCompactor).ranges[0] name, value := "foo", "bar" @@ -2528,8 +2464,7 @@ func TestBlockRanges(t *testing.T) { func TestDBReadOnly(t *testing.T) { t.Parallel() var ( - dbDir string - logger = promslog.New(&promslog.Config{}) + dbDir = t.TempDir() expBlocks []*Block expBlock *Block expSeries map[string][]chunks.Sample @@ -2541,8 +2476,6 @@ func TestDBReadOnly(t *testing.T) { // Bootstrap the db. { - dbDir = t.TempDir() - dbBlocks := []*BlockMeta{ // Create three 2-sample blocks. {MinTime: 10, MaxTime: 12}, @@ -2555,7 +2488,7 @@ func TestDBReadOnly(t *testing.T) { } // Add head to test DBReadOnly WAL reading capabilities. - w, err := wlog.New(logger, nil, filepath.Join(dbDir, "wal"), compression.Snappy) + w, err := wlog.New(nil, nil, filepath.Join(dbDir, "wal"), compression.Snappy) require.NoError(t, err) h := createHead(t, w, genSeries(1, 1, 16, 18), dbDir) require.NoError(t, h.Close()) @@ -2563,8 +2496,7 @@ func TestDBReadOnly(t *testing.T) { // Open a normal db to use for a comparison. { - dbWritable, err := Open(dbDir, logger, nil, nil, nil) - require.NoError(t, err) + dbWritable := newTestDB(t, withDir(dbDir)) dbWritable.DisableCompactions() dbSizeBeforeAppend, err := fileutil.DirSize(dbWritable.Dir()) @@ -2592,7 +2524,7 @@ func TestDBReadOnly(t *testing.T) { } // Open a read only db and ensure that the API returns the same result as the normal DB. - dbReadOnly, err := OpenDBReadOnly(dbDir, "", logger) + dbReadOnly, err := OpenDBReadOnly(dbDir, "", nil) require.NoError(t, err) defer func() { require.NoError(t, dbReadOnly.Close()) }() @@ -2665,20 +2597,16 @@ func TestDBReadOnlyClosing(t *testing.T) { func TestDBReadOnly_FlushWAL(t *testing.T) { t.Parallel() var ( - dbDir string - logger = promslog.New(&promslog.Config{}) - err error - maxt int - ctx = context.Background() + dbDir = t.TempDir() + err error + maxt int + ctx = context.Background() ) // Bootstrap the db. { - dbDir = t.TempDir() - // Append data to the WAL. - db, err := Open(dbDir, logger, nil, nil, nil) - require.NoError(t, err) + db := newTestDB(t, withDir(dbDir)) db.DisableCompactions() app := db.Appender(ctx) maxt = 1000 @@ -2691,7 +2619,7 @@ func TestDBReadOnly_FlushWAL(t *testing.T) { } // Flush WAL. - db, err := OpenDBReadOnly(dbDir, "", logger) + db, err := OpenDBReadOnly(dbDir, "", nil) require.NoError(t, err) flush := t.TempDir() @@ -2699,7 +2627,7 @@ func TestDBReadOnly_FlushWAL(t *testing.T) { require.NoError(t, db.Close()) // Reopen the DB from the flushed WAL block. - db, err = OpenDBReadOnly(flush, "", logger) + db, err = OpenDBReadOnly(flush, "", nil) require.NoError(t, err) defer func() { require.NoError(t, db.Close()) }() blocks, err := db.Blocks() @@ -2760,10 +2688,7 @@ func TestDBReadOnly_Querier_NoAlteration(t *testing.T) { } t.Run("doesn't cut chunks while replaying WAL", func(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) // Append until the first mmapped head chunk. for i := range 121 { @@ -2773,33 +2698,31 @@ func TestDBReadOnly_Querier_NoAlteration(t *testing.T) { require.NoError(t, app.Commit()) } - spinUpQuerierAndCheck(db.dir, t.TempDir(), 0) + spinUpQuerierAndCheck(db.Dir(), t.TempDir(), 0) // The RW Head should have no problem cutting its own chunk, // this also proves that a chunk needed to be cut. require.NotPanics(t, func() { db.ForceHeadMMap() }) - require.Equal(t, 1, countChunks(db.dir)) + require.Equal(t, 1, countChunks(db.Dir())) }) t.Run("doesn't truncate corrupted chunks", func(t *testing.T) { - db := openTestDB(t, nil, nil) + db := newTestDB(t) require.NoError(t, db.Close()) // Simulate a corrupted chunk: without a header. - chunk, err := os.Create(path.Join(mmappedChunksDir(db.dir), "000001")) + chunk, err := os.Create(path.Join(mmappedChunksDir(db.Dir()), "000001")) require.NoError(t, err) require.NoError(t, chunk.Close()) - spinUpQuerierAndCheck(db.dir, t.TempDir(), 1) + spinUpQuerierAndCheck(db.Dir(), t.TempDir(), 1) // The RW Head should have no problem truncating its corrupted file: // this proves that the chunk needed to be truncated. - db, err = Open(db.dir, nil, nil, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db = newTestDB(t, withDir(db.Dir())) + require.NoError(t, err) - require.Equal(t, 0, countChunks(db.dir)) + require.Equal(t, 0, countChunks(db.Dir())) }) } @@ -2808,11 +2731,7 @@ func TestDBCannotSeePartialCommits(t *testing.T) { t.Skip("skipping test since tsdb isolation is disabled") } - tmpdir := t.TempDir() - - db, err := Open(tmpdir, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() + db := newTestDB(t) stop := make(chan struct{}) firstInsert := make(chan struct{}) @@ -2828,8 +2747,7 @@ func TestDBCannotSeePartialCommits(t *testing.T) { _, err := app.Append(0, labels.FromStrings("foo", "bar", "a", strconv.Itoa(j)), int64(iter), float64(iter)) require.NoError(t, err) } - err = app.Commit() - require.NoError(t, err) + require.NoError(t, app.Commit()) if iter == 0 { close(firstInsert) @@ -2879,12 +2797,7 @@ func TestDBQueryDoesntSeeAppendsAfterCreation(t *testing.T) { t.Skip("skipping test since tsdb isolation is disabled") } - tmpdir := t.TempDir() - - db, err := Open(tmpdir, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() - + db := newTestDB(t) querierBeforeAdd, err := db.Querier(0, 1000000) require.NoError(t, err) defer querierBeforeAdd.Close() @@ -3202,19 +3115,16 @@ func TestChunkReader_ConcurrentReads(t *testing.T) { // * queries the db to ensure the samples are present from the compacted head. func TestCompactHead(t *testing.T) { t.Parallel() - dbDir := t.TempDir() // Open a DB and append data to the WAL. - tsdbCfg := &Options{ + opts := &Options{ RetentionDuration: int64(time.Hour * 24 * 15 / time.Millisecond), NoLockfile: true, MinBlockDuration: int64(time.Hour * 2 / time.Millisecond), MaxBlockDuration: int64(time.Hour * 2 / time.Millisecond), WALCompression: compression.Snappy, } - - db, err := Open(dbDir, promslog.NewNopLogger(), prometheus.NewRegistry(), tsdbCfg, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) ctx := context.Background() app := db.Appender(ctx) var expSamples []sample @@ -3234,8 +3144,7 @@ func TestCompactHead(t *testing.T) { // Delete everything but the new block and // reopen the db to query it to ensure it includes the head data. require.NoError(t, deleteNonBlocks(db.Dir())) - db, err = Open(dbDir, promslog.NewNopLogger(), prometheus.NewRegistry(), tsdbCfg, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.Len(t, db.Blocks(), 1) require.Equal(t, int64(maxt), db.Head().MinTime()) defer func() { require.NoError(t, db.Close()) }() @@ -3261,13 +3170,12 @@ func TestCompactHead(t *testing.T) { // TestCompactHeadWithDeletion tests https://github.com/prometheus/prometheus/issues/11585. func TestCompactHeadWithDeletion(t *testing.T) { - db, err := Open(t.TempDir(), promslog.NewNopLogger(), prometheus.NewRegistry(), nil, nil) - require.NoError(t, err) + db := newTestDB(t) ctx := context.Background() app := db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("a", "b"), 10, rand.Float64()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 10, rand.Float64()) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -3276,7 +3184,6 @@ func TestCompactHeadWithDeletion(t *testing.T) { // This recreates the bug. require.NoError(t, db.CompactHead(NewRangeHead(db.Head(), 0, 100))) - require.NoError(t, db.Close()) } func deleteNonBlocks(dbDir string) error { @@ -3386,9 +3293,7 @@ func TestOpen_VariousBlockStates(t *testing.T) { opts := DefaultOptions() opts.RetentionDuration = 0 - db, err := Open(tmpDir, promslog.New(&promslog.Config{}), nil, opts, nil) - require.NoError(t, err) - + db := newTestDB(t, withDir(tmpDir), withOpts(opts)) loadedBlocks := db.Blocks() var loaded int @@ -3421,21 +3326,16 @@ func TestOpen_VariousBlockStates(t *testing.T) { func TestOneCheckpointPerCompactCall(t *testing.T) { t.Parallel() blockRange := int64(1000) - tsdbCfg := &Options{ + opts := &Options{ RetentionDuration: blockRange * 1000, NoLockfile: true, MinBlockDuration: blockRange, MaxBlockDuration: blockRange, } - tmpDir := t.TempDir() ctx := context.Background() - db, err := Open(tmpDir, promslog.NewNopLogger(), prometheus.NewRegistry(), tsdbCfg, nil) - require.NoError(t, err) - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // Case 1: Lot's of uncompacted data in Head. @@ -3491,10 +3391,9 @@ func TestOneCheckpointPerCompactCall(t *testing.T) { newBlockMaxt := db.Head().MaxTime() + 1 require.NoError(t, db.Close()) - createBlock(t, db.dir, genSeries(1, 1, newBlockMint, newBlockMaxt)) + createBlock(t, db.Dir(), genSeries(1, 1, newBlockMint, newBlockMaxt)) - db, err = Open(db.dir, promslog.NewNopLogger(), prometheus.NewRegistry(), tsdbCfg, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) db.DisableCompactions() // 1 block more. @@ -3587,10 +3486,7 @@ func testQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t maxStressAllocationBytes = 512 * 1024 ) - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) // Disable compactions so we can control it. db.DisableCompactions() @@ -3723,10 +3619,7 @@ func testChunkQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChun maxStressAllocationBytes = 512 * 1024 ) - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) // Disable compactions so we can control it. db.DisableCompactions() @@ -3828,10 +3721,7 @@ func testChunkQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChun func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration - db := openTestDB(t, opts, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts)) // Disable compactions so we can control it. db.DisableCompactions() @@ -3922,10 +3812,7 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier(t *test func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration - db := openTestDB(t, opts, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts)) // Disable compactions so we can control it. db.DisableCompactions() @@ -4004,10 +3891,7 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting(t *testing.T) { func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration - db := openTestDB(t, opts, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts)) // Disable compactions so we can control it. db.DisableCompactions() @@ -4083,17 +3967,6 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators(t *te require.Eventually(t, compactionComplete.Load, time.Second, 10*time.Millisecond, "compaction should complete after querier was closed") } -func newTestDB(t *testing.T) *DB { - dir := t.TempDir() - - db, err := Open(dir, nil, nil, DefaultOptions(), nil) - require.NoError(t, err) - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) - return db -} - func TestOOOWALWrite(t *testing.T) { minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } @@ -4578,18 +4451,10 @@ func testOOOWALWrite(t *testing.T, expectedOOORecords []any, expectedInORecords []any, ) { - dir := t.TempDir() - opts := DefaultOptions() opts.OutOfOrderCapMax = 2 opts.OutOfOrderTimeWindow = 30 * time.Minute.Milliseconds() - - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) - - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) s1, s2 := labels.FromStrings("l", "v1"), labels.FromStrings("l", "v2") @@ -4673,21 +4538,20 @@ func testOOOWALWrite(t *testing.T, } // The normal WAL. - actRecs := getRecords(path.Join(dir, "wal")) + actRecs := getRecords(path.Join(db.Dir(), "wal")) require.Equal(t, expectedInORecords, actRecs) // The WBL. - actRecs = getRecords(path.Join(dir, wlog.WblDirName)) + actRecs = getRecords(path.Join(db.Dir(), wlog.WblDirName)) require.Equal(t, expectedOOORecords, actRecs) } // Tests https://github.com/prometheus/prometheus/issues/10291#issuecomment-1044373110. func TestDBPanicOnMmappingHeadChunk(t *testing.T) { - dir := t.TempDir() + var err error ctx := context.Background() - db, err := Open(dir, nil, nil, DefaultOptions(), nil) - require.NoError(t, err) + db := newTestDB(t) db.DisableCompactions() // Choosing scrape interval of 45s to have chunk larger than 1h. @@ -4721,8 +4585,7 @@ func TestDBPanicOnMmappingHeadChunk(t *testing.T) { // Restarting. require.NoError(t, db.Close()) - db, err = Open(dir, nil, nil, DefaultOptions(), nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir())) db.DisableCompactions() // Ingest samples upto 20m more to make the head compact. @@ -4915,7 +4778,7 @@ func TestMetadataAssertInMemoryData(t *testing.T) { require.NoError(t, err) } - db := openTestDB(t, nil, nil) + db := newTestDB(t) ctx := context.Background() // Add some series so we can append metadata to them. @@ -4976,19 +4839,14 @@ func TestMetadataAssertInMemoryData(t *testing.T) { // Reopen the DB, replaying the WAL. The Head must have been replayed // correctly in memory. - reopenDB, err := Open(db.Dir(), nil, nil, nil, nil) - require.NoError(t, err) - t.Cleanup(func() { - require.NoError(t, reopenDB.Close()) - }) - - _, err = reopenDB.head.wal.Size() + db = newTestDB(t, withDir(db.Dir())) + _, err := db.head.wal.Size() require.NoError(t, err) - require.Equal(t, *reopenDB.head.series.getByHash(s1.Hash(), s1).meta, m1) - require.Equal(t, *reopenDB.head.series.getByHash(s2.Hash(), s2).meta, m5) - require.Equal(t, *reopenDB.head.series.getByHash(s3.Hash(), s3).meta, m3) - require.Equal(t, *reopenDB.head.series.getByHash(s4.Hash(), s4).meta, m4) + require.Equal(t, *db.head.series.getByHash(s1.Hash(), s1).meta, m1) + require.Equal(t, *db.head.series.getByHash(s2.Hash(), s2).meta, m5) + require.Equal(t, *db.head.series.getByHash(s3.Hash(), s3).meta, m3) + require.Equal(t, *db.head.series.getByHash(s4.Hash(), s4).meta, m4) } // TestMultipleEncodingsCommitOrder mainly serves to demonstrate when happens when committing a batch of samples for the @@ -4998,14 +4856,10 @@ func TestMultipleEncodingsCommitOrder(t *testing.T) { opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() - series1 := labels.FromStrings("foo", "bar1") - - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - defer func() { - require.NoError(t, db.Close()) - }() + series1 := labels.FromStrings("foo", "bar1") addSample := func(app storage.Appender, ts int64, valType chunkenc.ValueType) chunks.Sample { if valType == chunkenc.ValFloat { _, err := app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) @@ -5148,19 +5002,13 @@ func TestOOOCompaction(t *testing.T) { } func testOOOCompaction(t *testing.T, scenario sampleTypeScenario, addExtraSamples bool) { - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() - - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") series2 := labels.FromStrings("foo", "bar2") @@ -5351,19 +5199,14 @@ func TestOOOCompactionWithNormalCompaction(t *testing.T) { func testOOOCompactionWithNormalCompaction(t *testing.T, scenario sampleTypeScenario) { t.Parallel() - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") series2 := labels.FromStrings("foo", "bar2") @@ -5461,7 +5304,6 @@ func TestOOOCompactionWithDisabledWriteLog(t *testing.T) { func testOOOCompactionWithDisabledWriteLog(t *testing.T, scenario sampleTypeScenario) { t.Parallel() - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() @@ -5469,12 +5311,8 @@ func testOOOCompactionWithDisabledWriteLog(t *testing.T, scenario sampleTypeScen opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() opts.WALSegmentSize = -1 // disabled WAL and WBL - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") series2 := labels.FromStrings("foo", "bar2") @@ -5571,7 +5409,6 @@ func TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T) { } func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sampleTypeScenario) { - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() @@ -5579,12 +5416,8 @@ func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sa opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() opts.EnableMemorySnapshotOnShutdown = true - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") series2 := labels.FromStrings("foo", "bar2") @@ -5620,10 +5453,9 @@ func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sa require.NoError(t, db.Close()) // For some reason wbl goes missing. - require.NoError(t, os.RemoveAll(path.Join(dir, "wbl"))) + require.NoError(t, os.RemoveAll(path.Join(db.Dir(), "wbl"))) - db, err = Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir())) db.DisableCompactions() // We want to manually call it. // Check ooo m-map chunks again. @@ -5940,11 +5772,8 @@ func testQuerierOOOQuery(t *testing.T, for _, tc := range tests { t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { opts.OutOfOrderCapMax = tc.oooCap - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - defer func() { - require.NoError(t, db.Close()) - }() var expSamples []chunks.Sample var oooSamples, appendedCount int @@ -6269,11 +6098,8 @@ func testChunkQuerierOOOQuery(t *testing.T, for _, tc := range tests { t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { opts.OutOfOrderCapMax = tc.oooCap - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - defer func() { - require.NoError(t, db.Close()) - }() var expSamples []chunks.Sample var oooSamples, appendedCount int @@ -6449,11 +6275,8 @@ func testOOONativeHistogramsWithCounterResets(t *testing.T, scenario sampleTypeS } for _, tc := range tests { t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - defer func() { - require.NoError(t, db.Close()) - }() app := db.Appender(context.Background()) @@ -6686,11 +6509,8 @@ func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario opts.OutOfOrderCapMax = tc.oooCap opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - defer func() { - require.NoError(t, db.Close()) - }() app := db.Appender(context.Background()) for _, s := range tc.samples { @@ -6787,11 +6607,8 @@ func testOOOAppendAndQuery(t *testing.T, scenario sampleTypeScenario) { opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 4 * time.Hour.Milliseconds() - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) s1 := labels.FromStrings("foo", "bar1") s2 := labels.FromStrings("foo", "bar2") @@ -6918,11 +6735,8 @@ func TestOOODisabled(t *testing.T) { func testOOODisabled(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 0 - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) s1 := labels.FromStrings("foo", "bar1") minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } @@ -6993,11 +6807,8 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 4 * time.Hour.Milliseconds() - db := openTestDB(t, opts, nil) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) s1 := labels.FromStrings("foo", "bar1") @@ -7078,38 +6889,32 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { } t.Run("Restart DB with both WBL and M-map files for ooo data", func(t *testing.T) { - db, err = Open(db.dir, nil, nil, opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.Equal(t, oooMint, db.head.MinOOOTime()) require.Equal(t, oooMaxt, db.head.MaxOOOTime()) testQuery(expSamples) - require.NoError(t, db.Close()) }) t.Run("Restart DB with only WBL for ooo data", func(t *testing.T) { require.NoError(t, os.RemoveAll(mmapDir)) - db, err = Open(db.dir, nil, nil, opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.Equal(t, oooMint, db.head.MinOOOTime()) require.Equal(t, oooMaxt, db.head.MaxOOOTime()) testQuery(expSamples) - require.NoError(t, db.Close()) }) t.Run("Restart DB with only M-map files for ooo data", func(t *testing.T) { require.NoError(t, os.RemoveAll(wblDir)) resetMmapToOriginal() - db, err = Open(db.dir, nil, nil, opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.Equal(t, oooMint, db.head.MinOOOTime()) require.Equal(t, oooMaxt, db.head.MaxOOOTime()) inOrderSample := expSamples[s1.String()][len(expSamples[s1.String()])-1] testQuery(map[string][]chunks.Sample{ s1.String(): append(s1MmapSamples, inOrderSample), }) - require.NoError(t, db.Close()) }) t.Run("Restart DB with WBL+Mmap while increasing the OOOCapMax", func(t *testing.T) { @@ -7117,24 +6922,22 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { resetMmapToOriginal() opts.OutOfOrderCapMax = 60 - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, oooMint, db.head.MinOOOTime()) require.Equal(t, oooMaxt, db.head.MaxOOOTime()) testQuery(expSamples) - require.NoError(t, db.Close()) }) t.Run("Restart DB with WBL+Mmap while decreasing the OOOCapMax", func(t *testing.T) { resetMmapToOriginal() // We need to reset because new duplicate chunks can be written above. opts.OutOfOrderCapMax = 10 - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, oooMint, db.head.MinOOOTime()) require.Equal(t, oooMaxt, db.head.MaxOOOTime()) testQuery(expSamples) - require.NoError(t, db.Close()) }) t.Run("Restart DB with WBL+Mmap while having no m-map markers in WBL", func(t *testing.T) { @@ -7164,7 +6967,7 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { require.NoError(t, os.Rename(newWbl.Dir(), wblDir)) opts.OutOfOrderCapMax = 30 - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, oooMint, db.head.MinOOOTime()) require.Equal(t, oooMaxt, db.head.MaxOOOTime()) @@ -7174,19 +6977,14 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { for _, floatHistogram := range []bool{false, true} { - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 500 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") series2 := labels.FromStrings("foo", "bar2") @@ -7199,7 +6997,7 @@ func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { if floatHistogram { h := tsdbutil.GenerateTestFloatHistogram(int64(val)) h.CounterResetHint = hint - _, err = app.AppendHistogram(0, l, tsMs, nil, h) + _, err := app.AppendHistogram(0, l, tsMs, nil, h) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, fh: h.Copy()} @@ -7207,7 +7005,7 @@ func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { h := tsdbutil.GenerateTestHistogram(int64(val)) h.CounterResetHint = hint - _, err = app.AppendHistogram(0, l, tsMs, h, nil) + _, err := app.AppendHistogram(0, l, tsMs, h, nil) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, h: h.Copy()} @@ -7534,19 +7332,14 @@ func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing.T) { for _, floatHistogram := range []bool{false, true} { - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 500 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") @@ -7555,14 +7348,14 @@ func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing tsMs := ts if floatHistogram { h := tsdbutil.GenerateTestFloatHistogram(int64(val)) - _, err = app.AppendHistogram(0, l, tsMs, nil, h) + _, err := app.AppendHistogram(0, l, tsMs, nil, h) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, fh: h.Copy()} } h := tsdbutil.GenerateTestHistogram(int64(val)) - _, err = app.AppendHistogram(0, l, tsMs, h, nil) + _, err := app.AppendHistogram(0, l, tsMs, h, nil) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, h: h.Copy()} @@ -7610,8 +7403,7 @@ func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing // Compact the in-order head and expect another block. // Since this is a forced compaction, this block is not aligned with 2h. - err = db.CompactHead(NewRangeHead(db.head, 0, 3)) - require.NoError(t, err) + require.NoError(t, db.CompactHead(NewRangeHead(db.head, 0, 3))) require.Len(t, db.Blocks(), 2) // Blocks created out of normal and OOO head now. But not merged. @@ -7649,19 +7441,13 @@ func TestOOOCompactionFailure(t *testing.T) { } func testOOOCompactionFailure(t *testing.T, scenario sampleTypeScenario) { - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() - - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") @@ -7787,18 +7573,11 @@ func testOOOCompactionFailure(t *testing.T, scenario sampleTypeScenario) { } func TestWBLCorruption(t *testing.T) { - dir := t.TempDir() - opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) - db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) series1 := labels.FromStrings("foo", "bar1") var allSamples, expAfterRestart []chunks.Sample @@ -7825,7 +7604,7 @@ func TestWBLCorruption(t *testing.T) { addSamples(120, 130, true) // Moving onto the second file. - _, err = db.head.wbl.NextSegment() + _, err := db.head.wbl.NextSegment() require.NoError(t, err) // More OOO samples. @@ -7897,7 +7676,7 @@ func TestWBLCorruption(t *testing.T) { require.NoError(t, os.RemoveAll(mmappedChunksDir(db.head.opts.ChunkDirRoot))) // Restart does the replay and repair. - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) require.Less(t, len(expAfterRestart), len(allSamples)) @@ -7926,7 +7705,7 @@ func TestWBLCorruption(t *testing.T) { // Another restart, everything normal with no repair. require.NoError(t, db.Close()) - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) verifySamples(expAfterRestart) @@ -7941,18 +7720,11 @@ func TestOOOMmapCorruption(t *testing.T) { } func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { - dir := t.TempDir() - opts := DefaultOptions() opts.OutOfOrderCapMax = 10 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) - db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) series1 := labels.FromStrings("foo", "bar1") var allSamples, expInMmapChunks []chunks.Sample @@ -8029,7 +7801,7 @@ func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { require.NoError(t, os.Rename(wblDir, wblDirTmp)) // Restart does the replay and repair of m-map files. - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.mmapChunkCorruptionTotal)) require.Less(t, len(expInMmapChunks), len(allSamples)) @@ -8049,7 +7821,7 @@ func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { // Another restart, everything normal with no repair. require.NoError(t, db.Close()) - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.mmapChunkCorruptionTotal)) verifySamples(expInMmapChunks) @@ -8058,7 +7830,7 @@ func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { require.NoError(t, db.Close()) require.NoError(t, os.RemoveAll(wblDir)) require.NoError(t, os.Rename(wblDirTmp, wblDir)) - db, err = Open(db.dir, nil, nil, opts, nil) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) require.NoError(t, err) verifySamples(allSamples) } @@ -8076,18 +7848,10 @@ func testOutOfOrderRuntimeConfig(t *testing.T, scenario sampleTypeScenario) { ctx := context.Background() getDB := func(oooTimeWindow int64) *DB { - dir := t.TempDir() - opts := DefaultOptions() opts.OutOfOrderTimeWindow = oooTimeWindow - - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) - return db } @@ -8369,18 +8133,12 @@ func testNoGapAfterRestartWithOOO(t *testing.T, scenario sampleTypeScenario) { for i, c := range cases { t.Run(fmt.Sprintf("case=%d", i), func(t *testing.T) { - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderTimeWindow = 30 * time.Minute.Milliseconds() - - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) // 3h10m=190m worth in-order data. addSamples(t, db, c.inOrderMint, c.inOrderMaxt, true) @@ -8407,8 +8165,7 @@ func testNoGapAfterRestartWithOOO(t *testing.T, scenario sampleTypeScenario) { // Restart and expect all samples to be present. require.NoError(t, db.Close()) - db, err = Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) db.DisableCompactions() verifyBlockRanges() @@ -8428,17 +8185,10 @@ func TestWblReplayAfterOOODisableAndRestart(t *testing.T) { } func testWblReplayAfterOOODisableAndRestart(t *testing.T, scenario sampleTypeScenario) { - dir := t.TempDir() - opts := DefaultOptions() opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) - db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) series1 := labels.FromStrings("foo", "bar1") var allSamples []chunks.Sample @@ -8478,9 +8228,9 @@ func testWblReplayAfterOOODisableAndRestart(t *testing.T, scenario sampleTypeSce // Restart DB with OOO disabled. require.NoError(t, db.Close()) + opts.OutOfOrderTimeWindow = 0 - db, err = Open(db.dir, nil, nil, opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) // We can still query OOO samples when OOO is disabled. verifySamples(allSamples) @@ -8495,17 +8245,10 @@ func TestPanicOnApplyConfig(t *testing.T) { } func testPanicOnApplyConfig(t *testing.T, scenario sampleTypeScenario) { - dir := t.TempDir() - opts := DefaultOptions() opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) - db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) series1 := labels.FromStrings("foo", "bar1") var allSamples []chunks.Sample @@ -8527,12 +8270,12 @@ func testPanicOnApplyConfig(t *testing.T, scenario sampleTypeScenario) { // Restart DB with OOO disabled. require.NoError(t, db.Close()) + opts.OutOfOrderTimeWindow = 0 - db, err = Open(db.dir, nil, prometheus.NewRegistry(), opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) // ApplyConfig with OOO enabled and expect no panic. - err = db.ApplyConfig(&config.Config{ + err := db.ApplyConfig(&config.Config{ StorageConfig: config.StorageConfig{ TSDBConfig: &config.TSDBConfig{ OutOfOrderTimeWindow: 60 * time.Minute.Milliseconds(), @@ -8553,18 +8296,13 @@ func TestDiskFillingUpAfterDisablingOOO(t *testing.T) { func testDiskFillingUpAfterDisablingOOO(t *testing.T, scenario sampleTypeScenario) { t.Parallel() - dir := t.TempDir() ctx := context.Background() opts := DefaultOptions() opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) + db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) series1 := labels.FromStrings("foo", "bar1") var allSamples []chunks.Sample @@ -8586,9 +8324,9 @@ func testDiskFillingUpAfterDisablingOOO(t *testing.T, scenario sampleTypeScenari // Restart DB with OOO disabled. require.NoError(t, db.Close()) + opts.OutOfOrderTimeWindow = 0 - db, err = Open(db.dir, nil, prometheus.NewRegistry(), opts, nil) - require.NoError(t, err) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) db.DisableCompactions() ms := db.head.series.getByHash(series1.Hash(), series1) @@ -8649,11 +8387,8 @@ func TestHistogramAppendAndQuery(t *testing.T) { func testHistogramAppendAndQueryHelper(t *testing.T, floatHistogram bool) { t.Helper() - db := openTestDB(t, nil, nil) + db := newTestDB(t) minute := func(m int) int64 { return int64(m) * time.Minute.Milliseconds() } - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) ctx := context.Background() appendHistogram := func(t *testing.T, @@ -8920,10 +8655,7 @@ func TestQueryHistogramFromBlocksWithCompaction(t *testing.T) { t.Helper() opts := DefaultOptions() - db := openTestDB(t, opts, nil) - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) + db := newTestDB(t, withOpts(opts)) var it chunkenc.Iterator exp := make(map[string][]chunks.Sample) @@ -9066,10 +8798,7 @@ func TestOOONativeHistogramsSettings(t *testing.T) { t.Run("Test OOO native histograms if OOO is disabled and Native Histograms is enabled", func(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 0 - db := openTestDB(t, opts, []int64{100}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts), withRngs(100)) app := db.Appender(context.Background()) _, err := app.AppendHistogram(0, l, 100, h, nil) @@ -9090,10 +8819,7 @@ func TestOOONativeHistogramsSettings(t *testing.T) { t.Run("Test OOO native histograms when both OOO and Native Histograms are enabled", func(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 100 - db := openTestDB(t, opts, []int64{100}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts), withRngs(100)) // Add in-order samples app := db.Appender(context.Background()) @@ -9183,10 +8909,7 @@ func compareSeries(t require.TestingT, expected, actual map[string][]chunks.Samp // worrying about the parallel write. func TestChunkQuerierReadWriteRace(t *testing.T) { t.Parallel() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) lbls := labels.FromStrings("foo", "bar") @@ -9272,10 +8995,7 @@ func (c *mockCompactorFn) Write(string, BlockReader, int64, int64, *BlockMeta) ( // Regression test for https://github.com/prometheus/prometheus/pull/13754 func TestAbortBlockCompactions(t *testing.T) { // Create a test DB - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t) // It should NOT be compactable at the beginning of the test require.False(t, db.head.compactable(), "head should NOT be compactable") @@ -9328,10 +9048,8 @@ func TestNewCompactorFunc(t *testing.T) { }, }, nil } - db := openTestDB(t, opts, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts)) + plans, err := db.compactor.Plan("") require.NoError(t, err) require.Equal(t, []string{block1.String(), block2.String()}, plans) @@ -9362,10 +9080,7 @@ func TestBlockQuerierAndBlockChunkQuerier(t *testing.T) { return storage.NoopChunkedQuerier(), nil } - db := openTestDB(t, opts, nil) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts)) metas := []BlockMeta{ {Compaction: BlockMetaCompaction{Hints: []string{"test-hint"}}}, @@ -9450,10 +9165,8 @@ func TestGenerateCompactionDelay(t *testing.T) { for _, c := range cases { opts.CompactionDelayMaxPercent = c.compactionDelayPercent - db := openTestDB(t, opts, []int64{60000}) - defer func() { - require.NoError(t, db.Close()) - }() + db := newTestDB(t, withOpts(opts), withRngs(60000)) + // The offset is generated and changed while opening. assertDelay(db.opts.CompactionDelay, c.compactionDelayPercent) @@ -9496,15 +9209,17 @@ func TestBlockClosingBlockedDuringRemoteRead(t *testing.T) { dir := t.TempDir() createBlock(t, dir, genSeries(2, 1, 0, 10)) + + // Not using newTestDB as db.Close is expected to return error. db, err := Open(dir, nil, nil, nil, nil) require.NoError(t, err) - // No error checking as manually closing the block is supposed to make this fail. defer db.Close() - readAPI := remote.NewReadHandler(nil, nil, db, func() config.Config { - return config.Config{} - }, - 0, 1, 0, + readAPI := remote.NewReadHandler( + nil, nil, db, + func() config.Config { + return config.Config{} + }, 0, 1, 0, ) matcher, err := labels.NewMatcher(labels.MatchRegexp, "__name__", ".*") diff --git a/tsdb/ooo_head_read_test.go b/tsdb/ooo_head_read_test.go index d197eacb56..5e754b59b8 100644 --- a/tsdb/ooo_head_read_test.go +++ b/tsdb/ooo_head_read_test.go @@ -498,7 +498,7 @@ func testOOOHeadChunkReader_Chunk(t *testing.T, scenario sampleTypeScenario) { minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } t.Run("Getting a non existing chunk fails with not found error", func(t *testing.T) { - db := newTestDBWithOpts(t, opts) + db := newTestDB(t, withOpts(opts)) cr := NewHeadAndOOOChunkReader(db.head, 0, 1000, nil, nil, 0) defer cr.Close() @@ -837,7 +837,7 @@ func testOOOHeadChunkReader_Chunk(t *testing.T, scenario sampleTypeScenario) { for _, tc := range tests { t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { - db := newTestDBWithOpts(t, opts) + db := newTestDB(t, withOpts(opts)) app := db.Appender(context.Background()) s1Ref, _, err := scenario.appendFunc(app, s1, tc.firstInOrderSampleAt, tc.firstInOrderSampleAt/1*time.Minute.Milliseconds()) @@ -1006,7 +1006,7 @@ func testOOOHeadChunkReader_Chunk_ConsistentQueryResponseDespiteOfHeadExpanding( for _, tc := range tests { t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { - db := newTestDBWithOpts(t, opts) + db := newTestDB(t, withOpts(opts)) app := db.Appender(context.Background()) s1Ref, _, err := scenario.appendFunc(app, s1, tc.firstInOrderSampleAt, tc.firstInOrderSampleAt/1*time.Minute.Milliseconds()) @@ -1118,16 +1118,3 @@ func TestSortMetaByMinTimeAndMinRef(t *testing.T) { }) } } - -func newTestDBWithOpts(t *testing.T, opts *Options) *DB { - dir := t.TempDir() - - db, err := Open(dir, nil, nil, opts, nil) - require.NoError(t, err) - - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) - - return db -} diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go index a5efa35ceb..5f5af441ca 100644 --- a/tsdb/querier_test.go +++ b/tsdb/querier_test.go @@ -3094,11 +3094,8 @@ func TestQuerierIndexQueriesRace(t *testing.T) { for _, c := range testCases { t.Run(fmt.Sprintf("%v", c.matchers), func(t *testing.T) { t.Parallel() - db := openTestDB(t, DefaultOptions(), nil) + db := newTestDB(t) h := db.Head() - t.Cleanup(func() { - require.NoError(t, db.Close()) - }) ctx, cancel := context.WithCancel(context.Background()) wg := &sync.WaitGroup{} wg.Add(1) @@ -3496,10 +3493,7 @@ func TestBlockBaseSeriesSet(t *testing.T) { } func BenchmarkHeadChunkQuerier(b *testing.B) { - db := openTestDB(b, nil, nil) - defer func() { - require.NoError(b, db.Close()) - }() + db := newTestDB(b) // 3h of data. numTimeseries := 100 @@ -3541,10 +3535,7 @@ func BenchmarkHeadChunkQuerier(b *testing.B) { } func BenchmarkHeadQuerier(b *testing.B) { - db := openTestDB(b, nil, nil) - defer func() { - require.NoError(b, db.Close()) - }() + db := newTestDB(b) // 3h of data. numTimeseries := 100 @@ -3606,12 +3597,8 @@ func TestQueryWithDeletedHistograms(t *testing.T) { for name, tc := range testcases { t.Run(name, func(t *testing.T) { - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() - - appender := db.Appender(context.Background()) + db := newTestDB(t) + app := db.Appender(context.Background()) var ( err error @@ -3621,12 +3608,11 @@ func TestQueryWithDeletedHistograms(t *testing.T) { for i := range 100 { h, fh := tc(i) - seriesRef, err = appender.AppendHistogram(seriesRef, lbs, int64(i), h, fh) + seriesRef, err = app.AppendHistogram(seriesRef, lbs, int64(i), h, fh) require.NoError(t, err) } - err = appender.Commit() - require.NoError(t, err) + require.NoError(t, app.Commit()) matcher, err := labels.NewMatcher(labels.MatchEqual, "__name__", "test") require.NoError(t, err) @@ -3664,12 +3650,8 @@ func TestQueryWithDeletedHistograms(t *testing.T) { func TestQueryWithOneChunkCompletelyDeleted(t *testing.T) { ctx := context.Background() - db := openTestDB(t, nil, nil) - defer func() { - require.NoError(t, db.Close()) - }() - - appender := db.Appender(context.Background()) + db := newTestDB(t) + app := db.Appender(context.Background()) var ( err error @@ -3680,12 +3662,12 @@ func TestQueryWithOneChunkCompletelyDeleted(t *testing.T) { // Create an int histogram chunk with samples between 0 - 20 and 30 - 40. for i := range 20 { h := tsdbutil.GenerateTestHistogram(1) - seriesRef, err = appender.AppendHistogram(seriesRef, lbs, int64(i), h, nil) + seriesRef, err = app.AppendHistogram(seriesRef, lbs, int64(i), h, nil) require.NoError(t, err) } for i := 30; i < 40; i++ { h := tsdbutil.GenerateTestHistogram(1) - seriesRef, err = appender.AppendHistogram(seriesRef, lbs, int64(i), h, nil) + seriesRef, err = app.AppendHistogram(seriesRef, lbs, int64(i), h, nil) require.NoError(t, err) } @@ -3693,12 +3675,11 @@ func TestQueryWithOneChunkCompletelyDeleted(t *testing.T) { // type from int histograms so a new chunk is created. for i := 60; i < 100; i++ { fh := tsdbutil.GenerateTestFloatHistogram(1) - seriesRef, err = appender.AppendHistogram(seriesRef, lbs, int64(i), nil, fh) + seriesRef, err = app.AppendHistogram(seriesRef, lbs, int64(i), nil, fh) require.NoError(t, err) } - err = appender.Commit() - require.NoError(t, err) + require.NoError(t, app.Commit()) matcher, err := labels.NewMatcher(labels.MatchEqual, "__name__", "test") require.NoError(t, err) From 1a853e23db411c93f552d46888bd8033486b860b Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Tue, 2 Dec 2025 16:48:31 +0100 Subject: [PATCH 105/439] Add start_timestamp field for unit tests. This commit adds support for configuring a custom start timestamp for Prometheus unit tests, allowing tests to use realistic timestamps instead of starting at Unix epoch 0. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- cmd/promtool/testdata/start-time-test.yml | 76 +++++++++++++++++++++++ cmd/promtool/unittest.go | 49 ++++++++++++--- cmd/promtool/unittest_test.go | 10 +++ docs/configuration/unit_testing_rules.md | 46 ++++++++------ promql/promqltest/test.go | 18 ++++-- 5 files changed, 168 insertions(+), 31 deletions(-) create mode 100644 cmd/promtool/testdata/start-time-test.yml diff --git a/cmd/promtool/testdata/start-time-test.yml b/cmd/promtool/testdata/start-time-test.yml new file mode 100644 index 0000000000..b7365366f4 --- /dev/null +++ b/cmd/promtool/testdata/start-time-test.yml @@ -0,0 +1,76 @@ +rule_files: + - rules.yml + +evaluation_interval: 1m + +tests: + # Test with default start_time (0 / Unix epoch). + - name: default_start_time + interval: 1m + promql_expr_test: + - expr: time() + eval_time: 0m + exp_samples: + - value: 0 + - expr: time() + eval_time: 5m + exp_samples: + - value: 300 + + # Test with RFC3339 start_timestamp. + - name: rfc3339_start_timestamp + interval: 1m + start_timestamp: "2024-01-01T00:00:00Z" + promql_expr_test: + - expr: time() + eval_time: 0m + exp_samples: + - value: 1704067200 + - expr: time() + eval_time: 5m + exp_samples: + - value: 1704067500 + + # Test with Unix timestamp start_timestamp. + - name: unix_timestamp_start_timestamp + interval: 1m + start_timestamp: 1609459200 + input_series: + - series: test_metric + values: "1 1 1" + promql_expr_test: + - expr: time() + eval_time: 0m + exp_samples: + - value: 1609459200 + - expr: time() + eval_time: 10m + exp_samples: + - value: 1609459800 + + # Test that input series samples are correctly timestamped with custom start_timestamp. + - name: samples_with_start_timestamp + interval: 1m + start_timestamp: "2024-01-01T00:00:00Z" + input_series: + - series: 'my_metric{label="test"}' + values: "10+10x15" + promql_expr_test: + # Query at absolute timestamp (start_timestamp = 1704067200). + - expr: my_metric@1704067200 + eval_time: 5m + exp_samples: + - labels: 'my_metric{label="test"}' + value: 10 + # Query at 2 minutes after start_timestamp (1704067200 + 120 = 1704067320). + - expr: my_metric@1704067320 + eval_time: 5m + exp_samples: + - labels: 'my_metric{label="test"}' + value: 30 + # Verify timestamp() function returns the absolute timestamp. + - expr: timestamp(my_metric) + eval_time: 5m + exp_samples: + - labels: '{label="test"}' + value: 1704067500 diff --git a/cmd/promtool/unittest.go b/cmd/promtool/unittest.go index 15b5171645..75da96c2eb 100644 --- a/cmd/promtool/unittest.go +++ b/cmd/promtool/unittest.go @@ -188,15 +188,37 @@ func resolveAndGlobFilepaths(baseDir string, utf *unitTestFile) error { return nil } +// testStartTimestamp wraps time.Time to support custom YAML unmarshaling. +// It can parse both RFC3339 timestamps and Unix timestamps. +type testStartTimestamp struct { + time.Time +} + +// UnmarshalYAML implements custom YAML unmarshaling for testStartTimestamp. +// It accepts both RFC3339 formatted strings and numeric Unix timestamps. +func (t *testStartTimestamp) UnmarshalYAML(unmarshal func(interface{}) error) error { + var s string + if err := unmarshal(&s); err != nil { + return err + } + parsed, err := parseTime(s) + if err != nil { + return err + } + t.Time = parsed + return nil +} + // testGroup is a group of input series and tests associated with it. type testGroup struct { - Interval model.Duration `yaml:"interval"` - InputSeries []series `yaml:"input_series"` - AlertRuleTests []alertTestCase `yaml:"alert_rule_test,omitempty"` - PromqlExprTests []promqlTestCase `yaml:"promql_expr_test,omitempty"` - ExternalLabels labels.Labels `yaml:"external_labels,omitempty"` - ExternalURL string `yaml:"external_url,omitempty"` - TestGroupName string `yaml:"name,omitempty"` + Interval model.Duration `yaml:"interval"` + InputSeries []series `yaml:"input_series"` + AlertRuleTests []alertTestCase `yaml:"alert_rule_test,omitempty"` + PromqlExprTests []promqlTestCase `yaml:"promql_expr_test,omitempty"` + ExternalLabels labels.Labels `yaml:"external_labels,omitempty"` + ExternalURL string `yaml:"external_url,omitempty"` + TestGroupName string `yaml:"name,omitempty"` + StartTimestamp testStartTimestamp `yaml:"start_timestamp,omitempty"` } // test performs the unit tests. @@ -209,6 +231,12 @@ func (tg *testGroup) test(testname string, evalInterval time.Duration, groupOrde }() } // Setup testing suite. + // Set the start time from the test group. + if tg.StartTimestamp.IsZero() { + queryOpts.StartTime = time.Unix(0, 0).UTC() + } else { + queryOpts.StartTime = tg.StartTimestamp.Time + } suite, err := promqltest.NewLazyLoader(tg.seriesLoadingString(), queryOpts) if err != nil { return []error{err} @@ -237,7 +265,12 @@ func (tg *testGroup) test(testname string, evalInterval time.Duration, groupOrde groups := orderedGroups(groupsMap, groupOrderMap) // Bounds for evaluating the rules. - mint := time.Unix(0, 0).UTC() + var mint time.Time + if tg.StartTimestamp.IsZero() { + mint = time.Unix(0, 0).UTC() + } else { + mint = tg.StartTimestamp.Time + } maxt := mint.Add(tg.maxEvalTime()) // Optional floating point compare fuzzing. diff --git a/cmd/promtool/unittest_test.go b/cmd/promtool/unittest_test.go index 566e0acbc6..bf4de02ccd 100644 --- a/cmd/promtool/unittest_test.go +++ b/cmd/promtool/unittest_test.go @@ -129,6 +129,16 @@ func TestRulesUnitTest(t *testing.T) { }, want: 0, }, + { + name: "Start time tests", + args: args{ + files: []string{"./testdata/start-time-test.yml"}, + }, + queryOpts: promqltest.LazyLoaderOpts{ + EnableAtModifier: true, + }, + want: 0, + }, } reuseFiles := []string{} reuseCount := [2]int{} diff --git a/docs/configuration/unit_testing_rules.md b/docs/configuration/unit_testing_rules.md index 13b0445c7c..af94c414f0 100644 --- a/docs/configuration/unit_testing_rules.md +++ b/docs/configuration/unit_testing_rules.md @@ -48,6 +48,18 @@ input_series: # Name of the test group [ name: ] +# Start timestamp for the test group. This sets the base time for all samples +# and evaluations in this test group. +# Accepts either a Unix timestamp (e.g., 1609459200) or an RFC3339 formatted +# timestamp (e.g., "2021-01-01T00:00:00Z"). +# Default: 0 (Unix epoch: 1970-01-01 00:00:00 UTC) +# +# When set: +# - All input_series samples are timestamped starting from start_timestamp +# - The eval_time in test cases is relative to start_timestamp +# - The time() function returns start_timestamp + eval_time +[ start_timestamp: | | default = 0 ] + # Unit tests for the above data. # Unit tests for alerting rules. We consider the alerting rules from the input file. @@ -137,7 +149,8 @@ values: Prometheus allows you to have same alertname for different alerting rules. Hence in this unit testing, you have to list the union of all the firing alerts for the alertname under a single ``. ``` yaml -# The time elapsed from time=0s when the alerts have to be checked. +# The time elapsed from start_timestamp when the alerts have to be checked. +# This is a duration relative to start_timestamp (which defaults to 0). eval_time: # Name of the alert to be tested. @@ -168,7 +181,8 @@ exp_annotations: # Expression to evaluate expr: -# The time elapsed from time=0s when the expression has to be evaluated. +# The time elapsed from start_timestamp when the expression has to be evaluated. +# This is a duration relative to start_timestamp (which defaults to 0). eval_time: # Expected samples at the given evaluation time. @@ -283,22 +297,16 @@ It should be noted that in all tests, either in `alert_test_case` or for example the `time()` and `day_of_*()` functions, will output a consistent value for tests. -At the start of the test evaluation, `time()` returns 0 and therefore when under test -`time()` will return a value of `0 + eval_time`. +By default, at the start of the test evaluation, `time()` returns 0 (Unix epoch: +January 1, 1970 00:00:00 UTC). The `eval_time` field specifies a duration relative +to `start_timestamp`, so by default `time()` will return a value of `0 + eval_time`. -If you need to write tests for alerts that use functions relating to the current -time, make sure that the values given to your `input_series` are placed far -enough in the past, relative to the evaluation time described above. The values -can for example be negative timestamps so that with a very small `eval_time` the -alert can be expected to trigger. +You can configure a custom start timestamp for your tests by setting the `start_timestamp` +field in your test group. This field accepts either: +- A Unix timestamp (e.g., `1609459200` for January 1, 2021 00:00:00 UTC) +- An RFC3339 formatted timestamp (e.g., `"2021-01-01T00:00:00Z"`) -Another method that's known to work is to instead bump `eval_time` in the future -so that the timestamp output by `time()` will be a higher value and the values -in `input_series` will be far enough apart from that point in time so that the -alerts will trigger. This method has the downside of making promtool generate a -timeseries database that contains a value for each `input_series` for each -`interval` for the given test. This can become very slow relatively easily and -can end up consuming a lot of RAM for running your test. By instead using values -for `input_series` relative to the timestamp described above even though the -values go into negative numbers, you can keep `eval_time` fairly lower and avoid -making your tests run very slowly. +When you set `start_timestamp`: +- All `input_series` samples will be timestamped starting from `start_timestamp` +- The `eval_time` field in test cases is interpreted as a duration relative to `start_timestamp` +- The `time()` function will return `start_timestamp + eval_time` diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go index b16433c14e..d4a11b9e50 100644 --- a/promql/promqltest/test.go +++ b/promql/promqltest/test.go @@ -231,7 +231,7 @@ func raise(line int, format string, v ...any) error { } } -func parseLoad(lines []string, i int) (int, *loadCmd, error) { +func parseLoad(lines []string, i int, startTime time.Time) (int, *loadCmd, error) { if !patLoad.MatchString(lines[i]) { return i, nil, raise(i, "invalid load command. (load[_with_nhcb] )") } @@ -245,6 +245,7 @@ func parseLoad(lines []string, i int) (int, *loadCmd, error) { return i, nil, raise(i, "invalid step definition %q: %s", step, err) } cmd := newLoadCmd(time.Duration(gap), withNHCB) + cmd.startTime = startTime for i+1 < len(lines) { i++ defLine := lines[i] @@ -579,7 +580,7 @@ func (t *test) parse(input string) error { case c == "clear": cmd = &clearCmd{} case strings.HasPrefix(c, "load"): - i, cmd, err = parseLoad(lines, i) + i, cmd, err = parseLoad(lines, i, testStartTime) case strings.HasPrefix(c, "eval"): i, cmd, err = t.parseEval(lines, i) default: @@ -611,6 +612,7 @@ type loadCmd struct { defs map[uint64][]promql.Sample exemplars map[uint64][]exemplar.Exemplar withNHCB bool + startTime time.Time } func newLoadCmd(gap time.Duration, withNHCB bool) *loadCmd { @@ -620,6 +622,7 @@ func newLoadCmd(gap time.Duration, withNHCB bool) *loadCmd { defs: map[uint64][]promql.Sample{}, exemplars: map[uint64][]exemplar.Exemplar{}, withNHCB: withNHCB, + startTime: testStartTime, } } @@ -632,7 +635,7 @@ func (cmd *loadCmd) set(m labels.Labels, vals ...parser.SequenceValue) { h := m.Hash() samples := make([]promql.Sample, 0, len(vals)) - ts := testStartTime + ts := cmd.startTime for _, v := range vals { if !v.Omitted { samples = append(samples, promql.Sample{ @@ -1627,6 +1630,8 @@ type LazyLoaderOpts struct { // Currently defaults to false, matches the "promql-delayed-name-removal" // feature flag. EnableDelayedNameRemoval bool + // StartTime is the start time for the test. If zero, defaults to Unix epoch. + StartTime time.Time } // NewLazyLoader returns an initialized empty LazyLoader. @@ -1652,7 +1657,12 @@ func (ll *LazyLoader) parse(input string) error { continue } if strings.HasPrefix(strings.ToLower(patSpace.Split(l, 2)[0]), "load") { - _, cmd, err := parseLoad(lines, i) + // Determine the start time to use for loading samples. + startTime := testStartTime + if !ll.opts.StartTime.IsZero() { + startTime = ll.opts.StartTime + } + _, cmd, err := parseLoad(lines, i, startTime) if err != nil { return err } From 39e11f50b257066cdc4d72d6df9582f3c3824242 Mon Sep 17 00:00:00 2001 From: Julius Volz Date: Wed, 3 Dec 2025 14:14:35 +0100 Subject: [PATCH 106/439] Fix serialization for empty `ignoring()` in combination with `group_x()` Currently both the backend and frontend printers/formatters/serializers incorrectly transform the following expression: ``` up * ignoring() group_left(__name__) node_boot_time_seconds ``` ...into: ``` up * node_boot_time_seconds ``` ...which yields a different result (including the metric name in the result vs. no metric name). We need to keep empty `ignoring()` modifiers if there is a grouping modifier present. Signed-off-by: Julius Volz --- promql/parser/printer.go | 12 +++--- promql/parser/printer_test.go | 19 ++++++++ web/ui/mantine-ui/src/promql/format.tsx | 23 +++++----- web/ui/mantine-ui/src/promql/serialize.ts | 13 +++--- .../src/promql/serializeAndFormat.test.ts | 43 ++++++++++++++++++- 5 files changed, 85 insertions(+), 25 deletions(-) diff --git a/promql/parser/printer.go b/promql/parser/printer.go index a562b88044..c315bface7 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -147,12 +147,14 @@ func (node *BinaryExpr) ShortString() string { func (node *BinaryExpr) getMatchingStr() string { matching := "" vm := node.VectorMatching - if vm != nil && (len(vm.MatchingLabels) > 0 || vm.On) { - vmTag := "ignoring" - if vm.On { - vmTag = "on" + if vm != nil { + if len(vm.MatchingLabels) > 0 || vm.On || vm.Card == CardManyToOne || vm.Card == CardOneToMany { + vmTag := "ignoring" + if vm.On { + vmTag = "on" + } + matching = fmt.Sprintf(" %s (%s)", vmTag, strings.Join(vm.MatchingLabels, ", ")) } - matching = fmt.Sprintf(" %s (%s)", vmTag, strings.Join(vm.MatchingLabels, ", ")) if vm.Card == CardManyToOne || vm.Card == CardOneToMany { vmCard := "right" diff --git a/promql/parser/printer_test.go b/promql/parser/printer_test.go index aadfd5688a..b28da988da 100644 --- a/promql/parser/printer_test.go +++ b/promql/parser/printer_test.go @@ -94,6 +94,25 @@ func TestExprString(t *testing.T) { in: `a - ignoring() c`, out: `a - c`, }, + { + // This is a bit of an odd case, but valid. If the user specifies ignoring() with + // no labels, it means that both label sets have to be exactly the same on both + // sides (except for the metric name). This is the same behavior as specifying + // no matching modifier at all, but if the user wants to include the metric name + // from either side in the output via group_x(__name__), they have to specify + // ignoring() explicitly to be able to do so, since the grammar does not allow + // grouping modifiers without either ignoring(...) or on(...). So we need to + // preserve the empty ignoring() clause in this case. + // + // a - group_left(__name__) c <--- Parse error + // a - ignoring() group_left(__name__) c <--- Valid + in: `a - ignoring() group_left(__metric__) c`, + out: `a - ignoring () group_left (__metric__) c`, + }, + { + in: `a - ignoring() group_left c`, + out: `a - ignoring () group_left () c`, + }, { in: `up > bool 0`, }, diff --git a/web/ui/mantine-ui/src/promql/format.tsx b/web/ui/mantine-ui/src/promql/format.tsx index f4b883f678..75b1965b35 100644 --- a/web/ui/mantine-ui/src/promql/format.tsx +++ b/web/ui/mantine-ui/src/promql/format.tsx @@ -266,22 +266,19 @@ const formatNodeInternal = ( let matching = <>; let grouping = <>; const vm = node.matching; - if (vm !== null && (vm.labels.length > 0 || vm.on)) { - if (vm.on) { + if (vm !== null) { + if ( + vm.labels.length > 0 || + vm.on || + vm.card === vectorMatchCardinality.manyToOne || + vm.card === vectorMatchCardinality.oneToMany + ) { matching = ( <> {" "} - on - ( - {labelNameList(vm.labels)} - ) - - ); - } else { - matching = ( - <> - {" "} - ignoring + + {vm.on ? "on" : "ignoring"} + ( {labelNameList(vm.labels)} ) diff --git a/web/ui/mantine-ui/src/promql/serialize.ts b/web/ui/mantine-ui/src/promql/serialize.ts index bbccede708..584e1ae9ff 100644 --- a/web/ui/mantine-ui/src/promql/serialize.ts +++ b/web/ui/mantine-ui/src/promql/serialize.ts @@ -136,11 +136,14 @@ const serializeNode = ( let matching = ""; let grouping = ""; const vm = node.matching; - if (vm !== null && (vm.labels.length > 0 || vm.on)) { - if (vm.on) { - matching = ` on(${labelNameList(vm.labels)})`; - } else { - matching = ` ignoring(${labelNameList(vm.labels)})`; + if (vm !== null) { + if ( + vm.labels.length > 0 || + vm.on || + vm.card === vectorMatchCardinality.manyToOne || + vm.card === vectorMatchCardinality.oneToMany + ) { + matching = ` ${vm.on ? "on" : "ignoring"}(${labelNameList(vm.labels)})`; } if ( diff --git a/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts b/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts index 62b10cd781..a3734d311f 100644 --- a/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts +++ b/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts @@ -192,8 +192,7 @@ describe("serializeNode and formatNode", () => { anchored: false, smoothed: false, }, - output: - '{label1="value1"}', + output: '{label1="value1"}', }, // Anchored and smoothed modifiers. @@ -722,6 +721,46 @@ describe("serializeNode and formatNode", () => { output: "… + ignoring(label1, label2) …", prettyOutput: ` … + ignoring(label1, label2) + …`, + }, + { + // Empty ignoring() without group modifiers can be stripped away. + node: { + type: nodeType.binaryExpr, + op: binaryOperatorType.add, + lhs: { type: nodeType.placeholder, children: [] }, + rhs: { type: nodeType.placeholder, children: [] }, + matching: { + card: vectorMatchCardinality.oneToOne, + labels: [], + on: false, + include: [], + }, + bool: false, + }, + output: "… + …", + prettyOutput: ` … ++ + …`, + }, + { + // Empty ignoring() with group modifiers may not be stripped away. + node: { + type: nodeType.binaryExpr, + op: binaryOperatorType.add, + lhs: { type: nodeType.placeholder, children: [] }, + rhs: { type: nodeType.placeholder, children: [] }, + matching: { + card: vectorMatchCardinality.manyToOne, + labels: [], + on: false, + include: ["__name__"], + }, + bool: false, + }, + output: "… + ignoring() group_left(__name__) …", + prettyOutput: ` … ++ ignoring() group_left(__name__) …`, }, { From 4620c8ac71842a15fc9d74d52017c3e012b5bd80 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Wed, 3 Dec 2025 14:20:00 +0100 Subject: [PATCH 107/439] Simplify StartTime assignment in unit test setup. Remove redundant IsZero check since promqltest.LazyLoader already handles zero StartTime by defaulting to Unix epoch. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- cmd/promtool/unittest.go | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/cmd/promtool/unittest.go b/cmd/promtool/unittest.go index 75da96c2eb..14557793c5 100644 --- a/cmd/promtool/unittest.go +++ b/cmd/promtool/unittest.go @@ -232,11 +232,7 @@ func (tg *testGroup) test(testname string, evalInterval time.Duration, groupOrde } // Setup testing suite. // Set the start time from the test group. - if tg.StartTimestamp.IsZero() { - queryOpts.StartTime = time.Unix(0, 0).UTC() - } else { - queryOpts.StartTime = tg.StartTimestamp.Time - } + queryOpts.StartTime = tg.StartTimestamp.Time suite, err := promqltest.NewLazyLoader(tg.seriesLoadingString(), queryOpts) if err != nil { return []error{err} From f0325c5875cdc83ce62025bdab6bbfe0b95bc164 Mon Sep 17 00:00:00 2001 From: Jorge Turrado Date: Thu, 4 Dec 2025 16:00:18 +0100 Subject: [PATCH 108/439] apply feedback Signed-off-by: Jorge Turrado --- docs/configuration/configuration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md index 09f71b5d3c..7773c23bd9 100644 --- a/docs/configuration/configuration.md +++ b/docs/configuration/configuration.md @@ -779,11 +779,11 @@ client_id: # It is mutually exclusive with `client_secret`. [ client_secret_file: ] -# RSA key to sign JWT with. Only used if +# Secret key to sign JWT with. Only used if # GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". [ client_certificate_key: ] -# Read the RSA key from a file. +# Read the secret key from a file. # It is mutually exclusive with `client_certificate_key`. [ client_certificate_key_file: ] @@ -791,7 +791,7 @@ client_id: # GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". [ client_certificate_key_id: ] -# RSA algorithm used to sign JWT token. Only used if +# Signature algorithm used to sign JWT token. Only used if # GrantType is set to "urn:ietf:params:oauth:grant-type:jwt-bearer". # Default value is RS256 and valid values RS256, RS384, RS512 [ signature_algorithm: ] From 3239723098143242b6ab5419e88e2e9ff75ba14e Mon Sep 17 00:00:00 2001 From: dongjiang Date: Fri, 5 Dec 2025 16:29:10 +0800 Subject: [PATCH 109/439] Update golangci-lint and add modernize check (#17640) * add modernize check Signed-off-by: dongjiang1989 * fix golangci lint Signed-off-by: dongjiang1989 --------- Signed-off-by: dongjiang1989 --- .golangci.yml | 7 +++++++ Makefile.common | 2 +- cmd/promtool/unittest.go | 18 ++++++++++-------- discovery/aws/ecs.go | 7 ++----- .../remote_storage_adapter/graphite/escape.go | 2 +- .../remote_storage_adapter/influxdb/client.go | 19 ++++++++++--------- .../opentsdb/tagvalue.go | 2 +- model/labels/labels_slicelabels.go | 12 +++--------- model/labels/labels_slicelabels_test.go | 4 ++-- promql/parser/printer.go | 7 ++++--- tsdb/head_test.go | 6 +++--- tsdb/querier_test.go | 9 +++++---- 12 files changed, 49 insertions(+), 46 deletions(-) diff --git a/.golangci.yml b/.golangci.yml index 22c89a6beb..6dbbcc433d 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -31,6 +31,7 @@ linters: - govet - loggercheck - misspell + - modernize - nilnesserr # TODO(bwplotka): Enable once https://github.com/golangci/golangci-lint/issues/3228 is fixed. # - nolintlint @@ -117,6 +118,12 @@ linters: - shadow - fieldalignment enable-all: true + modernize: + disable: + # Suggest replacing omitempty with omitzero for struct fields. + # Disable this check for now since it introduces too many changes in our existing codebase. + # See https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/modernize#hdr-Analyzer_omitzero for more details. + - omitzero perfsprint: # Optimizes even if it requires an int or uint type cast. int-conversion: true diff --git a/Makefile.common b/Makefile.common index 3ed717b460..840bc0ea71 100644 --- a/Makefile.common +++ b/Makefile.common @@ -61,7 +61,7 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_ SKIP_GOLANGCI_LINT := GOLANGCI_LINT := GOLANGCI_LINT_OPTS ?= -GOLANGCI_LINT_VERSION ?= v2.6.0 +GOLANGCI_LINT_VERSION ?= v2.6.2 GOLANGCI_FMT_OPTS ?= # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64. # windows isn't included here because of the path separator being different. diff --git a/cmd/promtool/unittest.go b/cmd/promtool/unittest.go index 14557793c5..944ffc9d7c 100644 --- a/cmd/promtool/unittest.go +++ b/cmd/promtool/unittest.go @@ -196,7 +196,7 @@ type testStartTimestamp struct { // UnmarshalYAML implements custom YAML unmarshaling for testStartTimestamp. // It accepts both RFC3339 formatted strings and numeric Unix timestamps. -func (t *testStartTimestamp) UnmarshalYAML(unmarshal func(interface{}) error) error { +func (t *testStartTimestamp) UnmarshalYAML(unmarshal func(any) error) error { var s string if err := unmarshal(&s); err != nil { return err @@ -660,13 +660,14 @@ func (la labelsAndAnnotations) String() string { if len(la) == 0 { return "[]" } - s := "[\n0:" + indentLines("\n"+la[0].String(), " ") + var s strings.Builder + s.WriteString("[\n0:" + indentLines("\n"+la[0].String(), " ")) for i, l := range la[1:] { - s += ",\n" + strconv.Itoa(i+1) + ":" + indentLines("\n"+l.String(), " ") + s.WriteString(",\n" + strconv.Itoa(i+1) + ":" + indentLines("\n"+l.String(), " ")) } - s += "\n]" + s.WriteString("\n]") - return s + return s.String() } type labelAndAnnotation struct { @@ -717,11 +718,12 @@ func parsedSamplesString(pss []parsedSample) string { if len(pss) == 0 { return "nil" } - s := pss[0].String() + var s strings.Builder + s.WriteString(pss[0].String()) for _, ps := range pss[1:] { - s += ", " + ps.String() + s.WriteString(", " + ps.String()) } - return s + return s.String() } func (ps *parsedSample) String() string { diff --git a/discovery/aws/ecs.go b/discovery/aws/ecs.go index 3794ad178d..d6b36a7980 100644 --- a/discovery/aws/ecs.go +++ b/discovery/aws/ecs.go @@ -122,7 +122,7 @@ func (c *ECSSDConfig) NewDiscoverer(opts discovery.DiscovererOptions) (discovery } // UnmarshalYAML implements the yaml.Unmarshaler interface for the ECS Config. -func (c *ECSSDConfig) UnmarshalYAML(unmarshal func(interface{}) error) error { +func (c *ECSSDConfig) UnmarshalYAML(unmarshal func(any) error) error { *c = DefaultECSSDConfig type plain ECSSDConfig err := unmarshal((*plain)(c)) @@ -461,10 +461,7 @@ func (d *ECSDiscovery) describeTasks(ctx context.Context, clusterARN string, tas func batchSlice[T any](a []T, size int) [][]T { batches := make([][]T, 0, len(a)/size+1) for i := 0; i < len(a); i += size { - end := i + size - if end > len(a) { - end = len(a) - } + end := min(i+size, len(a)) batches = append(batches, a[i:end]) } return batches diff --git a/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go b/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go index 1386f46761..3793973b7b 100644 --- a/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go +++ b/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go @@ -82,7 +82,7 @@ const ( func escape(tv model.LabelValue) string { length := len(tv) result := bytes.NewBuffer(make([]byte, 0, length)) - for i := 0; i < length; i++ { + for i := range length { b := tv[i] switch { // . is reserved by graphite, % is used to escape other bytes. diff --git a/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go b/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go index 005f8d534d..ffd81802c1 100644 --- a/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go +++ b/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go @@ -96,7 +96,7 @@ func (c *Client) Write(samples model.Samples) error { p := influx.NewPoint( string(s.Metric[model.MetricNameLabel]), tagsFromMetric(s.Metric), - map[string]interface{}{"value": v}, + map[string]any{"value": v}, s.Timestamp.Time(), ) points = append(points, p) @@ -158,16 +158,17 @@ func (c *Client) buildCommand(q *prompb.Query) (string, error) { // If we don't find a metric name matcher, query all metrics // (InfluxDB measurements) by default. - measurement := `r._measurement` + var measurement strings.Builder + measurement.WriteString(`r._measurement`) matchers := make([]string, 0, len(q.Matchers)) var joinedMatchers string for _, m := range q.Matchers { if m.Name == model.MetricNameLabel { switch m.Type { case prompb.LabelMatcher_EQ: - measurement += fmt.Sprintf(" == \"%s\"", m.Value) + measurement.WriteString(fmt.Sprintf(" == \"%s\"", m.Value)) case prompb.LabelMatcher_RE: - measurement += fmt.Sprintf(" =~ /%s/", escapeSlashes(m.Value)) + measurement.WriteString(fmt.Sprintf(" =~ /%s/", escapeSlashes(m.Value))) default: // TODO: Figure out how to support these efficiently. return "", errors.New("non-equal or regex-non-equal matchers are not supported on the metric name yet") @@ -195,7 +196,7 @@ func (c *Client) buildCommand(q *prompb.Query) (string, error) { // _measurement must be retained, otherwise "invalid metric name" shall be thrown command := fmt.Sprintf( "from(bucket: \"%s\") |> range(%s) |> filter(fn: (r) => %s%s)", - c.bucket, rangeInNs, measurement, joinedMatchers, + c.bucket, rangeInNs, measurement.String(), joinedMatchers, ) return command, nil @@ -237,7 +238,7 @@ func mergeResult(labelsToSeries map[string]*prompb.TimeSeries, record *query.Flu return nil } -func filterOutBuiltInLabels(labels map[string]interface{}) { +func filterOutBuiltInLabels(labels map[string]any) { delete(labels, "table") delete(labels, "_start") delete(labels, "_stop") @@ -248,7 +249,7 @@ func filterOutBuiltInLabels(labels map[string]interface{}) { delete(labels, "_measurement") } -func concatLabels(labels map[string]interface{}) string { +func concatLabels(labels map[string]any) string { // 0xff cannot occur in valid UTF-8 sequences, so use it // as a separator here. separator := "\xff" @@ -259,7 +260,7 @@ func concatLabels(labels map[string]interface{}) string { return strings.Join(pairs, separator) } -func tagsToLabelPairs(name string, tags map[string]interface{}) []prompb.Label { +func tagsToLabelPairs(name string, tags map[string]any) []prompb.Label { pairs := make([]prompb.Label, 0, len(tags)) for k, v := range tags { if v == nil { @@ -283,7 +284,7 @@ func tagsToLabelPairs(name string, tags map[string]interface{}) []prompb.Label { return pairs } -func valuesToSamples(timestamp time.Time, value interface{}) (prompb.Sample, error) { +func valuesToSamples(timestamp time.Time, value any) (prompb.Sample, error) { var valueFloat64 float64 var valueInt64 int64 var ok bool diff --git a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go index 6a691778af..c40f829a56 100644 --- a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go +++ b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go @@ -66,7 +66,7 @@ func (tv TagValue) MarshalJSON() ([]byte, error) { // Need at least two more bytes than in tv. result := bytes.NewBuffer(make([]byte, 0, length+2)) result.WriteByte('"') - for i := 0; i < length; i++ { + for i := range length { b := tv[i] switch { case (b >= '-' && b <= '9') || // '-', '.', '/', 0-9 diff --git a/model/labels/labels_slicelabels.go b/model/labels/labels_slicelabels.go index 21ad145c1c..e999432bf4 100644 --- a/model/labels/labels_slicelabels.go +++ b/model/labels/labels_slicelabels.go @@ -297,12 +297,9 @@ func FromStrings(ss ...string) Labels { // Compare compares the two label sets. // The result will be 0 if a==b, <0 if a < b, and >0 if a > b. func Compare(a, b Labels) int { - l := len(a) - if len(b) < l { - l = len(b) - } + l := min(len(b), len(a)) - for i := 0; i < l; i++ { + for i := range l { if a[i].Name != b[i].Name { if a[i].Name < b[i].Name { return -1 @@ -419,10 +416,7 @@ func (b *Builder) Labels() Labels { return b.base } - expectedSize := len(b.base) + len(b.add) - len(b.del) - if expectedSize < 1 { - expectedSize = 1 - } + expectedSize := max(len(b.base)+len(b.add)-len(b.del), 1) res := make(Labels, 0, expectedSize) for _, l := range b.base { if slices.Contains(b.del, l.Name) || contains(b.add, l.Name) { diff --git a/model/labels/labels_slicelabels_test.go b/model/labels/labels_slicelabels_test.go index 7961828378..0e55730082 100644 --- a/model/labels/labels_slicelabels_test.go +++ b/model/labels/labels_slicelabels_test.go @@ -77,8 +77,8 @@ func BenchmarkScratchBuilderUnsafeAdd(b *testing.B) { l.SetUnsafeAdd(true) b.ReportAllocs() - b.ResetTimer() - for i := 0; i < b.N; i++ { + + for b.Loop() { l.Add("__name__", "metric1") l.add = l.add[:0] // Reset slice so add can be repeated without side effects. } diff --git a/promql/parser/printer.go b/promql/parser/printer.go index c315bface7..961167428b 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -37,15 +37,16 @@ func tree(node Node, level string) string { } typs := strings.Split(fmt.Sprintf("%T", node), ".")[1] - t := fmt.Sprintf("%s |---- %s :: %s\n", level, typs, node) + var t strings.Builder + t.WriteString(fmt.Sprintf("%s |---- %s :: %s\n", level, typs, node)) level += " · · ·" for e := range ChildrenIter(node) { - t += tree(e, level) + t.WriteString(tree(e, level)) } - return t + return t.String() } func (node *EvalStmt) String() string { diff --git a/tsdb/head_test.go b/tsdb/head_test.go index 552db13d07..d3e6ca9bcc 100644 --- a/tsdb/head_test.go +++ b/tsdb/head_test.go @@ -6525,19 +6525,19 @@ func TestWALSampleAndExemplarOrder(t *testing.T) { appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { return app.Append(0, lbls, ts, 1.0) }, - expectedType: reflect.TypeOf([]record.RefSample{}), + expectedType: reflect.TypeFor[[]record.RefSample](), }, "histogram sample": { appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { return app.AppendHistogram(0, lbls, ts, tsdbutil.GenerateTestHistogram(1), nil) }, - expectedType: reflect.TypeOf([]record.RefHistogramSample{}), + expectedType: reflect.TypeFor[[]record.RefHistogramSample](), }, "float histogram sample": { appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { return app.AppendHistogram(0, lbls, ts, nil, tsdbutil.GenerateTestFloatHistogram(1)) }, - expectedType: reflect.TypeOf([]record.RefFloatHistogramSample{}), + expectedType: reflect.TypeFor[[]record.RefFloatHistogramSample](), }, } diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go index 5f5af441ca..6c3e37792f 100644 --- a/tsdb/querier_test.go +++ b/tsdb/querier_test.go @@ -23,6 +23,7 @@ import ( "slices" "sort" "strconv" + "strings" "sync" "testing" "time" @@ -3037,14 +3038,14 @@ func TestPostingsForMatchers(t *testing.T) { require.NoError(t, err) for _, c := range cases { - name := "" + var name strings.Builder for i, matcher := range c.matchers { if i > 0 { - name += "," + name.WriteString(",") } - name += matcher.String() + name.WriteString(matcher.String()) } - t.Run(name, func(t *testing.T) { + t.Run(name.String(), func(t *testing.T) { exp := map[string]struct{}{} for _, l := range c.exp { exp[l.String()] = struct{}{} From 025628f272ad1319288241c3e4111d4897d6e552 Mon Sep 17 00:00:00 2001 From: intojhanurag Date: Sat, 6 Dec 2025 19:34:14 +0000 Subject: [PATCH 110/439] unregistering RefreshMetrics instances Signed-off-by: intojhanurag --- discovery/triton/triton_test.go | 31 ++++++++++++++++++------------- discovery/xds/kuma_test.go | 18 ++++++++++-------- 2 files changed, 28 insertions(+), 21 deletions(-) diff --git a/discovery/triton/triton_test.go b/discovery/triton/triton_test.go index 731303677d..97a20f95d8 100644 --- a/discovery/triton/triton_test.go +++ b/discovery/triton/triton_test.go @@ -80,14 +80,13 @@ var ( } ) -func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, error) { +func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, discovery.RefreshMetricsManager, error) { reg := prometheus.NewRegistry() refreshMetrics := discovery.NewRefreshMetrics(reg) - // TODO(ptodev): Add the ability to unregister refresh metrics. metrics := c.NewDiscovererMetrics(reg, refreshMetrics) err := metrics.Register() if err != nil { - return nil, nil, err + return nil, nil, nil, err } d, err := New(&c, discovery.DiscovererOptions{ @@ -96,14 +95,14 @@ func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, er SetName: "triton", }) if err != nil { - return nil, nil, err + return nil, nil, nil, err } - return d, metrics, nil + return d, metrics, refreshMetrics, nil } func TestTritonSDNew(t *testing.T) { - td, m, err := newTritonDiscovery(conf) + td, m, rm, err := newTritonDiscovery(conf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -114,16 +113,17 @@ func TestTritonSDNew(t *testing.T) { require.Equal(t, conf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, conf.Port, td.sdConfig.Port) m.Unregister() + rm.Unregister() } func TestTritonSDNewBadConfig(t *testing.T) { - td, _, err := newTritonDiscovery(badconf) + td, _, _, err := newTritonDiscovery(badconf) require.Error(t, err) require.Nil(t, td) } func TestTritonSDNewGroupsConfig(t *testing.T) { - td, m, err := newTritonDiscovery(groupsconf) + td, m, rm, err := newTritonDiscovery(groupsconf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -135,10 +135,11 @@ func TestTritonSDNewGroupsConfig(t *testing.T) { require.Equal(t, groupsconf.Groups, td.sdConfig.Groups) require.Equal(t, groupsconf.Port, td.sdConfig.Port) m.Unregister() + rm.Unregister() } func TestTritonSDNewCNConfig(t *testing.T) { - td, m, err := newTritonDiscovery(cnconf) + td, m, rm, err := newTritonDiscovery(cnconf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -150,6 +151,7 @@ func TestTritonSDNewCNConfig(t *testing.T) { require.Equal(t, cnconf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, cnconf.Port, td.sdConfig.Port) m.Unregister() + rm.Unregister() } func TestTritonSDRefreshNoTargets(t *testing.T) { @@ -182,21 +184,23 @@ func TestTritonSDRefreshMultipleTargets(t *testing.T) { } func TestTritonSDRefreshNoServer(t *testing.T) { - td, m, _ := newTritonDiscovery(conf) + td, m, rm, _ := newTritonDiscovery(conf) _, err := td.refresh(context.Background()) require.ErrorContains(t, err, "an error occurred when requesting targets from the discovery endpoint") m.Unregister() + rm.Unregister() } func TestTritonSDRefreshCancelled(t *testing.T) { - td, m, _ := newTritonDiscovery(conf) + td, m, rm, _ := newTritonDiscovery(conf) ctx, cancel := context.WithCancel(context.Background()) cancel() _, err := td.refresh(ctx) require.ErrorContains(t, err, context.Canceled.Error()) m.Unregister() + rm.Unregister() } func TestTritonSDRefreshCNsUUIDOnly(t *testing.T) { @@ -233,8 +237,8 @@ func TestTritonSDRefreshCNsWithHostname(t *testing.T) { func testTritonSDRefresh(t *testing.T, c SDConfig, dstr string) []model.LabelSet { var ( - td, m, _ = newTritonDiscovery(c) - s = httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + td, m, rm, _ = newTritonDiscovery(c) + s = httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { fmt.Fprintln(w, dstr) })) ) @@ -263,6 +267,7 @@ func testTritonSDRefresh(t *testing.T, c SDConfig, dstr string) []model.LabelSet require.NotNil(t, tg) m.Unregister() + rm.Unregister() return tg.Targets } diff --git a/discovery/xds/kuma_test.go b/discovery/xds/kuma_test.go index 533a31dcf3..d54b83ea61 100644 --- a/discovery/xds/kuma_test.go +++ b/discovery/xds/kuma_test.go @@ -108,26 +108,25 @@ func getKumaMadsV1DiscoveryResponse(resources ...*MonitoringAssignment) (*v3.Dis }, nil } -func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, error) { +func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, discovery.RefreshMetricsManager, error) { reg := prometheus.NewRegistry() refreshMetrics := discovery.NewRefreshMetrics(reg) - // TODO(ptodev): Add the ability to unregister refresh metrics. metrics := c.NewDiscovererMetrics(reg, refreshMetrics) err := metrics.Register() if err != nil { - return nil, err + return nil, nil, err } kd, err := NewKumaHTTPDiscovery(&c, nopLogger, metrics) if err != nil { - return nil, err + return nil, nil, err } pd, ok := kd.(*fetchDiscovery) if !ok { - return nil, errors.New("not a fetchDiscovery") + return nil, nil, errors.New("not a fetchDiscovery") } - return pd, nil + return pd, refreshMetrics, nil } func TestKumaMadsV1ResourceParserInvalidTypeURL(t *testing.T) { @@ -216,7 +215,7 @@ func TestKumaMadsV1ResourceParserInvalidResources(t *testing.T) { func TestNewKumaHTTPDiscovery(t *testing.T) { t.Parallel() - kd, err := newKumaTestHTTPDiscovery(kumaConf) + kd, rm, err := newKumaTestHTTPDiscovery(kumaConf) require.NoError(t, err) require.NotNil(t, kd) @@ -228,6 +227,7 @@ func TestNewKumaHTTPDiscovery(t *testing.T) { require.Equal(t, KumaMadsV1ResourceType, resClient.config.ResourceType) kd.metrics.Unregister() + rm.Unregister() } func TestKumaHTTPDiscoveryRefresh(t *testing.T) { @@ -259,7 +259,7 @@ tls_config: var cfg KumaSDConfig require.NoError(t, yaml.Unmarshal([]byte(cfgString), &cfg)) - kd, err := newKumaTestHTTPDiscovery(cfg) + kd, rm, err := newKumaTestHTTPDiscovery(cfg) require.NoError(t, err) require.NotNil(t, kd) @@ -320,10 +320,12 @@ tls_config: kd.poll(ctx, ch) select { case <-ctx.Done(): + rm.Unregister() return case <-ch: require.Fail(t, "no update expected") } kd.metrics.Unregister() + rm.Unregister() } From 0251e888f96e853ae503d03de7e65f975a1fdfe1 Mon Sep 17 00:00:00 2001 From: intojhanurag Date: Mon, 8 Dec 2025 08:25:34 +0000 Subject: [PATCH 111/439] Refactor Triton tests to simplify metrics cleanup using cleanup closure Signed-off-by: intojhanurag --- discovery/triton/triton_test.go | 58 ++++++++++++++++----------------- discovery/xds/kuma_test.go | 1 + 2 files changed, 30 insertions(+), 29 deletions(-) diff --git a/discovery/triton/triton_test.go b/discovery/triton/triton_test.go index 97a20f95d8..453cdd2e62 100644 --- a/discovery/triton/triton_test.go +++ b/discovery/triton/triton_test.go @@ -80,13 +80,13 @@ var ( } ) -func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, discovery.RefreshMetricsManager, error) { +func newTritonDiscovery(c SDConfig) (*Discovery, func(), error) { reg := prometheus.NewRegistry() refreshMetrics := discovery.NewRefreshMetrics(reg) metrics := c.NewDiscovererMetrics(reg, refreshMetrics) err := metrics.Register() if err != nil { - return nil, nil, nil, err + return nil, nil, err } d, err := New(&c, discovery.DiscovererOptions{ @@ -95,14 +95,19 @@ func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, di SetName: "triton", }) if err != nil { - return nil, nil, nil, err + return nil, nil, err } - return d, metrics, refreshMetrics, nil + cleanup := func() { + metrics.Unregister() + refreshMetrics.Unregister() + } + + return d, cleanup, nil } func TestTritonSDNew(t *testing.T) { - td, m, rm, err := newTritonDiscovery(conf) + td, cleanup, err := newTritonDiscovery(conf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -112,18 +117,20 @@ func TestTritonSDNew(t *testing.T) { require.Equal(t, conf.DNSSuffix, td.sdConfig.DNSSuffix) require.Equal(t, conf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, conf.Port, td.sdConfig.Port) - m.Unregister() - rm.Unregister() + defer cleanup() } func TestTritonSDNewBadConfig(t *testing.T) { - td, _, _, err := newTritonDiscovery(badconf) + td, cleanup, err := newTritonDiscovery(badconf) require.Error(t, err) require.Nil(t, td) + if cleanup != nil { + defer cleanup() + } } func TestTritonSDNewGroupsConfig(t *testing.T) { - td, m, rm, err := newTritonDiscovery(groupsconf) + td, cleanup, err := newTritonDiscovery(groupsconf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -134,12 +141,11 @@ func TestTritonSDNewGroupsConfig(t *testing.T) { require.Equal(t, groupsconf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, groupsconf.Groups, td.sdConfig.Groups) require.Equal(t, groupsconf.Port, td.sdConfig.Port) - m.Unregister() - rm.Unregister() + defer cleanup() } func TestTritonSDNewCNConfig(t *testing.T) { - td, m, rm, err := newTritonDiscovery(cnconf) + td, cleanup, err := newTritonDiscovery(cnconf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -150,8 +156,7 @@ func TestTritonSDNewCNConfig(t *testing.T) { require.Equal(t, cnconf.DNSSuffix, td.sdConfig.DNSSuffix) require.Equal(t, cnconf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, cnconf.Port, td.sdConfig.Port) - m.Unregister() - rm.Unregister() + defer cleanup() } func TestTritonSDRefreshNoTargets(t *testing.T) { @@ -184,23 +189,21 @@ func TestTritonSDRefreshMultipleTargets(t *testing.T) { } func TestTritonSDRefreshNoServer(t *testing.T) { - td, m, rm, _ := newTritonDiscovery(conf) + td, cleanup, _ := newTritonDiscovery(conf) + defer cleanup() _, err := td.refresh(context.Background()) require.ErrorContains(t, err, "an error occurred when requesting targets from the discovery endpoint") - m.Unregister() - rm.Unregister() } func TestTritonSDRefreshCancelled(t *testing.T) { - td, m, rm, _ := newTritonDiscovery(conf) + td, cleanup, _ := newTritonDiscovery(conf) + defer cleanup() ctx, cancel := context.WithCancel(context.Background()) cancel() _, err := td.refresh(ctx) require.ErrorContains(t, err, context.Canceled.Error()) - m.Unregister() - rm.Unregister() } func TestTritonSDRefreshCNsUUIDOnly(t *testing.T) { @@ -236,12 +239,12 @@ func TestTritonSDRefreshCNsWithHostname(t *testing.T) { } func testTritonSDRefresh(t *testing.T, c SDConfig, dstr string) []model.LabelSet { - var ( - td, m, rm, _ = newTritonDiscovery(c) - s = httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { - fmt.Fprintln(w, dstr) - })) - ) + td, cleanup, _ := newTritonDiscovery(c) + defer cleanup() + + s := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + fmt.Fprintln(w, dstr) + })) defer s.Close() @@ -266,8 +269,5 @@ func testTritonSDRefresh(t *testing.T, c SDConfig, dstr string) []model.LabelSet tg := tgs[0] require.NotNil(t, tg) - m.Unregister() - rm.Unregister() - return tg.Targets } diff --git a/discovery/xds/kuma_test.go b/discovery/xds/kuma_test.go index d54b83ea61..b49bb478bd 100644 --- a/discovery/xds/kuma_test.go +++ b/discovery/xds/kuma_test.go @@ -320,6 +320,7 @@ tls_config: kd.poll(ctx, ch) select { case <-ctx.Done(): + kd.metrics.Unregister() rm.Unregister() return case <-ch: From ea072fd56af820cd7f50f2ec12340b983febae34 Mon Sep 17 00:00:00 2001 From: intojhanurag Date: Mon, 8 Dec 2025 08:35:16 +0000 Subject: [PATCH 112/439] Added cleanup closure in kuma test Signed-off-by: intojhanurag --- discovery/xds/kuma_test.go | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/discovery/xds/kuma_test.go b/discovery/xds/kuma_test.go index b49bb478bd..848c1826c8 100644 --- a/discovery/xds/kuma_test.go +++ b/discovery/xds/kuma_test.go @@ -108,7 +108,7 @@ func getKumaMadsV1DiscoveryResponse(resources ...*MonitoringAssignment) (*v3.Dis }, nil } -func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, discovery.RefreshMetricsManager, error) { +func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, func(), error) { reg := prometheus.NewRegistry() refreshMetrics := discovery.NewRefreshMetrics(reg) metrics := c.NewDiscovererMetrics(reg, refreshMetrics) @@ -126,7 +126,13 @@ func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, discovery.Refres if !ok { return nil, nil, errors.New("not a fetchDiscovery") } - return pd, refreshMetrics, nil + + cleanup := func() { + metrics.Unregister() + refreshMetrics.Unregister() + } + + return pd, cleanup, nil } func TestKumaMadsV1ResourceParserInvalidTypeURL(t *testing.T) { @@ -215,9 +221,10 @@ func TestKumaMadsV1ResourceParserInvalidResources(t *testing.T) { func TestNewKumaHTTPDiscovery(t *testing.T) { t.Parallel() - kd, rm, err := newKumaTestHTTPDiscovery(kumaConf) + kd, cleanup, err := newKumaTestHTTPDiscovery(kumaConf) require.NoError(t, err) require.NotNil(t, kd) + defer cleanup() resClient, ok := kd.client.(*HTTPResourceClient) require.True(t, ok) @@ -225,9 +232,6 @@ func TestNewKumaHTTPDiscovery(t *testing.T) { require.Equal(t, KumaMadsV1ResourceTypeURL, resClient.ResourceTypeURL()) require.Equal(t, kumaConf.ClientID, resClient.ID()) require.Equal(t, KumaMadsV1ResourceType, resClient.config.ResourceType) - - kd.metrics.Unregister() - rm.Unregister() } func TestKumaHTTPDiscoveryRefresh(t *testing.T) { @@ -259,9 +263,10 @@ tls_config: var cfg KumaSDConfig require.NoError(t, yaml.Unmarshal([]byte(cfgString), &cfg)) - kd, rm, err := newKumaTestHTTPDiscovery(cfg) + kd, cleanup, err := newKumaTestHTTPDiscovery(cfg) require.NoError(t, err) require.NotNil(t, kd) + defer cleanup() ch := make(chan []*targetgroup.Group, 1) kd.poll(context.Background(), ch) @@ -320,13 +325,8 @@ tls_config: kd.poll(ctx, ch) select { case <-ctx.Done(): - kd.metrics.Unregister() - rm.Unregister() return case <-ch: require.Fail(t, "no update expected") } - - kd.metrics.Unregister() - rm.Unregister() } From 1ccc0fed8141f4edf99270310132bb24ad5606da Mon Sep 17 00:00:00 2001 From: intojhanurag Date: Mon, 8 Dec 2025 16:50:54 +0000 Subject: [PATCH 113/439] Revert triton_test.go and kuma_test.go to main version Signed-off-by: intojhanurag --- discovery/triton/triton_test.go | 48 +++++++++++++++------------------ discovery/xds/kuma_test.go | 26 ++++++++---------- 2 files changed, 32 insertions(+), 42 deletions(-) diff --git a/discovery/triton/triton_test.go b/discovery/triton/triton_test.go index 453cdd2e62..6cbc52d020 100644 --- a/discovery/triton/triton_test.go +++ b/discovery/triton/triton_test.go @@ -80,7 +80,7 @@ var ( } ) -func newTritonDiscovery(c SDConfig) (*Discovery, func(), error) { +func newTritonDiscovery(c SDConfig) (*Discovery, discovery.DiscovererMetrics, error) { reg := prometheus.NewRegistry() refreshMetrics := discovery.NewRefreshMetrics(reg) metrics := c.NewDiscovererMetrics(reg, refreshMetrics) @@ -98,16 +98,11 @@ func newTritonDiscovery(c SDConfig) (*Discovery, func(), error) { return nil, nil, err } - cleanup := func() { - metrics.Unregister() - refreshMetrics.Unregister() - } - - return d, cleanup, nil + return d, metrics, nil } func TestTritonSDNew(t *testing.T) { - td, cleanup, err := newTritonDiscovery(conf) + td, m, err := newTritonDiscovery(conf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -117,20 +112,17 @@ func TestTritonSDNew(t *testing.T) { require.Equal(t, conf.DNSSuffix, td.sdConfig.DNSSuffix) require.Equal(t, conf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, conf.Port, td.sdConfig.Port) - defer cleanup() + m.Unregister() } func TestTritonSDNewBadConfig(t *testing.T) { - td, cleanup, err := newTritonDiscovery(badconf) + td, _, err := newTritonDiscovery(badconf) require.Error(t, err) require.Nil(t, td) - if cleanup != nil { - defer cleanup() - } } func TestTritonSDNewGroupsConfig(t *testing.T) { - td, cleanup, err := newTritonDiscovery(groupsconf) + td, m, err := newTritonDiscovery(groupsconf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -141,11 +133,11 @@ func TestTritonSDNewGroupsConfig(t *testing.T) { require.Equal(t, groupsconf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, groupsconf.Groups, td.sdConfig.Groups) require.Equal(t, groupsconf.Port, td.sdConfig.Port) - defer cleanup() + m.Unregister() } func TestTritonSDNewCNConfig(t *testing.T) { - td, cleanup, err := newTritonDiscovery(cnconf) + td, m, err := newTritonDiscovery(cnconf) require.NoError(t, err) require.NotNil(t, td) require.NotNil(t, td.client) @@ -156,7 +148,7 @@ func TestTritonSDNewCNConfig(t *testing.T) { require.Equal(t, cnconf.DNSSuffix, td.sdConfig.DNSSuffix) require.Equal(t, cnconf.Endpoint, td.sdConfig.Endpoint) require.Equal(t, cnconf.Port, td.sdConfig.Port) - defer cleanup() + m.Unregister() } func TestTritonSDRefreshNoTargets(t *testing.T) { @@ -189,21 +181,21 @@ func TestTritonSDRefreshMultipleTargets(t *testing.T) { } func TestTritonSDRefreshNoServer(t *testing.T) { - td, cleanup, _ := newTritonDiscovery(conf) - defer cleanup() + td, m, _ := newTritonDiscovery(conf) _, err := td.refresh(context.Background()) require.ErrorContains(t, err, "an error occurred when requesting targets from the discovery endpoint") + m.Unregister() } func TestTritonSDRefreshCancelled(t *testing.T) { - td, cleanup, _ := newTritonDiscovery(conf) - defer cleanup() + td, m, _ := newTritonDiscovery(conf) ctx, cancel := context.WithCancel(context.Background()) cancel() _, err := td.refresh(ctx) require.ErrorContains(t, err, context.Canceled.Error()) + m.Unregister() } func TestTritonSDRefreshCNsUUIDOnly(t *testing.T) { @@ -239,12 +231,12 @@ func TestTritonSDRefreshCNsWithHostname(t *testing.T) { } func testTritonSDRefresh(t *testing.T, c SDConfig, dstr string) []model.LabelSet { - td, cleanup, _ := newTritonDiscovery(c) - defer cleanup() - - s := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { - fmt.Fprintln(w, dstr) - })) + var ( + td, m, _ = newTritonDiscovery(c) + s = httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + fmt.Fprintln(w, dstr) + })) + ) defer s.Close() @@ -269,5 +261,7 @@ func testTritonSDRefresh(t *testing.T, c SDConfig, dstr string) []model.LabelSet tg := tgs[0] require.NotNil(t, tg) + m.Unregister() + return tg.Targets } diff --git a/discovery/xds/kuma_test.go b/discovery/xds/kuma_test.go index 848c1826c8..3f8a769fe1 100644 --- a/discovery/xds/kuma_test.go +++ b/discovery/xds/kuma_test.go @@ -108,31 +108,25 @@ func getKumaMadsV1DiscoveryResponse(resources ...*MonitoringAssignment) (*v3.Dis }, nil } -func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, func(), error) { +func newKumaTestHTTPDiscovery(c KumaSDConfig) (*fetchDiscovery, error) { reg := prometheus.NewRegistry() refreshMetrics := discovery.NewRefreshMetrics(reg) metrics := c.NewDiscovererMetrics(reg, refreshMetrics) err := metrics.Register() if err != nil { - return nil, nil, err + return nil, err } kd, err := NewKumaHTTPDiscovery(&c, nopLogger, metrics) if err != nil { - return nil, nil, err + return nil, err } pd, ok := kd.(*fetchDiscovery) if !ok { - return nil, nil, errors.New("not a fetchDiscovery") + return nil, errors.New("not a fetchDiscovery") } - - cleanup := func() { - metrics.Unregister() - refreshMetrics.Unregister() - } - - return pd, cleanup, nil + return pd, nil } func TestKumaMadsV1ResourceParserInvalidTypeURL(t *testing.T) { @@ -221,10 +215,9 @@ func TestKumaMadsV1ResourceParserInvalidResources(t *testing.T) { func TestNewKumaHTTPDiscovery(t *testing.T) { t.Parallel() - kd, cleanup, err := newKumaTestHTTPDiscovery(kumaConf) + kd, err := newKumaTestHTTPDiscovery(kumaConf) require.NoError(t, err) require.NotNil(t, kd) - defer cleanup() resClient, ok := kd.client.(*HTTPResourceClient) require.True(t, ok) @@ -232,6 +225,8 @@ func TestNewKumaHTTPDiscovery(t *testing.T) { require.Equal(t, KumaMadsV1ResourceTypeURL, resClient.ResourceTypeURL()) require.Equal(t, kumaConf.ClientID, resClient.ID()) require.Equal(t, KumaMadsV1ResourceType, resClient.config.ResourceType) + + kd.metrics.Unregister() } func TestKumaHTTPDiscoveryRefresh(t *testing.T) { @@ -263,10 +258,9 @@ tls_config: var cfg KumaSDConfig require.NoError(t, yaml.Unmarshal([]byte(cfgString), &cfg)) - kd, cleanup, err := newKumaTestHTTPDiscovery(cfg) + kd, err := newKumaTestHTTPDiscovery(cfg) require.NoError(t, err) require.NotNil(t, kd) - defer cleanup() ch := make(chan []*targetgroup.Group, 1) kd.poll(context.Background(), ch) @@ -329,4 +323,6 @@ tls_config: case <-ch: require.Fail(t, "no update expected") } + + kd.metrics.Unregister() } From 04696703fe253001571b4dbcb012a575f3eec18b Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 28 Nov 2025 12:13:38 +0000 Subject: [PATCH 114/439] refactor(appenderV2): add AppenderV2 interface Signed-off-by: bwplotka --- storage/interface.go | 47 ++++++---- storage/interface_append.go | 169 ++++++++++++++++++++++++++++++++++++ 2 files changed, 198 insertions(+), 18 deletions(-) create mode 100644 storage/interface_append.go diff --git a/storage/interface.go b/storage/interface.go index 19b4db4210..fe9b3fa6e8 100644 --- a/storage/interface.go +++ b/storage/interface.go @@ -58,11 +58,14 @@ var ( // their own reference types. type SeriesRef uint64 -// Appendable allows creating appenders. +// Appendable allows creating Appender. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type Appendable interface { - // Appender returns a new appender for the storage. The implementation - // can choose whether or not to use the context, for deadlines or to check - // for errors. + // Appender returns a new appender for the storage. + // + // Implementations CAN choose whether to use the context e.g. for deadlines, + // but it's not mandatory. Appender(ctx context.Context) Appender } @@ -255,7 +258,13 @@ func (f QueryableFunc) Querier(mint, maxt int64) (Querier, error) { return f(mint, maxt) } +// AppendOptions provides options for implementations of the Appender interface. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type AppendOptions struct { + // DiscardOutOfOrder tells implementation that this append should not be out + // of order. An OOO append MUST be rejected with storage.ErrOutOfOrderSample + // error. DiscardOutOfOrder bool } @@ -267,7 +276,11 @@ type AppendOptions struct { // The order of samples appended via the Appender is preserved within each // series. I.e. samples are not reordered per timestamp, or by float/histogram // type. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type Appender interface { + AppenderTransaction + // Append adds a sample pair for the given series. // An optional series reference can be provided to accelerate calls. // A series reference number is returned which can be used to add further @@ -278,16 +291,6 @@ type Appender interface { // If the reference is 0 it must not be used for caching. Append(ref SeriesRef, l labels.Labels, t int64, v float64) (SeriesRef, error) - // Commit submits the collected samples and purges the batch. If Commit - // returns a non-nil error, it also rolls back all modifications made in - // the appender so far, as Rollback would do. In any case, an Appender - // must not be used anymore after Commit has been called. - Commit() error - - // Rollback rolls back all modifications made in the appender so far. - // Appender has to be discarded after rollback. - Rollback() error - // SetOptions configures the appender with specific append options such as // discarding out-of-order samples even if out-of-order is enabled in the TSDB. SetOptions(opts *AppendOptions) @@ -301,8 +304,8 @@ type Appender interface { // GetRef is an extra interface on Appenders used by downstream projects // (e.g. Cortex) to avoid maintaining a parallel set of references. type GetRef interface { - // Returns reference number that can be used to pass to Appender.Append(), - // and a set of labels that will not cause another copy when passed to Appender.Append(). + // GetRef returns a reference number that can be used to pass to AppenderV2.Append(), + // and a set of labels that will not cause another copy when passed to AppenderV2.Append(). // 0 means the appender does not have a reference to this series. // hash should be a hash of lset. GetRef(lset labels.Labels, hash uint64) (SeriesRef, labels.Labels) @@ -310,6 +313,8 @@ type GetRef interface { // ExemplarAppender provides an interface for adding samples to exemplar storage, which // within Prometheus is in-memory only. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type ExemplarAppender interface { // AppendExemplar adds an exemplar for the given series labels. // An optional reference number can be provided to accelerate calls. @@ -326,6 +331,8 @@ type ExemplarAppender interface { } // HistogramAppender provides an interface for appending histograms to the storage. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type HistogramAppender interface { // AppendHistogram adds a histogram for the given series labels. An // optional reference number can be provided to accelerate calls. A @@ -356,6 +363,8 @@ type HistogramAppender interface { } // MetadataUpdater provides an interface for associating metadata to stored series. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type MetadataUpdater interface { // UpdateMetadata updates a metadata entry for the given series and labels. // A series reference number is returned which can be used to modify the @@ -368,6 +377,8 @@ type MetadataUpdater interface { } // StartTimestampAppender provides an interface for appending ST to storage. +// +// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). type StartTimestampAppender interface { // AppendSTZeroSample adds synthetic zero sample for the given st timestamp, // which will be associated with given series, labels and the incoming @@ -390,10 +401,10 @@ type SeriesSet interface { Next() bool // At returns full series. Returned series should be iterable even after Next is called. At() Series - // The error that iteration has failed with. + // Err returns the error that iteration has failed with. // When an error occurs, set cannot continue to iterate. Err() error - // A collection of warnings for the whole set. + // Warnings returns a collection of warnings for the whole set. // Warnings could be return even iteration has not failed with error. Warnings() annotations.Annotations } diff --git a/storage/interface_append.go b/storage/interface_append.go new file mode 100644 index 0000000000..c8d1b46ce8 --- /dev/null +++ b/storage/interface_append.go @@ -0,0 +1,169 @@ +package storage + +import ( + "context" + "errors" + + "github.com/prometheus/prometheus/model/exemplar" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/model/metadata" +) + +// AppendableV2 allows creating AppenderV2. +type AppendableV2 interface { + // AppenderV2 returns a new appender for the storage. + // + // Implementations CAN choose whether to use the context e.g. for deadlines, + // but it's not mandatory. + AppenderV2(ctx context.Context) AppenderV2 +} + +// AOptions is a shorthand for AppendV2Options. +// NOTE: AppendOption is used already. +type AOptions = AppendV2Options + +// AppendV2Options provides optional, auxiliary data and configuration for AppenderV2.Append. +type AppendV2Options struct { + // MetricFamilyName (optional) provides metric family name for the appended sample's + // series. If the client of the AppenderV2 has this information + // (e.g. from scrape) it's recommended to pass it to the appender. + // + // Provided string bytes are unsafe to reuse, it only lives for the duration of the Append call. + // + // Some implementations use this to avoid slow and prone to error metric family detection for: + // * Metadata per metric family storages (e.g. Prometheus metadata WAL/API/RW1) + // * Strictly complex types storages (e.g. OpenTelemetry Collector). + // + // NOTE(krajorama): Example purpose is highlighted in OTLP ingestion: OTLP calculates the + // metric family name for all metrics and uses it for generating summary, + // histogram series by adding the magic suffixes. The metric family name is + // passed down to the appender in case the storage needs it for metadata updates. + // Known user of this is Mimir that implements /api/v1/metadata and uses + // Remote-Write 1.0 for this. Might be removed later if no longer + // needed by any downstream project. + // NOTE(bwplotka): Long term, once Prometheus uses complex types on storage level + // the MetricFamilyName can be removed as MetricFamilyName will equal to __name__ always. + MetricFamilyName string + + // Metadata (optional) attached to the appended sample. + // Metadata strings are safe for reuse. + // IMPORTANT: Appender v1 was only providing update. This field MUST be + // set (if known) even if it didn't change since the last iteration. + // This moves the responsibility for metadata storage options to TSDB. + Metadata metadata.Metadata + + // Exemplars (optional) attached to the appended sample. + // Exemplar slice MUST be sorted by Exemplar.TS. + // Exemplar slice is unsafe for reuse. + Exemplars []exemplar.Exemplar + + // RejectOutOfOrder tells implementation that this append should not be out + // of order. An OOO append MUST be rejected with storage.ErrOutOfOrderSample + // error. + RejectOutOfOrder bool +} + +// AppendPartialError represents an AppenderV2.Append error that tells +// callers sample was written but some auxiliary optional data (e.g. exemplars) +// was not (or partially written) +// +// It's up to the caller to decide if it's an ignorable error or not, plus +// it allows extra reporting (e.g. for Remote Write 2.0 X-Remote-Write-Written headers). +type AppendPartialError struct { + ExemplarErrors []error +} + +// Error returns combined error string. +func (e *AppendPartialError) Error() string { + errs := errors.Join(e.ExemplarErrors...) + if errs == nil { + return "" + } + return errs.Error() +} + +var _ error = &AppendPartialError{} + +// AppenderV2 provides appends against a storage for all types of samples. +// It must be completed with a call to Commit or Rollback and must not be reused afterwards. +// +// Operations on the AppenderV2 interface are not goroutine-safe. +// +// The order of samples appended via the AppenderV2 is preserved within each +// series. I.e. samples are not reordered per timestamp, or by float/histogram +// type. +type AppenderV2 interface { + AppenderTransaction + + // Append appends a sample and related exemplars, metadata, and start timestamp (st) to the storage. + // + // ref (optional) represents the stable ID for the given series identified by ls (excluding metadata). + // Callers MAY provide the ref to help implementation avoid ls -> ref computation, otherwise ref MUST be 0 (unknown). + // + // ls represents labels for the sample's series. + // + // st (optional) represents sample start timestamp. 0 means unknown. Implementations + // are responsible for any potential ST storage logic (e.g. ST zero injections). + // + // t represents sample timestamp. + // + // v, h, fh represents sample value for each sample type. + // Callers MUST only provide one of the sample types (either v, h or fh). + // Implementations can detect the type of the sample with the following switch: + // + // switch { + // case fh != nil: It's a float histogram append. + // case h != nil: It's a histogram append. + // default: It's a float append. + // } + // TODO(bwplotka): We plan to experiment on using generics for complex sampleType, but do it after we unify interface (derisk) and before we add native summaries. + // + // Implementations MUST attempt to append sample even if metadata, exemplar or (st) start timestamp appends fail. + // Implementations MAY return AppendPartialError as an error. Use errors.As to detect. + // For the successful Append, Implementations MUST return valid SeriesRef that represents ls. + // NOTE(bwplotka): Given OTLP and native histograms and the relaxation of the requirement for + // type and unit suffixes in metric names we start to hit cases of ls being not enough for id + // of the series (metadata matters). Current solution is to enable 'type-and-unit-label' features for those cases, but we may + // start to extend the id with metadata one day. + Append(ref SeriesRef, ls labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts AppendV2Options) (SeriesRef, error) +} + +// AppenderTransaction allows transactional appends. +type AppenderTransaction interface { + // Commit submits the collected samples and purges the batch. If Commit + // returns a non-nil error, it also rolls back all modifications made in + // the appender so far, as Rollback would do. In any case, an Appender + // must not be used anymore after Commit has been called. + Commit() error + + // Rollback rolls back all modifications made in the appender so far. + // Appender has to be discarded after rollback. + Rollback() error +} + +// LimitedAppenderV1 is an Appender that only supports appending float and histogram samples. +// This is to support migration to AppenderV2. +// TODO(bwplotka): Remove once migration to AppenderV2 is fully complete. +type LimitedAppenderV1 interface { + Append(ref SeriesRef, l labels.Labels, t int64, v float64) (SeriesRef, error) + AppendHistogram(ref SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) +} + +// AppenderV2AsLimitedV1 returns appender that exposes AppenderV2 as LimitedAppenderV1 +// TODO(bwplotka): Remove once migration to AppenderV2 is fully complete. +func AppenderV2AsLimitedV1(app AppenderV2) LimitedAppenderV1 { + return &limitedAppenderV1{AppenderV2: app} +} + +type limitedAppenderV1 struct { + AppenderV2 +} + +func (a *limitedAppenderV1) Append(ref SeriesRef, l labels.Labels, t int64, v float64) (SeriesRef, error) { + return a.AppenderV2.Append(ref, l, 0, t, v, nil, nil, AppendV2Options{}) +} + +func (a *limitedAppenderV1) AppendHistogram(ref SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (SeriesRef, error) { + return a.AppenderV2.Append(ref, l, 0, t, 0, h, fh, AppendV2Options{}) +} From 129650df9d910e6288f37754140f842046a92d60 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 28 Nov 2025 12:41:41 +0000 Subject: [PATCH 115/439] refactor(appenderV2): 1:1 copy of head_append.go -> head_append_v2.go (starting point) Signed-off-by: bwplotka --- storage/interface_append.go | 13 + tsdb/head_append_v2.go | 2285 +++++++++++++++++++++++++++++++++++ 2 files changed, 2298 insertions(+) create mode 100644 tsdb/head_append_v2.go diff --git a/storage/interface_append.go b/storage/interface_append.go index c8d1b46ce8..880e57f194 100644 --- a/storage/interface_append.go +++ b/storage/interface_append.go @@ -1,3 +1,16 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + package storage import ( diff --git a/tsdb/head_append_v2.go b/tsdb/head_append_v2.go new file mode 100644 index 0000000000..942c3ce974 --- /dev/null +++ b/tsdb/head_append_v2.go @@ -0,0 +1,2285 @@ +// Copyright 2021 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package tsdb + +import ( + "context" + "errors" + "fmt" + "log/slog" + "math" + + "github.com/prometheus/prometheus/model/exemplar" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/model/metadata" + "github.com/prometheus/prometheus/model/value" + "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/tsdb/chunkenc" + "github.com/prometheus/prometheus/tsdb/chunks" + "github.com/prometheus/prometheus/tsdb/record" +) + +// initAppender is a helper to initialize the time bounds of the head +// upon the first sample it receives. +type initAppender struct { + app storage.Appender + head *Head +} + +var _ storage.GetRef = &initAppender{} + +func (a *initAppender) SetOptions(opts *storage.AppendOptions) { + if a.app != nil { + a.app.SetOptions(opts) + } +} + +func (a *initAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64, v float64) (storage.SeriesRef, error) { + if a.app != nil { + return a.app.Append(ref, lset, t, v) + } + + a.head.initTime(t) + a.app = a.head.appender() + return a.app.Append(ref, lset, t, v) +} + +func (a *initAppender) AppendExemplar(ref storage.SeriesRef, l labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { + // Check if exemplar storage is enabled. + if !a.head.opts.EnableExemplarStorage || a.head.opts.MaxExemplars.Load() <= 0 { + return 0, nil + } + + if a.app != nil { + return a.app.AppendExemplar(ref, l, e) + } + // We should never reach here given we would call Append before AppendExemplar + // and we probably want to always base head/WAL min time on sample times. + a.head.initTime(e.Ts) + a.app = a.head.appender() + + return a.app.AppendExemplar(ref, l, e) +} + +func (a *initAppender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + if a.app != nil { + return a.app.AppendHistogram(ref, l, t, h, fh) + } + a.head.initTime(t) + a.app = a.head.appender() + + return a.app.AppendHistogram(ref, l, t, h, fh) +} + +func (a *initAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + if a.app != nil { + return a.app.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) + } + a.head.initTime(t) + a.app = a.head.appender() + + return a.app.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) +} + +func (a *initAppender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) { + if a.app != nil { + return a.app.UpdateMetadata(ref, l, m) + } + + a.app = a.head.appender() + return a.app.UpdateMetadata(ref, l, m) +} + +func (a *initAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64) (storage.SeriesRef, error) { + if a.app != nil { + return a.app.AppendSTZeroSample(ref, lset, t, st) + } + + a.head.initTime(t) + a.app = a.head.appender() + + return a.app.AppendSTZeroSample(ref, lset, t, st) +} + +// initTime initializes a head with the first timestamp. This only needs to be called +// for a completely fresh head with an empty WAL. +func (h *Head) initTime(t int64) { + if !h.minTime.CompareAndSwap(math.MaxInt64, t) { + return + } + // Ensure that max time is initialized to at least the min time we just set. + // Concurrent appenders may already have set it to a higher value. + h.maxTime.CompareAndSwap(math.MinInt64, t) +} + +func (a *initAppender) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { + if g, ok := a.app.(storage.GetRef); ok { + return g.GetRef(lset, hash) + } + return 0, labels.EmptyLabels() +} + +func (a *initAppender) Commit() error { + if a.app == nil { + a.head.metrics.activeAppenders.Dec() + return nil + } + return a.app.Commit() +} + +func (a *initAppender) Rollback() error { + if a.app == nil { + a.head.metrics.activeAppenders.Dec() + return nil + } + return a.app.Rollback() +} + +// Appender returns a new Appender on the database. +func (h *Head) Appender(context.Context) storage.Appender { + h.metrics.activeAppenders.Inc() + + // The head cache might not have a starting point yet. The init appender + // picks up the first appended timestamp as the base. + if !h.initialized() { + return &initAppender{ + head: h, + } + } + return h.appender() +} + +func (h *Head) appender() *headAppender { + minValidTime := h.appendableMinValidTime() + appendID, cleanupAppendIDsBelow := h.iso.newAppendID(minValidTime) // Every appender gets an ID that is cleared upon commit/rollback. + return &headAppender{ + head: h, + minValidTime: minValidTime, + mint: math.MaxInt64, + maxt: math.MinInt64, + headMaxt: h.MaxTime(), + oooTimeWindow: h.opts.OutOfOrderTimeWindow.Load(), + seriesRefs: h.getRefSeriesBuffer(), + series: h.getSeriesBuffer(), + typesInBatch: h.getTypeMap(), + appendID: appendID, + cleanupAppendIDsBelow: cleanupAppendIDsBelow, + } +} + +// appendableMinValidTime returns the minimum valid timestamp for appends, +// such that samples stay ahead of prior blocks and the head compaction window. +func (h *Head) appendableMinValidTime() int64 { + // This boundary ensures that no samples will be added to the compaction window. + // This allows race-free, concurrent appending and compaction. + cwEnd := h.MaxTime() - h.chunkRange.Load()/2 + + // This boundary ensures that we avoid overlapping timeframes from one block to the next. + // While not necessary for correctness, it means we're not required to use vertical compaction. + minValid := h.minValidTime.Load() + + return max(cwEnd, minValid) +} + +// AppendableMinValidTime returns the minimum valid time for samples to be appended to the Head. +// Returns false if Head hasn't been initialized yet and the minimum time isn't known yet. +func (h *Head) AppendableMinValidTime() (int64, bool) { + if !h.initialized() { + return 0, false + } + + return h.appendableMinValidTime(), true +} + +func (h *Head) getRefSeriesBuffer() []record.RefSeries { + b := h.refSeriesPool.Get() + if b == nil { + return make([]record.RefSeries, 0, 512) + } + return b +} + +func (h *Head) putRefSeriesBuffer(b []record.RefSeries) { + h.refSeriesPool.Put(b[:0]) +} + +func (h *Head) getFloatBuffer() []record.RefSample { + b := h.floatsPool.Get() + if b == nil { + return make([]record.RefSample, 0, 512) + } + return b +} + +func (h *Head) putFloatBuffer(b []record.RefSample) { + h.floatsPool.Put(b[:0]) +} + +func (h *Head) getExemplarBuffer() []exemplarWithSeriesRef { + b := h.exemplarsPool.Get() + if b == nil { + return make([]exemplarWithSeriesRef, 0, 512) + } + return b +} + +func (h *Head) putExemplarBuffer(b []exemplarWithSeriesRef) { + if b == nil { + return + } + for i := range b { // Zero out to avoid retaining label data. + b[i].exemplar.Labels = labels.EmptyLabels() + } + + h.exemplarsPool.Put(b[:0]) +} + +func (h *Head) getHistogramBuffer() []record.RefHistogramSample { + b := h.histogramsPool.Get() + if b == nil { + return make([]record.RefHistogramSample, 0, 512) + } + return b +} + +func (h *Head) putHistogramBuffer(b []record.RefHistogramSample) { + h.histogramsPool.Put(b[:0]) +} + +func (h *Head) getFloatHistogramBuffer() []record.RefFloatHistogramSample { + b := h.floatHistogramsPool.Get() + if b == nil { + return make([]record.RefFloatHistogramSample, 0, 512) + } + return b +} + +func (h *Head) putFloatHistogramBuffer(b []record.RefFloatHistogramSample) { + h.floatHistogramsPool.Put(b[:0]) +} + +func (h *Head) getMetadataBuffer() []record.RefMetadata { + b := h.metadataPool.Get() + if b == nil { + return make([]record.RefMetadata, 0, 512) + } + return b +} + +func (h *Head) putMetadataBuffer(b []record.RefMetadata) { + h.metadataPool.Put(b[:0]) +} + +func (h *Head) getSeriesBuffer() []*memSeries { + b := h.seriesPool.Get() + if b == nil { + return make([]*memSeries, 0, 512) + } + return b +} + +func (h *Head) putSeriesBuffer(b []*memSeries) { + for i := range b { // Zero out to avoid retaining data. + b[i] = nil + } + h.seriesPool.Put(b[:0]) +} + +func (h *Head) getTypeMap() map[chunks.HeadSeriesRef]sampleType { + b := h.typeMapPool.Get() + if b == nil { + return make(map[chunks.HeadSeriesRef]sampleType) + } + return b +} + +func (h *Head) putTypeMap(b map[chunks.HeadSeriesRef]sampleType) { + clear(b) + h.typeMapPool.Put(b) +} + +func (h *Head) getBytesBuffer() []byte { + b := h.bytesPool.Get() + if b == nil { + return make([]byte, 0, 1024) + } + return b +} + +func (h *Head) putBytesBuffer(b []byte) { + h.bytesPool.Put(b[:0]) +} + +type exemplarWithSeriesRef struct { + ref storage.SeriesRef + exemplar exemplar.Exemplar +} + +// sampleType describes sample types we need to distinguish for append batching. +// We need separate types for everything that goes into a different WAL record +// type or into a different chunk encoding. +type sampleType byte + +const ( + stNone sampleType = iota // To mark that the sample type does not matter. + stFloat // All simple floats (counters, gauges, untyped). Goes to `floats`. + stHistogram // Native integer histograms with a standard exponential schema. Goes to `histograms`. + stCustomBucketHistogram // Native integer histograms with custom bucket boundaries. Goes to `histograms`. + stFloatHistogram // Native float histograms. Goes to `floatHistograms`. + stCustomBucketFloatHistogram // Native float histograms with custom bucket boundaries. Goes to `floatHistograms`. +) + +// appendBatch is used to partition all the appended data into batches that are +// "type clean", i.e. every series receives only samples of one type within the +// batch. Types in this regard are defined by the sampleType enum above. +// TODO(beorn7): The same concept could be extended to make sure every series in +// the batch has at most one metadata record. This is currently not implemented +// because it is unclear if it is needed at all. (Maybe we will remove metadata +// records altogether, see issue #15911.) +type appendBatch struct { + floats []record.RefSample // New float samples held by this appender. + floatSeries []*memSeries // Float series corresponding to the samples held by this appender (using corresponding slice indices - same series may appear more than once). + histograms []record.RefHistogramSample // New histogram samples held by this appender. + histogramSeries []*memSeries // HistogramSamples series corresponding to the samples held by this appender (using corresponding slice indices - same series may appear more than once). + floatHistograms []record.RefFloatHistogramSample // New float histogram samples held by this appender. + floatHistogramSeries []*memSeries // FloatHistogramSamples series corresponding to the samples held by this appender (using corresponding slice indices - same series may appear more than once). + metadata []record.RefMetadata // New metadata held by this appender. + metadataSeries []*memSeries // Series corresponding to the metadata held by this appender. + exemplars []exemplarWithSeriesRef // New exemplars held by this appender. +} + +// close returns all the slices to the pools in Head and nil's them. +func (b *appendBatch) close(h *Head) { + h.putFloatBuffer(b.floats) + b.floats = nil + h.putSeriesBuffer(b.floatSeries) + b.floatSeries = nil + h.putHistogramBuffer(b.histograms) + b.histograms = nil + h.putSeriesBuffer(b.histogramSeries) + b.histogramSeries = nil + h.putFloatHistogramBuffer(b.floatHistograms) + b.floatHistograms = nil + h.putSeriesBuffer(b.floatHistogramSeries) + b.floatHistogramSeries = nil + h.putMetadataBuffer(b.metadata) + b.metadata = nil + h.putSeriesBuffer(b.metadataSeries) + b.metadataSeries = nil + h.putExemplarBuffer(b.exemplars) + b.exemplars = nil +} + +type headAppender struct { + head *Head + minValidTime int64 // No samples below this timestamp are allowed. + mint, maxt int64 + headMaxt int64 // We track it here to not take the lock for every sample appended. + oooTimeWindow int64 // Use the same for the entire append, and don't load the atomic for each sample. + + seriesRefs []record.RefSeries // New series records held by this appender. + series []*memSeries // New series held by this appender (using corresponding slices indexes from seriesRefs) + batches []*appendBatch // Holds all the other data to append. (In regular cases, there should be only one of these.) + + typesInBatch map[chunks.HeadSeriesRef]sampleType // Which (one) sample type each series holds in the most recent batch. + + appendID, cleanupAppendIDsBelow uint64 + closed bool + hints *storage.AppendOptions +} + +func (a *headAppender) SetOptions(opts *storage.AppendOptions) { + a.hints = opts +} + +func (a *headAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64, v float64) (storage.SeriesRef, error) { + // Fail fast if OOO is disabled and the sample is out of bounds. + // Otherwise a full check will be done later to decide if the sample is in-order or out-of-order. + if a.oooTimeWindow == 0 && t < a.minValidTime { + a.head.metrics.outOfBoundSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + return 0, storage.ErrOutOfBounds + } + + s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) + if s == nil { + var err error + s, _, err = a.getOrCreate(lset) + if err != nil { + return 0, err + } + } + + if value.IsStaleNaN(v) { + // If we have added a sample before with this same appender, we + // can check the previously used type and turn a stale float + // sample into a stale histogram sample or stale float histogram + // sample as appropriate. This prevents an unnecessary creation + // of a new batch. However, since other appenders might append + // to the same series concurrently, this is not perfect but just + // an optimization for the more likely case. + switch a.typesInBatch[s.ref] { + case stHistogram, stCustomBucketHistogram: + return a.AppendHistogram(ref, lset, t, &histogram.Histogram{Sum: v}, nil) + case stFloatHistogram, stCustomBucketFloatHistogram: + return a.AppendHistogram(ref, lset, t, nil, &histogram.FloatHistogram{Sum: v}) + } + // Note that a series reference not yet in the map will come out + // as stNone, but since we do not handle that case separately, + // we do not need to check for the difference between "unknown + // series" and "known series with stNone". + } + + s.Lock() + defer s.Unlock() + // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise + // to skip that sample from the WAL and write only in the WBL. + isOOO, delta, err := s.appendable(t, v, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err == nil { + if isOOO && a.hints != nil && a.hints.DiscardOutOfOrder { + a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + return 0, storage.ErrOutOfOrderSample + } + s.pendingCommit = true + } + if delta > 0 { + a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) + } + if err != nil { + switch { + case errors.Is(err, storage.ErrOutOfOrderSample): + a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + case errors.Is(err, storage.ErrTooOldSample): + a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + } + return 0, err + } + + if t < a.mint { + a.mint = t + } + if t > a.maxt { + a.maxt = t + } + + b := a.getCurrentBatch(stFloat, s.ref) + b.floats = append(b.floats, record.RefSample{ + Ref: s.ref, + T: t, + V: v, + }) + b.floatSeries = append(b.floatSeries, s) + return storage.SeriesRef(s.ref), nil +} + +// AppendSTZeroSample appends synthetic zero sample for st timestamp. It returns +// error when sample can't be appended. See +// storage.StartTimestampAppender.AppendSTZeroSample for further documentation. +func (a *headAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64) (storage.SeriesRef, error) { + if st >= t { + return 0, storage.ErrSTNewerThanSample + } + + s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) + if s == nil { + var err error + s, _, err = a.getOrCreate(lset) + if err != nil { + return 0, err + } + } + + // Check if ST wouldn't be OOO vs samples we already might have for this series. + // NOTE(bwplotka): This will be often hit as it's expected for long living + // counters to share the same ST. + s.Lock() + isOOO, _, err := s.appendable(st, 0, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err == nil { + s.pendingCommit = true + } + s.Unlock() + if err != nil { + return 0, err + } + if isOOO { + return storage.SeriesRef(s.ref), storage.ErrOutOfOrderST + } + + if st > a.maxt { + a.maxt = st + } + b := a.getCurrentBatch(stFloat, s.ref) + b.floats = append(b.floats, record.RefSample{Ref: s.ref, T: st, V: 0.0}) + b.floatSeries = append(b.floatSeries, s) + return storage.SeriesRef(s.ref), nil +} + +func (a *headAppender) getOrCreate(lset labels.Labels) (s *memSeries, created bool, err error) { + // Ensure no empty labels have gotten through. + lset = lset.WithoutEmpty() + if lset.IsEmpty() { + return nil, false, fmt.Errorf("empty labelset: %w", ErrInvalidSample) + } + if l, dup := lset.HasDuplicateLabelNames(); dup { + return nil, false, fmt.Errorf(`label name "%s" is not unique: %w`, l, ErrInvalidSample) + } + s, created, err = a.head.getOrCreate(lset.Hash(), lset, true) + if err != nil { + return nil, false, err + } + if created { + a.seriesRefs = append(a.seriesRefs, record.RefSeries{ + Ref: s.ref, + Labels: lset, + }) + a.series = append(a.series, s) + } + return s, created, nil +} + +// getCurrentBatch returns the current batch if it fits the provided sampleType +// for the provided series. Otherwise, it adds a new batch and returns it. +func (a *headAppender) getCurrentBatch(st sampleType, s chunks.HeadSeriesRef) *appendBatch { + h := a.head + + newBatch := func() *appendBatch { + b := appendBatch{ + floats: h.getFloatBuffer(), + floatSeries: h.getSeriesBuffer(), + histograms: h.getHistogramBuffer(), + histogramSeries: h.getSeriesBuffer(), + floatHistograms: h.getFloatHistogramBuffer(), + floatHistogramSeries: h.getSeriesBuffer(), + metadata: h.getMetadataBuffer(), + metadataSeries: h.getSeriesBuffer(), + } + + // Allocate the exemplars buffer only if exemplars are enabled. + if h.opts.EnableExemplarStorage { + b.exemplars = h.getExemplarBuffer() + } + clear(a.typesInBatch) + switch st { + case stHistogram, stFloatHistogram, stCustomBucketHistogram, stCustomBucketFloatHistogram: + // We only record histogram sample types in the map. + // Floats are implicit. + a.typesInBatch[s] = st + } + a.batches = append(a.batches, &b) + return &b + } + + // First batch ever. Create it. + if len(a.batches) == 0 { + return newBatch() + } + + // TODO(beorn7): If we ever see that the a.typesInBatch map grows so + // large that it matters for total memory consumption, we could limit + // the batch size here, i.e. cut a new batch even without a type change. + // Something like: + // if len(a.typesInBatch > limit) { + // return newBatch() + // } + + lastBatch := a.batches[len(a.batches)-1] + if st == stNone { + // Type doesn't matter, last batch will always do. + return lastBatch + } + prevST, ok := a.typesInBatch[s] + switch { + case prevST == st: + // An old series of some histogram type with the same type being appended. + // Continue the batch. + return lastBatch + case !ok && st == stFloat: + // A new float series, or an old float series that gets floats appended. + // Note that we do not track stFloat in typesInBatch. + // Continue the batch. + return lastBatch + case st == stFloat: + // A float being appended to a histogram series. + // Start a new batch. + return newBatch() + case !ok: + // A new series of some histogram type, or some histogram type + // being appended to on old float series. Even in the latter + // case, we don't need to start a new batch because histograms + // after floats are fine. + // Add new sample type to the map and continue batch. + a.typesInBatch[s] = st + return lastBatch + default: + // One histogram type changed to another. + // Start a new batch. + return newBatch() + } +} + +// appendable checks whether the given sample is valid for appending to the series. +// If the sample is valid and in-order, it returns false with no error. +// If the sample belongs to the out-of-order chunk, it returns true with no error. +// If the sample cannot be handled, it returns an error. +func (s *memSeries) appendable(t int64, v float64, headMaxt, minValidTime, oooTimeWindow int64) (isOOO bool, oooDelta int64, err error) { + // Check if we can append in the in-order chunk. + if t >= minValidTime { + if s.headChunks == nil { + // The series has no sample and was freshly created. + return false, 0, nil + } + msMaxt := s.maxTime() + if t > msMaxt { + return false, 0, nil + } + if t == msMaxt { + // We are allowing exact duplicates as we can encounter them in valid cases + // like federation and erroring out at that time would be extremely noisy. + // This only checks against the latest in-order sample. + // The OOO headchunk has its own method to detect these duplicates. + if s.lastHistogramValue != nil || s.lastFloatHistogramValue != nil { + return false, 0, storage.NewDuplicateHistogramToFloatErr(t, v) + } + if math.Float64bits(s.lastValue) != math.Float64bits(v) { + return false, 0, storage.NewDuplicateFloatErr(t, s.lastValue, v) + } + // Sample is identical (ts + value) with most current (highest ts) sample in sampleBuf. + return false, 0, nil + } + } + + // The sample cannot go in the in-order chunk. Check if it can go in the out-of-order chunk. + if oooTimeWindow > 0 && t >= headMaxt-oooTimeWindow { + return true, headMaxt - t, nil + } + + // The sample cannot go in both in-order and out-of-order chunk. + if oooTimeWindow > 0 { + return true, headMaxt - t, storage.ErrTooOldSample + } + if t < minValidTime { + return false, headMaxt - t, storage.ErrOutOfBounds + } + return false, headMaxt - t, storage.ErrOutOfOrderSample +} + +// appendableHistogram checks whether the given histogram sample is valid for appending to the series. (if we return false and no error) +// The sample belongs to the out of order chunk if we return true and no error. +// An error signifies the sample cannot be handled. +func (s *memSeries) appendableHistogram(t int64, h *histogram.Histogram, headMaxt, minValidTime, oooTimeWindow int64) (isOOO bool, oooDelta int64, err error) { + // Check if we can append in the in-order chunk. + if t >= minValidTime { + if s.headChunks == nil { + // The series has no sample and was freshly created. + return false, 0, nil + } + msMaxt := s.maxTime() + if t > msMaxt { + return false, 0, nil + } + if t == msMaxt { + // We are allowing exact duplicates as we can encounter them in valid cases + // like federation and erroring out at that time would be extremely noisy. + // This only checks against the latest in-order sample. + // The OOO headchunk has its own method to detect these duplicates. + if !h.Equals(s.lastHistogramValue) { + return false, 0, storage.ErrDuplicateSampleForTimestamp + } + // Sample is identical (ts + value) with most current (highest ts) sample in sampleBuf. + return false, 0, nil + } + } + + // The sample cannot go in the in-order chunk. Check if it can go in the out-of-order chunk. + if oooTimeWindow > 0 && t >= headMaxt-oooTimeWindow { + return true, headMaxt - t, nil + } + + // The sample cannot go in both in-order and out-of-order chunk. + if oooTimeWindow > 0 { + return true, headMaxt - t, storage.ErrTooOldSample + } + if t < minValidTime { + return false, headMaxt - t, storage.ErrOutOfBounds + } + return false, headMaxt - t, storage.ErrOutOfOrderSample +} + +// appendableFloatHistogram checks whether the given float histogram sample is valid for appending to the series. (if we return false and no error) +// The sample belongs to the out of order chunk if we return true and no error. +// An error signifies the sample cannot be handled. +func (s *memSeries) appendableFloatHistogram(t int64, fh *histogram.FloatHistogram, headMaxt, minValidTime, oooTimeWindow int64) (isOOO bool, oooDelta int64, err error) { + // Check if we can append in the in-order chunk. + if t >= minValidTime { + if s.headChunks == nil { + // The series has no sample and was freshly created. + return false, 0, nil + } + msMaxt := s.maxTime() + if t > msMaxt { + return false, 0, nil + } + if t == msMaxt { + // We are allowing exact duplicates as we can encounter them in valid cases + // like federation and erroring out at that time would be extremely noisy. + // This only checks against the latest in-order sample. + // The OOO headchunk has its own method to detect these duplicates. + if !fh.Equals(s.lastFloatHistogramValue) { + return false, 0, storage.ErrDuplicateSampleForTimestamp + } + // Sample is identical (ts + value) with most current (highest ts) sample in sampleBuf. + return false, 0, nil + } + } + + // The sample cannot go in the in-order chunk. Check if it can go in the out-of-order chunk. + if oooTimeWindow > 0 && t >= headMaxt-oooTimeWindow { + return true, headMaxt - t, nil + } + + // The sample cannot go in both in-order and out-of-order chunk. + if oooTimeWindow > 0 { + return true, headMaxt - t, storage.ErrTooOldSample + } + if t < minValidTime { + return false, headMaxt - t, storage.ErrOutOfBounds + } + return false, headMaxt - t, storage.ErrOutOfOrderSample +} + +// AppendExemplar for headAppender assumes the series ref already exists, and so it doesn't +// use getOrCreate or make any of the lset validity checks that Append does. +func (a *headAppender) AppendExemplar(ref storage.SeriesRef, lset labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { + // Check if exemplar storage is enabled. + if !a.head.opts.EnableExemplarStorage || a.head.opts.MaxExemplars.Load() <= 0 { + return 0, nil + } + + // Get Series + s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) + if s == nil { + s = a.head.series.getByHash(lset.Hash(), lset) + if s != nil { + ref = storage.SeriesRef(s.ref) + } + } + if s == nil { + return 0, fmt.Errorf("unknown HeadSeriesRef when trying to add exemplar: %d", ref) + } + + // Ensure no empty labels have gotten through. + e.Labels = e.Labels.WithoutEmpty() + + err := a.head.exemplars.ValidateExemplar(s.labels(), e) + if err != nil { + if errors.Is(err, storage.ErrDuplicateExemplar) || errors.Is(err, storage.ErrExemplarsDisabled) { + // Duplicate, don't return an error but don't accept the exemplar. + return 0, nil + } + return 0, err + } + + b := a.getCurrentBatch(stNone, chunks.HeadSeriesRef(ref)) + b.exemplars = append(b.exemplars, exemplarWithSeriesRef{ref, e}) + + return storage.SeriesRef(s.ref), nil +} + +func (a *headAppender) AppendHistogram(ref storage.SeriesRef, lset labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + // Fail fast if OOO is disabled and the sample is out of bounds. + // Otherwise a full check will be done later to decide if the sample is in-order or out-of-order. + if a.oooTimeWindow == 0 && t < a.minValidTime { + a.head.metrics.outOfBoundSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + return 0, storage.ErrOutOfBounds + } + + if h != nil { + if err := h.Validate(); err != nil { + return 0, err + } + } + + if fh != nil { + if err := fh.Validate(); err != nil { + return 0, err + } + } + + s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) + if s == nil { + var err error + s, _, err = a.getOrCreate(lset) + if err != nil { + return 0, err + } + } + + switch { + case h != nil: + s.Lock() + // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise + // to skip that sample from the WAL and write only in the WBL. + _, delta, err := s.appendableHistogram(t, h, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err != nil { + s.pendingCommit = true + } + s.Unlock() + if delta > 0 { + a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) + } + if err != nil { + switch { + case errors.Is(err, storage.ErrOutOfOrderSample): + a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + case errors.Is(err, storage.ErrTooOldSample): + a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + } + return 0, err + } + st := stHistogram + if h.UsesCustomBuckets() { + st = stCustomBucketHistogram + } + b := a.getCurrentBatch(st, s.ref) + b.histograms = append(b.histograms, record.RefHistogramSample{ + Ref: s.ref, + T: t, + H: h, + }) + b.histogramSeries = append(b.histogramSeries, s) + case fh != nil: + s.Lock() + // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise + // to skip that sample from the WAL and write only in the WBL. + _, delta, err := s.appendableFloatHistogram(t, fh, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err == nil { + s.pendingCommit = true + } + s.Unlock() + if delta > 0 { + a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) + } + if err != nil { + switch { + case errors.Is(err, storage.ErrOutOfOrderSample): + a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + case errors.Is(err, storage.ErrTooOldSample): + a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + } + return 0, err + } + st := stFloatHistogram + if fh.UsesCustomBuckets() { + st = stCustomBucketFloatHistogram + } + b := a.getCurrentBatch(st, s.ref) + b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ + Ref: s.ref, + T: t, + FH: fh, + }) + b.floatHistogramSeries = append(b.floatHistogramSeries, s) + } + + if t < a.mint { + a.mint = t + } + if t > a.maxt { + a.maxt = t + } + + return storage.SeriesRef(s.ref), nil +} + +func (a *headAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + if st >= t { + return 0, storage.ErrSTNewerThanSample + } + + s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) + if s == nil { + var err error + s, _, err = a.getOrCreate(lset) + if err != nil { + return 0, err + } + } + + switch { + case h != nil: + zeroHistogram := &histogram.Histogram{ + // The STZeroSample represents a counter reset by definition. + CounterResetHint: histogram.CounterReset, + // Replicate other fields to avoid needless chunk creation. + Schema: h.Schema, + ZeroThreshold: h.ZeroThreshold, + CustomValues: h.CustomValues, + } + s.Lock() + // For STZeroSamples OOO is not allowed. + // We set it to true to make this implementation as close as possible to the float implementation. + isOOO, _, err := s.appendableHistogram(st, zeroHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err != nil { + s.Unlock() + if errors.Is(err, storage.ErrOutOfOrderSample) { + return 0, storage.ErrOutOfOrderST + } + + return 0, err + } + + // OOO is not allowed because after the first scrape, ST will be the same for most (if not all) future samples. + // This is to prevent the injected zero from being marked as OOO forever. + if isOOO { + s.Unlock() + return 0, storage.ErrOutOfOrderST + } + + s.pendingCommit = true + s.Unlock() + sTyp := stHistogram + if h.UsesCustomBuckets() { + sTyp = stCustomBucketHistogram + } + b := a.getCurrentBatch(sTyp, s.ref) + b.histograms = append(b.histograms, record.RefHistogramSample{ + Ref: s.ref, + T: st, + H: zeroHistogram, + }) + b.histogramSeries = append(b.histogramSeries, s) + case fh != nil: + zeroFloatHistogram := &histogram.FloatHistogram{ + // The STZeroSample represents a counter reset by definition. + CounterResetHint: histogram.CounterReset, + // Replicate other fields to avoid needless chunk creation. + Schema: fh.Schema, + ZeroThreshold: fh.ZeroThreshold, + CustomValues: fh.CustomValues, + } + s.Lock() + // We set it to true to make this implementation as close as possible to the float implementation. + isOOO, _, err := s.appendableFloatHistogram(st, zeroFloatHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) // OOO is not allowed for STZeroSamples. + if err != nil { + s.Unlock() + if errors.Is(err, storage.ErrOutOfOrderSample) { + return 0, storage.ErrOutOfOrderST + } + + return 0, err + } + + // OOO is not allowed because after the first scrape, ST will be the same for most (if not all) future samples. + // This is to prevent the injected zero from being marked as OOO forever. + if isOOO { + s.Unlock() + return 0, storage.ErrOutOfOrderST + } + + s.pendingCommit = true + s.Unlock() + sTyp := stFloatHistogram + if fh.UsesCustomBuckets() { + sTyp = stCustomBucketFloatHistogram + } + b := a.getCurrentBatch(sTyp, s.ref) + b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ + Ref: s.ref, + T: st, + FH: zeroFloatHistogram, + }) + b.floatHistogramSeries = append(b.floatHistogramSeries, s) + } + + if st > a.maxt { + a.maxt = st + } + + return storage.SeriesRef(s.ref), nil +} + +// UpdateMetadata for headAppender assumes the series ref already exists, and so it doesn't +// use getOrCreate or make any of the lset sanity checks that Append does. +func (a *headAppender) UpdateMetadata(ref storage.SeriesRef, lset labels.Labels, meta metadata.Metadata) (storage.SeriesRef, error) { + s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) + if s == nil { + s = a.head.series.getByHash(lset.Hash(), lset) + if s != nil { + ref = storage.SeriesRef(s.ref) + } + } + if s == nil { + return 0, fmt.Errorf("unknown series when trying to add metadata with HeadSeriesRef: %d and labels: %s", ref, lset) + } + + s.Lock() + hasNewMetadata := s.meta == nil || *s.meta != meta + s.Unlock() + + if hasNewMetadata { + b := a.getCurrentBatch(stNone, s.ref) + b.metadata = append(b.metadata, record.RefMetadata{ + Ref: s.ref, + Type: record.GetMetricType(meta.Type), + Unit: meta.Unit, + Help: meta.Help, + }) + b.metadataSeries = append(b.metadataSeries, s) + } + + return ref, nil +} + +var _ storage.GetRef = &headAppender{} + +func (a *headAppender) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { + s := a.head.series.getByHash(hash, lset) + if s == nil { + return 0, labels.EmptyLabels() + } + // returned labels must be suitable to pass to Append() + return storage.SeriesRef(s.ref), s.labels() +} + +// log writes all headAppender's data to the WAL. +func (a *headAppender) log() error { + if a.head.wal == nil { + return nil + } + + buf := a.head.getBytesBuffer() + defer func() { a.head.putBytesBuffer(buf) }() + + var rec []byte + var enc record.Encoder + + if len(a.seriesRefs) > 0 { + rec = enc.Series(a.seriesRefs, buf) + buf = rec[:0] + + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log series: %w", err) + } + } + for _, b := range a.batches { + if len(b.metadata) > 0 { + rec = enc.Metadata(b.metadata, buf) + buf = rec[:0] + + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log metadata: %w", err) + } + } + // It's important to do (float) Samples before histogram samples + // to end up with the correct order. + if len(b.floats) > 0 { + rec = enc.Samples(b.floats, buf) + buf = rec[:0] + + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log samples: %w", err) + } + } + if len(b.histograms) > 0 { + var customBucketsHistograms []record.RefHistogramSample + rec, customBucketsHistograms = enc.HistogramSamples(b.histograms, buf) + buf = rec[:0] + if len(rec) > 0 { + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log histograms: %w", err) + } + } + + if len(customBucketsHistograms) > 0 { + rec = enc.CustomBucketsHistogramSamples(customBucketsHistograms, buf) + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log custom buckets histograms: %w", err) + } + } + } + if len(b.floatHistograms) > 0 { + var customBucketsFloatHistograms []record.RefFloatHistogramSample + rec, customBucketsFloatHistograms = enc.FloatHistogramSamples(b.floatHistograms, buf) + buf = rec[:0] + if len(rec) > 0 { + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log float histograms: %w", err) + } + } + + if len(customBucketsFloatHistograms) > 0 { + rec = enc.CustomBucketsFloatHistogramSamples(customBucketsFloatHistograms, buf) + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log custom buckets float histograms: %w", err) + } + } + } + // Exemplars should be logged after samples (float/native histogram/etc), + // otherwise it might happen that we send the exemplars in a remote write + // batch before the samples, which in turn means the exemplar is rejected + // for missing series, since series are created due to samples. + if len(b.exemplars) > 0 { + rec = enc.Exemplars(exemplarsForEncoding(b.exemplars), buf) + buf = rec[:0] + + if err := a.head.wal.Log(rec); err != nil { + return fmt.Errorf("log exemplars: %w", err) + } + } + } + return nil +} + +func exemplarsForEncoding(es []exemplarWithSeriesRef) []record.RefExemplar { + ret := make([]record.RefExemplar, 0, len(es)) + for _, e := range es { + ret = append(ret, record.RefExemplar{ + Ref: chunks.HeadSeriesRef(e.ref), + T: e.exemplar.Ts, + V: e.exemplar.Value, + Labels: e.exemplar.Labels, + }) + } + return ret +} + +type appenderCommitContext struct { + floatsAppended int + histogramsAppended int + // Number of samples out of order but accepted: with ooo enabled and within time window. + oooFloatsAccepted int + oooHistogramAccepted int + // Number of samples rejected due to: out of order but OOO support disabled. + floatOOORejected int + histoOOORejected int + // Number of samples rejected due to: out of order but too old (OOO support enabled, but outside time window). + floatTooOldRejected int + histoTooOldRejected int + // Number of samples rejected due to: out of bounds: with t < minValidTime (OOO support disabled). + floatOOBRejected int + histoOOBRejected int + inOrderMint int64 + inOrderMaxt int64 + oooMinT int64 + oooMaxT int64 + wblSamples []record.RefSample + wblHistograms []record.RefHistogramSample + wblFloatHistograms []record.RefFloatHistogramSample + oooMmapMarkers map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef + oooMmapMarkersCount int + oooRecords [][]byte + oooCapMax int64 + appendChunkOpts chunkOpts + enc record.Encoder +} + +// commitExemplars adds all exemplars from the provided batch to the head's exemplar storage. +func (a *headAppender) commitExemplars(b *appendBatch) { + // No errors logging to WAL, so pass the exemplars along to the in memory storage. + for _, e := range b.exemplars { + s := a.head.series.getByID(chunks.HeadSeriesRef(e.ref)) + if s == nil { + // This is very unlikely to happen, but we have seen it in the wild. + // It means that the series was truncated between AppendExemplar and Commit. + // See TestHeadCompactionWhileAppendAndCommitExemplar. + continue + } + // We don't instrument exemplar appends here, all is instrumented by storage. + if err := a.head.exemplars.AddExemplar(s.labels(), e.exemplar); err != nil { + if errors.Is(err, storage.ErrOutOfOrderExemplar) { + continue + } + a.head.logger.Debug("Unknown error while adding exemplar", "err", err) + } + } +} + +func (acc *appenderCommitContext) collectOOORecords(a *headAppender) { + if a.head.wbl == nil { + // WBL is not enabled. So no need to collect. + acc.wblSamples = nil + acc.wblHistograms = nil + acc.wblFloatHistograms = nil + acc.oooMmapMarkers = nil + acc.oooMmapMarkersCount = 0 + return + } + + // The m-map happens before adding a new sample. So we collect + // the m-map markers first, and then samples. + // WBL Graphically: + // WBL Before this Commit(): [old samples before this commit for chunk 1] + // WBL After this Commit(): [old samples before this commit for chunk 1][new samples in this commit for chunk 1]mmapmarker1[samples for chunk 2]mmapmarker2[samples for chunk 3] + if acc.oooMmapMarkers != nil { + markers := make([]record.RefMmapMarker, 0, acc.oooMmapMarkersCount) + for ref, mmapRefs := range acc.oooMmapMarkers { + for _, mmapRef := range mmapRefs { + markers = append(markers, record.RefMmapMarker{ + Ref: ref, + MmapRef: mmapRef, + }) + } + } + r := acc.enc.MmapMarkers(markers, a.head.getBytesBuffer()) + acc.oooRecords = append(acc.oooRecords, r) + } + + if len(acc.wblSamples) > 0 { + r := acc.enc.Samples(acc.wblSamples, a.head.getBytesBuffer()) + acc.oooRecords = append(acc.oooRecords, r) + } + if len(acc.wblHistograms) > 0 { + r, customBucketsHistograms := acc.enc.HistogramSamples(acc.wblHistograms, a.head.getBytesBuffer()) + if len(r) > 0 { + acc.oooRecords = append(acc.oooRecords, r) + } + if len(customBucketsHistograms) > 0 { + r := acc.enc.CustomBucketsHistogramSamples(customBucketsHistograms, a.head.getBytesBuffer()) + acc.oooRecords = append(acc.oooRecords, r) + } + } + if len(acc.wblFloatHistograms) > 0 { + r, customBucketsFloatHistograms := acc.enc.FloatHistogramSamples(acc.wblFloatHistograms, a.head.getBytesBuffer()) + if len(r) > 0 { + acc.oooRecords = append(acc.oooRecords, r) + } + if len(customBucketsFloatHistograms) > 0 { + r := acc.enc.CustomBucketsFloatHistogramSamples(customBucketsFloatHistograms, a.head.getBytesBuffer()) + acc.oooRecords = append(acc.oooRecords, r) + } + } + + acc.wblSamples = nil + acc.wblHistograms = nil + acc.wblFloatHistograms = nil + acc.oooMmapMarkers = nil +} + +// handleAppendableError processes errors encountered during sample appending and updates +// the provided counters accordingly. +// +// Parameters: +// - err: The error encountered during appending. +// - appended: Pointer to the counter tracking the number of successfully appended samples. +// - oooRejected: Pointer to the counter tracking the number of out-of-order samples rejected. +// - oobRejected: Pointer to the counter tracking the number of out-of-bounds samples rejected. +// - tooOldRejected: Pointer to the counter tracking the number of too-old samples rejected. +func handleAppendableError(err error, appended, oooRejected, oobRejected, tooOldRejected *int) { + switch { + case errors.Is(err, storage.ErrOutOfOrderSample): + *appended-- + *oooRejected++ + case errors.Is(err, storage.ErrOutOfBounds): + *appended-- + *oobRejected++ + case errors.Is(err, storage.ErrTooOldSample): + *appended-- + *tooOldRejected++ + default: + *appended-- + } +} + +// commitFloats processes and commits the samples in the provided batch to the +// series. It handles both in-order and out-of-order samples, updating the +// appenderCommitContext with the results of the append operations. +// +// The function iterates over the samples in the headAppender and attempts to append each sample +// to its corresponding series. It handles various error cases such as out-of-order samples, +// out-of-bounds samples, and too-old samples, updating the appenderCommitContext accordingly. +// +// For out-of-order samples, it checks if the sample can be inserted into the series and updates +// the out-of-order mmap markers if necessary. It also updates the write-ahead log (WBL) samples +// and the minimum and maximum timestamps for out-of-order samples. +// +// For in-order samples, it attempts to append the sample to the series and updates the minimum +// and maximum timestamps for in-order samples. +// +// The function also increments the chunk metrics if a new chunk is created and performs cleanup +// operations on the series after appending the samples. +// +// There are also specific functions to commit histograms and float histograms. +func (a *headAppender) commitFloats(b *appendBatch, acc *appenderCommitContext) { + var ok, chunkCreated bool + var series *memSeries + + for i, s := range b.floats { + series = b.floatSeries[i] + series.Lock() + + if value.IsStaleNaN(s.V) { + // If a float staleness marker had been appended for a + // series that got a histogram or float histogram + // appended before via this same appender, it would not + // show up here because we had already converted it. We + // end up here for two reasons: (1) This is the very + // first sample for this series appended via this + // appender. (2) A float sample was appended to this + // series before via this same appender. + // + // In either case, we need to check the previous sample + // in the memSeries to append the appropriately typed + // staleness marker. This is obviously so in case (1). + // In case (2), we would usually expect a float sample + // as the previous sample, but there might be concurrent + // appends that have added a histogram sample in the + // meantime. (This will probably lead to OOO shenanigans + // anyway, but that's a different story.) + // + // If the last sample in the memSeries is indeed a + // float, we don't have to do anything special here and + // just go on with the normal commit for a float sample. + // However, if the last sample in the memSeries is a + // histogram or float histogram, we have to convert the + // staleness marker to a histogram (or float histogram, + // respectively), and just add it at the end of the + // histograms (or float histograms) in the same batch, + // to be committed later in commitHistograms (or + // commitFloatHistograms). The latter is fine because we + // know there is no other histogram (or float histogram) + // sample for this same series in this same batch + // (because any such sample would have triggered a new + // batch). + switch { + case series.lastHistogramValue != nil: + b.histograms = append(b.histograms, record.RefHistogramSample{ + Ref: series.ref, + T: s.T, + H: &histogram.Histogram{Sum: s.V}, + }) + b.histogramSeries = append(b.histogramSeries, series) + // This sample was counted as a float but is now a histogram. + acc.floatsAppended-- + acc.histogramsAppended++ + series.Unlock() + continue + case series.lastFloatHistogramValue != nil: + b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ + Ref: series.ref, + T: s.T, + FH: &histogram.FloatHistogram{Sum: s.V}, + }) + b.floatHistogramSeries = append(b.floatHistogramSeries, series) + // This sample was counted as a float but is now a float histogram. + acc.floatsAppended-- + acc.histogramsAppended++ + series.Unlock() + continue + } + } + oooSample, _, err := series.appendable(s.T, s.V, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err != nil { + handleAppendableError(err, &acc.floatsAppended, &acc.floatOOORejected, &acc.floatOOBRejected, &acc.floatTooOldRejected) + } + + switch { + case err != nil: + // Do nothing here. + case oooSample: + // Sample is OOO and OOO handling is enabled + // and the delta is within the OOO tolerance. + var mmapRefs []chunks.ChunkDiskMapperRef + ok, chunkCreated, mmapRefs = series.insert(s.T, s.V, nil, nil, a.head.chunkDiskMapper, acc.oooCapMax, a.head.logger) + if chunkCreated { + r, ok := acc.oooMmapMarkers[series.ref] + if !ok || r != nil { + // !ok means there are no markers collected for these samples yet. So we first flush the samples + // before setting this m-map marker. + + // r != nil means we have already m-mapped a chunk for this series in the same Commit(). + // Hence, before we m-map again, we should add the samples and m-map markers + // seen till now to the WBL records. + acc.collectOOORecords(a) + } + + if acc.oooMmapMarkers == nil { + acc.oooMmapMarkers = make(map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef) + } + if len(mmapRefs) > 0 { + acc.oooMmapMarkers[series.ref] = mmapRefs + acc.oooMmapMarkersCount += len(mmapRefs) + } else { + // No chunk was written to disk, so we need to set an initial marker for this series. + acc.oooMmapMarkers[series.ref] = []chunks.ChunkDiskMapperRef{0} + acc.oooMmapMarkersCount++ + } + } + if ok { + acc.wblSamples = append(acc.wblSamples, s) + if s.T < acc.oooMinT { + acc.oooMinT = s.T + } + if s.T > acc.oooMaxT { + acc.oooMaxT = s.T + } + acc.oooFloatsAccepted++ + } else { + // Sample is an exact duplicate of the last sample. + // NOTE: We can only detect updates if they clash with a sample in the OOOHeadChunk, + // not with samples in already flushed OOO chunks. + // TODO(codesome): Add error reporting? It depends on addressing https://github.com/prometheus/prometheus/discussions/10305. + acc.floatsAppended-- + } + default: + newlyStale := !value.IsStaleNaN(series.lastValue) && value.IsStaleNaN(s.V) + staleToNonStale := value.IsStaleNaN(series.lastValue) && !value.IsStaleNaN(s.V) + ok, chunkCreated = series.append(s.T, s.V, a.appendID, acc.appendChunkOpts) + if ok { + if s.T < acc.inOrderMint { + acc.inOrderMint = s.T + } + if s.T > acc.inOrderMaxt { + acc.inOrderMaxt = s.T + } + if newlyStale { + a.head.numStaleSeries.Inc() + } + if staleToNonStale { + a.head.numStaleSeries.Dec() + } + } else { + // The sample is an exact duplicate, and should be silently dropped. + acc.floatsAppended-- + } + } + + if chunkCreated { + a.head.metrics.chunks.Inc() + a.head.metrics.chunksCreated.Inc() + } + + series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) + series.pendingCommit = false + series.Unlock() + } +} + +// For details on the commitHistograms function, see the commitFloats docs. +func (a *headAppender) commitHistograms(b *appendBatch, acc *appenderCommitContext) { + var ok, chunkCreated bool + var series *memSeries + + for i, s := range b.histograms { + series = b.histogramSeries[i] + series.Lock() + + // At this point, we could encounter a histogram staleness + // marker that should better be a float staleness marker or a + // float histogram staleness marker. This can only happen with + // concurrent appenders appending to the same series _and_ doing + // so in a mixed-type scenario. This case is expected to be very + // rare, so we do not bother here to convert the staleness + // marker. The worst case is that we need to cut a new chunk + // just for the staleness marker. + + oooSample, _, err := series.appendableHistogram(s.T, s.H, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err != nil { + handleAppendableError(err, &acc.histogramsAppended, &acc.histoOOORejected, &acc.histoOOBRejected, &acc.histoTooOldRejected) + } + + switch { + case err != nil: + // Do nothing here. + case oooSample: + // Sample is OOO and OOO handling is enabled + // and the delta is within the OOO tolerance. + var mmapRefs []chunks.ChunkDiskMapperRef + ok, chunkCreated, mmapRefs = series.insert(s.T, 0, s.H, nil, a.head.chunkDiskMapper, acc.oooCapMax, a.head.logger) + if chunkCreated { + r, ok := acc.oooMmapMarkers[series.ref] + if !ok || r != nil { + // !ok means there are no markers collected for these samples yet. So we first flush the samples + // before setting this m-map marker. + + // r != 0 means we have already m-mapped a chunk for this series in the same Commit(). + // Hence, before we m-map again, we should add the samples and m-map markers + // seen till now to the WBL records. + acc.collectOOORecords(a) + } + + if acc.oooMmapMarkers == nil { + acc.oooMmapMarkers = make(map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef) + } + if len(mmapRefs) > 0 { + acc.oooMmapMarkers[series.ref] = mmapRefs + acc.oooMmapMarkersCount += len(mmapRefs) + } else { + // No chunk was written to disk, so we need to set an initial marker for this series. + acc.oooMmapMarkers[series.ref] = []chunks.ChunkDiskMapperRef{0} + acc.oooMmapMarkersCount++ + } + } + if ok { + acc.wblHistograms = append(acc.wblHistograms, s) + if s.T < acc.oooMinT { + acc.oooMinT = s.T + } + if s.T > acc.oooMaxT { + acc.oooMaxT = s.T + } + acc.oooHistogramAccepted++ + } else { + // Sample is an exact duplicate of the last sample. + // NOTE: We can only detect updates if they clash with a sample in the OOOHeadChunk, + // not with samples in already flushed OOO chunks. + // TODO(codesome): Add error reporting? It depends on addressing https://github.com/prometheus/prometheus/discussions/10305. + acc.histogramsAppended-- + } + default: + newlyStale := value.IsStaleNaN(s.H.Sum) + staleToNonStale := false + if series.lastHistogramValue != nil { + newlyStale = newlyStale && !value.IsStaleNaN(series.lastHistogramValue.Sum) + staleToNonStale = value.IsStaleNaN(series.lastHistogramValue.Sum) && !value.IsStaleNaN(s.H.Sum) + } + ok, chunkCreated = series.appendHistogram(s.T, s.H, a.appendID, acc.appendChunkOpts) + if ok { + if s.T < acc.inOrderMint { + acc.inOrderMint = s.T + } + if s.T > acc.inOrderMaxt { + acc.inOrderMaxt = s.T + } + if newlyStale { + a.head.numStaleSeries.Inc() + } + if staleToNonStale { + a.head.numStaleSeries.Dec() + } + } else { + acc.histogramsAppended-- + acc.histoOOORejected++ + } + } + + if chunkCreated { + a.head.metrics.chunks.Inc() + a.head.metrics.chunksCreated.Inc() + } + + series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) + series.pendingCommit = false + series.Unlock() + } +} + +// For details on the commitFloatHistograms function, see the commitFloats docs. +func (a *headAppender) commitFloatHistograms(b *appendBatch, acc *appenderCommitContext) { + var ok, chunkCreated bool + var series *memSeries + + for i, s := range b.floatHistograms { + series = b.floatHistogramSeries[i] + series.Lock() + + // At this point, we could encounter a float histogram staleness + // marker that should better be a float staleness marker or an + // integer histogram staleness marker. This can only happen with + // concurrent appenders appending to the same series _and_ doing + // so in a mixed-type scenario. This case is expected to be very + // rare, so we do not bother here to convert the staleness + // marker. The worst case is that we need to cut a new chunk + // just for the staleness marker. + + oooSample, _, err := series.appendableFloatHistogram(s.T, s.FH, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if err != nil { + handleAppendableError(err, &acc.histogramsAppended, &acc.histoOOORejected, &acc.histoOOBRejected, &acc.histoTooOldRejected) + } + + switch { + case err != nil: + // Do nothing here. + case oooSample: + // Sample is OOO and OOO handling is enabled + // and the delta is within the OOO tolerance. + var mmapRefs []chunks.ChunkDiskMapperRef + ok, chunkCreated, mmapRefs = series.insert(s.T, 0, nil, s.FH, a.head.chunkDiskMapper, acc.oooCapMax, a.head.logger) + if chunkCreated { + r, ok := acc.oooMmapMarkers[series.ref] + if !ok || r != nil { + // !ok means there are no markers collected for these samples yet. So we first flush the samples + // before setting this m-map marker. + + // r != 0 means we have already m-mapped a chunk for this series in the same Commit(). + // Hence, before we m-map again, we should add the samples and m-map markers + // seen till now to the WBL records. + acc.collectOOORecords(a) + } + + if acc.oooMmapMarkers == nil { + acc.oooMmapMarkers = make(map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef) + } + if len(mmapRefs) > 0 { + acc.oooMmapMarkers[series.ref] = mmapRefs + acc.oooMmapMarkersCount += len(mmapRefs) + } else { + // No chunk was written to disk, so we need to set an initial marker for this series. + acc.oooMmapMarkers[series.ref] = []chunks.ChunkDiskMapperRef{0} + acc.oooMmapMarkersCount++ + } + } + if ok { + acc.wblFloatHistograms = append(acc.wblFloatHistograms, s) + if s.T < acc.oooMinT { + acc.oooMinT = s.T + } + if s.T > acc.oooMaxT { + acc.oooMaxT = s.T + } + acc.oooHistogramAccepted++ + } else { + // Sample is an exact duplicate of the last sample. + // NOTE: We can only detect updates if they clash with a sample in the OOOHeadChunk, + // not with samples in already flushed OOO chunks. + // TODO(codesome): Add error reporting? It depends on addressing https://github.com/prometheus/prometheus/discussions/10305. + acc.histogramsAppended-- + } + default: + newlyStale := value.IsStaleNaN(s.FH.Sum) + staleToNonStale := false + if series.lastFloatHistogramValue != nil { + newlyStale = newlyStale && !value.IsStaleNaN(series.lastFloatHistogramValue.Sum) + staleToNonStale = value.IsStaleNaN(series.lastFloatHistogramValue.Sum) && !value.IsStaleNaN(s.FH.Sum) + } + ok, chunkCreated = series.appendFloatHistogram(s.T, s.FH, a.appendID, acc.appendChunkOpts) + if ok { + if s.T < acc.inOrderMint { + acc.inOrderMint = s.T + } + if s.T > acc.inOrderMaxt { + acc.inOrderMaxt = s.T + } + if newlyStale { + a.head.numStaleSeries.Inc() + } + if staleToNonStale { + a.head.numStaleSeries.Dec() + } + } else { + acc.histogramsAppended-- + acc.histoOOORejected++ + } + } + + if chunkCreated { + a.head.metrics.chunks.Inc() + a.head.metrics.chunksCreated.Inc() + } + + series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) + series.pendingCommit = false + series.Unlock() + } +} + +// commitMetadata commits the metadata for each series in the provided batch. +// It iterates over the metadata slice and updates the corresponding series +// with the new metadata information. The series is locked during the update +// to ensure thread safety. +func commitMetadata(b *appendBatch) { + var series *memSeries + for i, m := range b.metadata { + series = b.metadataSeries[i] + series.Lock() + series.meta = &metadata.Metadata{Type: record.ToMetricType(m.Type), Unit: m.Unit, Help: m.Help} + series.Unlock() + } +} + +func (a *headAppender) unmarkCreatedSeriesAsPendingCommit() { + for _, s := range a.series { + s.Lock() + s.pendingCommit = false + s.Unlock() + } +} + +// Commit writes to the WAL and adds the data to the Head. +// TODO(codesome): Refactor this method to reduce indentation and make it more readable. +func (a *headAppender) Commit() (err error) { + if a.closed { + return ErrAppenderClosed + } + + h := a.head + + defer func() { + if a.closed { + // Don't double-close in case Rollback() was called. + return + } + h.putRefSeriesBuffer(a.seriesRefs) + h.putSeriesBuffer(a.series) + h.putTypeMap(a.typesInBatch) + a.closed = true + }() + + if err := a.log(); err != nil { + _ = a.Rollback() // Most likely the same error will happen again. + return fmt.Errorf("write to WAL: %w", err) + } + + if h.writeNotified != nil { + h.writeNotified.Notify() + } + + acc := &appenderCommitContext{ + inOrderMint: math.MaxInt64, + inOrderMaxt: math.MinInt64, + oooMinT: math.MaxInt64, + oooMaxT: math.MinInt64, + oooCapMax: h.opts.OutOfOrderCapMax.Load(), + appendChunkOpts: chunkOpts{ + chunkDiskMapper: h.chunkDiskMapper, + chunkRange: h.chunkRange.Load(), + samplesPerChunk: h.opts.SamplesPerChunk, + }, + } + + for _, b := range a.batches { + acc.floatsAppended += len(b.floats) + acc.histogramsAppended += len(b.histograms) + len(b.floatHistograms) + a.commitExemplars(b) + defer b.close(h) + } + defer h.metrics.activeAppenders.Dec() + defer h.iso.closeAppend(a.appendID) + + defer func() { + for i := range acc.oooRecords { + h.putBytesBuffer(acc.oooRecords[i][:0]) + } + }() + + for _, b := range a.batches { + // Do not change the order of these calls. We depend on it for + // correct commit order of samples and for the staleness marker + // handling. + a.commitFloats(b, acc) + a.commitHistograms(b, acc) + a.commitFloatHistograms(b, acc) + commitMetadata(b) + } + // Unmark all series as pending commit after all samples have been committed. + a.unmarkCreatedSeriesAsPendingCommit() + + h.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatOOORejected)) + h.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeHistogram).Add(float64(acc.histoOOORejected)) + h.metrics.outOfBoundSamples.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatOOBRejected)) + h.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatTooOldRejected)) + h.metrics.samplesAppended.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatsAppended)) + h.metrics.samplesAppended.WithLabelValues(sampleMetricTypeHistogram).Add(float64(acc.histogramsAppended)) + h.metrics.outOfOrderSamplesAppended.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.oooFloatsAccepted)) + h.metrics.outOfOrderSamplesAppended.WithLabelValues(sampleMetricTypeHistogram).Add(float64(acc.oooHistogramAccepted)) + h.updateMinMaxTime(acc.inOrderMint, acc.inOrderMaxt) + h.updateMinOOOMaxOOOTime(acc.oooMinT, acc.oooMaxT) + + acc.collectOOORecords(a) + if h.wbl != nil { + if err := h.wbl.Log(acc.oooRecords...); err != nil { + // TODO(codesome): Currently WBL logging of ooo samples is best effort here since we cannot try logging + // until we have found what samples become OOO. We can try having a metric for this failure. + // Returning the error here is not correct because we have already put the samples into the memory, + // hence the append/insert was a success. + h.logger.Error("Failed to log out of order samples into the WAL", "err", err) + } + } + return nil +} + +// insert is like append, except it inserts. Used for OOO samples. +func (s *memSeries) insert(t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, chunkDiskMapper *chunks.ChunkDiskMapper, oooCapMax int64, logger *slog.Logger) (inserted, chunkCreated bool, mmapRefs []chunks.ChunkDiskMapperRef) { + if s.ooo == nil { + s.ooo = &memSeriesOOOFields{} + } + c := s.ooo.oooHeadChunk + if c == nil || c.chunk.NumSamples() == int(oooCapMax) { + // Note: If no new samples come in then we rely on compaction to clean up stale in-memory OOO chunks. + c, mmapRefs = s.cutNewOOOHeadChunk(t, chunkDiskMapper, logger) + chunkCreated = true + } + + ok := c.chunk.Insert(t, v, h, fh) + if ok { + if chunkCreated || t < c.minTime { + c.minTime = t + } + if chunkCreated || t > c.maxTime { + c.maxTime = t + } + } + return ok, chunkCreated, mmapRefs +} + +// chunkOpts are chunk-level options that are passed when appending to a memSeries. +type chunkOpts struct { + chunkDiskMapper *chunks.ChunkDiskMapper + chunkRange int64 + samplesPerChunk int +} + +// append adds the sample (t, v) to the series. The caller also has to provide +// the appendID for isolation. (The appendID can be zero, which results in no +// isolation for this append.) +// Series lock must be held when calling. +func (s *memSeries) append(t int64, v float64, appendID uint64, o chunkOpts) (sampleInOrder, chunkCreated bool) { + c, sampleInOrder, chunkCreated := s.appendPreprocessor(t, chunkenc.EncXOR, o) + if !sampleInOrder { + return sampleInOrder, chunkCreated + } + s.app.Append(t, v) + + c.maxTime = t + + s.lastValue = v + s.lastHistogramValue = nil + s.lastFloatHistogramValue = nil + + if appendID > 0 { + s.txs.add(appendID) + } + + return true, chunkCreated +} + +// appendHistogram adds the histogram. +// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. +// In case of recoding the existing chunk, a new chunk is allocated and the old chunk is dropped. +// To keep the meaning of prometheus_tsdb_head_chunks and prometheus_tsdb_head_chunks_created_total +// consistent, we return chunkCreated=false in this case. +func (s *memSeries) appendHistogram(t int64, h *histogram.Histogram, appendID uint64, o chunkOpts) (sampleInOrder, chunkCreated bool) { + // Head controls the execution of recoding, so that we own the proper + // chunk reference afterwards and mmap used up chunks. + + // Ignoring ok is ok, since we don't want to compare to the wrong previous appender anyway. + prevApp, _ := s.app.(*chunkenc.HistogramAppender) + + c, sampleInOrder, chunkCreated := s.histogramsAppendPreprocessor(t, chunkenc.EncHistogram, o) + if !sampleInOrder { + return sampleInOrder, chunkCreated + } + + var ( + newChunk chunkenc.Chunk + recoded bool + ) + + if !chunkCreated { + // Ignore the previous appender if we continue the current chunk. + prevApp = nil + } + + newChunk, recoded, s.app, _ = s.app.AppendHistogram(prevApp, t, h, false) // false=request a new chunk if needed + + s.lastHistogramValue = h + s.lastFloatHistogramValue = nil + + if appendID > 0 { + s.txs.add(appendID) + } + + if newChunk == nil { // Sample was appended to existing chunk or is the first sample in a new chunk. + c.maxTime = t + return true, chunkCreated + } + + if recoded { // The appender needed to recode the chunk. + c.maxTime = t + c.chunk = newChunk + return true, false + } + + s.headChunks = &memChunk{ + chunk: newChunk, + minTime: t, + maxTime: t, + prev: s.headChunks, + } + s.nextAt = rangeForTimestamp(t, o.chunkRange) + return true, true +} + +// appendFloatHistogram adds the float histogram. +// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. +// In case of recoding the existing chunk, a new chunk is allocated and the old chunk is dropped. +// To keep the meaning of prometheus_tsdb_head_chunks and prometheus_tsdb_head_chunks_created_total +// consistent, we return chunkCreated=false in this case. +func (s *memSeries) appendFloatHistogram(t int64, fh *histogram.FloatHistogram, appendID uint64, o chunkOpts) (sampleInOrder, chunkCreated bool) { + // Head controls the execution of recoding, so that we own the proper + // chunk reference afterwards and mmap used up chunks. + + // Ignoring ok is ok, since we don't want to compare to the wrong previous appender anyway. + prevApp, _ := s.app.(*chunkenc.FloatHistogramAppender) + + c, sampleInOrder, chunkCreated := s.histogramsAppendPreprocessor(t, chunkenc.EncFloatHistogram, o) + if !sampleInOrder { + return sampleInOrder, chunkCreated + } + + var ( + newChunk chunkenc.Chunk + recoded bool + ) + + if !chunkCreated { + // Ignore the previous appender if we continue the current chunk. + prevApp = nil + } + + newChunk, recoded, s.app, _ = s.app.AppendFloatHistogram(prevApp, t, fh, false) // False means request a new chunk if needed. + + s.lastHistogramValue = nil + s.lastFloatHistogramValue = fh + + if appendID > 0 { + s.txs.add(appendID) + } + + if newChunk == nil { // Sample was appended to existing chunk or is the first sample in a new chunk. + c.maxTime = t + return true, chunkCreated + } + + if recoded { // The appender needed to recode the chunk. + c.maxTime = t + c.chunk = newChunk + return true, false + } + + s.headChunks = &memChunk{ + chunk: newChunk, + minTime: t, + maxTime: t, + prev: s.headChunks, + } + s.nextAt = rangeForTimestamp(t, o.chunkRange) + return true, true +} + +// appendPreprocessor takes care of cutting new XOR chunks and m-mapping old ones. XOR chunks are cut based on the +// number of samples they contain with a soft cap in bytes. +// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. +// This should be called only when appending data. +func (s *memSeries) appendPreprocessor(t int64, e chunkenc.Encoding, o chunkOpts) (c *memChunk, sampleInOrder, chunkCreated bool) { + // We target chunkenc.MaxBytesPerXORChunk as a hard for the size of an XOR chunk. We must determine whether to cut + // a new head chunk without knowing the size of the next sample, however, so we assume the next sample will be a + // maximally-sized sample (19 bytes). + const maxBytesPerXORChunk = chunkenc.MaxBytesPerXORChunk - 19 + + c = s.headChunks + + if c == nil { + if len(s.mmappedChunks) > 0 && s.mmappedChunks[len(s.mmappedChunks)-1].maxTime >= t { + // Out of order sample. Sample timestamp is already in the mmapped chunks, so ignore it. + return c, false, false + } + // There is no head chunk in this series yet, create the first chunk for the sample. + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + // Out of order sample. + if c.maxTime >= t { + return c, false, chunkCreated + } + + // Check the chunk size, unless we just created it and if the chunk is too large, cut a new one. + if !chunkCreated && len(c.chunk.Bytes()) > maxBytesPerXORChunk { + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + if c.chunk.Encoding() != e { + // The chunk encoding expected by this append is different than the head chunk's + // encoding. So we cut a new chunk with the expected encoding. + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + numSamples := c.chunk.NumSamples() + if numSamples == 0 { + // It could be the new chunk created after reading the chunk snapshot, + // hence we fix the minTime of the chunk here. + c.minTime = t + s.nextAt = rangeForTimestamp(c.minTime, o.chunkRange) + } + + // If we reach 25% of a chunk's desired sample count, predict an end time + // for this chunk that will try to make samples equally distributed within + // the remaining chunks in the current chunk range. + // At latest it must happen at the timestamp set when the chunk was cut. + if numSamples == o.samplesPerChunk/4 { + s.nextAt = computeChunkEndTime(c.minTime, c.maxTime, s.nextAt, 4) + } + // If numSamples > samplesPerChunk*2 then our previous prediction was invalid, + // most likely because samples rate has changed and now they are arriving more frequently. + // Since we assume that the rate is higher, we're being conservative and cutting at 2*samplesPerChunk + // as we expect more chunks to come. + // Note that next chunk will have its nextAt recalculated for the new rate. + if t >= s.nextAt || numSamples >= o.samplesPerChunk*2 { + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + return c, true, chunkCreated +} + +// histogramsAppendPreprocessor takes care of cutting new histogram chunks and m-mapping old ones. Histogram chunks are +// cut based on their size in bytes. +// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. +// This should be called only when appending data. +func (s *memSeries) histogramsAppendPreprocessor(t int64, e chunkenc.Encoding, o chunkOpts) (c *memChunk, sampleInOrder, chunkCreated bool) { + c = s.headChunks + + if c == nil { + if len(s.mmappedChunks) > 0 && s.mmappedChunks[len(s.mmappedChunks)-1].maxTime >= t { + // Out of order sample. Sample timestamp is already in the mmapped chunks, so ignore it. + return c, false, false + } + // There is no head chunk in this series yet, create the first chunk for the sample. + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + // Out of order sample. + if c.maxTime >= t { + return c, false, chunkCreated + } + + if c.chunk.Encoding() != e { + // The chunk encoding expected by this append is different than the head chunk's + // encoding. So we cut a new chunk with the expected encoding. + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + numSamples := c.chunk.NumSamples() + targetBytes := chunkenc.TargetBytesPerHistogramChunk + numBytes := len(c.chunk.Bytes()) + + if numSamples == 0 { + // It could be the new chunk created after reading the chunk snapshot, + // hence we fix the minTime of the chunk here. + c.minTime = t + s.nextAt = rangeForTimestamp(c.minTime, o.chunkRange) + } + + // Below, we will enforce chunkenc.MinSamplesPerHistogramChunk. There are, however, two cases that supersede it: + // - The current chunk range is ending before chunkenc.MinSamplesPerHistogramChunk will be satisfied. + // - s.nextAt was set while loading a chunk snapshot with the intent that a new chunk be cut on the next append. + var nextChunkRangeStart int64 + if s.histogramChunkHasComputedEndTime { + nextChunkRangeStart = rangeForTimestamp(c.minTime, o.chunkRange) + } else { + // If we haven't yet computed an end time yet, s.nextAt is either set to + // rangeForTimestamp(c.minTime, o.chunkRange) or was set while loading a chunk snapshot. Either way, we want to + // skip enforcing chunkenc.MinSamplesPerHistogramChunk. + nextChunkRangeStart = s.nextAt + } + + // If we reach 25% of a chunk's desired maximum size, predict an end time + // for this chunk that will try to make samples equally distributed within + // the remaining chunks in the current chunk range. + // At the latest it must happen at the timestamp set when the chunk was cut. + if !s.histogramChunkHasComputedEndTime && numBytes >= targetBytes/4 { + ratioToFull := float64(targetBytes) / float64(numBytes) + s.nextAt = computeChunkEndTime(c.minTime, c.maxTime, s.nextAt, ratioToFull) + s.histogramChunkHasComputedEndTime = true + } + // If numBytes > targetBytes*2 then our previous prediction was invalid. This could happen if the sample rate has + // increased or if the bucket/span count has increased. + // Note that next chunk will have its nextAt recalculated for the new rate. + if (t >= s.nextAt || numBytes >= targetBytes*2) && (numSamples >= chunkenc.MinSamplesPerHistogramChunk || t >= nextChunkRangeStart) { + c = s.cutNewHeadChunk(t, e, o.chunkRange) + chunkCreated = true + } + + // The new chunk will also need a new computed end time. + if chunkCreated { + s.histogramChunkHasComputedEndTime = false + } + + return c, true, chunkCreated +} + +// computeChunkEndTime estimates the end timestamp based the beginning of a +// chunk, its current timestamp and the upper bound up to which we insert data. +// It assumes that the time range is 1/ratioToFull full. +// Assuming that the samples will keep arriving at the same rate, it will make the +// remaining n chunks within this chunk range (before max) equally sized. +func computeChunkEndTime(start, cur, maxT int64, ratioToFull float64) int64 { + n := float64(maxT-start) / (float64(cur-start+1) * ratioToFull) + if n <= 1 { + return maxT + } + return int64(float64(start) + float64(maxT-start)/math.Floor(n)) +} + +func (s *memSeries) cutNewHeadChunk(mint int64, e chunkenc.Encoding, chunkRange int64) *memChunk { + // When cutting a new head chunk we create a new memChunk instance with .prev + // pointing at the current .headChunks, so it forms a linked list. + // All but first headChunks list elements will be m-mapped as soon as possible + // so this is a single element list most of the time. + s.headChunks = &memChunk{ + minTime: mint, + maxTime: math.MinInt64, + prev: s.headChunks, + } + + if chunkenc.IsValidEncoding(e) { + var err error + s.headChunks.chunk, err = chunkenc.NewEmptyChunk(e) + if err != nil { + panic(err) // This should never happen. + } + } else { + s.headChunks.chunk = chunkenc.NewXORChunk() + } + + // Set upper bound on when the next chunk must be started. An earlier timestamp + // may be chosen dynamically at a later point. + s.nextAt = rangeForTimestamp(mint, chunkRange) + + app, err := s.headChunks.chunk.Appender() + if err != nil { + panic(err) + } + s.app = app + return s.headChunks +} + +// cutNewOOOHeadChunk cuts a new OOO chunk and m-maps the old chunk. +// The caller must ensure that s is locked and s.ooo is not nil. +func (s *memSeries) cutNewOOOHeadChunk(mint int64, chunkDiskMapper *chunks.ChunkDiskMapper, logger *slog.Logger) (*oooHeadChunk, []chunks.ChunkDiskMapperRef) { + ref := s.mmapCurrentOOOHeadChunk(chunkDiskMapper, logger) + + s.ooo.oooHeadChunk = &oooHeadChunk{ + chunk: NewOOOChunk(), + minTime: mint, + maxTime: math.MinInt64, + } + + return s.ooo.oooHeadChunk, ref +} + +// s must be locked when calling. +func (s *memSeries) mmapCurrentOOOHeadChunk(chunkDiskMapper *chunks.ChunkDiskMapper, logger *slog.Logger) []chunks.ChunkDiskMapperRef { + if s.ooo == nil || s.ooo.oooHeadChunk == nil { + // OOO is not enabled or there is no head chunk, so nothing to m-map here. + return nil + } + chks, err := s.ooo.oooHeadChunk.chunk.ToEncodedChunks(math.MinInt64, math.MaxInt64) + if err != nil { + handleChunkWriteError(err) + return nil + } + chunkRefs := make([]chunks.ChunkDiskMapperRef, 0, len(chks)) + for _, memchunk := range chks { + if len(s.ooo.oooMmappedChunks) >= (oooChunkIDMask - 1) { + logger.Error("Too many OOO chunks, dropping data", "series", s.lset.String()) + break + } + chunkRef := chunkDiskMapper.WriteChunk(s.ref, memchunk.minTime, memchunk.maxTime, memchunk.chunk, true, handleChunkWriteError) + chunkRefs = append(chunkRefs, chunkRef) + s.ooo.oooMmappedChunks = append(s.ooo.oooMmappedChunks, &mmappedChunk{ + ref: chunkRef, + numSamples: uint16(memchunk.chunk.NumSamples()), + minTime: memchunk.minTime, + maxTime: memchunk.maxTime, + }) + } + s.ooo.oooHeadChunk = nil + return chunkRefs +} + +// mmapChunks will m-map all but first chunk on s.headChunks list. +func (s *memSeries) mmapChunks(chunkDiskMapper *chunks.ChunkDiskMapper) (count int) { + if s.headChunks == nil || s.headChunks.prev == nil { + // There is none or only one head chunk, so nothing to m-map here. + return count + } + + // Write chunks starting from the oldest one and stop before we get to current s.headChunks. + // If we have this chain: s.headChunks{t4} -> t3 -> t2 -> t1 -> t0 + // then we need to write chunks t0 to t3, but skip s.headChunks. + for i := s.headChunks.len() - 1; i > 0; i-- { + chk := s.headChunks.atOffset(i) + chunkRef := chunkDiskMapper.WriteChunk(s.ref, chk.minTime, chk.maxTime, chk.chunk, false, handleChunkWriteError) + s.mmappedChunks = append(s.mmappedChunks, &mmappedChunk{ + ref: chunkRef, + numSamples: uint16(chk.chunk.NumSamples()), + minTime: chk.minTime, + maxTime: chk.maxTime, + }) + count++ + } + + // Once we've written out all chunks except s.headChunks we need to unlink these from s.headChunk. + s.headChunks.prev = nil + + return count +} + +func handleChunkWriteError(err error) { + if err != nil && !errors.Is(err, chunks.ErrChunkDiskMapperClosed) { + panic(err) + } +} + +// Rollback removes the samples and exemplars from headAppender and writes any series to WAL. +func (a *headAppender) Rollback() (err error) { + if a.closed { + return ErrAppenderClosed + } + h := a.head + defer func() { + a.unmarkCreatedSeriesAsPendingCommit() + h.iso.closeAppend(a.appendID) + h.metrics.activeAppenders.Dec() + a.closed = true + h.putRefSeriesBuffer(a.seriesRefs) + h.putSeriesBuffer(a.series) + h.putTypeMap(a.typesInBatch) + }() + + var series *memSeries + for _, b := range a.batches { + for i := range b.floats { + series = b.floatSeries[i] + series.Lock() + series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) + series.pendingCommit = false + series.Unlock() + } + for i := range b.histograms { + series = b.histogramSeries[i] + series.Lock() + series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) + series.pendingCommit = false + series.Unlock() + } + for i := range b.floatHistograms { + series = b.floatHistogramSeries[i] + series.Lock() + series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) + series.pendingCommit = false + series.Unlock() + } + b.close(h) + } + a.batches = a.batches[:0] + // Series are created in the head memory regardless of rollback. Thus we have + // to log them to the WAL in any case. + return a.log() +} From efdfb8fed626ebbcddeeb60ea9ba8b29c13dbdf2 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 28 Nov 2025 12:47:43 +0000 Subject: [PATCH 116/439] refactor(appenderV2): 1:1 copy of head append test files for v2 (starting point) Signed-off-by: bwplotka --- tsdb/head_append_v2_test.go | 7332 +++++++++++++++++++++++++++++++++++ tsdb/head_bench_v2_test.go | 173 + 2 files changed, 7505 insertions(+) create mode 100644 tsdb/head_append_v2_test.go create mode 100644 tsdb/head_bench_v2_test.go diff --git a/tsdb/head_append_v2_test.go b/tsdb/head_append_v2_test.go new file mode 100644 index 0000000000..552db13d07 --- /dev/null +++ b/tsdb/head_append_v2_test.go @@ -0,0 +1,7332 @@ +// Copyright 2017 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package tsdb + +import ( + "context" + "fmt" + "io" + "math" + "math/rand" + "os" + "path" + "path/filepath" + "reflect" + "slices" + "sort" + "strconv" + "strings" + "sync" + "testing" + "time" + + "github.com/google/go-cmp/cmp" + "github.com/prometheus/client_golang/prometheus" + prom_testutil "github.com/prometheus/client_golang/prometheus/testutil" + dto "github.com/prometheus/client_model/go" + "github.com/prometheus/common/promslog" + "github.com/stretchr/testify/require" + "go.uber.org/atomic" + "golang.org/x/sync/errgroup" + + "github.com/prometheus/prometheus/config" + "github.com/prometheus/prometheus/model/exemplar" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/model/value" + "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/tsdb/chunkenc" + "github.com/prometheus/prometheus/tsdb/chunks" + "github.com/prometheus/prometheus/tsdb/fileutil" + "github.com/prometheus/prometheus/tsdb/index" + "github.com/prometheus/prometheus/tsdb/record" + "github.com/prometheus/prometheus/tsdb/tombstones" + "github.com/prometheus/prometheus/tsdb/tsdbutil" + "github.com/prometheus/prometheus/tsdb/wlog" + "github.com/prometheus/prometheus/util/compression" + "github.com/prometheus/prometheus/util/testutil" +) + +// newTestHeadDefaultOptions returns the HeadOptions that should be used by default in unit tests. +func newTestHeadDefaultOptions(chunkRange int64, oooEnabled bool) *HeadOptions { + opts := DefaultHeadOptions() + opts.ChunkRange = chunkRange + opts.EnableExemplarStorage = true + opts.MaxExemplars.Store(config.DefaultExemplarsConfig.MaxExemplars) + if oooEnabled { + opts.OutOfOrderTimeWindow.Store(10 * time.Minute.Milliseconds()) + } + return opts +} + +func newTestHead(t testing.TB, chunkRange int64, compressWAL compression.Type, oooEnabled bool) (*Head, *wlog.WL) { + return newTestHeadWithOptions(t, compressWAL, newTestHeadDefaultOptions(chunkRange, oooEnabled)) +} + +func newTestHeadWithOptions(t testing.TB, compressWAL compression.Type, opts *HeadOptions) (*Head, *wlog.WL) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compressWAL) + require.NoError(t, err) + + // Override the chunks dir with the testing one. + opts.ChunkDirRoot = dir + + h, err := NewHead(nil, nil, wal, nil, opts, nil) + require.NoError(t, err) + + require.NoError(t, h.chunkDiskMapper.IterateAllChunks(func(chunks.HeadSeriesRef, chunks.ChunkDiskMapperRef, int64, int64, uint16, chunkenc.Encoding, bool) error { + return nil + })) + + return h, wal +} + +func BenchmarkCreateSeries(b *testing.B) { + series := genSeries(b.N, 10, 0, 0) + h, _ := newTestHead(b, 10000, compression.None, false) + b.Cleanup(func() { + require.NoError(b, h.Close()) + }) + + b.ReportAllocs() + b.ResetTimer() + + for _, s := range series { + h.getOrCreate(s.Labels().Hash(), s.Labels(), false) + } +} + +func BenchmarkHeadAppender_Append_Commit_ExistingSeries(b *testing.B) { + seriesCounts := []int{100, 1000, 10000} + series := genSeries(10000, 10, 0, 0) + + for _, seriesCount := range seriesCounts { + b.Run(fmt.Sprintf("%d series", seriesCount), func(b *testing.B) { + for _, samplesPerAppend := range []int64{1, 2, 5, 100} { + b.Run(fmt.Sprintf("%d samples per append", samplesPerAppend), func(b *testing.B) { + h, _ := newTestHead(b, 10000, compression.None, false) + b.Cleanup(func() { require.NoError(b, h.Close()) }) + + ts := int64(1000) + appendSamples := func() error { + var err error + app := h.Appender(context.Background()) + for _, s := range series[:seriesCount] { + var ref storage.SeriesRef + for sampleIndex := range samplesPerAppend { + ref, err = app.Append(ref, s.Labels(), ts+sampleIndex, float64(ts+sampleIndex)) + if err != nil { + return err + } + } + } + ts += 1000 // should increment more than highest samplesPerAppend + return app.Commit() + } + + // Init series, that's not what we're benchmarking here. + require.NoError(b, appendSamples()) + + b.ReportAllocs() + b.ResetTimer() + + for b.Loop() { + require.NoError(b, appendSamples()) + } + }) + } + }) + } +} + +func populateTestWL(t testing.TB, w *wlog.WL, recs []any, buf []byte) []byte { + var enc record.Encoder + for _, r := range recs { + buf = buf[:0] + switch v := r.(type) { + case []record.RefSeries: + buf = enc.Series(v, buf) + case []record.RefSample: + buf = enc.Samples(v, buf) + case []tombstones.Stone: + buf = enc.Tombstones(v, buf) + case []record.RefExemplar: + buf = enc.Exemplars(v, buf) + case []record.RefHistogramSample: + buf, _ = enc.HistogramSamples(v, buf) + case []record.RefFloatHistogramSample: + buf, _ = enc.FloatHistogramSamples(v, buf) + case []record.RefMmapMarker: + buf = enc.MmapMarkers(v, buf) + case []record.RefMetadata: + buf = enc.Metadata(v, buf) + default: + continue + } + require.NoError(t, w.Log(buf)) + } + return buf +} + +func readTestWAL(t testing.TB, dir string) (recs []any) { + sr, err := wlog.NewSegmentsReader(dir) + require.NoError(t, err) + defer func() { + require.NoError(t, sr.Close()) + }() + + dec := record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + r := wlog.NewReader(sr) + + for r.Next() { + rec := r.Record() + + switch dec.Type(rec) { + case record.Series: + series, err := dec.Series(rec, nil) + require.NoError(t, err) + recs = append(recs, series) + case record.Samples: + samples, err := dec.Samples(rec, nil) + require.NoError(t, err) + recs = append(recs, samples) + case record.HistogramSamples, record.CustomBucketsHistogramSamples: + samples, err := dec.HistogramSamples(rec, nil) + require.NoError(t, err) + recs = append(recs, samples) + case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: + samples, err := dec.FloatHistogramSamples(rec, nil) + require.NoError(t, err) + recs = append(recs, samples) + case record.Tombstones: + tstones, err := dec.Tombstones(rec, nil) + require.NoError(t, err) + recs = append(recs, tstones) + case record.Metadata: + meta, err := dec.Metadata(rec, nil) + require.NoError(t, err) + recs = append(recs, meta) + case record.Exemplars: + exemplars, err := dec.Exemplars(rec, nil) + require.NoError(t, err) + recs = append(recs, exemplars) + default: + require.Fail(t, "unknown record type") + } + } + require.NoError(t, r.Err()) + return recs +} + +func BenchmarkLoadWLs(b *testing.B) { + cases := []struct { + // Total series is (batches*seriesPerBatch). + batches int + seriesPerBatch int + samplesPerSeries int + mmappedChunkT int64 + // The first oooSeriesPct*seriesPerBatch series in a batch are selected as "OOO" series. + oooSeriesPct float64 + // The first oooSamplesPct*samplesPerSeries samples in an OOO series are written as OOO samples. + oooSamplesPct float64 + oooCapMax int64 + }{ + { // Less series and more samples. 2 hour WAL with 1 second scrape interval. + batches: 10, + seriesPerBatch: 100, + samplesPerSeries: 7200, + }, + { // More series and less samples. + batches: 10, + seriesPerBatch: 10000, + samplesPerSeries: 50, + }, + { // In between. + batches: 10, + seriesPerBatch: 1000, + samplesPerSeries: 480, + }, + { // 2 hour WAL with 15 second scrape interval, and mmapped chunks up to last 100 samples. + batches: 100, + seriesPerBatch: 1000, + samplesPerSeries: 480, + mmappedChunkT: 3800, + }, + { // A lot of OOO samples (50% series with 50% of samples being OOO). + batches: 10, + seriesPerBatch: 1000, + samplesPerSeries: 480, + oooSeriesPct: 0.5, + oooSamplesPct: 0.5, + oooCapMax: DefaultOutOfOrderCapMax, + }, + { // Fewer OOO samples (10% of series with 10% of samples being OOO). + batches: 10, + seriesPerBatch: 1000, + samplesPerSeries: 480, + oooSeriesPct: 0.1, + oooSamplesPct: 0.1, + }, + { // 2 hour WAL with 15 second scrape interval, and mmapped chunks up to last 100 samples. + // Four mmap markers per OOO series: 480 * 0.3 = 144, 144 / 32 (DefaultOutOfOrderCapMax) = 4. + batches: 100, + seriesPerBatch: 1000, + samplesPerSeries: 480, + mmappedChunkT: 3800, + oooSeriesPct: 0.2, + oooSamplesPct: 0.3, + oooCapMax: DefaultOutOfOrderCapMax, + }, + } + + labelsPerSeries := 5 + // Rough estimates of most common % of samples that have an exemplar for each scrape. + exemplarsPercentages := []float64{0, 0.5, 1, 5} + lastExemplarsPerSeries := -1 + for _, c := range cases { + missingSeriesPercentages := []float64{0, 0.1} + for _, missingSeriesPct := range missingSeriesPercentages { + for _, p := range exemplarsPercentages { + exemplarsPerSeries := int(math.RoundToEven(float64(c.samplesPerSeries) * p / 100)) + // For tests with low samplesPerSeries we could end up testing with 0 exemplarsPerSeries + // multiple times without this check. + if exemplarsPerSeries == lastExemplarsPerSeries { + continue + } + lastExemplarsPerSeries = exemplarsPerSeries + b.Run(fmt.Sprintf("batches=%d,seriesPerBatch=%d,samplesPerSeries=%d,exemplarsPerSeries=%d,mmappedChunkT=%d,oooSeriesPct=%.3f,oooSamplesPct=%.3f,oooCapMax=%d,missingSeriesPct=%.3f", c.batches, c.seriesPerBatch, c.samplesPerSeries, exemplarsPerSeries, c.mmappedChunkT, c.oooSeriesPct, c.oooSamplesPct, c.oooCapMax, missingSeriesPct), + func(b *testing.B) { + dir := b.TempDir() + + wal, err := wlog.New(nil, nil, dir, compression.None) + require.NoError(b, err) + var wbl *wlog.WL + if c.oooSeriesPct != 0 { + wbl, err = wlog.New(nil, nil, dir, compression.None) + require.NoError(b, err) + } + + // Write series. + refSeries := make([]record.RefSeries, 0, c.seriesPerBatch) + var buf []byte + builder := labels.NewBuilder(labels.EmptyLabels()) + for j := 1; j < labelsPerSeries; j++ { + builder.Set(defaultLabelName+strconv.Itoa(j), defaultLabelValue+strconv.Itoa(j)) + } + for k := 0; k < c.batches; k++ { + refSeries = refSeries[:0] + for i := k * c.seriesPerBatch; i < (k+1)*c.seriesPerBatch; i++ { + builder.Set(defaultLabelName, strconv.Itoa(i)) + refSeries = append(refSeries, record.RefSeries{Ref: chunks.HeadSeriesRef(i) * 101, Labels: builder.Labels()}) + } + + writeSeries := refSeries + if missingSeriesPct > 0 { + newWriteSeries := make([]record.RefSeries, 0, int(float64(len(refSeries))*(1.0-missingSeriesPct))) + keepRatio := 1.0 - missingSeriesPct + // Keep approximately every 1/keepRatio series. + for i, s := range refSeries { + if int(float64(i)*keepRatio) != int(float64(i+1)*keepRatio) { + newWriteSeries = append(newWriteSeries, s) + } + } + writeSeries = newWriteSeries + } + + buf = populateTestWL(b, wal, []any{writeSeries}, buf) + } + + // Write samples. + refSamples := make([]record.RefSample, 0, c.seriesPerBatch) + + oooSeriesPerBatch := int(float64(c.seriesPerBatch) * c.oooSeriesPct) + oooSamplesPerSeries := int(float64(c.samplesPerSeries) * c.oooSamplesPct) + + for i := 0; i < c.samplesPerSeries; i++ { + for j := 0; j < c.batches; j++ { + refSamples = refSamples[:0] + + k := j * c.seriesPerBatch + // Skip appending the first oooSamplesPerSeries samples for the series in the batch that + // should have OOO samples. OOO samples are appended after all the in-order samples. + if i < oooSamplesPerSeries { + k += oooSeriesPerBatch + } + for ; k < (j+1)*c.seriesPerBatch; k++ { + refSamples = append(refSamples, record.RefSample{ + Ref: chunks.HeadSeriesRef(k) * 101, + T: int64(i) * 10, + V: float64(i) * 100, + }) + } + buf = populateTestWL(b, wal, []any{refSamples}, buf) + } + } + + // Write mmapped chunks. + if c.mmappedChunkT != 0 { + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, mmappedChunksDir(dir), chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(b, err) + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: c.mmappedChunkT, + samplesPerChunk: DefaultSamplesPerChunk, + } + for k := 0; k < c.batches*c.seriesPerBatch; k++ { + // Create one mmapped chunk per series, with one sample at the given time. + s := newMemSeries(labels.Labels{}, chunks.HeadSeriesRef(k)*101, 0, defaultIsolationDisabled, false) + s.append(c.mmappedChunkT, 42, 0, cOpts) + // There's only one head chunk because only a single sample is appended. mmapChunks() + // ignores the latest chunk, so we need to cut a new head chunk to guarantee the chunk with + // the sample at c.mmappedChunkT is mmapped. + s.cutNewHeadChunk(c.mmappedChunkT, chunkenc.EncXOR, c.mmappedChunkT) + s.mmapChunks(chunkDiskMapper) + } + require.NoError(b, chunkDiskMapper.Close()) + } + + // Write exemplars. + refExemplars := make([]record.RefExemplar, 0, c.seriesPerBatch) + for i := range exemplarsPerSeries { + for j := 0; j < c.batches; j++ { + refExemplars = refExemplars[:0] + for k := j * c.seriesPerBatch; k < (j+1)*c.seriesPerBatch; k++ { + refExemplars = append(refExemplars, record.RefExemplar{ + Ref: chunks.HeadSeriesRef(k) * 101, + T: int64(i) * 10, + V: float64(i) * 100, + Labels: labels.FromStrings("trace_id", fmt.Sprintf("trace-%d", i)), + }) + } + buf = populateTestWL(b, wal, []any{refExemplars}, buf) + } + } + + // Write OOO samples and mmap markers. + refMarkers := make([]record.RefMmapMarker, 0, oooSeriesPerBatch) + refSamples = make([]record.RefSample, 0, oooSeriesPerBatch) + for i := range oooSamplesPerSeries { + shouldAddMarkers := c.oooCapMax != 0 && i != 0 && int64(i)%c.oooCapMax == 0 + + for j := 0; j < c.batches; j++ { + refSamples = refSamples[:0] + if shouldAddMarkers { + refMarkers = refMarkers[:0] + } + for k := j * c.seriesPerBatch; k < (j*c.seriesPerBatch)+oooSeriesPerBatch; k++ { + ref := chunks.HeadSeriesRef(k) * 101 + if shouldAddMarkers { + // loadWBL() checks that the marker's MmapRef is less than or equal to the ref + // for the last mmap chunk. Setting MmapRef to 0 to always pass that check. + refMarkers = append(refMarkers, record.RefMmapMarker{Ref: ref, MmapRef: 0}) + } + refSamples = append(refSamples, record.RefSample{ + Ref: ref, + T: int64(i) * 10, + V: float64(i) * 100, + }) + } + if shouldAddMarkers { + populateTestWL(b, wbl, []any{refMarkers}, buf) + } + buf = populateTestWL(b, wal, []any{refSamples}, buf) + buf = populateTestWL(b, wbl, []any{refSamples}, buf) + } + } + + b.ResetTimer() + + // Load the WAL. + for b.Loop() { + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = dir + if c.oooCapMax > 0 { + opts.OutOfOrderCapMax.Store(c.oooCapMax) + } + h, err := NewHead(nil, nil, wal, wbl, opts, nil) + require.NoError(b, err) + h.Init(0) + } + b.StopTimer() + wal.Close() + if wbl != nil { + wbl.Close() + } + }) + } + } + } +} + +// BenchmarkLoadRealWLs will be skipped unless the BENCHMARK_LOAD_REAL_WLS_DIR environment variable is set. +// BENCHMARK_LOAD_REAL_WLS_DIR should be the folder where `wal` and `chunks_head` are located. +// +// Using an absolute path for BENCHMARK_LOAD_REAL_WLS_DIR is recommended. +// +// Because WLs loading may alter BENCHMARK_LOAD_REAL_WLS_DIR which can affect benchmark results and to ensure consistency, +// a copy of BENCHMARK_LOAD_REAL_WLS_DIR is made for each iteration and deleted at the end. +// Make sure there is sufficient disk space for that. +func BenchmarkLoadRealWLs(b *testing.B) { + srcDir := os.Getenv("BENCHMARK_LOAD_REAL_WLS_DIR") + if srcDir == "" { + b.SkipNow() + } + + // Load the WAL. + for b.Loop() { + b.StopTimer() + dir := b.TempDir() + require.NoError(b, fileutil.CopyDirs(srcDir, dir)) + + wal, err := wlog.New(nil, nil, filepath.Join(dir, "wal"), compression.None) + require.NoError(b, err) + b.Cleanup(func() { wal.Close() }) + + wbl, err := wlog.New(nil, nil, filepath.Join(dir, "wbl"), compression.None) + require.NoError(b, err) + b.Cleanup(func() { wbl.Close() }) + b.StartTimer() + + opts := DefaultHeadOptions() + opts.ChunkDirRoot = dir + h, err := NewHead(nil, nil, wal, wbl, opts, nil) + require.NoError(b, err) + require.NoError(b, h.Init(0)) + + b.StopTimer() + require.NoError(b, os.RemoveAll(dir)) + } +} + +// TestHead_HighConcurrencyReadAndWrite generates 1000 series with a step of 15s and fills a whole block with samples, +// this means in total it generates 4000 chunks because with a step of 15s there are 4 chunks per block per series. +// While appending the samples to the head it concurrently queries them from multiple go routines and verifies that the +// returned results are correct. +func TestHead_HighConcurrencyReadAndWrite(t *testing.T) { + head, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + seriesCnt := 1000 + readConcurrency := 2 + writeConcurrency := 10 + startTs := uint64(DefaultBlockDuration) // start at the second block relative to the unix epoch. + qryRange := uint64(5 * time.Minute.Milliseconds()) + step := uint64(15 * time.Second / time.Millisecond) + endTs := startTs + uint64(DefaultBlockDuration) + + labelSets := make([]labels.Labels, seriesCnt) + for i := range seriesCnt { + labelSets[i] = labels.FromStrings("seriesId", strconv.Itoa(i)) + } + + head.Init(0) + + g, ctx := errgroup.WithContext(context.Background()) + whileNotCanceled := func(f func() (bool, error)) error { + for ctx.Err() == nil { + cont, err := f() + if err != nil { + return err + } + if !cont { + return nil + } + } + return nil + } + + // Create one channel for each write worker, the channels will be used by the coordinator + // go routine to coordinate which timestamps each write worker has to write. + writerTsCh := make([]chan uint64, writeConcurrency) + for writerTsChIdx := range writerTsCh { + writerTsCh[writerTsChIdx] = make(chan uint64) + } + + // workerReadyWg is used to synchronize the start of the test, + // we only start the test once all workers signal that they're ready. + var workerReadyWg sync.WaitGroup + workerReadyWg.Add(writeConcurrency + readConcurrency) + + // Start the write workers. + for wid := range writeConcurrency { + // Create copy of workerID to be used by worker routine. + workerID := wid + + g.Go(func() error { + // The label sets which this worker will write. + workerLabelSets := labelSets[(seriesCnt/writeConcurrency)*workerID : (seriesCnt/writeConcurrency)*(workerID+1)] + + // Signal that this worker is ready. + workerReadyWg.Done() + + return whileNotCanceled(func() (bool, error) { + ts, ok := <-writerTsCh[workerID] + if !ok { + return false, nil + } + + app := head.Appender(ctx) + for i := range workerLabelSets { + // We also use the timestamp as the sample value. + _, err := app.Append(0, workerLabelSets[i], int64(ts), float64(ts)) + if err != nil { + return false, fmt.Errorf("Error when appending to head: %w", err) + } + } + + return true, app.Commit() + }) + }) + } + + // queryHead is a helper to query the head for a given time range and labelset. + queryHead := func(mint, maxt uint64, label labels.Label) (map[string][]chunks.Sample, error) { + q, err := NewBlockQuerier(head, int64(mint), int64(maxt)) + if err != nil { + return nil, err + } + return query(t, q, labels.MustNewMatcher(labels.MatchEqual, label.Name, label.Value)), nil + } + + // readerTsCh will be used by the coordinator go routine to coordinate which timestamps the reader should read. + readerTsCh := make(chan uint64) + + // Start the read workers. + for wid := range readConcurrency { + // Create copy of threadID to be used by worker routine. + workerID := wid + + g.Go(func() error { + querySeriesRef := (seriesCnt / readConcurrency) * workerID + + // Signal that this worker is ready. + workerReadyWg.Done() + + return whileNotCanceled(func() (bool, error) { + ts, ok := <-readerTsCh + if !ok { + return false, nil + } + + querySeriesRef = (querySeriesRef + 1) % seriesCnt + lbls := labelSets[querySeriesRef] + // lbls has a single entry; extract it so we can run a query. + var lbl labels.Label + lbls.Range(func(l labels.Label) { + lbl = l + }) + samples, err := queryHead(ts-qryRange, ts, lbl) + if err != nil { + return false, err + } + + if len(samples) != 1 { + return false, fmt.Errorf("expected 1 series, got %d", len(samples)) + } + + series := lbls.String() + expectSampleCnt := qryRange/step + 1 + if expectSampleCnt != uint64(len(samples[series])) { + return false, fmt.Errorf("expected %d samples, got %d", expectSampleCnt, len(samples[series])) + } + + for sampleIdx, sample := range samples[series] { + expectedValue := ts - qryRange + (uint64(sampleIdx) * step) + if sample.T() != int64(expectedValue) { + return false, fmt.Errorf("expected sample %d to have ts %d, got %d", sampleIdx, expectedValue, sample.T()) + } + if sample.F() != float64(expectedValue) { + return false, fmt.Errorf("expected sample %d to have value %d, got %f", sampleIdx, expectedValue, sample.F()) + } + } + + return true, nil + }) + }) + } + + // Start the coordinator go routine. + g.Go(func() error { + currTs := startTs + + defer func() { + // End of the test, close all channels to stop the workers. + for _, ch := range writerTsCh { + close(ch) + } + close(readerTsCh) + }() + + // Wait until all workers are ready to start the test. + workerReadyWg.Wait() + return whileNotCanceled(func() (bool, error) { + // Send the current timestamp to each of the writers. + for _, ch := range writerTsCh { + select { + case ch <- currTs: + case <-ctx.Done(): + return false, nil + } + } + + // Once data for at least has been ingested, send the current timestamp to the readers. + if currTs > startTs+qryRange { + select { + case readerTsCh <- currTs - step: + case <-ctx.Done(): + return false, nil + } + } + + currTs += step + if currTs > endTs { + return false, nil + } + + return true, nil + }) + }) + + require.NoError(t, g.Wait()) +} + +func TestHead_ReadWAL(t *testing.T) { + for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { + t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { + entries := []any{ + []record.RefSeries{ + {Ref: 10, Labels: labels.FromStrings("a", "1")}, + {Ref: 11, Labels: labels.FromStrings("a", "2")}, + {Ref: 100, Labels: labels.FromStrings("a", "3")}, + }, + []record.RefSample{ + {Ref: 0, T: 99, V: 1}, + {Ref: 10, T: 100, V: 2}, + {Ref: 100, T: 100, V: 3}, + }, + []record.RefSeries{ + {Ref: 50, Labels: labels.FromStrings("a", "4")}, + // This series has two refs pointing to it. + {Ref: 101, Labels: labels.FromStrings("a", "3")}, + }, + []record.RefSample{ + {Ref: 10, T: 101, V: 5}, + {Ref: 50, T: 101, V: 6}, + // Sample for duplicate series record. + {Ref: 101, T: 101, V: 7}, + }, + []tombstones.Stone{ + {Ref: 0, Intervals: []tombstones.Interval{{Mint: 99, Maxt: 101}}}, + // Tombstone for duplicate series record. + {Ref: 101, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 100}}}, + }, + []record.RefExemplar{ + {Ref: 10, T: 100, V: 1, Labels: labels.FromStrings("trace_id", "asdf")}, + // Exemplar for duplicate series record. + {Ref: 101, T: 101, V: 7, Labels: labels.FromStrings("trace_id", "zxcv")}, + }, + []record.RefMetadata{ + // Metadata for duplicate series record. + {Ref: 101, Type: uint8(record.Counter), Unit: "foo", Help: "total foo"}, + }, + } + + head, w := newTestHead(t, 1000, compress, false) + defer func() { + require.NoError(t, head.Close()) + }() + + populateTestWL(t, w, entries, nil) + + require.NoError(t, head.Init(math.MinInt64)) + require.Equal(t, uint64(101), head.lastSeriesID.Load()) + + s10 := head.series.getByID(10) + s11 := head.series.getByID(11) + s50 := head.series.getByID(50) + s100 := head.series.getByID(100) + s101 := head.series.getByID(101) + + testutil.RequireEqual(t, labels.FromStrings("a", "1"), s10.lset) + require.Nil(t, s11) // Series without samples should be garbage collected at head.Init(). + testutil.RequireEqual(t, labels.FromStrings("a", "4"), s50.lset) + testutil.RequireEqual(t, labels.FromStrings("a", "3"), s100.lset) + + // Duplicate series record should not be written to the head. + require.Nil(t, s101) + // But it should have a WAL expiry set. + keepUntil, ok := head.getWALExpiry(101) + require.True(t, ok) + require.Equal(t, int64(101), keepUntil) + // Only the duplicate series record should have a WAL expiry set. + _, ok = head.getWALExpiry(50) + require.False(t, ok) + + expandChunk := func(c chunkenc.Iterator) (x []sample) { + for c.Next() == chunkenc.ValFloat { + t, v := c.At() + x = append(x, sample{t: t, f: v}) + } + require.NoError(t, c.Err()) + return x + } + + // Verify samples and exemplar for series 10. + c, _, _, err := s10.chunk(0, head.chunkDiskMapper, &head.memChunkPool) + require.NoError(t, err) + require.Equal(t, []sample{{100, 2, nil, nil}, {101, 5, nil, nil}}, expandChunk(c.chunk.Iterator(nil))) + + q, err := head.ExemplarQuerier(context.Background()) + require.NoError(t, err) + e, err := q.Select(0, 1000, []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "1")}) + require.NoError(t, err) + require.NotEmpty(t, e) + require.NotEmpty(t, e[0].Exemplars) + require.True(t, exemplar.Exemplar{Ts: 100, Value: 1, Labels: labels.FromStrings("trace_id", "asdf")}.Equals(e[0].Exemplars[0])) + + // Verify samples for series 50 + c, _, _, err = s50.chunk(0, head.chunkDiskMapper, &head.memChunkPool) + require.NoError(t, err) + require.Equal(t, []sample{{101, 6, nil, nil}}, expandChunk(c.chunk.Iterator(nil))) + + // Verify records for series 100 and its duplicate, series 101. + // The samples before the new series record should be discarded since a duplicate record + // is only possible when old samples were compacted. + c, _, _, err = s100.chunk(0, head.chunkDiskMapper, &head.memChunkPool) + require.NoError(t, err) + require.Equal(t, []sample{{101, 7, nil, nil}}, expandChunk(c.chunk.Iterator(nil))) + + q, err = head.ExemplarQuerier(context.Background()) + require.NoError(t, err) + e, err = q.Select(0, 1000, []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "3")}) + require.NoError(t, err) + require.NotEmpty(t, e) + require.NotEmpty(t, e[0].Exemplars) + require.True(t, exemplar.Exemplar{Ts: 101, Value: 7, Labels: labels.FromStrings("trace_id", "zxcv")}.Equals(e[0].Exemplars[0])) + + require.NotNil(t, s100.meta) + require.Equal(t, "foo", s100.meta.Unit) + require.Equal(t, "total foo", s100.meta.Help) + + intervals, err := head.tombstones.Get(storage.SeriesRef(s100.ref)) + require.NoError(t, err) + require.Equal(t, tombstones.Intervals{{Mint: 0, Maxt: 100}}, intervals) + }) + } +} + +func TestHead_WALMultiRef(t *testing.T) { + head, w := newTestHead(t, 1000, compression.None, false) + + require.NoError(t, head.Init(0)) + + app := head.Appender(context.Background()) + ref1, err := app.Append(0, labels.FromStrings("foo", "bar"), 100, 1) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) + + // Add another sample outside chunk range to mmap a chunk. + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 1500, 2) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 2.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) + + require.NoError(t, head.Truncate(1600)) + + app = head.Appender(context.Background()) + ref2, err := app.Append(0, labels.FromStrings("foo", "bar"), 1700, 3) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 3.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) + + // Add another sample outside chunk range to mmap a chunk. + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 2000, 4) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 4.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) + + require.NotEqual(t, ref1, ref2, "Refs are the same") + require.NoError(t, head.Close()) + + w, err = wlog.New(nil, nil, w.Dir(), compression.None) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = head.opts.ChunkDirRoot + head, err = NewHead(nil, nil, w, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(0)) + defer func() { + require.NoError(t, head.Close()) + }() + + q, err := NewBlockQuerier(head, 0, 2100) + require.NoError(t, err) + series := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + // The samples before the new ref should be discarded since Head truncation + // happens only after compacting the Head. + require.Equal(t, map[string][]chunks.Sample{`{foo="bar"}`: { + sample{1700, 3, nil, nil}, + sample{2000, 4, nil, nil}, + }}, series) +} + +func TestHead_WALCheckpointMultiRef(t *testing.T) { + cases := []struct { + name string + walEntries []any + expectedWalExpiry int64 + walTruncateMinT int64 + expectedWalEntries []any + }{ + { + name: "Samples only; keep needed duplicate series record", + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefSample{ + {Ref: 1, T: 100, V: 1}, + {Ref: 2, T: 200, V: 2}, + {Ref: 2, T: 500, V: 3}, + }, + }, + expectedWalExpiry: 500, + walTruncateMinT: 500, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefSample{ + {Ref: 2, T: 500, V: 3}, + }, + }, + }, + { + name: "Tombstones only; keep needed duplicate series record", + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []tombstones.Stone{ + {Ref: 1, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 100}}}, + {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 200}}}, + {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 500}}}, + }, + }, + expectedWalExpiry: 500, + walTruncateMinT: 500, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []tombstones.Stone{ + {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 500}}}, + }, + }, + }, + { + name: "Exemplars only; keep needed duplicate series record", + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefExemplar{ + {Ref: 1, T: 100, V: 1, Labels: labels.FromStrings("trace_id", "asdf")}, + {Ref: 2, T: 200, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, + {Ref: 2, T: 500, V: 3, Labels: labels.FromStrings("trace_id", "asdf")}, + }, + }, + expectedWalExpiry: 500, + walTruncateMinT: 500, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefExemplar{ + {Ref: 2, T: 500, V: 3, Labels: labels.FromStrings("trace_id", "asdf")}, + }, + }, + }, + { + name: "Histograms only; keep needed duplicate series record", + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: 100, H: &histogram.Histogram{}}, + {Ref: 2, T: 200, H: &histogram.Histogram{}}, + {Ref: 2, T: 500, H: &histogram.Histogram{}}, + }, + }, + expectedWalExpiry: 500, + walTruncateMinT: 500, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: 500, H: &histogram.Histogram{}}, + }, + }, + }, + { + name: "Float histograms only; keep needed duplicate series record", + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: 100, FH: &histogram.FloatHistogram{}}, + {Ref: 2, T: 200, FH: &histogram.FloatHistogram{}}, + {Ref: 2, T: 500, FH: &histogram.FloatHistogram{}}, + }, + }, + expectedWalExpiry: 500, + walTruncateMinT: 500, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: 500, FH: &histogram.FloatHistogram{}}, + }, + }, + }, + { + name: "All record types; keep needed duplicate series record until last record", + // Series with 2 refs and samples for both + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefSample{ + {Ref: 2, T: 500, V: 3}, + }, + []tombstones.Stone{ + {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 500}}}, + }, + []record.RefExemplar{ + {Ref: 2, T: 800, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: 500, H: &histogram.Histogram{}}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: 500, FH: &histogram.FloatHistogram{}}, + }, + }, + expectedWalExpiry: 800, + walTruncateMinT: 700, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefExemplar{ + {Ref: 2, T: 800, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, + }, + }, + }, + { + name: "All record types; drop expired duplicate series record", + // Series with 2 refs and samples for both + walEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + {Ref: 2, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefSample{ + {Ref: 2, T: 500, V: 2}, + {Ref: 1, T: 900, V: 3}, + }, + []tombstones.Stone{ + {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 750}}}, + }, + []record.RefExemplar{ + {Ref: 2, T: 800, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: 600, H: &histogram.Histogram{}}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: 700, FH: &histogram.FloatHistogram{}}, + }, + }, + expectedWalExpiry: 800, + walTruncateMinT: 900, + expectedWalEntries: []any{ + []record.RefSeries{ + {Ref: 1, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefSample{ + {Ref: 1, T: 900, V: 3}, + }, + }, + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + h, w := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, h.Close()) + }) + + populateTestWL(t, w, tc.walEntries, nil) + first, _, err := wlog.Segments(w.Dir()) + require.NoError(t, err) + + require.NoError(t, h.Init(0)) + + keepUntil, ok := h.getWALExpiry(2) + require.True(t, ok) + require.Equal(t, tc.expectedWalExpiry, keepUntil) + + // Each truncation creates a new segment, so attempt truncations until a checkpoint is created + for { + h.lastWALTruncationTime.Store(0) // Reset so that it's always time to truncate the WAL + err := h.truncateWAL(tc.walTruncateMinT) + require.NoError(t, err) + f, _, err := wlog.Segments(w.Dir()) + require.NoError(t, err) + if f > first { + break + } + } + + // Read test WAL , checkpoint first + checkpointDir, _, err := wlog.LastCheckpoint(w.Dir()) + require.NoError(t, err) + cprecs := readTestWAL(t, checkpointDir) + recs := readTestWAL(t, w.Dir()) + recs = append(cprecs, recs...) + + // Use testutil.RequireEqual which handles labels properly with dedupelabels + testutil.RequireEqual(t, tc.expectedWalEntries, recs) + }) + } +} + +func TestHead_KeepSeriesInWALCheckpoint(t *testing.T) { + existingRef := 1 + existingLbls := labels.FromStrings("foo", "bar") + keepUntil := int64(10) + + cases := []struct { + name string + prepare func(t *testing.T, h *Head) + mint int64 + expected bool + }{ + { + name: "keep series still in the head", + prepare: func(t *testing.T, h *Head) { + _, _, err := h.getOrCreateWithOptionalID(chunks.HeadSeriesRef(existingRef), existingLbls.Hash(), existingLbls, false) + require.NoError(t, err) + }, + expected: true, + }, + { + name: "keep series with keepUntil > mint", + mint: keepUntil - 1, + expected: true, + }, + { + name: "keep series with keepUntil = mint", + mint: keepUntil, + expected: true, + }, + { + name: "drop series with keepUntil < mint", + mint: keepUntil + 1, + expected: false, + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, h.Close()) + }) + + if tc.prepare != nil { + tc.prepare(t, h) + } else { + h.updateWALExpiry(chunks.HeadSeriesRef(existingRef), keepUntil) + } + + keep := h.keepSeriesInWALCheckpointFn(tc.mint) + require.Equal(t, tc.expected, keep(chunks.HeadSeriesRef(existingRef))) + }) + } +} + +func TestHead_ActiveAppenders(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + defer head.Close() + + require.NoError(t, head.Init(0)) + + // First rollback with no samples. + app := head.Appender(context.Background()) + require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) + require.NoError(t, app.Rollback()) + require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) + + // Then commit with no samples. + app = head.Appender(context.Background()) + require.NoError(t, app.Commit()) + require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) + + // Now rollback with one sample. + app = head.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 100, 1) + require.NoError(t, err) + require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) + require.NoError(t, app.Rollback()) + require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) + + // Now commit with one sample. + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 100, 1) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) +} + +func TestHead_RaceBetweenSeriesCreationAndGC(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { _ = head.Close() }) + require.NoError(t, head.Init(0)) + + const totalSeries = 100_000 + series := make([]labels.Labels, totalSeries) + for i := range totalSeries { + series[i] = labels.FromStrings("foo", strconv.Itoa(i)) + } + done := atomic.NewBool(false) + + go func() { + defer done.Store(true) + app := head.Appender(context.Background()) + defer func() { + if err := app.Commit(); err != nil { + t.Errorf("Failed to commit: %v", err) + } + }() + for i := range totalSeries { + _, err := app.Append(0, series[i], 100, 1) + if err != nil { + t.Errorf("Failed to append: %v", err) + return + } + } + }() + + // Don't check the atomic.Bool on all iterations in order to perform more gc iterations and make the race condition more likely. + for i := 1; i%128 != 0 || !done.Load(); i++ { + head.gc() + } + + require.Equal(t, totalSeries, int(head.NumSeries())) +} + +func TestHead_CanGarbagecollectSeriesCreatedWithoutSamples(t *testing.T) { + for op, finishTxn := range map[string]func(app storage.Appender) error{ + "after commit": func(app storage.Appender) error { return app.Commit() }, + "after rollback": func(app storage.Appender) error { return app.Rollback() }, + } { + t.Run(op, func(t *testing.T) { + chunkRange := time.Hour.Milliseconds() + head, _ := newTestHead(t, chunkRange, compression.None, true) + t.Cleanup(func() { _ = head.Close() }) + + require.NoError(t, head.Init(0)) + + firstSampleTime := 10 * chunkRange + { + // Append first sample, it should init head max time to firstSampleTime. + app := head.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("lbl", "ok"), firstSampleTime, 1) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 1, int(head.NumSeries())) + } + + // Append a sample in a time range that is not covered by the chunk range, + // We would create series first and then append no sample. + app := head.Appender(context.Background()) + invalidSampleTime := firstSampleTime - chunkRange + _, err := app.Append(0, labels.FromStrings("foo", "bar"), invalidSampleTime, 2) + require.Error(t, err) + // These are our assumptions: we're not testing them, we're just checking them to make debugging a failed + // test easier if someone refactors the code and breaks these assumptions. + // If these assumptions fail after a refactor, feel free to remove them but make sure that the test is still what we intended to test. + require.NotErrorIs(t, err, storage.ErrOutOfBounds, "Failed to append sample shouldn't take the shortcut that returns storage.ErrOutOfBounds") + require.ErrorIs(t, err, storage.ErrTooOldSample, "Failed to append sample should return storage.ErrTooOldSample, because OOO window was enabled but this sample doesn't fall into it.") + // Do commit or rollback, depending on what we're testing. + require.NoError(t, finishTxn(app)) + + // Garbage-collect, since we finished the transaction and series has no samples, it should be collectable. + head.gc() + require.Equal(t, 1, int(head.NumSeries())) + }) + } +} + +func TestHead_UnknownWALRecord(t *testing.T) { + head, w := newTestHead(t, 1000, compression.None, false) + w.Log([]byte{255, 42}) + require.NoError(t, head.Init(0)) + require.NoError(t, head.Close()) +} + +// BenchmarkHead_Truncate is quite heavy, so consider running it with +// -benchtime=10x or similar to get more stable and comparable results. +func BenchmarkHead_Truncate(b *testing.B) { + const total = 1e6 + + prepare := func(b *testing.B, churn int) *Head { + h, _ := newTestHead(b, 1000, compression.None, false) + b.Cleanup(func() { + require.NoError(b, h.Close()) + }) + + h.initTime(0) + + internedItoa := map[int]string{} + var mtx sync.RWMutex + itoa := func(i int) string { + mtx.RLock() + s, ok := internedItoa[i] + mtx.RUnlock() + if ok { + return s + } + mtx.Lock() + s = strconv.Itoa(i) + internedItoa[i] = s + mtx.Unlock() + return s + } + + allSeries := [total]labels.Labels{} + nameValues := make([]string, 0, 100) + for i := range int(total) { + nameValues = nameValues[:0] + + // A thousand labels like lbl_x_of_1000, each with total/1000 values + thousand := "lbl_" + itoa(i%1000) + "_of_1000" + nameValues = append(nameValues, thousand, itoa(i/1000)) + // A hundred labels like lbl_x_of_100, each with total/100 values. + hundred := "lbl_" + itoa(i%100) + "_of_100" + nameValues = append(nameValues, hundred, itoa(i/100)) + + if i%13 == 0 { + ten := "lbl_" + itoa(i%10) + "_of_10" + nameValues = append(nameValues, ten, itoa(i%10)) + } + + allSeries[i] = labels.FromStrings(append(nameValues, "first", "a", "second", "a", "third", "a")...) + s, _, _ := h.getOrCreate(allSeries[i].Hash(), allSeries[i], false) + s.mmappedChunks = []*mmappedChunk{ + {minTime: 1000 * int64(i/churn), maxTime: 999 + 1000*int64(i/churn)}, + } + } + + return h + } + + for _, churn := range []int{10, 100, 1000} { + b.Run(fmt.Sprintf("churn=%d", churn), func(b *testing.B) { + if b.N > total/churn { + // Just to make sure that benchmark still makes sense. + panic("benchmark not prepared") + } + h := prepare(b, churn) + b.ResetTimer() + + for i := 0; b.Loop(); i++ { + require.NoError(b, h.Truncate(1000*int64(i))) + // Make sure the benchmark is meaningful and it's actually truncating the expected amount of series. + require.Equal(b, total-churn*i, int(h.NumSeries())) + } + }) + } +} + +func TestHead_Truncate(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + h.initTime(0) + + ctx := context.Background() + + s1, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1", "b", "1"), false) + s2, _, _ := h.getOrCreate(2, labels.FromStrings("a", "2", "b", "1"), false) + s3, _, _ := h.getOrCreate(3, labels.FromStrings("a", "1", "b", "2"), false) + s4, _, _ := h.getOrCreate(4, labels.FromStrings("a", "2", "b", "2", "c", "1"), false) + + s1.mmappedChunks = []*mmappedChunk{ + {minTime: 0, maxTime: 999}, + {minTime: 1000, maxTime: 1999}, + {minTime: 2000, maxTime: 2999}, + } + s2.mmappedChunks = []*mmappedChunk{ + {minTime: 1000, maxTime: 1999}, + {minTime: 2000, maxTime: 2999}, + {minTime: 3000, maxTime: 3999}, + } + s3.mmappedChunks = []*mmappedChunk{ + {minTime: 0, maxTime: 999}, + {minTime: 1000, maxTime: 1999}, + } + s4.mmappedChunks = []*mmappedChunk{} + + // Truncation need not be aligned. + require.NoError(t, h.Truncate(1)) + + require.NoError(t, h.Truncate(2000)) + + require.Equal(t, []*mmappedChunk{ + {minTime: 2000, maxTime: 2999}, + }, h.series.getByID(s1.ref).mmappedChunks) + + require.Equal(t, []*mmappedChunk{ + {minTime: 2000, maxTime: 2999}, + {minTime: 3000, maxTime: 3999}, + }, h.series.getByID(s2.ref).mmappedChunks) + + require.Nil(t, h.series.getByID(s3.ref)) + require.Nil(t, h.series.getByID(s4.ref)) + + postingsA1, _ := index.ExpandPostings(h.postings.Postings(ctx, "a", "1")) + postingsA2, _ := index.ExpandPostings(h.postings.Postings(ctx, "a", "2")) + postingsB1, _ := index.ExpandPostings(h.postings.Postings(ctx, "b", "1")) + postingsB2, _ := index.ExpandPostings(h.postings.Postings(ctx, "b", "2")) + postingsC1, _ := index.ExpandPostings(h.postings.Postings(ctx, "c", "1")) + postingsAll, _ := index.ExpandPostings(h.postings.Postings(ctx, "", "")) + + require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s1.ref)}, postingsA1) + require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s2.ref)}, postingsA2) + require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s1.ref), storage.SeriesRef(s2.ref)}, postingsB1) + require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s1.ref), storage.SeriesRef(s2.ref)}, postingsAll) + require.Nil(t, postingsB2) + require.Nil(t, postingsC1) + + iter := h.postings.Symbols() + symbols := []string{} + for iter.Next() { + symbols = append(symbols, iter.At()) + } + require.Equal(t, + []string{"" /* from 'all' postings list */, "1", "2", "a", "b"}, + symbols) + + values := map[string]map[string]struct{}{} + for _, name := range h.postings.LabelNames() { + ss, ok := values[name] + if !ok { + ss = map[string]struct{}{} + values[name] = ss + } + for _, value := range h.postings.LabelValues(ctx, name, nil) { + ss[value] = struct{}{} + } + } + require.Equal(t, map[string]map[string]struct{}{ + "a": {"1": struct{}{}, "2": struct{}{}}, + "b": {"1": struct{}{}}, + }, values) +} + +// Validate various behaviors brought on by firstChunkID accounting for +// garbage collected chunks. +func TestMemSeries_truncateChunks(t *testing.T) { + dir := t.TempDir() + // This is usually taken from the Head, but passing manually here. + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(t, err) + defer func() { + require.NoError(t, chunkDiskMapper.Close()) + }() + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: 2000, + samplesPerChunk: DefaultSamplesPerChunk, + } + + memChunkPool := sync.Pool{ + New: func() any { + return &memChunk{} + }, + } + + s := newMemSeries(labels.FromStrings("a", "b"), 1, 0, defaultIsolationDisabled, false) + + for i := 0; i < 4000; i += 5 { + ok, _ := s.append(int64(i), float64(i), 0, cOpts) + require.True(t, ok, "sample append failed") + } + s.mmapChunks(chunkDiskMapper) + + // Check that truncate removes half of the chunks and afterwards + // that the ID of the last chunk still gives us the same chunk afterwards. + countBefore := len(s.mmappedChunks) + 1 // +1 for the head chunk. + lastID := s.headChunkID(countBefore - 1) + lastChunk, _, _, err := s.chunk(lastID, chunkDiskMapper, &memChunkPool) + require.NoError(t, err) + require.NotNil(t, lastChunk) + + chk, _, _, err := s.chunk(0, chunkDiskMapper, &memChunkPool) + require.NotNil(t, chk) + require.NoError(t, err) + + s.truncateChunksBefore(2000, 0) + + require.Equal(t, int64(2000), s.mmappedChunks[0].minTime) + _, _, _, err = s.chunk(0, chunkDiskMapper, &memChunkPool) + require.Equal(t, storage.ErrNotFound, err, "first chunks not gone") + require.Equal(t, countBefore/2, len(s.mmappedChunks)+1) // +1 for the head chunk. + chk, _, _, err = s.chunk(lastID, chunkDiskMapper, &memChunkPool) + require.NoError(t, err) + require.Equal(t, lastChunk, chk) +} + +func TestMemSeries_truncateChunks_scenarios(t *testing.T) { + const chunkRange = 100 + const chunkStep = 5 + + tests := []struct { + name string + headChunks int // the number of head chunks to create on memSeries by appending enough samples + mmappedChunks int // the number of mmapped chunks to create on memSeries by appending enough samples + truncateBefore int64 // the mint to pass to truncateChunksBefore() + expectedTruncated int // the number of chunks that we're expecting be truncated and returned by truncateChunksBefore() + expectedHead int // the expected number of head chunks after truncation + expectedMmap int // the expected number of mmapped chunks after truncation + expectedFirstChunkID chunks.HeadChunkID // the expected series.firstChunkID after truncation + }{ + { + name: "empty memSeries", + truncateBefore: chunkRange * 10, + }, + { + name: "single head chunk, not truncated", + headChunks: 1, + expectedHead: 1, + }, + { + name: "single head chunk, truncated", + headChunks: 1, + truncateBefore: chunkRange, + expectedTruncated: 1, + expectedHead: 0, + expectedFirstChunkID: 1, + }, + { + name: "2 head chunks, not truncated", + headChunks: 2, + expectedHead: 2, + }, + { + name: "2 head chunks, first truncated", + headChunks: 2, + truncateBefore: chunkRange, + expectedTruncated: 1, + expectedHead: 1, + expectedFirstChunkID: 1, + }, + { + name: "2 head chunks, everything truncated", + headChunks: 2, + truncateBefore: chunkRange * 2, + expectedTruncated: 2, + expectedHead: 0, + expectedFirstChunkID: 2, + }, + { + name: "no head chunks, 3 mmap chunks, second mmap truncated", + headChunks: 0, + mmappedChunks: 3, + truncateBefore: chunkRange * 2, + expectedTruncated: 2, + expectedHead: 0, + expectedMmap: 1, + expectedFirstChunkID: 2, + }, + { + name: "single head chunk, single mmap chunk, not truncated", + headChunks: 1, + mmappedChunks: 1, + expectedHead: 1, + expectedMmap: 1, + }, + { + name: "single head chunk, single mmap chunk, mmap truncated", + headChunks: 1, + mmappedChunks: 1, + truncateBefore: chunkRange, + expectedTruncated: 1, + expectedHead: 1, + expectedMmap: 0, + expectedFirstChunkID: 1, + }, + { + name: "5 head chunk, 5 mmap chunk, third head truncated", + headChunks: 5, + mmappedChunks: 5, + truncateBefore: chunkRange * 7, + expectedTruncated: 7, + expectedHead: 3, + expectedMmap: 0, + expectedFirstChunkID: 7, + }, + { + name: "2 head chunks, 3 mmap chunks, second mmap truncated", + headChunks: 2, + mmappedChunks: 3, + truncateBefore: chunkRange * 2, + expectedTruncated: 2, + expectedHead: 2, + expectedMmap: 1, + expectedFirstChunkID: 2, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + dir := t.TempDir() + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(t, err) + defer func() { + require.NoError(t, chunkDiskMapper.Close()) + }() + + series := newMemSeries(labels.EmptyLabels(), 1, 0, true, false) + + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: chunkRange, + samplesPerChunk: DefaultSamplesPerChunk, + } + + var headStart int + if tc.mmappedChunks > 0 { + headStart = (tc.mmappedChunks + 1) * chunkRange + for i := 0; i < (tc.mmappedChunks+1)*chunkRange; i += chunkStep { + ok, _ := series.append(int64(i), float64(i), 0, cOpts) + require.True(t, ok, "sample append failed") + } + series.mmapChunks(chunkDiskMapper) + } + + if tc.headChunks == 0 { + series.headChunks = nil + } else { + for i := headStart; i < chunkRange*(tc.mmappedChunks+tc.headChunks); i += chunkStep { + ok, _ := series.append(int64(i), float64(i), 0, cOpts) + require.True(t, ok, "sample append failed: %d", i) + } + } + + if tc.headChunks > 0 { + require.NotNil(t, series.headChunks, "head chunk is missing") + require.Equal(t, tc.headChunks, series.headChunks.len(), "wrong number of head chunks") + } else { + require.Nil(t, series.headChunks, "head chunk is present") + } + require.Len(t, series.mmappedChunks, tc.mmappedChunks, "wrong number of mmapped chunks") + + truncated := series.truncateChunksBefore(tc.truncateBefore, 0) + require.Equal(t, tc.expectedTruncated, truncated, "wrong number of truncated chunks returned") + + require.Len(t, series.mmappedChunks, tc.expectedMmap, "wrong number of mmappedChunks after truncation") + + if tc.expectedHead > 0 { + require.NotNil(t, series.headChunks, "headChunks should is nil after truncation") + require.Equal(t, tc.expectedHead, series.headChunks.len(), "wrong number of head chunks after truncation") + require.Nil(t, series.headChunks.oldest().prev, "last head chunk cannot have any next chunk set") + } else { + require.Nil(t, series.headChunks, "headChunks should is non-nil after truncation") + } + + if series.headChunks != nil || len(series.mmappedChunks) > 0 { + require.GreaterOrEqual(t, series.maxTime(), tc.truncateBefore, "wrong value of series.maxTime() after truncation") + } else { + require.Equal(t, int64(math.MinInt64), series.maxTime(), "wrong value of series.maxTime() after truncation") + } + + require.Equal(t, tc.expectedFirstChunkID, series.firstChunkID, "wrong firstChunkID after truncation") + }) + } +} + +func TestHeadDeleteSeriesWithoutSamples(t *testing.T) { + for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { + t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { + entries := []any{ + []record.RefSeries{ + {Ref: 10, Labels: labels.FromStrings("a", "1")}, + }, + []record.RefSample{}, + []record.RefSeries{ + {Ref: 50, Labels: labels.FromStrings("a", "2")}, + }, + []record.RefSample{ + {Ref: 50, T: 80, V: 1}, + {Ref: 50, T: 90, V: 1}, + }, + } + head, w := newTestHead(t, 1000, compress, false) + defer func() { + require.NoError(t, head.Close()) + }() + + populateTestWL(t, w, entries, nil) + + require.NoError(t, head.Init(math.MinInt64)) + + require.NoError(t, head.Delete(context.Background(), 0, 100, labels.MustNewMatcher(labels.MatchEqual, "a", "1"))) + }) + } +} + +func TestHeadDeleteSimple(t *testing.T) { + buildSmpls := func(s []int64) []sample { + ss := make([]sample, 0, len(s)) + for _, t := range s { + ss = append(ss, sample{t: t, f: float64(t)}) + } + return ss + } + smplsAll := buildSmpls([]int64{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}) + lblDefault := labels.Label{Name: "a", Value: "b"} + lblsDefault := labels.FromStrings("a", "b") + + cases := []struct { + dranges tombstones.Intervals + addSamples []sample // Samples to add after delete. + smplsExp []sample + }{ + { + dranges: tombstones.Intervals{{Mint: 0, Maxt: 3}}, + smplsExp: buildSmpls([]int64{4, 5, 6, 7, 8, 9}), + }, + { + dranges: tombstones.Intervals{{Mint: 1, Maxt: 3}}, + smplsExp: buildSmpls([]int64{0, 4, 5, 6, 7, 8, 9}), + }, + { + dranges: tombstones.Intervals{{Mint: 1, Maxt: 3}, {Mint: 4, Maxt: 7}}, + smplsExp: buildSmpls([]int64{0, 8, 9}), + }, + { + dranges: tombstones.Intervals{{Mint: 1, Maxt: 3}, {Mint: 4, Maxt: 700}}, + smplsExp: buildSmpls([]int64{0}), + }, + { // This case is to ensure that labels and symbols are deleted. + dranges: tombstones.Intervals{{Mint: 0, Maxt: 9}}, + smplsExp: buildSmpls([]int64{}), + }, + { + dranges: tombstones.Intervals{{Mint: 1, Maxt: 3}}, + addSamples: buildSmpls([]int64{11, 13, 15}), + smplsExp: buildSmpls([]int64{0, 4, 5, 6, 7, 8, 9, 11, 13, 15}), + }, + { + // After delete, the appended samples in the deleted range should be visible + // as the tombstones are clamped to head min/max time. + dranges: tombstones.Intervals{{Mint: 7, Maxt: 20}}, + addSamples: buildSmpls([]int64{11, 13, 15}), + smplsExp: buildSmpls([]int64{0, 1, 2, 3, 4, 5, 6, 11, 13, 15}), + }, + } + + for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { + t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { + for _, c := range cases { + head, w := newTestHead(t, 1000, compress, false) + require.NoError(t, head.Init(0)) + + app := head.Appender(context.Background()) + for _, smpl := range smplsAll { + _, err := app.Append(0, lblsDefault, smpl.t, smpl.f) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Delete the ranges. + for _, r := range c.dranges { + require.NoError(t, head.Delete(context.Background(), r.Mint, r.Maxt, labels.MustNewMatcher(labels.MatchEqual, lblDefault.Name, lblDefault.Value))) + } + + // Add more samples. + app = head.Appender(context.Background()) + for _, smpl := range c.addSamples { + _, err := app.Append(0, lblsDefault, smpl.t, smpl.f) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Compare the samples for both heads - before and after the reloadBlocks. + reloadedW, err := wlog.New(nil, nil, w.Dir(), compress) // Use a new wal to ensure deleted samples are gone even after a reloadBlocks. + require.NoError(t, err) + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = reloadedW.Dir() + reloadedHead, err := NewHead(nil, nil, reloadedW, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, reloadedHead.Init(0)) + + // Compare the query results for both heads - before and after the reloadBlocks. + Outer: + for _, h := range []*Head{head, reloadedHead} { + q, err := NewBlockQuerier(h, h.MinTime(), h.MaxTime()) + require.NoError(t, err) + actSeriesSet := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, lblDefault.Name, lblDefault.Value)) + require.NoError(t, q.Close()) + expSeriesSet := newMockSeriesSet([]storage.Series{ + storage.NewListSeries(lblsDefault, func() []chunks.Sample { + ss := make([]chunks.Sample, 0, len(c.smplsExp)) + for _, s := range c.smplsExp { + ss = append(ss, s) + } + return ss + }(), + ), + }) + + for { + eok, rok := expSeriesSet.Next(), actSeriesSet.Next() + require.Equal(t, eok, rok) + + if !eok { + require.NoError(t, h.Close()) + require.NoError(t, actSeriesSet.Err()) + require.Empty(t, actSeriesSet.Warnings()) + continue Outer + } + expSeries := expSeriesSet.At() + actSeries := actSeriesSet.At() + + require.Equal(t, expSeries.Labels(), actSeries.Labels()) + + smplExp, errExp := storage.ExpandSamples(expSeries.Iterator(nil), nil) + smplRes, errRes := storage.ExpandSamples(actSeries.Iterator(nil), nil) + + require.Equal(t, errExp, errRes) + require.Equal(t, smplExp, smplRes) + } + } + } + }) + } +} + +func TestDeleteUntilCurMax(t *testing.T) { + hb, _ := newTestHead(t, 1000000, compression.None, false) + defer func() { + require.NoError(t, hb.Close()) + }() + + numSamples := int64(10) + app := hb.Appender(context.Background()) + smpls := make([]float64, numSamples) + for i := range numSamples { + smpls[i] = rand.Float64() + _, err := app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + require.NoError(t, hb.Delete(context.Background(), 0, 10000, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) + + // Test the series returns no samples. The series is cleared only after compaction. + q, err := NewBlockQuerier(hb, 0, 100000) + require.NoError(t, err) + res := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.True(t, res.Next(), "series is not present") + s := res.At() + it := s.Iterator(nil) + require.Equal(t, chunkenc.ValNone, it.Next(), "expected no samples") + for res.Next() { + } + require.NoError(t, res.Err()) + require.Empty(t, res.Warnings()) + + // Add again and test for presence. + app = hb.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("a", "b"), 11, 1) + require.NoError(t, err) + require.NoError(t, app.Commit()) + q, err = NewBlockQuerier(hb, 0, 100000) + require.NoError(t, err) + res = q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.True(t, res.Next(), "series don't exist") + exps := res.At() + it = exps.Iterator(nil) + resSamples, err := storage.ExpandSamples(it, newSample) + require.NoError(t, err) + require.Equal(t, []chunks.Sample{sample{11, 1, nil, nil}}, resSamples) + for res.Next() { + } + require.NoError(t, res.Err()) + require.Empty(t, res.Warnings()) +} + +func TestDeletedSamplesAndSeriesStillInWALAfterCheckpoint(t *testing.T) { + numSamples := 10000 + + // Enough samples to cause a checkpoint. + hb, w := newTestHead(t, int64(numSamples)*10, compression.None, false) + + for i := range numSamples { + app := hb.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), int64(i), 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + require.NoError(t, hb.Delete(context.Background(), 0, int64(numSamples), labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) + require.NoError(t, hb.Truncate(1)) + require.NoError(t, hb.Close()) + + // Confirm there's been a checkpoint. + cdir, _, err := wlog.LastCheckpoint(w.Dir()) + require.NoError(t, err) + // Read in checkpoint and WAL. + recs := readTestWAL(t, cdir) + recs = append(recs, readTestWAL(t, w.Dir())...) + + var series, samples, stones, metadata int + for _, rec := range recs { + switch rec.(type) { + case []record.RefSeries: + series++ + case []record.RefSample: + samples++ + case []tombstones.Stone: + stones++ + case []record.RefMetadata: + metadata++ + default: + require.Fail(t, "unknown record type") + } + } + require.Equal(t, 1, series) + require.Equal(t, 9999, samples) + require.Equal(t, 1, stones) + require.Equal(t, 0, metadata) +} + +func TestDelete_e2e(t *testing.T) { + numDatapoints := 1000 + numRanges := 1000 + timeInterval := int64(2) + // Create 8 series with 1000 data-points of different ranges, delete and run queries. + lbls := [][]labels.Label{ + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + } + seriesMap := map[string][]chunks.Sample{} + for _, l := range lbls { + seriesMap[labels.New(l...).String()] = []chunks.Sample{} + } + + hb, _ := newTestHead(t, 100000, compression.None, false) + defer func() { + require.NoError(t, hb.Close()) + }() + + app := hb.Appender(context.Background()) + for _, l := range lbls { + ls := labels.New(l...) + series := []chunks.Sample{} + ts := rand.Int63n(300) + for range numDatapoints { + v := rand.Float64() + _, err := app.Append(0, ls, ts, v) + require.NoError(t, err) + series = append(series, sample{ts, v, nil, nil}) + ts += rand.Int63n(timeInterval) + 1 + } + seriesMap[labels.New(l...).String()] = series + } + require.NoError(t, app.Commit()) + // Delete a time-range from each-selector. + dels := []struct { + ms []*labels.Matcher + drange tombstones.Intervals + }{ + { + ms: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "b")}, + drange: tombstones.Intervals{{Mint: 300, Maxt: 500}, {Mint: 600, Maxt: 670}}, + }, + { + ms: []*labels.Matcher{ + labels.MustNewMatcher(labels.MatchEqual, "a", "b"), + labels.MustNewMatcher(labels.MatchEqual, "job", "prom-k8s"), + }, + drange: tombstones.Intervals{{Mint: 300, Maxt: 500}, {Mint: 100, Maxt: 670}}, + }, + { + ms: []*labels.Matcher{ + labels.MustNewMatcher(labels.MatchEqual, "a", "c"), + labels.MustNewMatcher(labels.MatchEqual, "instance", "localhost:9090"), + labels.MustNewMatcher(labels.MatchEqual, "job", "prometheus"), + }, + drange: tombstones.Intervals{{Mint: 300, Maxt: 400}, {Mint: 100, Maxt: 6700}}, + }, + // TODO: Add Regexp Matchers. + } + for _, del := range dels { + for _, r := range del.drange { + require.NoError(t, hb.Delete(context.Background(), r.Mint, r.Maxt, del.ms...)) + } + matched := labels.Slice{} + for _, l := range lbls { + s := labels.Selector(del.ms) + ls := labels.New(l...) + if s.Matches(ls) { + matched = append(matched, ls) + } + } + sort.Sort(matched) + for range numRanges { + q, err := NewBlockQuerier(hb, 0, 100000) + require.NoError(t, err) + ss := q.Select(context.Background(), true, nil, del.ms...) + // Build the mockSeriesSet. + matchedSeries := make([]storage.Series, 0, len(matched)) + for _, m := range matched { + smpls := seriesMap[m.String()] + smpls = deletedSamples(smpls, del.drange) + // Only append those series for which samples exist as mockSeriesSet + // doesn't skip series with no samples. + // TODO: But sometimes SeriesSet returns an empty chunkenc.Iterator + if len(smpls) > 0 { + matchedSeries = append(matchedSeries, storage.NewListSeries(m, smpls)) + } + } + expSs := newMockSeriesSet(matchedSeries) + // Compare both SeriesSets. + for { + eok, rok := expSs.Next(), ss.Next() + // Skip a series if iterator is empty. + if rok { + for ss.At().Iterator(nil).Next() == chunkenc.ValNone { + rok = ss.Next() + if !rok { + break + } + } + } + require.Equal(t, eok, rok) + if !eok { + break + } + sexp := expSs.At() + sres := ss.At() + require.Equal(t, sexp.Labels(), sres.Labels()) + smplExp, errExp := storage.ExpandSamples(sexp.Iterator(nil), nil) + smplRes, errRes := storage.ExpandSamples(sres.Iterator(nil), nil) + require.Equal(t, errExp, errRes) + require.Equal(t, smplExp, smplRes) + } + require.NoError(t, ss.Err()) + require.Empty(t, ss.Warnings()) + require.NoError(t, q.Close()) + } + } +} + +func boundedSamples(full []chunks.Sample, mint, maxt int64) []chunks.Sample { + for len(full) > 0 { + if full[0].T() >= mint { + break + } + full = full[1:] + } + for i, s := range full { + // labels.Labelinate on the first sample larger than maxt. + if s.T() > maxt { + return full[:i] + } + } + // maxt is after highest sample. + return full +} + +func deletedSamples(full []chunks.Sample, dranges tombstones.Intervals) []chunks.Sample { + ds := make([]chunks.Sample, 0, len(full)) +Outer: + for _, s := range full { + for _, r := range dranges { + if r.InBounds(s.T()) { + continue Outer + } + } + ds = append(ds, s) + } + + return ds +} + +func TestComputeChunkEndTime(t *testing.T) { + cases := map[string]struct { + start, cur, max int64 + ratioToFull float64 + res int64 + }{ + "exactly 1/4 full, even increment": { + start: 0, + cur: 250, + max: 1000, + ratioToFull: 4, + res: 1000, + }, + "exactly 1/4 full, uneven increment": { + start: 100, + cur: 200, + max: 1000, + ratioToFull: 4, + res: 550, + }, + "decimal ratio to full": { + start: 5000, + cur: 5110, + max: 10000, + ratioToFull: 4.2, + res: 5500, + }, + // Case where we fit floored 0 chunks. Must catch division by 0 + // and default to maximum time. + "fit floored 0 chunks": { + start: 0, + cur: 500, + max: 1000, + ratioToFull: 4, + res: 1000, + }, + // Catch division by zero for cur == start. Strictly not a possible case. + "cur == start": { + start: 100, + cur: 100, + max: 1000, + ratioToFull: 4, + res: 104, + }, + } + + for testName, tc := range cases { + t.Run(testName, func(t *testing.T) { + got := computeChunkEndTime(tc.start, tc.cur, tc.max, tc.ratioToFull) + require.Equal(t, tc.res, got, "(start: %d, cur: %d, max: %d)", tc.start, tc.cur, tc.max) + }) + } +} + +func TestMemSeries_append(t *testing.T) { + dir := t.TempDir() + // This is usually taken from the Head, but passing manually here. + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(t, err) + defer func() { + require.NoError(t, chunkDiskMapper.Close()) + }() + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: 500, + samplesPerChunk: DefaultSamplesPerChunk, + } + + s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) + + // Add first two samples at the very end of a chunk range and the next two + // on and after it. + // New chunk must correctly be cut at 1000. + ok, chunkCreated := s.append(998, 1, 0, cOpts) + require.True(t, ok, "append failed") + require.True(t, chunkCreated, "first sample created chunk") + + ok, chunkCreated = s.append(999, 2, 0, cOpts) + require.True(t, ok, "append failed") + require.False(t, chunkCreated, "second sample should use same chunk") + s.mmapChunks(chunkDiskMapper) + + ok, chunkCreated = s.append(1000, 3, 0, cOpts) + require.True(t, ok, "append failed") + require.True(t, chunkCreated, "expected new chunk on boundary") + + ok, chunkCreated = s.append(1001, 4, 0, cOpts) + require.True(t, ok, "append failed") + require.False(t, chunkCreated, "second sample should use same chunk") + + s.mmapChunks(chunkDiskMapper) + require.Len(t, s.mmappedChunks, 1, "there should be only 1 mmapped chunk") + require.Equal(t, int64(998), s.mmappedChunks[0].minTime, "wrong chunk range") + require.Equal(t, int64(999), s.mmappedChunks[0].maxTime, "wrong chunk range") + require.Equal(t, int64(1000), s.headChunks.minTime, "wrong chunk range") + require.Equal(t, int64(1001), s.headChunks.maxTime, "wrong chunk range") + + // Fill the range [1000,2000) with many samples. Intermediate chunks should be cut + // at approximately 120 samples per chunk. + for i := 1; i < 1000; i++ { + ok, _ := s.append(1001+int64(i), float64(i), 0, cOpts) + require.True(t, ok, "append failed") + } + s.mmapChunks(chunkDiskMapper) + + require.Greater(t, len(s.mmappedChunks)+1, 7, "expected intermediate chunks") + + // All chunks but the first and last should now be moderately full. + for i, c := range s.mmappedChunks[1:] { + chk, err := chunkDiskMapper.Chunk(c.ref) + require.NoError(t, err) + require.Greater(t, chk.NumSamples(), 100, "unexpected small chunk %d of length %d", i, chk.NumSamples()) + } +} + +func TestMemSeries_appendHistogram(t *testing.T) { + dir := t.TempDir() + // This is usually taken from the Head, but passing manually here. + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(t, err) + defer func() { + require.NoError(t, chunkDiskMapper.Close()) + }() + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: int64(1000), + samplesPerChunk: DefaultSamplesPerChunk, + } + + s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) + + histograms := tsdbutil.GenerateTestHistograms(4) + histogramWithOneMoreBucket := histograms[3].Copy() + histogramWithOneMoreBucket.Count++ + histogramWithOneMoreBucket.Sum += 1.23 + histogramWithOneMoreBucket.PositiveSpans[1].Length = 3 + histogramWithOneMoreBucket.PositiveBuckets = append(histogramWithOneMoreBucket.PositiveBuckets, 1) + + // Add first two samples at the very end of a chunk range and the next two + // on and after it. + // New chunk must correctly be cut at 1000. + ok, chunkCreated := s.appendHistogram(998, histograms[0], 0, cOpts) + require.True(t, ok, "append failed") + require.True(t, chunkCreated, "first sample created chunk") + + ok, chunkCreated = s.appendHistogram(999, histograms[1], 0, cOpts) + require.True(t, ok, "append failed") + require.False(t, chunkCreated, "second sample should use same chunk") + + ok, chunkCreated = s.appendHistogram(1000, histograms[2], 0, cOpts) + require.True(t, ok, "append failed") + require.True(t, chunkCreated, "expected new chunk on boundary") + + ok, chunkCreated = s.appendHistogram(1001, histograms[3], 0, cOpts) + require.True(t, ok, "append failed") + require.False(t, chunkCreated, "second sample should use same chunk") + + s.mmapChunks(chunkDiskMapper) + require.Len(t, s.mmappedChunks, 1, "there should be only 1 mmapped chunk") + require.Equal(t, int64(998), s.mmappedChunks[0].minTime, "wrong chunk range") + require.Equal(t, int64(999), s.mmappedChunks[0].maxTime, "wrong chunk range") + require.Equal(t, int64(1000), s.headChunks.minTime, "wrong chunk range") + require.Equal(t, int64(1001), s.headChunks.maxTime, "wrong chunk range") + + ok, chunkCreated = s.appendHistogram(1002, histogramWithOneMoreBucket, 0, cOpts) + require.True(t, ok, "append failed") + require.False(t, chunkCreated, "third sample should trigger a re-encoded chunk") + + s.mmapChunks(chunkDiskMapper) + require.Len(t, s.mmappedChunks, 1, "there should be only 1 mmapped chunk") + require.Equal(t, int64(998), s.mmappedChunks[0].minTime, "wrong chunk range") + require.Equal(t, int64(999), s.mmappedChunks[0].maxTime, "wrong chunk range") + require.Equal(t, int64(1000), s.headChunks.minTime, "wrong chunk range") + require.Equal(t, int64(1002), s.headChunks.maxTime, "wrong chunk range") +} + +func TestMemSeries_append_atVariableRate(t *testing.T) { + const samplesPerChunk = 120 + dir := t.TempDir() + // This is usually taken from the Head, but passing manually here. + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, chunkDiskMapper.Close()) + }) + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: DefaultBlockDuration, + samplesPerChunk: samplesPerChunk, + } + + s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) + + // At this slow rate, we will fill the chunk in two block durations. + slowRate := (DefaultBlockDuration * 2) / samplesPerChunk + + var nextTs int64 + var totalAppendedSamples int + for i := range samplesPerChunk / 4 { + ok, _ := s.append(nextTs, float64(i), 0, cOpts) + require.Truef(t, ok, "slow sample %d was not appended", i) + nextTs += slowRate + totalAppendedSamples++ + } + require.Equal(t, DefaultBlockDuration, s.nextAt, "after appending a samplesPerChunk/4 samples at a slow rate, we should aim to cut a new block at the default block duration %d, but it's set to %d", DefaultBlockDuration, s.nextAt) + + // Suddenly, the rate increases and we receive a sample every millisecond. + for i := range math.MaxUint16 { + ok, _ := s.append(nextTs, float64(i), 0, cOpts) + require.Truef(t, ok, "quick sample %d was not appended", i) + nextTs++ + totalAppendedSamples++ + } + ok, chunkCreated := s.append(DefaultBlockDuration, float64(0), 0, cOpts) + require.True(t, ok, "new chunk sample was not appended") + require.True(t, chunkCreated, "sample at block duration timestamp should create a new chunk") + + s.mmapChunks(chunkDiskMapper) + var totalSamplesInChunks int + for i, c := range s.mmappedChunks { + totalSamplesInChunks += int(c.numSamples) + require.LessOrEqualf(t, c.numSamples, uint16(2*samplesPerChunk), "mmapped chunk %d has more than %d samples", i, 2*samplesPerChunk) + } + require.Equal(t, totalAppendedSamples, totalSamplesInChunks, "wrong number of samples in %d mmapped chunks", len(s.mmappedChunks)) +} + +func TestGCChunkAccess(t *testing.T) { + // Put a chunk, select it. GC it and then access it. + const chunkRange = 1000 + h, _ := newTestHead(t, chunkRange, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + cOpts := chunkOpts{ + chunkDiskMapper: h.chunkDiskMapper, + chunkRange: chunkRange, + samplesPerChunk: DefaultSamplesPerChunk, + } + + h.initTime(0) + + s, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) + + // Appending 2 samples for the first chunk. + ok, chunkCreated := s.append(0, 0, 0, cOpts) + require.True(t, ok, "series append failed") + require.True(t, chunkCreated, "chunks was not created") + ok, chunkCreated = s.append(999, 999, 0, cOpts) + require.True(t, ok, "series append failed") + require.False(t, chunkCreated, "chunks was created") + + // A new chunks should be created here as it's beyond the chunk range. + ok, chunkCreated = s.append(1000, 1000, 0, cOpts) + require.True(t, ok, "series append failed") + require.True(t, chunkCreated, "chunks was not created") + ok, chunkCreated = s.append(1999, 1999, 0, cOpts) + require.True(t, ok, "series append failed") + require.False(t, chunkCreated, "chunks was created") + + idx := h.indexRange(0, 1500) + var ( + chunks []chunks.Meta + builder labels.ScratchBuilder + ) + require.NoError(t, idx.Series(1, &builder, &chunks)) + + require.Equal(t, labels.FromStrings("a", "1"), builder.Labels()) + require.Len(t, chunks, 2) + + cr, err := h.chunksRange(0, 1500, nil) + require.NoError(t, err) + _, _, err = cr.ChunkOrIterable(chunks[0]) + require.NoError(t, err) + _, _, err = cr.ChunkOrIterable(chunks[1]) + require.NoError(t, err) + + require.NoError(t, h.Truncate(1500)) // Remove a chunk. + + _, _, err = cr.ChunkOrIterable(chunks[0]) + require.Equal(t, storage.ErrNotFound, err) + _, _, err = cr.ChunkOrIterable(chunks[1]) + require.NoError(t, err) +} + +func TestGCSeriesAccess(t *testing.T) { + // Put a series, select it. GC it and then access it. + const chunkRange = 1000 + h, _ := newTestHead(t, chunkRange, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + cOpts := chunkOpts{ + chunkDiskMapper: h.chunkDiskMapper, + chunkRange: chunkRange, + samplesPerChunk: DefaultSamplesPerChunk, + } + + h.initTime(0) + + s, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) + + // Appending 2 samples for the first chunk. + ok, chunkCreated := s.append(0, 0, 0, cOpts) + require.True(t, ok, "series append failed") + require.True(t, chunkCreated, "chunks was not created") + ok, chunkCreated = s.append(999, 999, 0, cOpts) + require.True(t, ok, "series append failed") + require.False(t, chunkCreated, "chunks was created") + + // A new chunks should be created here as it's beyond the chunk range. + ok, chunkCreated = s.append(1000, 1000, 0, cOpts) + require.True(t, ok, "series append failed") + require.True(t, chunkCreated, "chunks was not created") + ok, chunkCreated = s.append(1999, 1999, 0, cOpts) + require.True(t, ok, "series append failed") + require.False(t, chunkCreated, "chunks was created") + + idx := h.indexRange(0, 2000) + var ( + chunks []chunks.Meta + builder labels.ScratchBuilder + ) + require.NoError(t, idx.Series(1, &builder, &chunks)) + + require.Equal(t, labels.FromStrings("a", "1"), builder.Labels()) + require.Len(t, chunks, 2) + + cr, err := h.chunksRange(0, 2000, nil) + require.NoError(t, err) + _, _, err = cr.ChunkOrIterable(chunks[0]) + require.NoError(t, err) + _, _, err = cr.ChunkOrIterable(chunks[1]) + require.NoError(t, err) + + require.NoError(t, h.Truncate(2000)) // Remove the series. + + require.Equal(t, (*memSeries)(nil), h.series.getByID(1)) + + _, _, err = cr.ChunkOrIterable(chunks[0]) + require.Equal(t, storage.ErrNotFound, err) + _, _, err = cr.ChunkOrIterable(chunks[1]) + require.Equal(t, storage.ErrNotFound, err) +} + +func TestUncommittedSamplesNotLostOnTruncate(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + h.initTime(0) + + app := h.appender() + lset := labels.FromStrings("a", "1") + _, err := app.Append(0, lset, 2100, 1) + require.NoError(t, err) + + require.NoError(t, h.Truncate(2000)) + require.NotNil(t, h.series.getByHash(lset.Hash(), lset), "series should not have been garbage collected") + + require.NoError(t, app.Commit()) + + q, err := NewBlockQuerier(h, 1500, 2500) + require.NoError(t, err) + defer q.Close() + + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "1")) + require.True(t, ss.Next()) + for ss.Next() { + } + require.NoError(t, ss.Err()) + require.Empty(t, ss.Warnings()) +} + +func TestRemoveSeriesAfterRollbackAndTruncate(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + h.initTime(0) + + app := h.appender() + lset := labels.FromStrings("a", "1") + _, err := app.Append(0, lset, 2100, 1) + require.NoError(t, err) + + require.NoError(t, h.Truncate(2000)) + require.NotNil(t, h.series.getByHash(lset.Hash(), lset), "series should not have been garbage collected") + + require.NoError(t, app.Rollback()) + + q, err := NewBlockQuerier(h, 1500, 2500) + require.NoError(t, err) + + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "1")) + require.False(t, ss.Next()) + require.Empty(t, ss.Warnings()) + require.NoError(t, q.Close()) + + // Truncate again, this time the series should be deleted + require.NoError(t, h.Truncate(2050)) + require.Equal(t, (*memSeries)(nil), h.series.getByHash(lset.Hash(), lset)) +} + +func TestHead_LogRollback(t *testing.T) { + for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { + t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { + h, w := newTestHead(t, 1000, compress, false) + defer func() { + require.NoError(t, h.Close()) + }() + + app := h.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 1, 2) + require.NoError(t, err) + + require.NoError(t, app.Rollback()) + recs := readTestWAL(t, w.Dir()) + + require.Len(t, recs, 1) + + series, ok := recs[0].([]record.RefSeries) + require.True(t, ok, "expected series record but got %+v", recs[0]) + require.Equal(t, []record.RefSeries{{Ref: 1, Labels: labels.FromStrings("a", "b")}}, series) + }) + } +} + +func TestHead_ReturnsSortedLabelValues(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + h.initTime(0) + + app := h.appender() + for i := 100; i > 0; i-- { + for j := range 10 { + lset := labels.FromStrings( + "__name__", fmt.Sprintf("metric_%d", i), + "label", fmt.Sprintf("value_%d", j), + ) + _, err := app.Append(0, lset, 2100, 1) + require.NoError(t, err) + } + } + + q, err := NewBlockQuerier(h, 1500, 2500) + require.NoError(t, err) + + res, _, err := q.LabelValues(context.Background(), "__name__", nil) + require.NoError(t, err) + + require.True(t, slices.IsSorted(res)) + require.NoError(t, q.Close()) +} + +// TestWalRepair_DecodingError ensures that a repair is run for an error +// when decoding a record. +func TestWalRepair_DecodingError(t *testing.T) { + var enc record.Encoder + for name, test := range map[string]struct { + corrFunc func(rec []byte) []byte // Func that applies the corruption to a record. + rec []byte + totalRecs int + expRecs int + }{ + "decode_series": { + func(rec []byte) []byte { + return rec[:3] + }, + enc.Series([]record.RefSeries{{Ref: 1, Labels: labels.FromStrings("a", "b")}}, []byte{}), + 9, + 5, + }, + "decode_samples": { + func(rec []byte) []byte { + return rec[:3] + }, + enc.Samples([]record.RefSample{{Ref: 0, T: 99, V: 1}}, []byte{}), + 9, + 5, + }, + "decode_tombstone": { + func(rec []byte) []byte { + return rec[:3] + }, + enc.Tombstones([]tombstones.Stone{{Ref: 1, Intervals: tombstones.Intervals{}}}, []byte{}), + 9, + 5, + }, + } { + for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { + t.Run(fmt.Sprintf("%s,compress=%s", name, compress), func(t *testing.T) { + dir := t.TempDir() + + // Fill the wal and corrupt it. + { + w, err := wlog.New(nil, nil, filepath.Join(dir, "wal"), compress) + require.NoError(t, err) + + for i := 1; i <= test.totalRecs; i++ { + // At this point insert a corrupted record. + if i-1 == test.expRecs { + require.NoError(t, w.Log(test.corrFunc(test.rec))) + continue + } + require.NoError(t, w.Log(test.rec)) + } + + opts := DefaultHeadOptions() + opts.ChunkRange = 1 + opts.ChunkDirRoot = w.Dir() + h, err := NewHead(nil, nil, w, nil, opts, nil) + require.NoError(t, err) + require.Equal(t, 0.0, prom_testutil.ToFloat64(h.metrics.walCorruptionsTotal)) + initErr := h.Init(math.MinInt64) + + var cerr *wlog.CorruptionErr + require.ErrorAs(t, initErr, &cerr, "reading the wal didn't return corruption error") + require.NoError(t, h.Close()) // Head will close the wal as well. + } + + // Open the db to trigger a repair. + { + db, err := Open(dir, nil, nil, DefaultOptions(), nil) + require.NoError(t, err) + defer func() { + require.NoError(t, db.Close()) + }() + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) + } + + // Read the wal content after the repair. + { + sr, err := wlog.NewSegmentsReader(filepath.Join(dir, "wal")) + require.NoError(t, err) + defer sr.Close() + r := wlog.NewReader(sr) + + var actRec int + for r.Next() { + actRec++ + } + require.NoError(t, r.Err()) + require.Equal(t, test.expRecs, actRec, "Wrong number of intact records") + } + }) + } + } +} + +// TestWblRepair_DecodingError ensures that a repair is run for an error +// when decoding a record. +func TestWblRepair_DecodingError(t *testing.T) { + var enc record.Encoder + corrFunc := func(rec []byte) []byte { + return rec[:3] + } + rec := enc.Samples([]record.RefSample{{Ref: 0, T: 99, V: 1}}, []byte{}) + totalRecs := 9 + expRecs := 5 + dir := t.TempDir() + + // Fill the wbl and corrupt it. + { + wal, err := wlog.New(nil, nil, filepath.Join(dir, "wal"), compression.None) + require.NoError(t, err) + wbl, err := wlog.New(nil, nil, filepath.Join(dir, "wbl"), compression.None) + require.NoError(t, err) + + for i := 1; i <= totalRecs; i++ { + // At this point insert a corrupted record. + if i-1 == expRecs { + require.NoError(t, wbl.Log(corrFunc(rec))) + continue + } + require.NoError(t, wbl.Log(rec)) + } + + opts := DefaultHeadOptions() + opts.ChunkRange = 1 + opts.ChunkDirRoot = wal.Dir() + opts.OutOfOrderCapMax.Store(30) + opts.OutOfOrderTimeWindow.Store(1000 * time.Minute.Milliseconds()) + h, err := NewHead(nil, nil, wal, wbl, opts, nil) + require.NoError(t, err) + require.Equal(t, 0.0, prom_testutil.ToFloat64(h.metrics.walCorruptionsTotal)) + initErr := h.Init(math.MinInt64) + + var elb *errLoadWbl + require.ErrorAs(t, initErr, &elb) // Wbl errors are wrapped into errLoadWbl, make sure we can unwrap it. + + var cerr *wlog.CorruptionErr + require.ErrorAs(t, initErr, &cerr, "reading the wal didn't return corruption error") + require.NoError(t, h.Close()) // Head will close the wal as well. + } + + // Open the db to trigger a repair. + { + db, err := Open(dir, nil, nil, DefaultOptions(), nil) + require.NoError(t, err) + defer func() { + require.NoError(t, db.Close()) + }() + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) + } + + // Read the wbl content after the repair. + { + sr, err := wlog.NewSegmentsReader(filepath.Join(dir, "wbl")) + require.NoError(t, err) + defer sr.Close() + r := wlog.NewReader(sr) + + var actRec int + for r.Next() { + actRec++ + } + require.NoError(t, r.Err()) + require.Equal(t, expRecs, actRec, "Wrong number of intact records") + } +} + +func TestHeadReadWriterRepair(t *testing.T) { + dir := t.TempDir() + + const chunkRange = 1000 + + walDir := filepath.Join(dir, "wal") + // Fill the chunk segments and corrupt it. + { + w, err := wlog.New(nil, nil, walDir, compression.None) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkRange = chunkRange + opts.ChunkDirRoot = dir + opts.ChunkWriteQueueSize = 1 // We need to set this option so that we use the async queue. Upstream prometheus uses the queue directly. + h, err := NewHead(nil, nil, w, nil, opts, nil) + require.NoError(t, err) + require.Equal(t, 0.0, prom_testutil.ToFloat64(h.metrics.mmapChunkCorruptionTotal)) + require.NoError(t, h.Init(math.MinInt64)) + + cOpts := chunkOpts{ + chunkDiskMapper: h.chunkDiskMapper, + chunkRange: chunkRange, + samplesPerChunk: DefaultSamplesPerChunk, + } + + s, created, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) + require.True(t, created, "series was not created") + + for i := range 7 { + ok, chunkCreated := s.append(int64(i*chunkRange), float64(i*chunkRange), 0, cOpts) + require.True(t, ok, "series append failed") + require.True(t, chunkCreated, "chunk was not created") + ok, chunkCreated = s.append(int64(i*chunkRange)+chunkRange-1, float64(i*chunkRange), 0, cOpts) + require.True(t, ok, "series append failed") + require.False(t, chunkCreated, "chunk was created") + h.chunkDiskMapper.CutNewFile() + s.mmapChunks(h.chunkDiskMapper) + } + require.NoError(t, h.Close()) + + // Verify that there are 6 segment files. + // It should only be 6 because the last call to .CutNewFile() won't + // take effect without another chunk being written. + files, err := os.ReadDir(mmappedChunksDir(dir)) + require.NoError(t, err) + require.Len(t, files, 6) + + // Corrupt the 4th file by writing a random byte to series ref. + f, err := os.OpenFile(filepath.Join(mmappedChunksDir(dir), files[3].Name()), os.O_WRONLY, 0o666) + require.NoError(t, err) + n, err := f.WriteAt([]byte{67, 88}, chunks.HeadChunkFileHeaderSize+2) + require.NoError(t, err) + require.Equal(t, 2, n) + require.NoError(t, f.Close()) + } + + // Open the db to trigger a repair. + { + db, err := Open(dir, nil, nil, DefaultOptions(), nil) + require.NoError(t, err) + defer func() { + require.NoError(t, db.Close()) + }() + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.mmapChunkCorruptionTotal)) + } + + // Verify that there are 3 segment files after the repair. + // The segments from the corrupt segment should be removed. + { + files, err := os.ReadDir(mmappedChunksDir(dir)) + require.NoError(t, err) + require.Len(t, files, 3) + } +} + +func TestNewWalSegmentOnTruncate(t *testing.T) { + h, wal := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + add := func(ts int64) { + app := h.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), ts, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + add(0) + _, last, err := wlog.Segments(wal.Dir()) + require.NoError(t, err) + require.Equal(t, 0, last) + + add(1) + require.NoError(t, h.Truncate(1)) + _, last, err = wlog.Segments(wal.Dir()) + require.NoError(t, err) + require.Equal(t, 1, last) + + add(2) + require.NoError(t, h.Truncate(2)) + _, last, err = wlog.Segments(wal.Dir()) + require.NoError(t, err) + require.Equal(t, 2, last) +} + +func TestAddDuplicateLabelName(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + add := func(labels labels.Labels, labelName string) { + app := h.Appender(context.Background()) + _, err := app.Append(0, labels, 0, 0) + require.EqualError(t, err, fmt.Sprintf(`label name "%s" is not unique: invalid sample`, labelName)) + } + + add(labels.FromStrings("a", "c", "a", "b"), "a") + add(labels.FromStrings("a", "c", "a", "c"), "a") + add(labels.FromStrings("__name__", "up", "job", "prometheus", "le", "500", "le", "400", "unit", "s"), "le") +} + +func TestMemSeriesIsolation(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + // Put a series, select it. GC it and then access it. + lastValue := func(h *Head, maxAppendID uint64) int { + idx, err := h.Index() + + require.NoError(t, err) + + iso := h.iso.State(math.MinInt64, math.MaxInt64) + iso.maxAppendID = maxAppendID + + chunks, err := h.chunksRange(math.MinInt64, math.MaxInt64, iso) + require.NoError(t, err) + // Hm.. here direct block chunk querier might be required? + querier := blockQuerier{ + blockBaseQuerier: &blockBaseQuerier{ + index: idx, + chunks: chunks, + tombstones: tombstones.NewMemTombstones(), + + mint: 0, + maxt: 10000, + }, + } + + require.NoError(t, err) + defer querier.Close() + + ss := querier.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err := expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + + for _, series := range seriesSet { + return int(series[len(series)-1].f) + } + return -1 + } + + addSamples := func(h *Head) int { + i := 1 + for ; i <= 1000; i++ { + var app storage.Appender + // To initialize bounds. + if h.MinTime() == math.MaxInt64 { + app = &initAppender{head: h} + } else { + a := h.appender() + a.cleanupAppendIDsBelow = 0 + app = a + } + + _, err := app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + h.mmapHeadChunks() + } + return i + } + + testIsolation := func(*Head, int) { + } + + // Test isolation without restart of Head. + hb, _ := newTestHead(t, 1000, compression.None, false) + i := addSamples(hb) + testIsolation(hb, i) + + // Test simple cases in different chunks when no appendID cleanup has been performed. + require.Equal(t, 10, lastValue(hb, 10)) + require.Equal(t, 130, lastValue(hb, 130)) + require.Equal(t, 160, lastValue(hb, 160)) + require.Equal(t, 240, lastValue(hb, 240)) + require.Equal(t, 500, lastValue(hb, 500)) + require.Equal(t, 750, lastValue(hb, 750)) + require.Equal(t, 995, lastValue(hb, 995)) + require.Equal(t, 999, lastValue(hb, 999)) + + // Cleanup appendIDs below 500. + app := hb.appender() + app.cleanupAppendIDsBelow = 500 + _, err := app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + i++ + + // We should not get queries with a maxAppendID below 500 after the cleanup, + // but they only take the remaining appendIDs into account. + require.Equal(t, 499, lastValue(hb, 10)) + require.Equal(t, 499, lastValue(hb, 130)) + require.Equal(t, 499, lastValue(hb, 160)) + require.Equal(t, 499, lastValue(hb, 240)) + require.Equal(t, 500, lastValue(hb, 500)) + require.Equal(t, 995, lastValue(hb, 995)) + require.Equal(t, 999, lastValue(hb, 999)) + + // Cleanup appendIDs below 1000, which means the sample buffer is + // the only thing with appendIDs. + app = hb.appender() + app.cleanupAppendIDsBelow = 1000 + _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 999, lastValue(hb, 998)) + require.Equal(t, 999, lastValue(hb, 999)) + require.Equal(t, 1000, lastValue(hb, 1000)) + require.Equal(t, 1001, lastValue(hb, 1001)) + require.Equal(t, 1002, lastValue(hb, 1002)) + require.Equal(t, 1002, lastValue(hb, 1003)) + + i++ + // Cleanup appendIDs below 1001, but with a rollback. + app = hb.appender() + app.cleanupAppendIDsBelow = 1001 + _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + require.NoError(t, err) + require.NoError(t, app.Rollback()) + require.Equal(t, 1000, lastValue(hb, 999)) + require.Equal(t, 1000, lastValue(hb, 1000)) + require.Equal(t, 1001, lastValue(hb, 1001)) + require.Equal(t, 1002, lastValue(hb, 1002)) + require.Equal(t, 1002, lastValue(hb, 1003)) + + require.NoError(t, hb.Close()) + + // Test isolation with restart of Head. This is to verify the num samples of chunks after m-map chunk replay. + hb, w := newTestHead(t, 1000, compression.None, false) + i = addSamples(hb) + require.NoError(t, hb.Close()) + + wal, err := wlog.NewSize(nil, nil, w.Dir(), 32768, compression.None) + require.NoError(t, err) + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = wal.Dir() + hb, err = NewHead(nil, nil, wal, nil, opts, nil) + defer func() { require.NoError(t, hb.Close()) }() + require.NoError(t, err) + require.NoError(t, hb.Init(0)) + + // No appends after restarting. Hence all should return the last value. + require.Equal(t, 1000, lastValue(hb, 10)) + require.Equal(t, 1000, lastValue(hb, 130)) + require.Equal(t, 1000, lastValue(hb, 160)) + require.Equal(t, 1000, lastValue(hb, 240)) + require.Equal(t, 1000, lastValue(hb, 500)) + + // Cleanup appendIDs below 1000, which means the sample buffer is + // the only thing with appendIDs. + app = hb.appender() + _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + i++ + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, 1001, lastValue(hb, 998)) + require.Equal(t, 1001, lastValue(hb, 999)) + require.Equal(t, 1001, lastValue(hb, 1000)) + require.Equal(t, 1001, lastValue(hb, 1001)) + require.Equal(t, 1001, lastValue(hb, 1002)) + require.Equal(t, 1001, lastValue(hb, 1003)) + + // Cleanup appendIDs below 1002, but with a rollback. + app = hb.appender() + _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + require.NoError(t, err) + require.NoError(t, app.Rollback()) + require.Equal(t, 1001, lastValue(hb, 999)) + require.Equal(t, 1001, lastValue(hb, 1000)) + require.Equal(t, 1001, lastValue(hb, 1001)) + require.Equal(t, 1001, lastValue(hb, 1002)) + require.Equal(t, 1001, lastValue(hb, 1003)) +} + +func TestIsolationRollback(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + // Rollback after a failed append and test if the low watermark has progressed anyway. + hb, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, hb.Close()) + }() + + app := hb.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, uint64(1), hb.iso.lowWatermark()) + + app = hb.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 1, 1) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("foo", "bar", "foo", "baz"), 2, 2) + require.Error(t, err) + require.NoError(t, app.Rollback()) + require.Equal(t, uint64(2), hb.iso.lowWatermark()) + + app = hb.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 3, 3) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Equal(t, uint64(3), hb.iso.lowWatermark(), "Low watermark should proceed to 3 even if append #2 was rolled back.") +} + +func TestIsolationLowWatermarkMonotonous(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + hb, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, hb.Close()) + }() + + app1 := hb.Appender(context.Background()) + _, err := app1.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + require.NoError(t, err) + require.NoError(t, app1.Commit()) + require.Equal(t, uint64(1), hb.iso.lowWatermark(), "Low watermark should by 1 after 1st append.") + + app1 = hb.Appender(context.Background()) + _, err = app1.Append(0, labels.FromStrings("foo", "bar"), 1, 1) + require.NoError(t, err) + require.Equal(t, uint64(2), hb.iso.lowWatermark(), "Low watermark should be two, even if append is not committed yet.") + + app2 := hb.Appender(context.Background()) + _, err = app2.Append(0, labels.FromStrings("foo", "baz"), 1, 1) + require.NoError(t, err) + require.NoError(t, app2.Commit()) + require.Equal(t, uint64(2), hb.iso.lowWatermark(), "Low watermark should stay two because app1 is not committed yet.") + + is := hb.iso.State(math.MinInt64, math.MaxInt64) + require.Equal(t, uint64(2), hb.iso.lowWatermark(), "After simulated read (iso state retrieved), low watermark should stay at 2.") + + require.NoError(t, app1.Commit()) + require.Equal(t, uint64(2), hb.iso.lowWatermark(), "Even after app1 is committed, low watermark should stay at 2 because read is still ongoing.") + + is.Close() + require.Equal(t, uint64(3), hb.iso.lowWatermark(), "After read has finished (iso state closed), low watermark should jump to three.") +} + +func TestIsolationAppendIDZeroIsNoop(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + h.initTime(0) + + cOpts := chunkOpts{ + chunkDiskMapper: h.chunkDiskMapper, + chunkRange: h.chunkRange.Load(), + samplesPerChunk: DefaultSamplesPerChunk, + } + + s, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) + + ok, _ := s.append(0, 0, 0, cOpts) + require.True(t, ok, "Series append failed.") + require.Equal(t, 0, int(s.txs.txIDCount), "Series should not have an appendID after append with appendID=0.") +} + +func TestHeadSeriesChunkRace(t *testing.T) { + t.Parallel() + for range 100 { + testHeadSeriesChunkRace(t) + } +} + +func TestIsolationWithoutAdd(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + hb, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, hb.Close()) + }() + + app := hb.Appender(context.Background()) + require.NoError(t, app.Commit()) + + app = hb.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "baz"), 1, 1) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + require.Equal(t, hb.iso.lastAppendID(), hb.iso.lowWatermark(), "High watermark should be equal to the low watermark") +} + +func TestOutOfOrderSamplesMetric(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + options := DefaultOptions() + testOutOfOrderSamplesMetric(t, scenario, options, storage.ErrOutOfOrderSample) + }) + } +} + +func TestOutOfOrderSamplesMetricNativeHistogramOOODisabled(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + if scenario.sampleType != "histogram" { + continue + } + t.Run(name, func(t *testing.T) { + options := DefaultOptions() + options.OutOfOrderTimeWindow = 0 + testOutOfOrderSamplesMetric(t, scenario, options, storage.ErrOutOfOrderSample) + }) + } +} + +func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, options *Options, expectOutOfOrderError error) { + dir := t.TempDir() + db, err := Open(dir, nil, nil, options, nil) + require.NoError(t, err) + defer func() { + require.NoError(t, db.Close()) + }() + db.DisableCompactions() + + appendSample := func(appender storage.Appender, ts int64) (storage.SeriesRef, error) { + ref, _, err := scenario.appendFunc(appender, labels.FromStrings("a", "b"), ts, 99) + return ref, err + } + + ctx := context.Background() + app := db.Appender(ctx) + for i := 1; i <= 5; i++ { + _, err = appendSample(app, int64(i)) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Test out of order metric. + require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + app = db.Appender(ctx) + _, err = appendSample(app, 2) + require.Equal(t, expectOutOfOrderError, err) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + + _, err = appendSample(app, 3) + require.Equal(t, expectOutOfOrderError, err) + require.Equal(t, 2.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + + _, err = appendSample(app, 4) + require.Equal(t, expectOutOfOrderError, err) + require.Equal(t, 3.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + require.NoError(t, app.Commit()) + + // Compact Head to test out of bound metric. + app = db.Appender(ctx) + _, err = appendSample(app, DefaultBlockDuration*2) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + require.Equal(t, int64(math.MinInt64), db.head.minValidTime.Load()) + require.NoError(t, db.Compact(ctx)) + require.Positive(t, db.head.minValidTime.Load()) + + app = db.Appender(ctx) + _, err = appendSample(app, db.head.minValidTime.Load()-2) + require.Equal(t, storage.ErrOutOfBounds, err) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.outOfBoundSamples.WithLabelValues(scenario.sampleType))) + + _, err = appendSample(app, db.head.minValidTime.Load()-1) + require.Equal(t, storage.ErrOutOfBounds, err) + require.Equal(t, 2.0, prom_testutil.ToFloat64(db.head.metrics.outOfBoundSamples.WithLabelValues(scenario.sampleType))) + require.NoError(t, app.Commit()) + + // Some more valid samples for out of order. + app = db.Appender(ctx) + for i := 1; i <= 5; i++ { + _, err = appendSample(app, db.head.minValidTime.Load()+DefaultBlockDuration+int64(i)) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Test out of order metric. + app = db.Appender(ctx) + _, err = appendSample(app, db.head.minValidTime.Load()+DefaultBlockDuration+2) + require.Equal(t, expectOutOfOrderError, err) + require.Equal(t, 4.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + + _, err = appendSample(app, db.head.minValidTime.Load()+DefaultBlockDuration+3) + require.Equal(t, expectOutOfOrderError, err) + require.Equal(t, 5.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + + _, err = appendSample(app, db.head.minValidTime.Load()+DefaultBlockDuration+4) + require.Equal(t, expectOutOfOrderError, err) + require.Equal(t, 6.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) + require.NoError(t, app.Commit()) +} + +func testHeadSeriesChunkRace(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + require.NoError(t, h.Init(0)) + app := h.Appender(context.Background()) + + s2, err := app.Append(0, labels.FromStrings("foo2", "bar"), 5, 0) + require.NoError(t, err) + for ts := int64(6); ts < 11; ts++ { + _, err = app.Append(s2, labels.EmptyLabels(), ts, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + matcher := labels.MustNewMatcher(labels.MatchEqual, "", "") + q, err := NewBlockQuerier(h, 18, 22) + require.NoError(t, err) + defer q.Close() + + var wg sync.WaitGroup + wg.Add(1) + go func() { + defer wg.Done() + h.updateMinMaxTime(20, 25) + h.gc() + }() + ss := q.Select(context.Background(), false, nil, matcher) + for ss.Next() { + } + require.NoError(t, ss.Err()) + wg.Wait() +} + +func TestHeadLabelNamesValuesWithMinMaxRange(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + const ( + firstSeriesTimestamp int64 = 100 + secondSeriesTimestamp int64 = 200 + lastSeriesTimestamp int64 = 300 + ) + var ( + seriesTimestamps = []int64{ + firstSeriesTimestamp, + secondSeriesTimestamp, + lastSeriesTimestamp, + } + expectedLabelNames = []string{"a", "b", "c"} + expectedLabelValues = []string{"d", "e", "f"} + ctx = context.Background() + ) + + app := head.Appender(ctx) + for i, name := range expectedLabelNames { + _, err := app.Append(0, labels.FromStrings(name, expectedLabelValues[i]), seriesTimestamps[i], 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + require.Equal(t, firstSeriesTimestamp, head.MinTime()) + require.Equal(t, lastSeriesTimestamp, head.MaxTime()) + + testCases := []struct { + name string + mint int64 + maxt int64 + expectedNames []string + expectedValues []string + }{ + {"maxt less than head min", head.MaxTime() - 10, head.MinTime() - 10, []string{}, []string{}}, + {"mint less than head max", head.MaxTime() + 10, head.MinTime() + 10, []string{}, []string{}}, + {"mint and maxt outside head", head.MaxTime() + 10, head.MinTime() - 10, []string{}, []string{}}, + {"mint and maxt within head", head.MaxTime() - 10, head.MinTime() + 10, expectedLabelNames, expectedLabelValues}, + } + + for _, tt := range testCases { + t.Run(tt.name, func(t *testing.T) { + headIdxReader := head.indexRange(tt.mint, tt.maxt) + actualLabelNames, err := headIdxReader.LabelNames(ctx) + require.NoError(t, err) + require.Equal(t, tt.expectedNames, actualLabelNames) + if len(tt.expectedValues) > 0 { + for i, name := range expectedLabelNames { + actualLabelValue, err := headIdxReader.SortedLabelValues(ctx, name, nil) + require.NoError(t, err) + require.Equal(t, []string{tt.expectedValues[i]}, actualLabelValue) + } + } + }) + } +} + +func TestHeadLabelValuesWithMatchers(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { require.NoError(t, head.Close()) }) + + ctx := context.Background() + + app := head.Appender(context.Background()) + for i := range 100 { + _, err := app.Append(0, labels.FromStrings( + "tens", fmt.Sprintf("value%d", i/10), + "unique", fmt.Sprintf("value%d", i), + ), 100, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + var uniqueWithout30s []string + for i := range 100 { + if i/10 != 3 { + uniqueWithout30s = append(uniqueWithout30s, fmt.Sprintf("value%d", i)) + } + } + sort.Strings(uniqueWithout30s) + testCases := []struct { + name string + labelName string + matchers []*labels.Matcher + expectedValues []string + }{ + { + name: "get tens based on unique id", + labelName: "tens", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "unique", "value35")}, + expectedValues: []string{"value3"}, + }, { + name: "get unique ids based on a ten", + labelName: "unique", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "tens", "value1")}, + expectedValues: []string{"value10", "value11", "value12", "value13", "value14", "value15", "value16", "value17", "value18", "value19"}, + }, { + name: "get tens by pattern matching on unique id", + labelName: "tens", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "unique", "value[5-7]5")}, + expectedValues: []string{"value5", "value6", "value7"}, + }, { + name: "get tens by matching for presence of unique label", + labelName: "tens", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchNotEqual, "unique", "")}, + expectedValues: []string{"value0", "value1", "value2", "value3", "value4", "value5", "value6", "value7", "value8", "value9"}, + }, { + name: "get unique IDs based on tens not being equal to a certain value, while not empty", + labelName: "unique", + matchers: []*labels.Matcher{ + labels.MustNewMatcher(labels.MatchNotEqual, "tens", "value3"), + labels.MustNewMatcher(labels.MatchNotEqual, "tens", ""), + }, + expectedValues: uniqueWithout30s, + }, + } + + for _, tt := range testCases { + t.Run(tt.name, func(t *testing.T) { + headIdxReader := head.indexRange(0, 200) + + actualValues, err := headIdxReader.SortedLabelValues(ctx, tt.labelName, nil, tt.matchers...) + require.NoError(t, err) + require.Equal(t, tt.expectedValues, actualValues) + + actualValues, err = headIdxReader.LabelValues(ctx, tt.labelName, nil, tt.matchers...) + sort.Strings(actualValues) + require.NoError(t, err) + require.Equal(t, tt.expectedValues, actualValues) + }) + } +} + +func TestHeadLabelNamesWithMatchers(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + app := head.Appender(context.Background()) + for i := range 100 { + _, err := app.Append(0, labels.FromStrings( + "unique", fmt.Sprintf("value%d", i), + ), 100, 0) + require.NoError(t, err) + + if i%10 == 0 { + _, err := app.Append(0, labels.FromStrings( + "tens", fmt.Sprintf("value%d", i/10), + "unique", fmt.Sprintf("value%d", i), + ), 100, 0) + require.NoError(t, err) + } + + if i%20 == 0 { + _, err := app.Append(0, labels.FromStrings( + "tens", fmt.Sprintf("value%d", i/10), + "twenties", fmt.Sprintf("value%d", i/20), + "unique", fmt.Sprintf("value%d", i), + ), 100, 0) + require.NoError(t, err) + } + } + require.NoError(t, app.Commit()) + + testCases := []struct { + name string + labelName string + matchers []*labels.Matcher + expectedNames []string + }{ + { + name: "get with non-empty unique: all", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchNotEqual, "unique", "")}, + expectedNames: []string{"tens", "twenties", "unique"}, + }, { + name: "get with unique ending in 1: only unique", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "unique", "value.*1")}, + expectedNames: []string{"unique"}, + }, { + name: "get with unique = value20: all", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "unique", "value20")}, + expectedNames: []string{"tens", "twenties", "unique"}, + }, { + name: "get tens = 1: unique & tens", + matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "tens", "value1")}, + expectedNames: []string{"tens", "unique"}, + }, + } + + for _, tt := range testCases { + t.Run(tt.name, func(t *testing.T) { + headIdxReader := head.indexRange(0, 200) + + actualNames, err := headIdxReader.LabelNames(context.Background(), tt.matchers...) + require.NoError(t, err) + require.Equal(t, tt.expectedNames, actualNames) + }) + } +} + +func TestHeadShardedPostings(t *testing.T) { + headOpts := newTestHeadDefaultOptions(1000, false) + headOpts.EnableSharding = true + head, _ := newTestHeadWithOptions(t, compression.None, headOpts) + defer func() { + require.NoError(t, head.Close()) + }() + + ctx := context.Background() + + // Append some series. + app := head.Appender(ctx) + for i := range 100 { + _, err := app.Append(0, labels.FromStrings("unique", fmt.Sprintf("value%d", i), "const", "1"), 100, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + ir := head.indexRange(0, 200) + + // List all postings for a given label value. This is what we expect to get + // in output from all shards. + p, err := ir.Postings(ctx, "const", "1") + require.NoError(t, err) + + var expected []storage.SeriesRef + for p.Next() { + expected = append(expected, p.At()) + } + require.NoError(t, p.Err()) + require.NotEmpty(t, expected) + + // Query the same postings for each shard. + const shardCount = uint64(4) + actualShards := make(map[uint64][]storage.SeriesRef) + actualPostings := make([]storage.SeriesRef, 0, len(expected)) + + for shardIndex := range shardCount { + p, err = ir.Postings(ctx, "const", "1") + require.NoError(t, err) + + p = ir.ShardedPostings(p, shardIndex, shardCount) + for p.Next() { + ref := p.At() + + actualShards[shardIndex] = append(actualShards[shardIndex], ref) + actualPostings = append(actualPostings, ref) + } + require.NoError(t, p.Err()) + } + + // We expect the postings merged out of shards is the exact same of the non sharded ones. + require.ElementsMatch(t, expected, actualPostings) + + // We expect the series in each shard are the expected ones. + for shardIndex, ids := range actualShards { + for _, id := range ids { + var lbls labels.ScratchBuilder + + require.NoError(t, ir.Series(id, &lbls, nil)) + require.Equal(t, shardIndex, labels.StableHash(lbls.Labels())%shardCount) + } + } +} + +func TestErrReuseAppender(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + app := head.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("test", "test"), 0, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Error(t, app.Commit()) + require.Error(t, app.Rollback()) + + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("test", "test"), 1, 0) + require.NoError(t, err) + require.NoError(t, app.Rollback()) + require.Error(t, app.Rollback()) + require.Error(t, app.Commit()) + + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("test", "test"), 2, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.Error(t, app.Rollback()) + require.Error(t, app.Commit()) + + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("test", "test"), 3, 0) + require.NoError(t, err) + require.NoError(t, app.Rollback()) + require.Error(t, app.Commit()) + require.Error(t, app.Rollback()) +} + +func TestHeadMintAfterTruncation(t *testing.T) { + chunkRange := int64(2000) + head, _ := newTestHead(t, chunkRange, compression.None, false) + + app := head.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 100, 100) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("a", "b"), 4000, 200) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("a", "b"), 8000, 300) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // Truncating outside the appendable window and actual mint being outside + // appendable window should leave mint at the actual mint. + require.NoError(t, head.Truncate(3500)) + require.Equal(t, int64(4000), head.MinTime()) + require.Equal(t, int64(4000), head.minValidTime.Load()) + + // After truncation outside the appendable window if the actual min time + // is in the appendable window then we should leave mint at the start of appendable window. + require.NoError(t, head.Truncate(5000)) + require.Equal(t, head.appendableMinValidTime(), head.MinTime()) + require.Equal(t, head.appendableMinValidTime(), head.minValidTime.Load()) + + // If the truncation time is inside the appendable window, then the min time + // should be the truncation time. + require.NoError(t, head.Truncate(7500)) + require.Equal(t, int64(7500), head.MinTime()) + require.Equal(t, int64(7500), head.minValidTime.Load()) + + require.NoError(t, head.Close()) +} + +func TestHeadExemplars(t *testing.T) { + chunkRange := int64(2000) + head, _ := newTestHead(t, chunkRange, compression.None, false) + app := head.Appender(context.Background()) + + l := labels.FromStrings("trace_id", "123") + // It is perfectly valid to add Exemplars before the current start time - + // histogram buckets that haven't been update in a while could still be + // exported exemplars from an hour ago. + ref, err := app.Append(0, labels.FromStrings("a", "b"), 100, 100) + require.NoError(t, err) + _, err = app.AppendExemplar(ref, l, exemplar.Exemplar{ + Labels: l, + HasTs: true, + Ts: -1000, + Value: 1, + }) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.NoError(t, head.Close()) +} + +func BenchmarkHeadLabelValuesWithMatchers(b *testing.B) { + chunkRange := int64(2000) + head, _ := newTestHead(b, chunkRange, compression.None, false) + b.Cleanup(func() { require.NoError(b, head.Close()) }) + + ctx := context.Background() + + app := head.Appender(context.Background()) + + metricCount := 1000000 + for i := range metricCount { + _, err := app.Append(0, labels.FromStrings( + "a_unique", fmt.Sprintf("value%d", i), + "b_tens", fmt.Sprintf("value%d", i/(metricCount/10)), + "c_ninety", fmt.Sprintf("value%d", i/(metricCount/10)/9), // "0" for the first 90%, then "1" + ), 100, 0) + require.NoError(b, err) + } + require.NoError(b, app.Commit()) + + headIdxReader := head.indexRange(0, 200) + matchers := []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "c_ninety", "value0")} + + b.ReportAllocs() + + for b.Loop() { + actualValues, err := headIdxReader.LabelValues(ctx, "b_tens", nil, matchers...) + require.NoError(b, err) + require.Len(b, actualValues, 9) + } +} + +func TestIteratorSeekIntoBuffer(t *testing.T) { + dir := t.TempDir() + // This is usually taken from the Head, but passing manually here. + chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) + require.NoError(t, err) + defer func() { + require.NoError(t, chunkDiskMapper.Close()) + }() + cOpts := chunkOpts{ + chunkDiskMapper: chunkDiskMapper, + chunkRange: 500, + samplesPerChunk: DefaultSamplesPerChunk, + } + + s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) + + for i := range 7 { + ok, _ := s.append(int64(i), float64(i), 0, cOpts) + require.True(t, ok, "sample append failed") + } + + c, _, _, err := s.chunk(0, chunkDiskMapper, &sync.Pool{ + New: func() any { + return &memChunk{} + }, + }) + require.NoError(t, err) + it := c.chunk.Iterator(nil) + + // First point. + require.Equal(t, chunkenc.ValFloat, it.Seek(0)) + ts, val := it.At() + require.Equal(t, int64(0), ts) + require.Equal(t, float64(0), val) + + // Advance one point. + require.Equal(t, chunkenc.ValFloat, it.Next()) + ts, val = it.At() + require.Equal(t, int64(1), ts) + require.Equal(t, float64(1), val) + + // Seeking an older timestamp shouldn't cause the iterator to go backwards. + require.Equal(t, chunkenc.ValFloat, it.Seek(0)) + ts, val = it.At() + require.Equal(t, int64(1), ts) + require.Equal(t, float64(1), val) + + // Seek into the buffer. + require.Equal(t, chunkenc.ValFloat, it.Seek(3)) + ts, val = it.At() + require.Equal(t, int64(3), ts) + require.Equal(t, float64(3), val) + + // Iterate through the rest of the buffer. + for i := 4; i < 7; i++ { + require.Equal(t, chunkenc.ValFloat, it.Next()) + ts, val = it.At() + require.Equal(t, int64(i), ts) + require.Equal(t, float64(i), val) + } + + // Run out of elements in the iterator. + require.Equal(t, chunkenc.ValNone, it.Next()) + require.Equal(t, chunkenc.ValNone, it.Seek(7)) +} + +// Tests https://github.com/prometheus/prometheus/issues/8221. +func TestChunkNotFoundHeadGCRace(t *testing.T) { + t.Parallel() + db := newTestDB(t) + db.DisableCompactions() + ctx := context.Background() + + var ( + app = db.Appender(context.Background()) + ref = storage.SeriesRef(0) + mint, maxt = int64(0), int64(0) + err error + ) + + // Appends samples to span over 1.5 block ranges. + // 7 chunks with 15s scrape interval. + for i := int64(0); i <= 120*7; i++ { + ts := i * DefaultBlockDuration / (4 * 120) + ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i)) + require.NoError(t, err) + maxt = ts + } + require.NoError(t, app.Commit()) + + // Get a querier before compaction (or when compaction is about to begin). + q, err := db.Querier(mint, maxt) + require.NoError(t, err) + + // Query the compacted range and get the first series before compaction. + ss := q.Select(context.Background(), true, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.True(t, ss.Next()) + s := ss.At() + + var wg sync.WaitGroup + wg.Add(1) + go func() { + defer wg.Done() + // Compacting head while the querier spans the compaction time. + require.NoError(t, db.Compact(ctx)) + require.NotEmpty(t, db.Blocks()) + }() + + // Give enough time for compaction to finish. + // We expect it to be blocked until querier is closed. + <-time.After(3 * time.Second) + + // Now consume after compaction when it's gone. + it := s.Iterator(nil) + for it.Next() == chunkenc.ValFloat { + _, _ = it.At() + } + // It should error here without any fix for the mentioned issue. + require.NoError(t, it.Err()) + for ss.Next() { + s = ss.At() + it = s.Iterator(it) + for it.Next() == chunkenc.ValFloat { + _, _ = it.At() + } + require.NoError(t, it.Err()) + } + require.NoError(t, ss.Err()) + + require.NoError(t, q.Close()) + wg.Wait() +} + +// Tests https://github.com/prometheus/prometheus/issues/9079. +func TestDataMissingOnQueryDuringCompaction(t *testing.T) { + t.Parallel() + db := newTestDB(t) + db.DisableCompactions() + ctx := context.Background() + + var ( + app = db.Appender(context.Background()) + ref = storage.SeriesRef(0) + mint, maxt = int64(0), int64(0) + err error + ) + + // Appends samples to span over 1.5 block ranges. + expSamples := make([]chunks.Sample, 0) + // 7 chunks with 15s scrape interval. + for i := int64(0); i <= 120*7; i++ { + ts := i * DefaultBlockDuration / (4 * 120) + ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i)) + require.NoError(t, err) + maxt = ts + expSamples = append(expSamples, sample{ts, float64(i), nil, nil}) + } + require.NoError(t, app.Commit()) + + // Get a querier before compaction (or when compaction is about to begin). + q, err := db.Querier(mint, maxt) + require.NoError(t, err) + + var wg sync.WaitGroup + wg.Add(1) + go func() { + defer wg.Done() + // Compacting head while the querier spans the compaction time. + require.NoError(t, db.Compact(ctx)) + require.NotEmpty(t, db.Blocks()) + }() + + // Give enough time for compaction to finish. + // We expect it to be blocked until querier is closed. + <-time.After(3 * time.Second) + + // Querying the querier that was got before compaction. + series := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.Equal(t, map[string][]chunks.Sample{`{a="b"}`: expSamples}, series) + + wg.Wait() +} + +func TestIsQuerierCollidingWithTruncation(t *testing.T) { + db := newTestDB(t) + db.DisableCompactions() + + var ( + app = db.Appender(context.Background()) + ref = storage.SeriesRef(0) + err error + ) + + for i := int64(0); i <= 3000; i++ { + ref, err = app.Append(ref, labels.FromStrings("a", "b"), i, float64(i)) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // This mocks truncation. + db.head.memTruncationInProcess.Store(true) + db.head.lastMemoryTruncationTime.Store(2000) + + // Test that IsQuerierValid suggests correct querier ranges. + cases := []struct { + mint, maxt int64 // For the querier. + expShouldClose, expGetNew bool + expNewMint int64 + }{ + {-200, -100, true, false, 0}, + {-200, 300, true, false, 0}, + {100, 1900, true, false, 0}, + {1900, 2200, true, true, 2000}, + {2000, 2500, false, false, 0}, + } + + for _, c := range cases { + t.Run(fmt.Sprintf("mint=%d,maxt=%d", c.mint, c.maxt), func(t *testing.T) { + shouldClose, getNew, newMint := db.head.IsQuerierCollidingWithTruncation(c.mint, c.maxt) + require.Equal(t, c.expShouldClose, shouldClose) + require.Equal(t, c.expGetNew, getNew) + if getNew { + require.Equal(t, c.expNewMint, newMint) + } + }) + } +} + +func TestWaitForPendingReadersInTimeRange(t *testing.T) { + t.Parallel() + db := newTestDB(t) + db.DisableCompactions() + + sampleTs := func(i int64) int64 { return i * DefaultBlockDuration / (4 * 120) } + + var ( + app = db.Appender(context.Background()) + ref = storage.SeriesRef(0) + err error + ) + + for i := int64(0); i <= 3000; i++ { + ts := sampleTs(i) + ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i)) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + truncMint, truncMaxt := int64(1000), int64(2000) + cases := []struct { + mint, maxt int64 + shouldWait bool + }{ + {0, 500, false}, // Before truncation range. + {500, 1500, true}, // Overlaps with truncation at the start. + {1200, 1700, true}, // Within truncation range. + {1800, 2500, true}, // Overlaps with truncation at the end. + {2000, 2500, false}, // After truncation range. + {2100, 2500, false}, // After truncation range. + } + for _, c := range cases { + t.Run(fmt.Sprintf("mint=%d,maxt=%d,shouldWait=%t", c.mint, c.maxt, c.shouldWait), func(t *testing.T) { + checkWaiting := func(cl io.Closer) { + var waitOver atomic.Bool + go func() { + db.head.WaitForPendingReadersInTimeRange(truncMint, truncMaxt) + waitOver.Store(true) + }() + <-time.After(550 * time.Millisecond) + require.Equal(t, !c.shouldWait, waitOver.Load()) + require.NoError(t, cl.Close()) + <-time.After(550 * time.Millisecond) + require.True(t, waitOver.Load()) + } + + q, err := db.Querier(c.mint, c.maxt) + require.NoError(t, err) + checkWaiting(q) + + cq, err := db.ChunkQuerier(c.mint, c.maxt) + require.NoError(t, err) + checkWaiting(cq) + }) + } +} + +func TestQueryOOOHeadDuringTruncate(t *testing.T) { + testQueryOOOHeadDuringTruncate(t, + func(db *DB, minT, maxT int64) (storage.LabelQuerier, error) { + return db.Querier(minT, maxT) + }, + func(t *testing.T, lq storage.LabelQuerier, minT, _ int64) { + // Samples + q, ok := lq.(storage.Querier) + require.True(t, ok) + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.True(t, ss.Next()) + s := ss.At() + require.False(t, ss.Next()) // One series. + it := s.Iterator(nil) + require.NotEqual(t, chunkenc.ValNone, it.Next()) // Has some data. + require.Equal(t, minT, it.AtT()) // It is an in-order sample. + require.NotEqual(t, chunkenc.ValNone, it.Next()) // Has some data. + require.Equal(t, minT+50, it.AtT()) // it is an out-of-order sample. + require.NoError(t, it.Err()) + }, + ) +} + +func TestChunkQueryOOOHeadDuringTruncate(t *testing.T) { + testQueryOOOHeadDuringTruncate(t, + func(db *DB, minT, maxT int64) (storage.LabelQuerier, error) { + return db.ChunkQuerier(minT, maxT) + }, + func(t *testing.T, lq storage.LabelQuerier, minT, _ int64) { + // Chunks + q, ok := lq.(storage.ChunkQuerier) + require.True(t, ok) + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.True(t, ss.Next()) + s := ss.At() + require.False(t, ss.Next()) // One series. + metaIt := s.Iterator(nil) + require.True(t, metaIt.Next()) + meta := metaIt.At() + // Samples + it := meta.Chunk.Iterator(nil) + require.NotEqual(t, chunkenc.ValNone, it.Next()) // Has some data. + require.Equal(t, minT, it.AtT()) // It is an in-order sample. + require.NotEqual(t, chunkenc.ValNone, it.Next()) // Has some data. + require.Equal(t, minT+50, it.AtT()) // it is an out-of-order sample. + require.NoError(t, it.Err()) + }, + ) +} + +func testQueryOOOHeadDuringTruncate(t *testing.T, makeQuerier func(db *DB, minT, maxT int64) (storage.LabelQuerier, error), verify func(t *testing.T, q storage.LabelQuerier, minT, maxT int64)) { + const maxT int64 = 6000 + + dir := t.TempDir() + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = maxT + opts.MinBlockDuration = maxT / 2 // So that head will compact up to 3000. + + db, err := Open(dir, nil, nil, opts, nil) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, db.Close()) + }) + db.DisableCompactions() + + var ( + ref = storage.SeriesRef(0) + app = db.Appender(context.Background()) + ) + // Add in-order samples at every 100ms starting at 0ms. + for i := int64(0); i < maxT; i += 100 { + _, err := app.Append(ref, labels.FromStrings("a", "b"), i, 0) + require.NoError(t, err) + } + // Add out-of-order samples at every 100ms starting at 50ms. + for i := int64(50); i < maxT; i += 100 { + _, err := app.Append(ref, labels.FromStrings("a", "b"), i, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + requireEqualOOOSamples(t, int(maxT/100-1), db) + + // Synchronization points. + allowQueryToStart := make(chan struct{}) + queryStarted := make(chan struct{}) + compactionFinished := make(chan struct{}) + + db.head.memTruncationCallBack = func() { + // Compaction has started, let the query start and wait for it to actually start to simulate race condition. + allowQueryToStart <- struct{}{} + <-queryStarted + } + + go func() { + db.Compact(context.Background()) // Compact and write blocks up to 3000 (maxtT/2). + compactionFinished <- struct{}{} + }() + + // Wait for the compaction to start. + <-allowQueryToStart + + q, err := makeQuerier(db, 1500, 2500) + require.NoError(t, err) + queryStarted <- struct{}{} // Unblock the compaction. + ctx := context.Background() + + // Label names. + res, annots, err := q.LabelNames(ctx, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.NoError(t, err) + require.Empty(t, annots) + require.Equal(t, []string{"a"}, res) + + // Label values. + res, annots, err = q.LabelValues(ctx, "a", nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.NoError(t, err) + require.Empty(t, annots) + require.Equal(t, []string{"b"}, res) + + verify(t, q, 1500, 2500) + + require.NoError(t, q.Close()) // Cannot be deferred as the compaction waits for queries to close before finishing. + + <-compactionFinished // Wait for compaction otherwise Go test finds stray goroutines. +} + +func TestAppendHistogram(t *testing.T) { + l := labels.FromStrings("a", "b") + for _, numHistograms := range []int{1, 10, 150, 200, 250, 300} { + t.Run(strconv.Itoa(numHistograms), func(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + + require.NoError(t, head.Init(0)) + ingestTs := int64(0) + app := head.Appender(context.Background()) + + expHistograms := make([]chunks.Sample, 0, 2*numHistograms) + + // Counter integer histograms. + for _, h := range tsdbutil.GenerateTestHistograms(numHistograms) { + _, err := app.AppendHistogram(0, l, ingestTs, h, nil) + require.NoError(t, err) + expHistograms = append(expHistograms, sample{t: ingestTs, h: h}) + ingestTs++ + if ingestTs%50 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + + // Gauge integer histograms. + for _, h := range tsdbutil.GenerateTestGaugeHistograms(numHistograms) { + _, err := app.AppendHistogram(0, l, ingestTs, h, nil) + require.NoError(t, err) + expHistograms = append(expHistograms, sample{t: ingestTs, h: h}) + ingestTs++ + if ingestTs%50 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + + expFloatHistograms := make([]chunks.Sample, 0, 2*numHistograms) + + // Counter float histograms. + for _, fh := range tsdbutil.GenerateTestFloatHistograms(numHistograms) { + _, err := app.AppendHistogram(0, l, ingestTs, nil, fh) + require.NoError(t, err) + expFloatHistograms = append(expFloatHistograms, sample{t: ingestTs, fh: fh}) + ingestTs++ + if ingestTs%50 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + + // Gauge float histograms. + for _, fh := range tsdbutil.GenerateTestGaugeFloatHistograms(numHistograms) { + _, err := app.AppendHistogram(0, l, ingestTs, nil, fh) + require.NoError(t, err) + expFloatHistograms = append(expFloatHistograms, sample{t: ingestTs, fh: fh}) + ingestTs++ + if ingestTs%50 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + + require.NoError(t, app.Commit()) + + q, err := NewBlockQuerier(head, head.MinTime(), head.MaxTime()) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, q.Close()) + }) + + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + require.True(t, ss.Next()) + s := ss.At() + require.False(t, ss.Next()) + + it := s.Iterator(nil) + actHistograms := make([]chunks.Sample, 0, len(expHistograms)) + actFloatHistograms := make([]chunks.Sample, 0, len(expFloatHistograms)) + for typ := it.Next(); typ != chunkenc.ValNone; typ = it.Next() { + switch typ { + case chunkenc.ValHistogram: + ts, h := it.AtHistogram(nil) + actHistograms = append(actHistograms, sample{t: ts, h: h}) + case chunkenc.ValFloatHistogram: + ts, fh := it.AtFloatHistogram(nil) + actFloatHistograms = append(actFloatHistograms, sample{t: ts, fh: fh}) + } + } + + compareSeries( + t, + map[string][]chunks.Sample{"dummy": expHistograms}, + map[string][]chunks.Sample{"dummy": actHistograms}, + ) + compareSeries( + t, + map[string][]chunks.Sample{"dummy": expFloatHistograms}, + map[string][]chunks.Sample{"dummy": actFloatHistograms}, + ) + }) + } +} + +func TestHistogramInWALAndMmapChunk(t *testing.T) { + head, _ := newTestHead(t, 3000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + // Series with only histograms. + s1 := labels.FromStrings("a", "b1") + k1 := s1.String() + numHistograms := 300 + exp := map[string][]chunks.Sample{} + ts := int64(0) + var app storage.Appender + for _, gauge := range []bool{true, false} { + app = head.Appender(context.Background()) + var hists []*histogram.Histogram + if gauge { + hists = tsdbutil.GenerateTestGaugeHistograms(numHistograms) + } else { + hists = tsdbutil.GenerateTestHistograms(numHistograms) + } + for _, h := range hists { + h.NegativeSpans = h.PositiveSpans + h.NegativeBuckets = h.PositiveBuckets + _, err := app.AppendHistogram(0, s1, ts, h, nil) + require.NoError(t, err) + exp[k1] = append(exp[k1], sample{t: ts, h: h.Copy()}) + ts++ + if ts%5 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + } + for _, gauge := range []bool{true, false} { + app = head.Appender(context.Background()) + var hists []*histogram.FloatHistogram + if gauge { + hists = tsdbutil.GenerateTestGaugeFloatHistograms(numHistograms) + } else { + hists = tsdbutil.GenerateTestFloatHistograms(numHistograms) + } + for _, h := range hists { + h.NegativeSpans = h.PositiveSpans + h.NegativeBuckets = h.PositiveBuckets + _, err := app.AppendHistogram(0, s1, ts, nil, h) + require.NoError(t, err) + exp[k1] = append(exp[k1], sample{t: ts, fh: h.Copy()}) + ts++ + if ts%5 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + head.mmapHeadChunks() + } + + // There should be 20 mmap chunks in s1. + ms := head.series.getByHash(s1.Hash(), s1) + require.Len(t, ms.mmappedChunks, 25) + expMmapChunks := make([]*mmappedChunk, 0, 20) + for _, mmap := range ms.mmappedChunks { + require.Positive(t, mmap.numSamples) + cpy := *mmap + expMmapChunks = append(expMmapChunks, &cpy) + } + expHeadChunkSamples := ms.headChunks.chunk.NumSamples() + require.Positive(t, expHeadChunkSamples) + + // Series with mix of histograms and float. + s2 := labels.FromStrings("a", "b2") + k2 := s2.String() + ts = 0 + for _, gauge := range []bool{true, false} { + app = head.Appender(context.Background()) + var hists []*histogram.Histogram + if gauge { + hists = tsdbutil.GenerateTestGaugeHistograms(100) + } else { + hists = tsdbutil.GenerateTestHistograms(100) + } + for _, h := range hists { + ts++ + h.NegativeSpans = h.PositiveSpans + h.NegativeBuckets = h.PositiveBuckets + _, err := app.AppendHistogram(0, s2, ts, h, nil) + require.NoError(t, err) + eh := h.Copy() + if !gauge && ts > 30 && (ts-10)%20 == 1 { + // Need "unknown" hint after float sample. + eh.CounterResetHint = histogram.UnknownCounterReset + } + exp[k2] = append(exp[k2], sample{t: ts, h: eh}) + if ts%20 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + // Add some float. + for range 10 { + ts++ + _, err := app.Append(0, s2, ts, float64(ts)) + require.NoError(t, err) + exp[k2] = append(exp[k2], sample{t: ts, f: float64(ts)}) + } + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + } + for _, gauge := range []bool{true, false} { + app = head.Appender(context.Background()) + var hists []*histogram.FloatHistogram + if gauge { + hists = tsdbutil.GenerateTestGaugeFloatHistograms(100) + } else { + hists = tsdbutil.GenerateTestFloatHistograms(100) + } + for _, h := range hists { + ts++ + h.NegativeSpans = h.PositiveSpans + h.NegativeBuckets = h.PositiveBuckets + _, err := app.AppendHistogram(0, s2, ts, nil, h) + require.NoError(t, err) + eh := h.Copy() + if !gauge && ts > 30 && (ts-10)%20 == 1 { + // Need "unknown" hint after float sample. + eh.CounterResetHint = histogram.UnknownCounterReset + } + exp[k2] = append(exp[k2], sample{t: ts, fh: eh}) + if ts%20 == 0 { + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + // Add some float. + for range 10 { + ts++ + _, err := app.Append(0, s2, ts, float64(ts)) + require.NoError(t, err) + exp[k2] = append(exp[k2], sample{t: ts, f: float64(ts)}) + } + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + } + + // Restart head. + require.NoError(t, head.Close()) + startHead := func() { + w, err := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(0)) + } + startHead() + + // Checking contents of s1. + ms = head.series.getByHash(s1.Hash(), s1) + require.Equal(t, expMmapChunks, ms.mmappedChunks) + require.Equal(t, expHeadChunkSamples, ms.headChunks.chunk.NumSamples()) + + testQuery := func() { + q, err := NewBlockQuerier(head, head.MinTime(), head.MaxTime()) + require.NoError(t, err) + act := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "a", "b.*")) + compareSeries(t, exp, act) + } + testQuery() + + // Restart with no mmap chunks to test WAL replay. + require.NoError(t, head.Close()) + require.NoError(t, os.RemoveAll(mmappedChunksDir(head.opts.ChunkDirRoot))) + startHead() + testQuery() +} + +func TestChunkSnapshot(t *testing.T) { + head, _ := newTestHead(t, 120*4, compression.None, false) + defer func() { + head.opts.EnableMemorySnapshotOnShutdown = false + require.NoError(t, head.Close()) + }() + + type ex struct { + seriesLabels labels.Labels + e exemplar.Exemplar + } + + numSeries := 10 + expSeries := make(map[string][]chunks.Sample) + expHist := make(map[string][]chunks.Sample) + expFloatHist := make(map[string][]chunks.Sample) + expTombstones := make(map[storage.SeriesRef]tombstones.Intervals) + expExemplars := make([]ex, 0) + histograms := tsdbutil.GenerateTestGaugeHistograms(481) + floatHistogram := tsdbutil.GenerateTestGaugeFloatHistograms(481) + + addExemplar := func(app storage.Appender, ref storage.SeriesRef, lbls labels.Labels, ts int64) { + e := ex{ + seriesLabels: lbls, + e: exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts, + }, + } + expExemplars = append(expExemplars, e) + _, err := app.AppendExemplar(ref, e.seriesLabels, e.e) + require.NoError(t, err) + } + + checkSamples := func() { + q, err := NewBlockQuerier(head, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + series := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + require.Equal(t, expSeries, series) + } + checkHistograms := func() { + q, err := NewBlockQuerier(head, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + series := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "hist", "baz.*")) + require.Equal(t, expHist, series) + } + checkFloatHistograms := func() { + q, err := NewBlockQuerier(head, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + series := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "floathist", "bat.*")) + require.Equal(t, expFloatHist, series) + } + checkTombstones := func() { + tr, err := head.Tombstones() + require.NoError(t, err) + actTombstones := make(map[storage.SeriesRef]tombstones.Intervals) + require.NoError(t, tr.Iter(func(ref storage.SeriesRef, itvs tombstones.Intervals) error { + for _, itv := range itvs { + actTombstones[ref].Add(itv) + } + return nil + })) + require.Equal(t, expTombstones, actTombstones) + } + checkExemplars := func() { + actExemplars := make([]ex, 0, len(expExemplars)) + err := head.exemplars.IterateExemplars(func(seriesLabels labels.Labels, e exemplar.Exemplar) error { + actExemplars = append(actExemplars, ex{ + seriesLabels: seriesLabels, + e: e, + }) + return nil + }) + require.NoError(t, err) + // Verifies both existence of right exemplars and order of exemplars in the buffer. + testutil.RequireEqualWithOptions(t, expExemplars, actExemplars, []cmp.Option{cmp.AllowUnexported(ex{})}) + } + + var ( + wlast, woffset int + err error + ) + + closeHeadAndCheckSnapshot := func() { + require.NoError(t, head.Close()) + + _, sidx, soffset, err := LastChunkSnapshot(head.opts.ChunkDirRoot) + require.NoError(t, err) + require.Equal(t, wlast, sidx) + require.Equal(t, woffset, soffset) + } + + openHeadAndCheckReplay := func() { + w, err := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(math.MinInt64)) + + checkSamples() + checkHistograms() + checkFloatHistograms() + checkTombstones() + checkExemplars() + } + + { // Initial data that goes into snapshot. + // Add some initial samples with >=1 m-map chunk. + app := head.Appender(context.Background()) + for i := 1; i <= numSeries; i++ { + lbls := labels.FromStrings("foo", fmt.Sprintf("bar%d", i)) + lblStr := lbls.String() + lblsHist := labels.FromStrings("hist", fmt.Sprintf("baz%d", i)) + lblsHistStr := lblsHist.String() + lblsFloatHist := labels.FromStrings("floathist", fmt.Sprintf("bat%d", i)) + lblsFloatHistStr := lblsFloatHist.String() + + // 240 samples should m-map at least 1 chunk. + for ts := int64(1); ts <= 240; ts++ { + val := rand.Float64() + expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil}) + ref, err := app.Append(0, lbls, ts, val) + require.NoError(t, err) + + hist := histograms[int(ts)] + expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil}) + _, err = app.AppendHistogram(0, lblsHist, ts, hist, nil) + require.NoError(t, err) + + floatHist := floatHistogram[int(ts)] + expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist}) + _, err = app.AppendHistogram(0, lblsFloatHist, ts, nil, floatHist) + require.NoError(t, err) + + // Add an exemplar and to create multiple WAL records. + if ts%10 == 0 { + addExemplar(app, ref, lbls, ts) + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + } + require.NoError(t, app.Commit()) + + // Add some tombstones. + var enc record.Encoder + for i := 1; i <= numSeries; i++ { + ref := storage.SeriesRef(i) + itvs := tombstones.Intervals{ + {Mint: 1234, Maxt: 2345}, + {Mint: 3456, Maxt: 4567}, + } + for _, itv := range itvs { + expTombstones[ref].Add(itv) + } + head.tombstones.AddInterval(ref, itvs...) + err := head.wal.Log(enc.Tombstones([]tombstones.Stone{ + {Ref: ref, Intervals: itvs}, + }, nil)) + require.NoError(t, err) + } + } + + // These references should be the ones used for the snapshot. + wlast, woffset, err = head.wal.LastSegmentAndOffset() + require.NoError(t, err) + if woffset != 0 && woffset < 32*1024 { + // The page is always filled before taking the snapshot. + woffset = 32 * 1024 + } + + { + // Creating snapshot and verifying it. + head.opts.EnableMemorySnapshotOnShutdown = true + closeHeadAndCheckSnapshot() // This will create a snapshot. + + // Test the replay of snapshot. + openHeadAndCheckReplay() + } + + { // Additional data to only include in WAL and m-mapped chunks and not snapshot. This mimics having an old snapshot on disk. + // Add more samples. + app := head.Appender(context.Background()) + for i := 1; i <= numSeries; i++ { + lbls := labels.FromStrings("foo", fmt.Sprintf("bar%d", i)) + lblStr := lbls.String() + lblsHist := labels.FromStrings("hist", fmt.Sprintf("baz%d", i)) + lblsHistStr := lblsHist.String() + lblsFloatHist := labels.FromStrings("floathist", fmt.Sprintf("bat%d", i)) + lblsFloatHistStr := lblsFloatHist.String() + + // 240 samples should m-map at least 1 chunk. + for ts := int64(241); ts <= 480; ts++ { + val := rand.Float64() + expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil}) + ref, err := app.Append(0, lbls, ts, val) + require.NoError(t, err) + + hist := histograms[int(ts)] + expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil}) + _, err = app.AppendHistogram(0, lblsHist, ts, hist, nil) + require.NoError(t, err) + + floatHist := floatHistogram[int(ts)] + expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist}) + _, err = app.AppendHistogram(0, lblsFloatHist, ts, nil, floatHist) + require.NoError(t, err) + + // Add an exemplar and to create multiple WAL records. + if ts%10 == 0 { + addExemplar(app, ref, lbls, ts) + require.NoError(t, app.Commit()) + app = head.Appender(context.Background()) + } + } + } + require.NoError(t, app.Commit()) + + // Add more tombstones. + var enc record.Encoder + for i := 1; i <= numSeries; i++ { + ref := storage.SeriesRef(i) + itvs := tombstones.Intervals{ + {Mint: 12345, Maxt: 23456}, + {Mint: 34567, Maxt: 45678}, + } + for _, itv := range itvs { + expTombstones[ref].Add(itv) + } + head.tombstones.AddInterval(ref, itvs...) + err := head.wal.Log(enc.Tombstones([]tombstones.Stone{ + {Ref: ref, Intervals: itvs}, + }, nil)) + require.NoError(t, err) + } + } + { + // Close Head and verify that new snapshot was not created. + head.opts.EnableMemorySnapshotOnShutdown = false + closeHeadAndCheckSnapshot() // This should not create a snapshot. + + // Test the replay of snapshot, m-map chunks, and WAL. + head.opts.EnableMemorySnapshotOnShutdown = true // Enabled to read from snapshot. + openHeadAndCheckReplay() + } + + // Creating another snapshot should delete the older snapshot and replay still works fine. + wlast, woffset, err = head.wal.LastSegmentAndOffset() + require.NoError(t, err) + if woffset != 0 && woffset < 32*1024 { + // The page is always filled before taking the snapshot. + woffset = 32 * 1024 + } + + { + // Close Head and verify that new snapshot was created. + closeHeadAndCheckSnapshot() + + // Verify that there is only 1 snapshot. + files, err := os.ReadDir(head.opts.ChunkDirRoot) + require.NoError(t, err) + snapshots := 0 + for i := len(files) - 1; i >= 0; i-- { + fi := files[i] + if strings.HasPrefix(fi.Name(), chunkSnapshotPrefix) { + snapshots++ + require.Equal(t, chunkSnapshotDir(wlast, woffset), fi.Name()) + } + } + require.Equal(t, 1, snapshots) + + // Test the replay of snapshot. + head.opts.EnableMemorySnapshotOnShutdown = true // Enabled to read from snapshot. + + // Disabling exemplars to check that it does not hard fail replay + // https://github.com/prometheus/prometheus/issues/9437#issuecomment-933285870. + head.opts.EnableExemplarStorage = false + head.opts.MaxExemplars.Store(0) + expExemplars = expExemplars[:0] + + openHeadAndCheckReplay() + + require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) + } +} + +func TestSnapshotError(t *testing.T) { + head, _ := newTestHead(t, 120*4, compression.None, false) + defer func() { + head.opts.EnableMemorySnapshotOnShutdown = false + require.NoError(t, head.Close()) + }() + + // Add a sample. + app := head.Appender(context.Background()) + lbls := labels.FromStrings("foo", "bar") + _, err := app.Append(0, lbls, 99, 99) + require.NoError(t, err) + + // Add histograms + hist := tsdbutil.GenerateTestGaugeHistograms(1)[0] + floatHist := tsdbutil.GenerateTestGaugeFloatHistograms(1)[0] + lblsHist := labels.FromStrings("hist", "bar") + lblsFloatHist := labels.FromStrings("floathist", "bar") + + _, err = app.AppendHistogram(0, lblsHist, 99, hist, nil) + require.NoError(t, err) + + _, err = app.AppendHistogram(0, lblsFloatHist, 99, nil, floatHist) + require.NoError(t, err) + + require.NoError(t, app.Commit()) + + // Add some tombstones. + itvs := tombstones.Intervals{ + {Mint: 1234, Maxt: 2345}, + {Mint: 3456, Maxt: 4567}, + } + head.tombstones.AddInterval(1, itvs...) + + // Check existence of data. + require.NotNil(t, head.series.getByHash(lbls.Hash(), lbls)) + tm, err := head.tombstones.Get(1) + require.NoError(t, err) + require.NotEmpty(t, tm) + + head.opts.EnableMemorySnapshotOnShutdown = true + require.NoError(t, head.Close()) // This will create a snapshot. + + // Remove the WAL so that we don't load from it. + require.NoError(t, os.RemoveAll(head.wal.Dir())) + + // Corrupt the snapshot. + snapDir, _, _, err := LastChunkSnapshot(head.opts.ChunkDirRoot) + require.NoError(t, err) + files, err := os.ReadDir(snapDir) + require.NoError(t, err) + f, err := os.OpenFile(path.Join(snapDir, files[0].Name()), os.O_RDWR, 0) + require.NoError(t, err) + // Create snapshot backup to be restored on future test cases. + snapshotBackup, err := io.ReadAll(f) + require.NoError(t, err) + _, err = f.WriteAt([]byte{0b11111111}, 18) + require.NoError(t, err) + require.NoError(t, f.Close()) + + // Create new Head which should replay this snapshot. + w, err := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + // Testing https://github.com/prometheus/prometheus/issues/9437 with the registry. + head, err = NewHead(prometheus.NewRegistry(), nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(math.MinInt64)) + + // There should be no series in the memory after snapshot error since WAL was removed. + require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) + require.Equal(t, uint64(0), head.NumSeries()) + require.Nil(t, head.series.getByHash(lbls.Hash(), lbls)) + tm, err = head.tombstones.Get(1) + require.NoError(t, err) + require.Empty(t, tm) + require.NoError(t, head.Close()) + + // Test corruption in the middle of the snapshot. + f, err = os.OpenFile(path.Join(snapDir, files[0].Name()), os.O_RDWR, 0) + require.NoError(t, err) + _, err = f.WriteAt(snapshotBackup, 0) + require.NoError(t, err) + _, err = f.WriteAt([]byte{0b11111111}, 300) + require.NoError(t, err) + require.NoError(t, f.Close()) + + c := &countSeriesLifecycleCallback{} + opts := head.opts + opts.SeriesCallback = c + + w, err = wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(prometheus.NewRegistry(), nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(math.MinInt64)) + + // There should be no series in the memory after snapshot error since WAL was removed. + require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) + require.Nil(t, head.series.getByHash(lbls.Hash(), lbls)) + require.Equal(t, uint64(0), head.NumSeries()) + + // Since the snapshot could replay certain series, we continue invoking the create hooks. + // In such instances, we need to ensure that we also trigger the delete hooks when resetting the memory. + require.Equal(t, int64(2), c.created.Load()) + require.Equal(t, int64(2), c.deleted.Load()) + + require.Equal(t, 2.0, prom_testutil.ToFloat64(head.metrics.seriesRemoved)) + require.Equal(t, 2.0, prom_testutil.ToFloat64(head.metrics.seriesCreated)) +} + +func TestHistogramMetrics(t *testing.T) { + numHistograms := 10 + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + expHSeries, expHSamples := 0, 0 + + for x := range 5 { + expHSeries++ + l := labels.FromStrings("a", fmt.Sprintf("b%d", x)) + for i, h := range tsdbutil.GenerateTestHistograms(numHistograms) { + app := head.Appender(context.Background()) + _, err := app.AppendHistogram(0, l, int64(i), h, nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + expHSamples++ + } + for i, fh := range tsdbutil.GenerateTestFloatHistograms(numHistograms) { + app := head.Appender(context.Background()) + _, err := app.AppendHistogram(0, l, int64(numHistograms+i), nil, fh) + require.NoError(t, err) + require.NoError(t, app.Commit()) + expHSamples++ + } + } + + require.Equal(t, float64(expHSamples), prom_testutil.ToFloat64(head.metrics.samplesAppended.WithLabelValues(sampleMetricTypeHistogram))) + + require.NoError(t, head.Close()) + w, err := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(0)) + + require.Equal(t, float64(0), prom_testutil.ToFloat64(head.metrics.samplesAppended.WithLabelValues(sampleMetricTypeHistogram))) // Counter reset. +} + +func TestHistogramStaleSample(t *testing.T) { + t.Run("integer histogram", func(t *testing.T) { + testHistogramStaleSampleHelper(t, false) + }) + t.Run("float histogram", func(t *testing.T) { + testHistogramStaleSampleHelper(t, true) + }) +} + +func testHistogramStaleSampleHelper(t *testing.T, floatHistogram bool) { + t.Helper() + l := labels.FromStrings("a", "b") + numHistograms := 20 + head, _ := newTestHead(t, 100000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + type timedHistogram struct { + t int64 + h *histogram.Histogram + fh *histogram.FloatHistogram + } + expHistograms := make([]timedHistogram, 0, numHistograms) + + testQuery := func(numStale int) { + q, err := NewBlockQuerier(head, head.MinTime(), head.MaxTime()) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, q.Close()) + }) + + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + require.True(t, ss.Next()) + s := ss.At() + require.False(t, ss.Next()) + + it := s.Iterator(nil) + actHistograms := make([]timedHistogram, 0, len(expHistograms)) + for typ := it.Next(); typ != chunkenc.ValNone; typ = it.Next() { + switch typ { + case chunkenc.ValHistogram: + t, h := it.AtHistogram(nil) + actHistograms = append(actHistograms, timedHistogram{t: t, h: h}) + case chunkenc.ValFloatHistogram: + t, h := it.AtFloatHistogram(nil) + actHistograms = append(actHistograms, timedHistogram{t: t, fh: h}) + } + } + + // We cannot compare StaleNAN with require.Equal, hence checking each histogram manually. + require.Len(t, actHistograms, len(expHistograms)) + actNumStale := 0 + for i, eh := range expHistograms { + ah := actHistograms[i] + if floatHistogram { + switch { + case value.IsStaleNaN(eh.fh.Sum): + actNumStale++ + require.True(t, value.IsStaleNaN(ah.fh.Sum)) + // To make require.Equal work. + ah.fh.Sum = 0 + eh.fh = eh.fh.Copy() + eh.fh.Sum = 0 + case i > 0: + prev := expHistograms[i-1] + if prev.fh == nil || value.IsStaleNaN(prev.fh.Sum) { + eh.fh.CounterResetHint = histogram.UnknownCounterReset + } + } + require.Equal(t, eh, ah) + } else { + switch { + case value.IsStaleNaN(eh.h.Sum): + actNumStale++ + require.True(t, value.IsStaleNaN(ah.h.Sum)) + // To make require.Equal work. + ah.h.Sum = 0 + eh.h = eh.h.Copy() + eh.h.Sum = 0 + case i > 0: + prev := expHistograms[i-1] + if prev.h == nil || value.IsStaleNaN(prev.h.Sum) { + eh.h.CounterResetHint = histogram.UnknownCounterReset + } + } + require.Equal(t, eh, ah) + } + } + require.Equal(t, numStale, actNumStale) + } + + // Adding stale in the same appender. + app := head.Appender(context.Background()) + for _, h := range tsdbutil.GenerateTestHistograms(numHistograms) { + var err error + if floatHistogram { + _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), nil, h.ToFloat(nil)) + expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), fh: h.ToFloat(nil)}) + } else { + _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), h, nil) + expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), h: h}) + } + require.NoError(t, err) + } + // +1 so that delta-of-delta is not 0. + _, err := app.Append(0, l, 100*int64(len(expHistograms))+1, math.Float64frombits(value.StaleNaN)) + require.NoError(t, err) + if floatHistogram { + expHistograms = append(expHistograms, timedHistogram{t: 100*int64(len(expHistograms)) + 1, fh: &histogram.FloatHistogram{Sum: math.Float64frombits(value.StaleNaN)}}) + } else { + expHistograms = append(expHistograms, timedHistogram{t: 100*int64(len(expHistograms)) + 1, h: &histogram.Histogram{Sum: math.Float64frombits(value.StaleNaN)}}) + } + require.NoError(t, app.Commit()) + + // Only 1 chunk in the memory, no m-mapped chunk. + s := head.series.getByHash(l.Hash(), l) + require.NotNil(t, s) + require.NotNil(t, s.headChunks) + require.Equal(t, 1, s.headChunks.len()) + require.Empty(t, s.mmappedChunks) + testQuery(1) + + // Adding stale in different appender and continuing series after a stale sample. + app = head.Appender(context.Background()) + for _, h := range tsdbutil.GenerateTestHistograms(2 * numHistograms)[numHistograms:] { + var err error + if floatHistogram { + _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), nil, h.ToFloat(nil)) + expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), fh: h.ToFloat(nil)}) + } else { + _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), h, nil) + expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), h: h}) + } + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + app = head.Appender(context.Background()) + // +1 so that delta-of-delta is not 0. + _, err = app.Append(0, l, 100*int64(len(expHistograms))+1, math.Float64frombits(value.StaleNaN)) + require.NoError(t, err) + if floatHistogram { + expHistograms = append(expHistograms, timedHistogram{t: 100*int64(len(expHistograms)) + 1, fh: &histogram.FloatHistogram{Sum: math.Float64frombits(value.StaleNaN)}}) + } else { + expHistograms = append(expHistograms, timedHistogram{t: 100*int64(len(expHistograms)) + 1, h: &histogram.Histogram{Sum: math.Float64frombits(value.StaleNaN)}}) + } + require.NoError(t, app.Commit()) + head.mmapHeadChunks() + + // Total 2 chunks, 1 m-mapped. + s = head.series.getByHash(l.Hash(), l) + require.NotNil(t, s) + require.NotNil(t, s.headChunks) + require.Equal(t, 1, s.headChunks.len()) + require.Len(t, s.mmappedChunks, 1) + testQuery(2) +} + +func TestHistogramCounterResetHeader(t *testing.T) { + for _, floatHisto := range []bool{true} { // FIXME + t.Run(fmt.Sprintf("floatHistogram=%t", floatHisto), func(t *testing.T) { + l := labels.FromStrings("a", "b") + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + ts := int64(0) + appendHistogram := func(h *histogram.Histogram) { + ts++ + app := head.Appender(context.Background()) + var err error + if floatHisto { + _, err = app.AppendHistogram(0, l, ts, nil, h.ToFloat(nil)) + } else { + _, err = app.AppendHistogram(0, l, ts, h.Copy(), nil) + } + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + var expHeaders []chunkenc.CounterResetHeader + checkExpCounterResetHeader := func(newHeaders ...chunkenc.CounterResetHeader) { + expHeaders = append(expHeaders, newHeaders...) + + ms, _, err := head.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + ms.mmapChunks(head.chunkDiskMapper) + require.Len(t, ms.mmappedChunks, len(expHeaders)-1) // One is the head chunk. + + for i, mmapChunk := range ms.mmappedChunks { + chk, err := head.chunkDiskMapper.Chunk(mmapChunk.ref) + require.NoError(t, err) + if floatHisto { + require.Equal(t, expHeaders[i], chk.(*chunkenc.FloatHistogramChunk).GetCounterResetHeader()) + } else { + require.Equal(t, expHeaders[i], chk.(*chunkenc.HistogramChunk).GetCounterResetHeader()) + } + } + if floatHisto { + require.Equal(t, expHeaders[len(expHeaders)-1], ms.headChunks.chunk.(*chunkenc.FloatHistogramChunk).GetCounterResetHeader()) + } else { + require.Equal(t, expHeaders[len(expHeaders)-1], ms.headChunks.chunk.(*chunkenc.HistogramChunk).GetCounterResetHeader()) + } + } + + h := tsdbutil.GenerateTestHistograms(1)[0] + h.PositiveBuckets = []int64{100, 1, 1, 1} + h.NegativeBuckets = []int64{100, 1, 1, 1} + h.Count = 1000 + + // First histogram is UnknownCounterReset. + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.UnknownCounterReset) + + // Another normal histogram. + h.Count++ + appendHistogram(h) + checkExpCounterResetHeader() + + // Counter reset via Count. + h.Count-- + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.CounterReset) + + // Add 2 non-counter reset histogram chunks (each chunk targets 1024 bytes which contains ~500 int histogram + // samples or ~1000 float histogram samples). + numAppend := 2000 + if floatHisto { + numAppend = 1000 + } + for i := 0; i < numAppend; i++ { + appendHistogram(h) + } + + checkExpCounterResetHeader(chunkenc.NotCounterReset, chunkenc.NotCounterReset) + + // Changing schema will cut a new chunk with unknown counter reset. + h.Schema++ + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.UnknownCounterReset) + + // Changing schema will zero threshold a new chunk with unknown counter reset. + h.ZeroThreshold += 0.01 + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.UnknownCounterReset) + + // Counter reset by removing a positive bucket. + h.PositiveSpans[1].Length-- + h.PositiveBuckets = h.PositiveBuckets[1:] + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.CounterReset) + + // Counter reset by removing a negative bucket. + h.NegativeSpans[1].Length-- + h.NegativeBuckets = h.NegativeBuckets[1:] + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.CounterReset) + + // Add 2 non-counter reset histogram chunks. Just to have some non-counter reset chunks in between. + for range 2000 { + appendHistogram(h) + } + checkExpCounterResetHeader(chunkenc.NotCounterReset, chunkenc.NotCounterReset) + + // Counter reset with counter reset in a positive bucket. + h.PositiveBuckets[len(h.PositiveBuckets)-1]-- + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.CounterReset) + + // Counter reset with counter reset in a negative bucket. + h.NegativeBuckets[len(h.NegativeBuckets)-1]-- + appendHistogram(h) + checkExpCounterResetHeader(chunkenc.CounterReset) + }) + } +} + +func TestOOOHistogramCounterResetHeaders(t *testing.T) { + for _, floatHisto := range []bool{true, false} { + t.Run(fmt.Sprintf("floatHistogram=%t", floatHisto), func(t *testing.T) { + l := labels.FromStrings("a", "b") + head, _ := newTestHead(t, 1000, compression.None, true) + head.opts.OutOfOrderCapMax.Store(5) + + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + appendHistogram := func(ts int64, h *histogram.Histogram) { + app := head.Appender(context.Background()) + var err error + if floatHisto { + _, err = app.AppendHistogram(0, l, ts, nil, h.ToFloat(nil)) + } else { + _, err = app.AppendHistogram(0, l, ts, h.Copy(), nil) + } + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + type expOOOMmappedChunks struct { + header chunkenc.CounterResetHeader + mint, maxt int64 + numSamples uint16 + } + + var expChunks []expOOOMmappedChunks + checkOOOExpCounterResetHeader := func(newChunks ...expOOOMmappedChunks) { + expChunks = append(expChunks, newChunks...) + + ms, _, err := head.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + + require.Len(t, ms.ooo.oooMmappedChunks, len(expChunks)) + + for i, mmapChunk := range ms.ooo.oooMmappedChunks { + chk, err := head.chunkDiskMapper.Chunk(mmapChunk.ref) + require.NoError(t, err) + if floatHisto { + require.Equal(t, expChunks[i].header, chk.(*chunkenc.FloatHistogramChunk).GetCounterResetHeader()) + } else { + require.Equal(t, expChunks[i].header, chk.(*chunkenc.HistogramChunk).GetCounterResetHeader()) + } + require.Equal(t, expChunks[i].mint, mmapChunk.minTime) + require.Equal(t, expChunks[i].maxt, mmapChunk.maxTime) + require.Equal(t, expChunks[i].numSamples, mmapChunk.numSamples) + } + } + + // Append an in-order histogram, so the rest of the samples can be detected as OOO. + appendHistogram(1000, tsdbutil.GenerateTestHistogram(1000)) + + // OOO histogram + for i := 1; i <= 5; i++ { + appendHistogram(100+int64(i), tsdbutil.GenerateTestHistogram(1000+int64(i))) + } + // Nothing mmapped yet. + checkOOOExpCounterResetHeader() + + // 6th observation (which triggers a head chunk mmapping). + appendHistogram(int64(112), tsdbutil.GenerateTestHistogram(1002)) + + // One mmapped chunk with (ts, val) [(101, 1001), (102, 1002), (103, 1003), (104, 1004), (105, 1005)]. + checkOOOExpCounterResetHeader(expOOOMmappedChunks{ + header: chunkenc.UnknownCounterReset, + mint: 101, + maxt: 105, + numSamples: 5, + }) + + // Add more samples, there's a counter reset at ts 122. + appendHistogram(int64(110), tsdbutil.GenerateTestHistogram(1001)) + appendHistogram(int64(124), tsdbutil.GenerateTestHistogram(904)) + appendHistogram(int64(123), tsdbutil.GenerateTestHistogram(903)) + appendHistogram(int64(122), tsdbutil.GenerateTestHistogram(902)) + + // New samples not mmapped yet. + checkOOOExpCounterResetHeader() + + // 11th observation (which triggers another head chunk mmapping). + appendHistogram(int64(200), tsdbutil.GenerateTestHistogram(2000)) + + // Two new mmapped chunks [(110, 1001), (112, 1002)], [(122, 902), (123, 903), (124, 904)]. + checkOOOExpCounterResetHeader( + expOOOMmappedChunks{ + header: chunkenc.UnknownCounterReset, + mint: 110, + maxt: 112, + numSamples: 2, + }, + expOOOMmappedChunks{ + header: chunkenc.CounterReset, + mint: 122, + maxt: 124, + numSamples: 3, + }, + ) + + // Count is lower than previous sample at ts 200, and NotCounterReset is always ignored on append. + appendHistogram(int64(205), tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(1000))) + + appendHistogram(int64(210), tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(2010))) + + appendHistogram(int64(220), tsdbutil.GenerateTestHistogram(2020)) + + appendHistogram(int64(215), tsdbutil.GenerateTestHistogram(2005)) + + // 16th observation (which triggers another head chunk mmapping). + appendHistogram(int64(350), tsdbutil.GenerateTestHistogram(4000)) + + // Four new mmapped chunks: [(200, 2000)] [(205, 1000)], [(210, 2010)], [(215, 2015), (220, 2020)] + checkOOOExpCounterResetHeader( + expOOOMmappedChunks{ + header: chunkenc.UnknownCounterReset, + mint: 200, + maxt: 200, + numSamples: 1, + }, + expOOOMmappedChunks{ + header: chunkenc.CounterReset, + mint: 205, + maxt: 205, + numSamples: 1, + }, + expOOOMmappedChunks{ + header: chunkenc.CounterReset, + mint: 210, + maxt: 210, + numSamples: 1, + }, + expOOOMmappedChunks{ + header: chunkenc.CounterReset, + mint: 215, + maxt: 220, + numSamples: 2, + }, + ) + + // Adding five more samples (21 in total), so another mmapped chunk is created. + appendHistogram(300, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(3000))) + + for i := 1; i <= 4; i++ { + appendHistogram(300+int64(i), tsdbutil.GenerateTestHistogram(3000+int64(i))) + } + + // One mmapped chunk with (ts, val) [(300, 3000), (301, 3001), (302, 3002), (303, 3003), (350, 4000)]. + checkOOOExpCounterResetHeader(expOOOMmappedChunks{ + header: chunkenc.CounterReset, + mint: 300, + maxt: 350, + numSamples: 5, + }) + }) + } +} + +func TestAppendingDifferentEncodingToSameSeries(t *testing.T) { + dir := t.TempDir() + opts := DefaultOptions() + db, err := Open(dir, nil, nil, opts, nil) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, db.Close()) + }) + db.DisableCompactions() + + hists := tsdbutil.GenerateTestHistograms(10) + floatHists := tsdbutil.GenerateTestFloatHistograms(10) + lbls := labels.FromStrings("a", "b") + + var expResult []chunks.Sample + checkExpChunks := func(count int) { + ms, created, err := db.Head().getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.NotNil(t, ms) + require.Equal(t, count, ms.headChunks.len()) + } + + appends := []struct { + samples []chunks.Sample + expChunks int + err error + }{ + // Histograms that end up in the expected samples are copied here so that we + // can independently set the CounterResetHint later. + { + samples: []chunks.Sample{sample{t: 100, h: hists[0].Copy()}}, + expChunks: 1, + }, + { + samples: []chunks.Sample{sample{t: 200, f: 2}}, + expChunks: 2, + }, + { + samples: []chunks.Sample{sample{t: 210, fh: floatHists[0].Copy()}}, + expChunks: 3, + }, + { + samples: []chunks.Sample{sample{t: 220, h: hists[1].Copy()}}, + expChunks: 4, + }, + { + samples: []chunks.Sample{sample{t: 230, fh: floatHists[3].Copy()}}, + expChunks: 5, + }, + { + samples: []chunks.Sample{sample{t: 100, h: hists[2].Copy()}}, + err: storage.ErrOutOfOrderSample, + }, + { + samples: []chunks.Sample{sample{t: 300, h: hists[3].Copy()}}, + expChunks: 6, + }, + { + samples: []chunks.Sample{sample{t: 100, f: 2}}, + err: storage.ErrOutOfOrderSample, + }, + { + samples: []chunks.Sample{sample{t: 100, fh: floatHists[4].Copy()}}, + err: storage.ErrOutOfOrderSample, + }, + // The three next tests all failed before #15177 was fixed. + { + samples: []chunks.Sample{ + sample{t: 400, f: 4}, + sample{t: 500, h: hists[5]}, + sample{t: 600, f: 6}, + }, + expChunks: 9, // Each of the three samples above creates a new chunk because the type changes. + }, + { + samples: []chunks.Sample{ + sample{t: 700, h: hists[7]}, + sample{t: 800, f: 8}, + sample{t: 900, h: hists[9]}, + }, + expChunks: 12, // Again each sample creates a new chunk. + }, + { + samples: []chunks.Sample{ + sample{t: 1000, fh: floatHists[7]}, + sample{t: 1100, h: hists[9]}, + }, + expChunks: 14, // Even changes between float and integer histogram create new chunks. + }, + } + + for _, a := range appends { + app := db.Appender(context.Background()) + for _, s := range a.samples { + var err error + if s.H() != nil || s.FH() != nil { + _, err = app.AppendHistogram(0, lbls, s.T(), s.H(), s.FH()) + } else { + _, err = app.Append(0, lbls, s.T(), s.F()) + } + require.Equal(t, a.err, err) + } + + if a.err == nil { + require.NoError(t, app.Commit()) + expResult = append(expResult, a.samples...) + checkExpChunks(a.expChunks) + } else { + require.NoError(t, app.Rollback()) + } + } + for i, s := range expResult[1:] { + switch { + case s.H() != nil && expResult[i].H() == nil: + s.(sample).h.CounterResetHint = histogram.UnknownCounterReset + case s.FH() != nil && expResult[i].FH() == nil: + s.(sample).fh.CounterResetHint = histogram.UnknownCounterReset + } + } + + // Query back and expect same order of samples. + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + series := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.Equal(t, map[string][]chunks.Sample{lbls.String(): expResult}, series) +} + +// Tests https://github.com/prometheus/prometheus/issues/9725. +func TestChunkSnapshotReplayBug(t *testing.T) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + + // Write few series records and samples such that the series references are not in order in the WAL + // for status_code="200". + var buf []byte + for i := 1; i <= 1000; i++ { + var ref chunks.HeadSeriesRef + if i <= 500 { + ref = chunks.HeadSeriesRef(i * 100) + } else { + ref = chunks.HeadSeriesRef((i - 500) * 50) + } + seriesRec := record.RefSeries{ + Ref: ref, + Labels: labels.FromStrings( + "__name__", "request_duration", + "status_code", "200", + "foo", fmt.Sprintf("baz%d", rand.Int()), + ), + } + // Add a sample so that the series is not garbage collected. + samplesRec := record.RefSample{Ref: ref, T: 1000, V: 1000} + var enc record.Encoder + + rec := enc.Series([]record.RefSeries{seriesRec}, buf) + buf = rec[:0] + require.NoError(t, wal.Log(rec)) + rec = enc.Samples([]record.RefSample{samplesRec}, buf) + buf = rec[:0] + require.NoError(t, wal.Log(rec)) + } + + // Write a corrupt snapshot to fail the replay on startup. + snapshotName := chunkSnapshotDir(0, 100) + cpdir := filepath.Join(dir, snapshotName) + require.NoError(t, os.MkdirAll(cpdir, 0o777)) + + err = os.WriteFile(filepath.Join(cpdir, "00000000"), []byte{1, 5, 3, 5, 6, 7, 4, 2, 2}, 0o777) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkDirRoot = dir + opts.EnableMemorySnapshotOnShutdown = true + head, err := NewHead(nil, nil, wal, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(math.MinInt64)) + defer func() { + require.NoError(t, head.Close()) + }() + + // Snapshot replay should error out. + require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) + + // Querying `request_duration{status_code!="200"}` should return no series since all of + // them have status_code="200". + q, err := NewBlockQuerier(head, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + series := query(t, q, + labels.MustNewMatcher(labels.MatchEqual, "__name__", "request_duration"), + labels.MustNewMatcher(labels.MatchNotEqual, "status_code", "200"), + ) + require.Empty(t, series, "there should be no series found") +} + +func TestChunkSnapshotTakenAfterIncompleteSnapshot(t *testing.T) { + dir := t.TempDir() + wlTemp, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + + // Write a snapshot with .tmp suffix. This used to fail taking any further snapshots or replay of snapshots. + snapshotName := chunkSnapshotDir(0, 100) + ".tmp" + cpdir := filepath.Join(dir, snapshotName) + require.NoError(t, os.MkdirAll(cpdir, 0o777)) + + opts := DefaultHeadOptions() + opts.ChunkDirRoot = dir + opts.EnableMemorySnapshotOnShutdown = true + head, err := NewHead(nil, nil, wlTemp, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(math.MinInt64)) + + require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) + + // Add some samples for the snapshot. + app := head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 10, 10) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // Should not return any error for a successful snapshot. + require.NoError(t, head.Close()) + + // Verify the snapshot. + name, idx, offset, err := LastChunkSnapshot(dir) + require.NoError(t, err) + require.NotEmpty(t, name) + require.Equal(t, 0, idx) + require.Positive(t, offset) +} + +// TestWBLReplay checks the replay at a low level. +func TestWBLReplay(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testWBLReplay(t, scenario) + }) + } +} + +func testWBLReplay(t *testing.T, scenario sampleTypeScenario) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + oooWlog, err := wlog.NewSize(nil, nil, filepath.Join(dir, wlog.WblDirName), 32768, compression.Snappy) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = dir + opts.OutOfOrderTimeWindow.Store(30 * time.Minute.Milliseconds()) + + h, err := NewHead(nil, nil, wal, oooWlog, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) + + var expOOOSamples []chunks.Sample + l := labels.FromStrings("foo", "bar") + appendSample := func(mins int64, _ float64, isOOO bool) { + app := h.Appender(context.Background()) + _, s, err := scenario.appendFunc(app, l, mins*time.Minute.Milliseconds(), mins) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + if isOOO { + expOOOSamples = append(expOOOSamples, s) + } + } + + // In-order sample. + appendSample(60, 60, false) + + // Out of order samples. + appendSample(40, 40, true) + appendSample(35, 35, true) + appendSample(50, 50, true) + appendSample(55, 55, true) + appendSample(59, 59, true) + appendSample(31, 31, true) + + // Check that Head's time ranges are set properly. + require.Equal(t, 60*time.Minute.Milliseconds(), h.MinTime()) + require.Equal(t, 60*time.Minute.Milliseconds(), h.MaxTime()) + require.Equal(t, 31*time.Minute.Milliseconds(), h.MinOOOTime()) + require.Equal(t, 59*time.Minute.Milliseconds(), h.MaxOOOTime()) + + // Restart head. + require.NoError(t, h.Close()) + wal, err = wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + oooWlog, err = wlog.NewSize(nil, nil, filepath.Join(dir, wlog.WblDirName), 32768, compression.Snappy) + require.NoError(t, err) + h, err = NewHead(nil, nil, wal, oooWlog, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) // Replay happens here. + + // Get the ooo samples from the Head. + ms, ok, err := h.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + require.False(t, ok) + require.NotNil(t, ms) + + chks, err := ms.ooo.oooHeadChunk.chunk.ToEncodedChunks(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + require.Len(t, chks, 1) + + it := chks[0].chunk.Iterator(nil) + actOOOSamples, err := storage.ExpandSamples(it, nil) + require.NoError(t, err) + + // OOO chunk will be sorted. Hence sort the expected samples. + sort.Slice(expOOOSamples, func(i, j int) bool { + return expOOOSamples[i].T() < expOOOSamples[j].T() + }) + + // Passing in true for the 'ignoreCounterResets' parameter prevents differences in counter reset headers + // from being factored in to the sample comparison + // TODO(fionaliao): understand counter reset behaviour, might want to modify this later + requireEqualSamples(t, l.String(), expOOOSamples, actOOOSamples, requireEqualSamplesIgnoreCounterResets) + + require.NoError(t, h.Close()) +} + +// TestOOOMmapReplay checks the replay at a low level. +func TestOOOMmapReplay(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOMmapReplay(t, scenario) + }) + } +} + +func testOOOMmapReplay(t *testing.T, scenario sampleTypeScenario) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + oooWlog, err := wlog.NewSize(nil, nil, filepath.Join(dir, wlog.WblDirName), 32768, compression.Snappy) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = dir + opts.OutOfOrderCapMax.Store(30) + opts.OutOfOrderTimeWindow.Store(1000 * time.Minute.Milliseconds()) + + h, err := NewHead(nil, nil, wal, oooWlog, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) + + l := labels.FromStrings("foo", "bar") + appendSample := func(mins int64) { + app := h.Appender(context.Background()) + _, _, err := scenario.appendFunc(app, l, mins*time.Minute.Milliseconds(), mins) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + // In-order sample. + appendSample(200) + + // Out of order samples. 92 samples to create 3 m-map chunks. + for mins := int64(100); mins <= 191; mins++ { + appendSample(mins) + } + + ms, ok, err := h.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + require.False(t, ok) + require.NotNil(t, ms) + + require.Len(t, ms.ooo.oooMmappedChunks, 3) + // Verify that we can access the chunks without error. + for _, m := range ms.ooo.oooMmappedChunks { + chk, err := h.chunkDiskMapper.Chunk(m.ref) + require.NoError(t, err) + require.Equal(t, int(m.numSamples), chk.NumSamples()) + } + + expMmapChunks := make([]*mmappedChunk, 3) + copy(expMmapChunks, ms.ooo.oooMmappedChunks) + + // Restart head. + require.NoError(t, h.Close()) + + wal, err = wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + oooWlog, err = wlog.NewSize(nil, nil, filepath.Join(dir, wlog.WblDirName), 32768, compression.Snappy) + require.NoError(t, err) + h, err = NewHead(nil, nil, wal, oooWlog, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) // Replay happens here. + + // Get the mmap chunks from the Head. + ms, ok, err = h.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + require.False(t, ok) + require.NotNil(t, ms) + + require.Len(t, ms.ooo.oooMmappedChunks, len(expMmapChunks)) + // Verify that we can access the chunks without error. + for _, m := range ms.ooo.oooMmappedChunks { + chk, err := h.chunkDiskMapper.Chunk(m.ref) + require.NoError(t, err) + require.Equal(t, int(m.numSamples), chk.NumSamples()) + } + + actMmapChunks := make([]*mmappedChunk, len(expMmapChunks)) + copy(actMmapChunks, ms.ooo.oooMmappedChunks) + + require.Equal(t, expMmapChunks, actMmapChunks) + + require.NoError(t, h.Close()) +} + +func TestHeadInit_DiscardChunksWithUnsupportedEncoding(t *testing.T) { + h, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + require.NoError(t, h.Init(0)) + + ctx := context.Background() + app := h.Appender(ctx) + seriesLabels := labels.FromStrings("a", "1") + var seriesRef storage.SeriesRef + var err error + for i := range 400 { + seriesRef, err = app.Append(0, seriesLabels, int64(i), float64(i)) + require.NoError(t, err) + } + + require.NoError(t, app.Commit()) + require.Greater(t, prom_testutil.ToFloat64(h.metrics.chunksCreated), 1.0) + + uc := newUnsupportedChunk() + // Make this chunk not overlap with the previous and the next + h.chunkDiskMapper.WriteChunk(chunks.HeadSeriesRef(seriesRef), 500, 600, uc, false, func(err error) { require.NoError(t, err) }) + + app = h.Appender(ctx) + for i := 700; i < 1200; i++ { + _, err := app.Append(0, seriesLabels, int64(i), float64(i)) + require.NoError(t, err) + } + + require.NoError(t, app.Commit()) + require.Greater(t, prom_testutil.ToFloat64(h.metrics.chunksCreated), 4.0) + + series, created, err := h.getOrCreate(seriesLabels.Hash(), seriesLabels, false) + require.NoError(t, err) + require.False(t, created, "should already exist") + require.NotNil(t, series, "should return the series we created above") + + series.mmapChunks(h.chunkDiskMapper) + expChunks := make([]*mmappedChunk, len(series.mmappedChunks)) + copy(expChunks, series.mmappedChunks) + + require.NoError(t, h.Close()) + + wal, err := wlog.NewSize(nil, nil, filepath.Join(h.opts.ChunkDirRoot, "wal"), 32768, compression.None) + require.NoError(t, err) + h, err = NewHead(nil, nil, wal, nil, h.opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) + + series, created, err = h.getOrCreate(seriesLabels.Hash(), seriesLabels, false) + require.NoError(t, err) + require.False(t, created, "should already exist") + require.NotNil(t, series, "should return the series we created above") + + require.Equal(t, expChunks, series.mmappedChunks) +} + +const ( + UnsupportedMask = 0b10000000 + EncUnsupportedXOR = chunkenc.EncXOR | UnsupportedMask +) + +// unsupportedChunk holds a XORChunk and overrides the Encoding() method. +type unsupportedChunk struct { + *chunkenc.XORChunk +} + +func newUnsupportedChunk() *unsupportedChunk { + return &unsupportedChunk{chunkenc.NewXORChunk()} +} + +func (*unsupportedChunk) Encoding() chunkenc.Encoding { + return EncUnsupportedXOR +} + +// Tests https://github.com/prometheus/prometheus/issues/10277. +func TestMmapPanicAfterMmapReplayCorruption(t *testing.T) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.None) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkRange = DefaultBlockDuration + opts.ChunkDirRoot = dir + opts.EnableExemplarStorage = true + opts.MaxExemplars.Store(config.DefaultExemplarsConfig.MaxExemplars) + + h, err := NewHead(nil, nil, wal, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) + + lastTs := int64(0) + var ref storage.SeriesRef + lbls := labels.FromStrings("__name__", "testing", "foo", "bar") + addChunks := func() { + interval := DefaultBlockDuration / (4 * 120) + app := h.Appender(context.Background()) + for i := range 250 { + ref, err = app.Append(ref, lbls, lastTs, float64(lastTs)) + lastTs += interval + if i%10 == 0 { + require.NoError(t, app.Commit()) + app = h.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + } + + addChunks() + + require.NoError(t, h.Close()) + wal, err = wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.None) + require.NoError(t, err) + + mmapFilePath := filepath.Join(dir, "chunks_head", "000001") + f, err := os.OpenFile(mmapFilePath, os.O_WRONLY, 0o666) + require.NoError(t, err) + _, err = f.WriteAt([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}, 17) + require.NoError(t, err) + require.NoError(t, f.Close()) + + h, err = NewHead(nil, nil, wal, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) + + addChunks() + + require.NoError(t, h.Close()) +} + +// Tests https://github.com/prometheus/prometheus/issues/10277. +func TestReplayAfterMmapReplayError(t *testing.T) { + dir := t.TempDir() + var h *Head + var err error + + openHead := func() { + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.None) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkRange = DefaultBlockDuration + opts.ChunkDirRoot = dir + opts.EnableMemorySnapshotOnShutdown = true + opts.MaxExemplars.Store(config.DefaultExemplarsConfig.MaxExemplars) + + h, err = NewHead(nil, nil, wal, nil, opts, nil) + require.NoError(t, err) + require.NoError(t, h.Init(0)) + } + + openHead() + + itvl := int64(15 * time.Second / time.Millisecond) + lastTs := int64(0) + lbls := labels.FromStrings("__name__", "testing", "foo", "bar") + var expSamples []chunks.Sample + addSamples := func(numSamples int) { + app := h.Appender(context.Background()) + var ref storage.SeriesRef + for i := range numSamples { + ref, err = app.Append(ref, lbls, lastTs, float64(lastTs)) + expSamples = append(expSamples, sample{t: lastTs, f: float64(lastTs)}) + require.NoError(t, err) + lastTs += itvl + if i%10 == 0 { + require.NoError(t, app.Commit()) + app = h.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + } + + // Creating multiple m-map files. + for i := range 5 { + addSamples(250) + require.NoError(t, h.Close()) + if i != 4 { + // Don't open head for the last iteration. + openHead() + } + } + + files, err := os.ReadDir(filepath.Join(dir, "chunks_head")) + require.Len(t, files, 5) + + // Corrupt a m-map file. + mmapFilePath := filepath.Join(dir, "chunks_head", "000002") + f, err := os.OpenFile(mmapFilePath, os.O_WRONLY, 0o666) + require.NoError(t, err) + _, err = f.WriteAt([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}, 17) + require.NoError(t, err) + require.NoError(t, f.Close()) + + openHead() + h.mmapHeadChunks() + + // There should be less m-map files due to corruption. + files, err = os.ReadDir(filepath.Join(dir, "chunks_head")) + require.Len(t, files, 2) + + // Querying should not panic. + q, err := NewBlockQuerier(h, 0, lastTs) + require.NoError(t, err) + res := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "__name__", "testing")) + require.Equal(t, map[string][]chunks.Sample{lbls.String(): expSamples}, res) + + require.NoError(t, h.Close()) +} + +func TestOOOAppendWithNoSeries(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOAppendWithNoSeries(t, scenario.appendFunc) + }) + } +} + +func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error)) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + oooWlog, err := wlog.NewSize(nil, nil, filepath.Join(dir, wlog.WblDirName), 32768, compression.Snappy) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkDirRoot = dir + opts.OutOfOrderCapMax.Store(30) + opts.OutOfOrderTimeWindow.Store(120 * time.Minute.Milliseconds()) + + h, err := NewHead(nil, nil, wal, oooWlog, opts, nil) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, h.Close()) + }) + require.NoError(t, h.Init(0)) + + appendSample := func(lbls labels.Labels, ts int64) { + app := h.Appender(context.Background()) + _, _, err := appendFunc(app, lbls, ts*time.Minute.Milliseconds(), ts) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + verifyOOOSamples := func(lbls labels.Labels, expSamples int) { + ms, created, err := h.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.NotNil(t, ms) + + require.Nil(t, ms.headChunks) + require.NotNil(t, ms.ooo.oooHeadChunk) + require.Equal(t, expSamples, ms.ooo.oooHeadChunk.chunk.NumSamples()) + } + + verifyInOrderSamples := func(lbls labels.Labels, expSamples int) { + ms, created, err := h.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.NotNil(t, ms) + + require.Nil(t, ms.ooo) + require.NotNil(t, ms.headChunks) + require.Equal(t, expSamples, ms.headChunks.chunk.NumSamples()) + } + + newLabels := func(idx int) labels.Labels { return labels.FromStrings("foo", strconv.Itoa(idx)) } + + s1 := newLabels(1) + appendSample(s1, 300) // At 300m. + verifyInOrderSamples(s1, 1) + + // At 239m, the sample cannot be appended to in-order chunk since it is + // beyond the minValidTime. So it should go in OOO chunk. + // Series does not exist for s2 yet. + s2 := newLabels(2) + appendSample(s2, 239) // OOO sample. + verifyOOOSamples(s2, 1) + + // Similar for 180m. + s3 := newLabels(3) + appendSample(s3, 180) // OOO sample. + verifyOOOSamples(s3, 1) + + // Now 179m is too old. + s4 := newLabels(4) + app := h.Appender(context.Background()) + _, _, err = appendFunc(app, s4, 179*time.Minute.Milliseconds(), 179) + require.Equal(t, storage.ErrTooOldSample, err) + require.NoError(t, app.Rollback()) + verifyOOOSamples(s3, 1) + + // Samples still go into in-order chunk for samples within + // appendable minValidTime. + s5 := newLabels(5) + appendSample(s5, 240) + verifyInOrderSamples(s5, 1) +} + +func TestHeadMinOOOTimeUpdate(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + if scenario.sampleType == sampleMetricTypeFloat { + testHeadMinOOOTimeUpdate(t, scenario) + } + }) + } +} + +func testHeadMinOOOTimeUpdate(t *testing.T, scenario sampleTypeScenario) { + dir := t.TempDir() + wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) + require.NoError(t, err) + oooWlog, err := wlog.NewSize(nil, nil, filepath.Join(dir, wlog.WblDirName), 32768, compression.Snappy) + require.NoError(t, err) + + opts := DefaultHeadOptions() + opts.ChunkDirRoot = dir + opts.OutOfOrderTimeWindow.Store(10 * time.Minute.Milliseconds()) + + h, err := NewHead(nil, nil, wal, oooWlog, opts, nil) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, h.Close()) + }) + require.NoError(t, h.Init(0)) + + appendSample := func(ts int64) { + app := h.Appender(context.Background()) + _, _, err = scenario.appendFunc(app, labels.FromStrings("a", "b"), ts*time.Minute.Milliseconds(), ts) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + appendSample(300) // In-order sample. + require.Equal(t, int64(math.MaxInt64), h.MinOOOTime()) + + appendSample(295) // OOO sample. + require.Equal(t, 295*time.Minute.Milliseconds(), h.MinOOOTime()) + + // Allowed window for OOO is >=290, which is before the earliest ooo sample 295, so it gets set to the lower value. + require.NoError(t, h.truncateOOO(0, 1)) + require.Equal(t, 290*time.Minute.Milliseconds(), h.MinOOOTime()) + + appendSample(310) // In-order sample. + appendSample(305) // OOO sample. + require.Equal(t, 290*time.Minute.Milliseconds(), h.MinOOOTime()) + + // Now the OOO sample 295 was not gc'ed yet. And allowed window for OOO is now >=300. + // So the lowest among them, 295, is set as minOOOTime. + require.NoError(t, h.truncateOOO(0, 2)) + require.Equal(t, 295*time.Minute.Milliseconds(), h.MinOOOTime()) +} + +func TestGaugeHistogramWALAndChunkHeader(t *testing.T) { + l := labels.FromStrings("a", "b") + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + ts := int64(0) + appendHistogram := func(h *histogram.Histogram) { + ts++ + app := head.Appender(context.Background()) + _, err := app.AppendHistogram(0, l, ts, h.Copy(), nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + hists := tsdbutil.GenerateTestGaugeHistograms(5) + hists[0].CounterResetHint = histogram.UnknownCounterReset + appendHistogram(hists[0]) + appendHistogram(hists[1]) + appendHistogram(hists[2]) + hists[3].CounterResetHint = histogram.UnknownCounterReset + appendHistogram(hists[3]) + appendHistogram(hists[3]) + appendHistogram(hists[4]) + + checkHeaders := func() { + head.mmapHeadChunks() + ms, _, err := head.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + require.Len(t, ms.mmappedChunks, 3) + expHeaders := []chunkenc.CounterResetHeader{ + chunkenc.UnknownCounterReset, + chunkenc.GaugeType, + chunkenc.NotCounterReset, + chunkenc.GaugeType, + } + for i, mmapChunk := range ms.mmappedChunks { + chk, err := head.chunkDiskMapper.Chunk(mmapChunk.ref) + require.NoError(t, err) + require.Equal(t, expHeaders[i], chk.(*chunkenc.HistogramChunk).GetCounterResetHeader()) + } + require.Equal(t, expHeaders[len(expHeaders)-1], ms.headChunks.chunk.(*chunkenc.HistogramChunk).GetCounterResetHeader()) + } + checkHeaders() + + recs := readTestWAL(t, head.wal.Dir()) + require.Equal(t, []any{ + []record.RefSeries{ + { + Ref: 1, + Labels: labels.FromStrings("a", "b"), + }, + }, + []record.RefHistogramSample{{Ref: 1, T: 1, H: hists[0]}}, + []record.RefHistogramSample{{Ref: 1, T: 2, H: hists[1]}}, + []record.RefHistogramSample{{Ref: 1, T: 3, H: hists[2]}}, + []record.RefHistogramSample{{Ref: 1, T: 4, H: hists[3]}}, + []record.RefHistogramSample{{Ref: 1, T: 5, H: hists[3]}}, + []record.RefHistogramSample{{Ref: 1, T: 6, H: hists[4]}}, + }, recs) + + // Restart Head without mmap chunks to expect the WAL replay to recognize gauge histograms. + require.NoError(t, head.Close()) + require.NoError(t, os.RemoveAll(mmappedChunksDir(head.opts.ChunkDirRoot))) + + w, err := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(0)) + + checkHeaders() +} + +func TestGaugeFloatHistogramWALAndChunkHeader(t *testing.T) { + l := labels.FromStrings("a", "b") + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + ts := int64(0) + appendHistogram := func(h *histogram.FloatHistogram) { + ts++ + app := head.Appender(context.Background()) + _, err := app.AppendHistogram(0, l, ts, nil, h.Copy()) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + hists := tsdbutil.GenerateTestGaugeFloatHistograms(5) + hists[0].CounterResetHint = histogram.UnknownCounterReset + appendHistogram(hists[0]) + appendHistogram(hists[1]) + appendHistogram(hists[2]) + hists[3].CounterResetHint = histogram.UnknownCounterReset + appendHistogram(hists[3]) + appendHistogram(hists[3]) + appendHistogram(hists[4]) + + checkHeaders := func() { + ms, _, err := head.getOrCreate(l.Hash(), l, false) + require.NoError(t, err) + head.mmapHeadChunks() + require.Len(t, ms.mmappedChunks, 3) + expHeaders := []chunkenc.CounterResetHeader{ + chunkenc.UnknownCounterReset, + chunkenc.GaugeType, + chunkenc.UnknownCounterReset, + chunkenc.GaugeType, + } + for i, mmapChunk := range ms.mmappedChunks { + chk, err := head.chunkDiskMapper.Chunk(mmapChunk.ref) + require.NoError(t, err) + require.Equal(t, expHeaders[i], chk.(*chunkenc.FloatHistogramChunk).GetCounterResetHeader()) + } + require.Equal(t, expHeaders[len(expHeaders)-1], ms.headChunks.chunk.(*chunkenc.FloatHistogramChunk).GetCounterResetHeader()) + } + checkHeaders() + + recs := readTestWAL(t, head.wal.Dir()) + require.Equal(t, []any{ + []record.RefSeries{ + { + Ref: 1, + Labels: labels.FromStrings("a", "b"), + }, + }, + []record.RefFloatHistogramSample{{Ref: 1, T: 1, FH: hists[0]}}, + []record.RefFloatHistogramSample{{Ref: 1, T: 2, FH: hists[1]}}, + []record.RefFloatHistogramSample{{Ref: 1, T: 3, FH: hists[2]}}, + []record.RefFloatHistogramSample{{Ref: 1, T: 4, FH: hists[3]}}, + []record.RefFloatHistogramSample{{Ref: 1, T: 5, FH: hists[3]}}, + []record.RefFloatHistogramSample{{Ref: 1, T: 6, FH: hists[4]}}, + }, recs) + + // Restart Head without mmap chunks to expect the WAL replay to recognize gauge histograms. + require.NoError(t, head.Close()) + require.NoError(t, os.RemoveAll(mmappedChunksDir(head.opts.ChunkDirRoot))) + + w, err := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(0)) + + checkHeaders() +} + +func TestSnapshotAheadOfWALError(t *testing.T) { + head, _ := newTestHead(t, 120*4, compression.None, false) + head.opts.EnableMemorySnapshotOnShutdown = true + // Add a sample to fill WAL. + app := head.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 10, 10) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // Increment snapshot index to create sufficiently large difference. + for range 2 { + _, err = head.wal.NextSegment() + require.NoError(t, err) + } + require.NoError(t, head.Close()) // This will create a snapshot. + + _, idx, _, err := LastChunkSnapshot(head.opts.ChunkDirRoot) + require.NoError(t, err) + require.Equal(t, 2, idx) + + // Restart the WAL while keeping the old snapshot. The new head is created manually in this case in order + // to keep using the same snapshot directory instead of a random one. + require.NoError(t, os.RemoveAll(head.wal.Dir())) + head.opts.EnableMemorySnapshotOnShutdown = false + w, _ := wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + // Add a sample to fill WAL. + app = head.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 10, 10) + require.NoError(t, err) + require.NoError(t, app.Commit()) + lastSegment, _, _ := w.LastSegmentAndOffset() + require.Equal(t, 0, lastSegment) + require.NoError(t, head.Close()) + + // New WAL is saved, but old snapshot still exists. + _, idx, _, err = LastChunkSnapshot(head.opts.ChunkDirRoot) + require.NoError(t, err) + require.Equal(t, 2, idx) + + // Create new Head which should detect the incorrect index and delete the snapshot. + head.opts.EnableMemorySnapshotOnShutdown = true + w, _ = wlog.NewSize(nil, nil, head.wal.Dir(), 32768, compression.None) + head, err = NewHead(nil, nil, w, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(math.MinInt64)) + + // Verify that snapshot directory does not exist anymore. + _, _, _, err = LastChunkSnapshot(head.opts.ChunkDirRoot) + require.Equal(t, record.ErrNotFound, err) + + require.NoError(t, head.Close()) +} + +func BenchmarkCuttingHeadHistogramChunks(b *testing.B) { + const ( + numSamples = 50000 + numBuckets = 100 + ) + samples := histogram.GenerateBigTestHistograms(numSamples, numBuckets) + + h, _ := newTestHead(b, DefaultBlockDuration, compression.None, false) + defer func() { + require.NoError(b, h.Close()) + }() + + a := h.Appender(context.Background()) + ts := time.Now().UnixMilli() + lbls := labels.FromStrings("foo", "bar") + + b.ResetTimer() + + for _, s := range samples { + _, err := a.AppendHistogram(0, lbls, ts, s, nil) + require.NoError(b, err) + } +} + +func TestCuttingNewHeadChunks(t *testing.T) { + ctx := context.Background() + testCases := map[string]struct { + numTotalSamples int + timestampJitter bool + floatValFunc func(i int) float64 + histValFunc func(i int) *histogram.Histogram + expectedChks []struct { + numSamples int + numBytes int + } + }{ + "float samples": { + numTotalSamples: 180, + floatValFunc: func(int) float64 { + return 1. + }, + expectedChks: []struct { + numSamples int + numBytes int + }{ + {numSamples: 120, numBytes: 46}, + {numSamples: 60, numBytes: 32}, + }, + }, + "large float samples": { + // Normally 120 samples would fit into a single chunk but these chunks violate the 1005 byte soft cap. + numTotalSamples: 120, + timestampJitter: true, + floatValFunc: func(i int) float64 { + // Flipping between these two make each sample val take at least 64 bits. + vals := []float64{math.MaxFloat64, 0x00} + return vals[i%len(vals)] + }, + expectedChks: []struct { + numSamples int + numBytes int + }{ + {99, 1008}, + {21, 219}, + }, + }, + "small histograms": { + numTotalSamples: 240, + histValFunc: func() func(i int) *histogram.Histogram { + hists := histogram.GenerateBigTestHistograms(240, 10) + return func(i int) *histogram.Histogram { + return hists[i] + } + }(), + expectedChks: []struct { + numSamples int + numBytes int + }{ + {120, 1087}, + {120, 1039}, + }, + }, + "large histograms": { + numTotalSamples: 240, + histValFunc: func() func(i int) *histogram.Histogram { + hists := histogram.GenerateBigTestHistograms(240, 100) + return func(i int) *histogram.Histogram { + return hists[i] + } + }(), + expectedChks: []struct { + numSamples int + numBytes int + }{ + {40, 896}, + {40, 899}, + {40, 896}, + {30, 690}, + {30, 691}, + {30, 694}, + {30, 693}, + }, + }, + "really large histograms": { + // Really large histograms; each chunk can only contain a single histogram but we have a 10 sample minimum + // per chunk. + numTotalSamples: 11, + histValFunc: func() func(i int) *histogram.Histogram { + hists := histogram.GenerateBigTestHistograms(11, 100000) + return func(i int) *histogram.Histogram { + return hists[i] + } + }(), + expectedChks: []struct { + numSamples int + numBytes int + }{ + {10, 200103}, + {1, 87540}, + }, + }, + } + for testName, tc := range testCases { + t.Run(testName, func(t *testing.T) { + h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + a := h.Appender(context.Background()) + + ts := int64(10000) + lbls := labels.FromStrings("foo", "bar") + jitter := []int64{0, 1} // A bit of jitter to prevent dod=0. + + for i := 0; i < tc.numTotalSamples; i++ { + if tc.floatValFunc != nil { + _, err := a.Append(0, lbls, ts, tc.floatValFunc(i)) + require.NoError(t, err) + } else if tc.histValFunc != nil { + _, err := a.AppendHistogram(0, lbls, ts, tc.histValFunc(i), nil) + require.NoError(t, err) + } + + ts += int64(60 * time.Second / time.Millisecond) + if tc.timestampJitter { + ts += jitter[i%len(jitter)] + } + } + + require.NoError(t, a.Commit()) + + idxReader, err := h.Index() + require.NoError(t, err) + + chkReader, err := h.Chunks() + require.NoError(t, err) + + p, err := idxReader.Postings(ctx, "foo", "bar") + require.NoError(t, err) + + var lblBuilder labels.ScratchBuilder + + for p.Next() { + sRef := p.At() + + chkMetas := make([]chunks.Meta, len(tc.expectedChks)) + require.NoError(t, idxReader.Series(sRef, &lblBuilder, &chkMetas)) + + require.Len(t, chkMetas, len(tc.expectedChks)) + + for i, expected := range tc.expectedChks { + chk, iterable, err := chkReader.ChunkOrIterable(chkMetas[i]) + require.NoError(t, err) + require.Nil(t, iterable) + + require.Equal(t, expected.numSamples, chk.NumSamples()) + require.Len(t, chk.Bytes(), expected.numBytes) + } + } + }) + } +} + +// TestHeadDetectsDuplicateSampleAtSizeLimit tests a regression where a duplicate sample +// is appended to the head, right when the head chunk is at the size limit. +// The test adds all samples as duplicate, thus expecting that the result has +// exactly half of the samples. +func TestHeadDetectsDuplicateSampleAtSizeLimit(t *testing.T) { + numSamples := 1000 + baseTS := int64(1695209650) + + h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + a := h.Appender(context.Background()) + var err error + vals := []float64{math.MaxFloat64, 0x00} // Use the worst case scenario for the XOR encoding. Otherwise we hit the sample limit before the size limit. + for i := range numSamples { + ts := baseTS + int64(i/2)*10000 + a.Append(0, labels.FromStrings("foo", "bar"), ts, vals[(i/2)%len(vals)]) + err = a.Commit() + require.NoError(t, err) + a = h.Appender(context.Background()) + } + + indexReader, err := h.Index() + require.NoError(t, err) + + var ( + chunks []chunks.Meta + builder labels.ScratchBuilder + ) + require.NoError(t, indexReader.Series(1, &builder, &chunks)) + + chunkReader, err := h.Chunks() + require.NoError(t, err) + + storedSampleCount := 0 + for _, chunkMeta := range chunks { + chunk, iterable, err := chunkReader.ChunkOrIterable(chunkMeta) + require.NoError(t, err) + require.Nil(t, iterable) + storedSampleCount += chunk.NumSamples() + } + + require.Equal(t, numSamples/2, storedSampleCount) +} + +func TestWALSampleAndExemplarOrder(t *testing.T) { + lbls := labels.FromStrings("foo", "bar") + testcases := map[string]struct { + appendF func(app storage.Appender, ts int64) (storage.SeriesRef, error) + expectedType reflect.Type + }{ + "float sample": { + appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { + return app.Append(0, lbls, ts, 1.0) + }, + expectedType: reflect.TypeOf([]record.RefSample{}), + }, + "histogram sample": { + appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { + return app.AppendHistogram(0, lbls, ts, tsdbutil.GenerateTestHistogram(1), nil) + }, + expectedType: reflect.TypeOf([]record.RefHistogramSample{}), + }, + "float histogram sample": { + appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { + return app.AppendHistogram(0, lbls, ts, nil, tsdbutil.GenerateTestFloatHistogram(1)) + }, + expectedType: reflect.TypeOf([]record.RefFloatHistogramSample{}), + }, + } + + for testName, tc := range testcases { + t.Run(testName, func(t *testing.T) { + h, w := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + + app := h.Appender(context.Background()) + ref, err := tc.appendF(app, 10) + require.NoError(t, err) + app.AppendExemplar(ref, lbls, exemplar.Exemplar{Value: 1.0, Ts: 5}) + + app.Commit() + + recs := readTestWAL(t, w.Dir()) + require.Len(t, recs, 3) + _, ok := recs[0].([]record.RefSeries) + require.True(t, ok, "expected first record to be a RefSeries") + actualType := reflect.TypeOf(recs[1]) + require.Equal(t, tc.expectedType, actualType, "expected second record to be a %s", tc.expectedType) + _, ok = recs[2].([]record.RefExemplar) + require.True(t, ok, "expected third record to be a RefExemplar") + }) + } +} + +// TestHeadCompactionWhileAppendAndCommitExemplar simulates a use case where +// a series is removed from the head while an exemplar is being appended to it. +// This can happen in theory by compacting the head at the right time due to +// a series being idle. +// The test cheats a little bit by not appending a sample with the exemplar. +// If you also add a sample and run Truncate in a concurrent goroutine and run +// the test around a million(!) times, you can get +// `unknown HeadSeriesRef when trying to add exemplar: 1` error on push. +// It is likely that running the test for much longer and with more time variations +// would trigger the +// `signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xbb03d1` +// panic, that we have seen in the wild once. +func TestHeadCompactionWhileAppendAndCommitExemplar(t *testing.T) { + h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + app := h.Appender(context.Background()) + lbls := labels.FromStrings("foo", "bar") + ref, err := app.Append(0, lbls, 1, 1) + require.NoError(t, err) + app.Commit() + // Not adding a sample here to trigger the fault. + app = h.Appender(context.Background()) + _, err = app.AppendExemplar(ref, lbls, exemplar.Exemplar{Value: 1, Ts: 20}) + require.NoError(t, err) + h.Truncate(10) + app.Commit() + h.Close() +} + +func labelsWithHashCollision() (labels.Labels, labels.Labels) { + // These two series have the same XXHash; thanks to https://github.com/pstibrany/labels_hash_collisions + ls1 := labels.FromStrings("__name__", "metric", "lbl", "HFnEaGl") + ls2 := labels.FromStrings("__name__", "metric", "lbl", "RqcXatm") + + if ls1.Hash() != ls2.Hash() { + // These ones are the same when using -tags slicelabels + ls1 = labels.FromStrings("__name__", "metric", "lbl1", "value", "lbl2", "l6CQ5y") + ls2 = labels.FromStrings("__name__", "metric", "lbl1", "value", "lbl2", "v7uDlF") + } + + if ls1.Hash() != ls2.Hash() { + panic("This code needs to be updated: find new labels with colliding hash values.") + } + + return ls1, ls2 +} + +// stripeSeriesWithCollidingSeries returns a stripeSeries with two memSeries having the same, colliding, hash. +func stripeSeriesWithCollidingSeries(t *testing.T) (*stripeSeries, *memSeries, *memSeries) { + t.Helper() + + lbls1, lbls2 := labelsWithHashCollision() + ms1 := memSeries{ + lset: lbls1, + } + ms2 := memSeries{ + lset: lbls2, + } + hash := lbls1.Hash() + s := newStripeSeries(1, noopSeriesLifecycleCallback{}) + + got, created := s.setUnlessAlreadySet(hash, lbls1, &ms1) + require.True(t, created) + require.Same(t, &ms1, got) + + // Add a conflicting series + got, created = s.setUnlessAlreadySet(hash, lbls2, &ms2) + require.True(t, created) + require.Same(t, &ms2, got) + + return s, &ms1, &ms2 +} + +func TestStripeSeries_getOrSet(t *testing.T) { + s, ms1, ms2 := stripeSeriesWithCollidingSeries(t) + hash := ms1.lset.Hash() + + // Verify that we can get both of the series despite the hash collision + got := s.getByHash(hash, ms1.lset) + require.Same(t, ms1, got) + got = s.getByHash(hash, ms2.lset) + require.Same(t, ms2, got) +} + +func TestStripeSeries_gc(t *testing.T) { + s, ms1, ms2 := stripeSeriesWithCollidingSeries(t) + hash := ms1.lset.Hash() + + s.gc(0, 0, nil) + + // Verify that we can get neither ms1 nor ms2 after gc-ing corresponding series + got := s.getByHash(hash, ms1.lset) + require.Nil(t, got) + got = s.getByHash(hash, ms2.lset) + require.Nil(t, got) +} + +func TestPostingsCardinalityStats(t *testing.T) { + head := &Head{postings: index.NewMemPostings()} + head.postings.Add(1, labels.FromStrings(labels.MetricName, "t", "n", "v1")) + head.postings.Add(2, labels.FromStrings(labels.MetricName, "t", "n", "v2")) + + statsForMetricName := head.PostingsCardinalityStats(labels.MetricName, 10) + head.postings.Add(3, labels.FromStrings(labels.MetricName, "t", "n", "v3")) + // Using cache. + require.Equal(t, statsForMetricName, head.PostingsCardinalityStats(labels.MetricName, 10)) + + statsForSomeLabel := head.PostingsCardinalityStats("n", 10) + // Cache should be evicted because of the change of label name. + require.NotEqual(t, statsForMetricName, statsForSomeLabel) + head.postings.Add(4, labels.FromStrings(labels.MetricName, "t", "n", "v4")) + // Using cache. + require.Equal(t, statsForSomeLabel, head.PostingsCardinalityStats("n", 10)) + // Cache should be evicted because of the change of limit parameter. + statsForSomeLabel1 := head.PostingsCardinalityStats("n", 1) + require.NotEqual(t, statsForSomeLabel1, statsForSomeLabel) + // Using cache. + require.Equal(t, statsForSomeLabel1, head.PostingsCardinalityStats("n", 1)) +} + +func TestHeadAppender_AppendFloatWithSameTimestampAsPreviousHistogram(t *testing.T) { + head, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + t.Cleanup(func() { head.Close() }) + + ls := labels.FromStrings(labels.MetricName, "test") + + { + // Append a float 10.0 @ 1_000 + app := head.Appender(context.Background()) + _, err := app.Append(0, ls, 1_000, 10.0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + { + // Append a float histogram @ 2_000 + app := head.Appender(context.Background()) + h := tsdbutil.GenerateTestHistogram(1) + _, err := app.AppendHistogram(0, ls, 2_000, h, nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + app := head.Appender(context.Background()) + _, err := app.Append(0, ls, 2_000, 10.0) + require.Error(t, err) + require.ErrorIs(t, err, storage.NewDuplicateHistogramToFloatErr(2_000, 10.0)) +} + +func TestHeadAppender_AppendST(t *testing.T) { + testHistogram := tsdbutil.GenerateTestHistogram(1) + testHistogram.CounterResetHint = histogram.NotCounterReset + testFloatHistogram := tsdbutil.GenerateTestFloatHistogram(1) + testFloatHistogram.CounterResetHint = histogram.NotCounterReset + // TODO(beorn7): Once issue #15346 is fixed, the CounterResetHint of the + // following two zero histograms should be histogram.CounterReset. + testZeroHistogram := &histogram.Histogram{ + Schema: testHistogram.Schema, + ZeroThreshold: testHistogram.ZeroThreshold, + PositiveSpans: testHistogram.PositiveSpans, + NegativeSpans: testHistogram.NegativeSpans, + PositiveBuckets: []int64{0, 0, 0, 0}, + NegativeBuckets: []int64{0, 0, 0, 0}, + } + testZeroFloatHistogram := &histogram.FloatHistogram{ + Schema: testFloatHistogram.Schema, + ZeroThreshold: testFloatHistogram.ZeroThreshold, + PositiveSpans: testFloatHistogram.PositiveSpans, + NegativeSpans: testFloatHistogram.NegativeSpans, + PositiveBuckets: []float64{0, 0, 0, 0}, + NegativeBuckets: []float64{0, 0, 0, 0}, + } + type appendableSamples struct { + ts int64 + fSample float64 + h *histogram.Histogram + fh *histogram.FloatHistogram + st int64 + } + for _, tc := range []struct { + name string + appendableSamples []appendableSamples + expectedSamples []chunks.Sample + }{ + { + name: "In order ct+normal sample/floatSample", + appendableSamples: []appendableSamples{ + {ts: 100, fSample: 10, st: 1}, + {ts: 101, fSample: 10, st: 1}, + }, + expectedSamples: []chunks.Sample{ + sample{t: 1, f: 0}, + sample{t: 100, f: 10}, + sample{t: 101, f: 10}, + }, + }, + { + name: "In order ct+normal sample/histogram", + appendableSamples: []appendableSamples{ + {ts: 100, h: testHistogram, st: 1}, + {ts: 101, h: testHistogram, st: 1}, + }, + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 1, h: testZeroHistogram}, + sample{t: 100, h: testHistogram}, + sample{t: 101, h: testHistogram}, + } + }(), + }, + { + name: "In order ct+normal sample/floathistogram", + appendableSamples: []appendableSamples{ + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 101, fh: testFloatHistogram, st: 1}, + }, + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 1, fh: testZeroFloatHistogram}, + sample{t: 100, fh: testFloatHistogram}, + sample{t: 101, fh: testFloatHistogram}, + } + }(), + }, + { + name: "Consecutive appends with same st ignore st/floatSample", + appendableSamples: []appendableSamples{ + {ts: 100, fSample: 10, st: 1}, + {ts: 101, fSample: 10, st: 1}, + }, + expectedSamples: []chunks.Sample{ + sample{t: 1, f: 0}, + sample{t: 100, f: 10}, + sample{t: 101, f: 10}, + }, + }, + { + name: "Consecutive appends with same st ignore st/histogram", + appendableSamples: []appendableSamples{ + {ts: 100, h: testHistogram, st: 1}, + {ts: 101, h: testHistogram, st: 1}, + }, + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 1, h: testZeroHistogram}, + sample{t: 100, h: testHistogram}, + sample{t: 101, h: testHistogram}, + } + }(), + }, + { + name: "Consecutive appends with same st ignore st/floathistogram", + appendableSamples: []appendableSamples{ + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 101, fh: testFloatHistogram, st: 1}, + }, + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 1, fh: testZeroFloatHistogram}, + sample{t: 100, fh: testFloatHistogram}, + sample{t: 101, fh: testFloatHistogram}, + } + }(), + }, + { + name: "Consecutive appends with newer st do not ignore st/floatSample", + appendableSamples: []appendableSamples{ + {ts: 100, fSample: 10, st: 1}, + {ts: 102, fSample: 10, st: 101}, + }, + expectedSamples: []chunks.Sample{ + sample{t: 1, f: 0}, + sample{t: 100, f: 10}, + sample{t: 101, f: 0}, + sample{t: 102, f: 10}, + }, + }, + { + name: "Consecutive appends with newer st do not ignore st/histogram", + appendableSamples: []appendableSamples{ + {ts: 100, h: testHistogram, st: 1}, + {ts: 102, h: testHistogram, st: 101}, + }, + expectedSamples: []chunks.Sample{ + sample{t: 1, h: testZeroHistogram}, + sample{t: 100, h: testHistogram}, + sample{t: 101, h: testZeroHistogram}, + sample{t: 102, h: testHistogram}, + }, + }, + { + name: "Consecutive appends with newer st do not ignore st/floathistogram", + appendableSamples: []appendableSamples{ + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 102, fh: testFloatHistogram, st: 101}, + }, + expectedSamples: []chunks.Sample{ + sample{t: 1, fh: testZeroFloatHistogram}, + sample{t: 100, fh: testFloatHistogram}, + sample{t: 101, fh: testZeroFloatHistogram}, + sample{t: 102, fh: testFloatHistogram}, + }, + }, + { + name: "ST equals to previous sample timestamp is ignored/floatSample", + appendableSamples: []appendableSamples{ + {ts: 100, fSample: 10, st: 1}, + {ts: 101, fSample: 10, st: 100}, + }, + expectedSamples: []chunks.Sample{ + sample{t: 1, f: 0}, + sample{t: 100, f: 10}, + sample{t: 101, f: 10}, + }, + }, + { + name: "ST equals to previous sample timestamp is ignored/histogram", + appendableSamples: []appendableSamples{ + {ts: 100, h: testHistogram, st: 1}, + {ts: 101, h: testHistogram, st: 100}, + }, + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 1, h: testZeroHistogram}, + sample{t: 100, h: testHistogram}, + sample{t: 101, h: testHistogram}, + } + }(), + }, + { + name: "ST equals to previous sample timestamp is ignored/floathistogram", + appendableSamples: []appendableSamples{ + {ts: 100, fh: testFloatHistogram, st: 1}, + {ts: 101, fh: testFloatHistogram, st: 100}, + }, + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 1, fh: testZeroFloatHistogram}, + sample{t: 100, fh: testFloatHistogram}, + sample{t: 101, fh: testFloatHistogram}, + } + }(), + }, + } { + t.Run(tc.name, func(t *testing.T) { + h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + defer func() { + require.NoError(t, h.Close()) + }() + a := h.Appender(context.Background()) + lbls := labels.FromStrings("foo", "bar") + for _, sample := range tc.appendableSamples { + // Append float if it's a float test case + if sample.fSample != 0 { + _, err := a.AppendSTZeroSample(0, lbls, sample.ts, sample.st) + require.NoError(t, err) + _, err = a.Append(0, lbls, sample.ts, sample.fSample) + require.NoError(t, err) + } + + // Append histograms if it's a histogram test case + if sample.h != nil || sample.fh != nil { + ref, err := a.AppendHistogramSTZeroSample(0, lbls, sample.ts, sample.st, sample.h, sample.fh) + require.NoError(t, err) + _, err = a.AppendHistogram(ref, lbls, sample.ts, sample.h, sample.fh) + require.NoError(t, err) + } + } + require.NoError(t, a.Commit()) + + q, err := NewBlockQuerier(h, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + result := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + require.Equal(t, tc.expectedSamples, result[`{foo="bar"}`]) + }) + } +} + +func TestHeadAppender_AppendHistogramSTZeroSample(t *testing.T) { + type appendableSamples struct { + ts int64 + h *histogram.Histogram + fh *histogram.FloatHistogram + st int64 // 0 if no created timestamp. + } + for _, tc := range []struct { + name string + appendableSamples []appendableSamples + expectedError error + }{ + { + name: "integer histogram ST lower than minValidTime initiates ErrOutOfBounds", + appendableSamples: []appendableSamples{ + {ts: 100, h: tsdbutil.GenerateTestHistogram(1), st: -1}, + }, + expectedError: storage.ErrOutOfBounds, + }, + { + name: "float histograms ST lower than minValidTime initiates ErrOutOfBounds", + appendableSamples: []appendableSamples{ + {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1), st: -1}, + }, + expectedError: storage.ErrOutOfBounds, + }, + { + name: "integer histogram ST duplicates an existing sample", + appendableSamples: []appendableSamples{ + {ts: 100, h: tsdbutil.GenerateTestHistogram(1)}, + {ts: 200, h: tsdbutil.GenerateTestHistogram(1), st: 100}, + }, + expectedError: storage.ErrDuplicateSampleForTimestamp, + }, + { + name: "float histogram ST duplicates an existing sample", + appendableSamples: []appendableSamples{ + {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1)}, + {ts: 200, fh: tsdbutil.GenerateTestFloatHistogram(1), st: 100}, + }, + expectedError: storage.ErrDuplicateSampleForTimestamp, + }, + } { + t.Run(tc.name, func(t *testing.T) { + h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + + defer func() { + require.NoError(t, h.Close()) + }() + + lbls := labels.FromStrings("foo", "bar") + + var ref storage.SeriesRef + for _, sample := range tc.appendableSamples { + a := h.Appender(context.Background()) + var err error + if sample.st != 0 { + ref, err = a.AppendHistogramSTZeroSample(ref, lbls, sample.ts, sample.st, sample.h, sample.fh) + require.ErrorIs(t, err, tc.expectedError) + } + + ref, err = a.AppendHistogram(ref, lbls, sample.ts, sample.h, sample.fh) + require.NoError(t, err) + require.NoError(t, a.Commit()) + } + }) + } +} + +func TestHeadCompactableDoesNotCompactEmptyHead(t *testing.T) { + // Use a chunk range of 1 here so that if we attempted to determine if the head + // was compactable using default values for min and max times, `Head.compactable()` + // would return true which is incorrect. This test verifies that we short-circuit + // the check when the head has not yet had any samples added. + head, _ := newTestHead(t, 1, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + require.False(t, head.compactable()) +} + +type countSeriesLifecycleCallback struct { + created atomic.Int64 + deleted atomic.Int64 +} + +func (*countSeriesLifecycleCallback) PreCreation(labels.Labels) error { return nil } +func (c *countSeriesLifecycleCallback) PostCreation(labels.Labels) { c.created.Inc() } +func (c *countSeriesLifecycleCallback) PostDeletion(s map[chunks.HeadSeriesRef]labels.Labels) { + c.deleted.Add(int64(len(s))) +} + +// Regression test for data race https://github.com/prometheus/prometheus/issues/15139. +func TestHeadAppendHistogramAndCommitConcurrency(t *testing.T) { + h := tsdbutil.GenerateTestHistogram(1) + fh := tsdbutil.GenerateTestFloatHistogram(1) + + testCases := map[string]func(storage.Appender, int) error{ + "integer histogram": func(app storage.Appender, i int) error { + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar", "serial", strconv.Itoa(i)), 1, h, nil) + return err + }, + "float histogram": func(app storage.Appender, i int) error { + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar", "serial", strconv.Itoa(i)), 1, nil, fh) + return err + }, + } + for name, tc := range testCases { + t.Run(name, func(t *testing.T) { + testHeadAppendHistogramAndCommitConcurrency(t, tc) + }) + } +} + +func testHeadAppendHistogramAndCommitConcurrency(t *testing.T, appendFn func(storage.Appender, int) error) { + head, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + wg := sync.WaitGroup{} + wg.Add(2) + + // How this works: Commit() should be atomic, thus one of the commits will + // be first and the other second. The first commit will create a new series + // and write a sample. The second commit will see an exact duplicate sample + // which it should ignore. Unless there's a race that causes the + // memSeries.lastHistogram to be corrupt and fail the duplicate check. + go func() { + defer wg.Done() + for i := range 10000 { + app := head.Appender(context.Background()) + require.NoError(t, appendFn(app, i)) + require.NoError(t, app.Commit()) + } + }() + + go func() { + defer wg.Done() + for i := range 10000 { + app := head.Appender(context.Background()) + require.NoError(t, appendFn(app, i)) + require.NoError(t, app.Commit()) + } + }() + + wg.Wait() +} + +func TestHead_NumStaleSeries(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + t.Cleanup(func() { + require.NoError(t, head.Close()) + }) + require.NoError(t, head.Init(0)) + + // Initially, no series should be stale. + require.Equal(t, uint64(0), head.NumStaleSeries()) + + appendSample := func(lbls labels.Labels, ts int64, val float64) { + app := head.Appender(context.Background()) + _, err := app.Append(0, lbls, ts, val) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + appendHistogram := func(lbls labels.Labels, ts int64, val *histogram.Histogram) { + app := head.Appender(context.Background()) + _, err := app.AppendHistogram(0, lbls, ts, val, nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + appendFloatHistogram := func(lbls labels.Labels, ts int64, val *histogram.FloatHistogram) { + app := head.Appender(context.Background()) + _, err := app.AppendHistogram(0, lbls, ts, nil, val) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + verifySeriesCounts := func(numStaleSeries, numSeries int) { + require.Equal(t, uint64(numStaleSeries), head.NumStaleSeries()) + require.Equal(t, uint64(numSeries), head.NumSeries()) + } + + restartHeadAndVerifySeriesCounts := func(numStaleSeries, numSeries int) { + verifySeriesCounts(numStaleSeries, numSeries) + + require.NoError(t, head.Close()) + + wal, err := wlog.NewSize(nil, nil, filepath.Join(head.opts.ChunkDirRoot, "wal"), 32768, compression.None) + require.NoError(t, err) + head, err = NewHead(nil, nil, wal, nil, head.opts, nil) + require.NoError(t, err) + require.NoError(t, head.Init(0)) + + verifySeriesCounts(numStaleSeries, numSeries) + } + + // Create some series with normal samples. + series1 := labels.FromStrings("name", "series1", "label", "value1") + series2 := labels.FromStrings("name", "series2", "label", "value2") + series3 := labels.FromStrings("name", "series3", "label", "value3") + + // Add normal samples to all series. + appendSample(series1, 100, 1) + appendSample(series2, 100, 2) + appendSample(series3, 100, 3) + // Still no stale series. + verifySeriesCounts(0, 3) + + // Make series1 stale by appending a stale sample. Now we should have 1 stale series. + appendSample(series1, 200, math.Float64frombits(value.StaleNaN)) + verifySeriesCounts(1, 3) + + // Make series2 stale as well. + appendSample(series2, 200, math.Float64frombits(value.StaleNaN)) + verifySeriesCounts(2, 3) + restartHeadAndVerifySeriesCounts(2, 3) + + // Add a non-stale sample to series1. It should not be counted as stale now. + appendSample(series1, 300, 10) + verifySeriesCounts(1, 3) + restartHeadAndVerifySeriesCounts(1, 3) + + // Test that series3 doesn't become stale when we add another normal sample. + appendSample(series3, 200, 10) + verifySeriesCounts(1, 3) + + // Test histogram stale samples as well. + series4 := labels.FromStrings("name", "series4", "type", "histogram") + h := tsdbutil.GenerateTestHistograms(1)[0] + appendHistogram(series4, 100, h) + verifySeriesCounts(1, 4) + + // Make histogram series stale. + staleHist := h.Copy() + staleHist.Sum = math.Float64frombits(value.StaleNaN) + appendHistogram(series4, 200, staleHist) + verifySeriesCounts(2, 4) + + // Test float histogram stale samples. + series5 := labels.FromStrings("name", "series5", "type", "float_histogram") + fh := tsdbutil.GenerateTestFloatHistograms(1)[0] + appendFloatHistogram(series5, 100, fh) + verifySeriesCounts(2, 5) + restartHeadAndVerifySeriesCounts(2, 5) + + // Make float histogram series stale. + staleFH := fh.Copy() + staleFH.Sum = math.Float64frombits(value.StaleNaN) + appendFloatHistogram(series5, 200, staleFH) + verifySeriesCounts(3, 5) + + // Make histogram sample non-stale and stale back again. + appendHistogram(series4, 210, h) + verifySeriesCounts(2, 5) + appendHistogram(series4, 220, staleHist) + verifySeriesCounts(3, 5) + + // Make float histogram sample non-stale and stale back again. + appendFloatHistogram(series5, 210, fh) + verifySeriesCounts(2, 5) + appendFloatHistogram(series5, 220, staleFH) + verifySeriesCounts(3, 5) + + // Series 1 and 3 are not stale at this point. Add a new sample to series 1 and series 5, + // so after the GC and removing series 2, 3, 4, we should be left with 1 stale and 1 non-stale series. + appendSample(series1, 400, 10) + appendFloatHistogram(series5, 400, staleFH) + restartHeadAndVerifySeriesCounts(3, 5) + + // This will test restarting with snapshot. + head.opts.EnableMemorySnapshotOnShutdown = true + restartHeadAndVerifySeriesCounts(3, 5) + + // Test garbage collection behavior - stale series should be decremented when GC'd. + // Force a garbage collection by truncating old data. + require.NoError(t, head.Truncate(300)) + + // After truncation, run GC to collect old chunks/series. + head.gc() + + // series 1 and series 5 are left. + verifySeriesCounts(1, 2) + + // Test creating a new series for each of float, histogram, float histogram that starts as stale. + // This should be counted as stale. + series6 := labels.FromStrings("name", "series6", "direct", "stale") + series7 := labels.FromStrings("name", "series7", "direct", "stale") + series8 := labels.FromStrings("name", "series8", "direct", "stale") + appendSample(series6, 400, math.Float64frombits(value.StaleNaN)) + verifySeriesCounts(2, 3) + appendHistogram(series7, 400, staleHist) + verifySeriesCounts(3, 4) + appendFloatHistogram(series8, 400, staleFH) + verifySeriesCounts(4, 5) +} + +// TestHistogramStalenessConversionMetrics verifies that staleness marker conversion correctly +// increments the right appender metrics for both histogram and float histogram scenarios. +func TestHistogramStalenessConversionMetrics(t *testing.T) { + testCases := []struct { + name string + setupHistogram func(app storage.Appender, lbls labels.Labels) error + }{ + { + name: "float_staleness_to_histogram", + setupHistogram: func(app storage.Appender, lbls labels.Labels) error { + _, err := app.AppendHistogram(0, lbls, 1000, tsdbutil.GenerateTestHistograms(1)[0], nil) + return err + }, + }, + { + name: "float_staleness_to_float_histogram", + setupHistogram: func(app storage.Appender, lbls labels.Labels) error { + _, err := app.AppendHistogram(0, lbls, 1000, nil, tsdbutil.GenerateTestFloatHistograms(1)[0]) + return err + }, + }, + } + + for _, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + head, _ := newTestHead(t, 1000, compression.None, false) + defer func() { + require.NoError(t, head.Close()) + }() + + lbls := labels.FromStrings("name", tc.name) + + // Helper to get counter values + getSampleCounter := func(sampleType string) float64 { + metric := &dto.Metric{} + err := head.metrics.samplesAppended.WithLabelValues(sampleType).Write(metric) + require.NoError(t, err) + return metric.GetCounter().GetValue() + } + + // Step 1: Establish a series with histogram data + app := head.Appender(context.Background()) + err := tc.setupHistogram(app, lbls) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // Step 2: Add a float staleness marker + app = head.Appender(context.Background()) + _, err = app.Append(0, lbls, 2000, math.Float64frombits(value.StaleNaN)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // Count what was actually stored by querying the series + q, err := NewBlockQuerier(head, 0, 3000) + require.NoError(t, err) + defer q.Close() + + ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "name", tc.name)) + require.True(t, ss.Next()) + series := ss.At() + + it := series.Iterator(nil) + + actualFloatSamples := 0 + actualHistogramSamples := 0 + + for valType := it.Next(); valType != chunkenc.ValNone; valType = it.Next() { + switch valType { + case chunkenc.ValFloat: + actualFloatSamples++ + case chunkenc.ValHistogram, chunkenc.ValFloatHistogram: + actualHistogramSamples++ + } + } + require.NoError(t, it.Err()) + + // Verify what was actually stored - should be 0 floats, 2 histograms (original + converted staleness marker) + require.Equal(t, 0, actualFloatSamples, "Should have 0 float samples stored") + require.Equal(t, 2, actualHistogramSamples, "Should have 2 histogram samples: original + converted staleness marker") + + // The metrics should match what was actually stored + require.Equal(t, float64(actualFloatSamples), getSampleCounter(sampleMetricTypeFloat), + "Float counter should match actual float samples stored") + require.Equal(t, float64(actualHistogramSamples), getSampleCounter(sampleMetricTypeHistogram), + "Histogram counter should match actual histogram samples stored") + }) + } +} diff --git a/tsdb/head_bench_v2_test.go b/tsdb/head_bench_v2_test.go new file mode 100644 index 0000000000..c98fb6613d --- /dev/null +++ b/tsdb/head_bench_v2_test.go @@ -0,0 +1,173 @@ +// Copyright 2018 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package tsdb + +import ( + "context" + "errors" + "fmt" + "math/rand" + "strconv" + "testing" + + "github.com/stretchr/testify/require" + "go.uber.org/atomic" + + "github.com/prometheus/prometheus/model/exemplar" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/tsdb/chunks" + "github.com/prometheus/prometheus/util/compression" +) + +func BenchmarkHeadStripeSeriesCreate(b *testing.B) { + chunkDir := b.TempDir() + // Put a series, select it. GC it and then access it. + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = chunkDir + h, err := NewHead(nil, nil, nil, nil, opts, nil) + require.NoError(b, err) + defer h.Close() + + for i := 0; b.Loop(); i++ { + h.getOrCreate(uint64(i), labels.FromStrings("a", strconv.Itoa(i)), false) + } +} + +func BenchmarkHeadStripeSeriesCreateParallel(b *testing.B) { + chunkDir := b.TempDir() + // Put a series, select it. GC it and then access it. + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = chunkDir + h, err := NewHead(nil, nil, nil, nil, opts, nil) + require.NoError(b, err) + defer h.Close() + + var count atomic.Int64 + + b.RunParallel(func(pb *testing.PB) { + for pb.Next() { + i := count.Inc() + h.getOrCreate(uint64(i), labels.FromStrings("a", strconv.Itoa(int(i))), false) + } + }) +} + +func BenchmarkHeadStripeSeriesCreate_PreCreationFailure(b *testing.B) { + chunkDir := b.TempDir() + // Put a series, select it. GC it and then access it. + opts := DefaultHeadOptions() + opts.ChunkRange = 1000 + opts.ChunkDirRoot = chunkDir + + // Mock the PreCreation() callback to fail on each series. + opts.SeriesCallback = failingSeriesLifecycleCallback{} + + h, err := NewHead(nil, nil, nil, nil, opts, nil) + require.NoError(b, err) + defer h.Close() + + for i := 0; b.Loop(); i++ { + h.getOrCreate(uint64(i), labels.FromStrings("a", strconv.Itoa(i)), false) + } +} + +func BenchmarkHead_WalCommit(b *testing.B) { + seriesCounts := []int{100, 1000, 10000} + series := genSeries(10000, 10, 0, 0) // Only using the generated labels. + + appendSamples := func(b *testing.B, app storage.Appender, seriesCount int, ts int64) { + var err error + for i, s := range series[:seriesCount] { + var ref storage.SeriesRef + // if i is even, append a sample, else append a histogram. + if i%2 == 0 { + ref, err = app.Append(ref, s.Labels(), ts, float64(ts)) + } else { + h := &histogram.Histogram{ + Count: 7 + uint64(ts*5), + ZeroCount: 2 + uint64(ts), + ZeroThreshold: 0.001, + Sum: 18.4 * rand.Float64(), + Schema: 1, + PositiveSpans: []histogram.Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + PositiveBuckets: []int64{ts + 1, 1, -1, 0}, + } + ref, err = app.AppendHistogram(ref, s.Labels(), ts, h, nil) + } + require.NoError(b, err) + + _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts, + }) + require.NoError(b, err) + } + } + + for _, seriesCount := range seriesCounts { + b.Run(fmt.Sprintf("%d series", seriesCount), func(b *testing.B) { + for _, commits := range []int64{1, 2} { // To test commits that create new series and when the series already exists. + b.Run(fmt.Sprintf("%d commits", commits), func(b *testing.B) { + b.ReportAllocs() + b.ResetTimer() + + for b.Loop() { + b.StopTimer() + h, w := newTestHead(b, 10000, compression.None, false) + b.Cleanup(func() { + if h != nil { + h.Close() + } + if w != nil { + w.Close() + } + }) + app := h.Appender(context.Background()) + + appendSamples(b, app, seriesCount, 0) + + b.StartTimer() + require.NoError(b, app.Commit()) + if commits == 2 { + b.StopTimer() + app = h.Appender(context.Background()) + appendSamples(b, app, seriesCount, 1) + b.StartTimer() + require.NoError(b, app.Commit()) + } + b.StopTimer() + h.Close() + h = nil + w.Close() + w = nil + } + }) + } + }) + } +} + +type failingSeriesLifecycleCallback struct{} + +func (failingSeriesLifecycleCallback) PreCreation(labels.Labels) error { return errors.New("failed") } +func (failingSeriesLifecycleCallback) PostCreation(labels.Labels) {} +func (failingSeriesLifecycleCallback) PostDeletion(map[chunks.HeadSeriesRef]labels.Labels) {} From a41e1144ddefafbc3de344b9d1c0b7f4b0f237af Mon Sep 17 00:00:00 2001 From: bwplotka Date: Tue, 2 Dec 2025 13:41:18 +0000 Subject: [PATCH 117/439] refactor(appenderV2): 1:1 copy of db_test.go to db_append_v2_test.go (starting point) Signed-off-by: bwplotka --- tsdb/db_append_v2_test.go | 9286 +++++++++++++++++++++++++++++++++++++ 1 file changed, 9286 insertions(+) create mode 100644 tsdb/db_append_v2_test.go diff --git a/tsdb/db_append_v2_test.go b/tsdb/db_append_v2_test.go new file mode 100644 index 0000000000..4e084ef0d8 --- /dev/null +++ b/tsdb/db_append_v2_test.go @@ -0,0 +1,9286 @@ +// Copyright 2017 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package tsdb + +import ( + "bufio" + "bytes" + "context" + "encoding/binary" + "errors" + "flag" + "fmt" + "hash/crc32" + "log/slog" + "math" + "math/rand" + "net/http" + "net/http/httptest" + "os" + "path" + "path/filepath" + "runtime" + "sort" + "strconv" + "sync" + "testing" + "time" + + "github.com/gogo/protobuf/proto" + "github.com/golang/snappy" + "github.com/oklog/ulid/v2" + "github.com/prometheus/client_golang/prometheus" + prom_testutil "github.com/prometheus/client_golang/prometheus/testutil" + "github.com/prometheus/common/model" + "github.com/prometheus/common/promslog" + "github.com/stretchr/testify/require" + "go.uber.org/atomic" + "go.uber.org/goleak" + + "github.com/prometheus/prometheus/config" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/model/metadata" + "github.com/prometheus/prometheus/prompb" + "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/storage/remote" + "github.com/prometheus/prometheus/tsdb/chunkenc" + "github.com/prometheus/prometheus/tsdb/chunks" + "github.com/prometheus/prometheus/tsdb/fileutil" + "github.com/prometheus/prometheus/tsdb/index" + "github.com/prometheus/prometheus/tsdb/record" + "github.com/prometheus/prometheus/tsdb/tombstones" + "github.com/prometheus/prometheus/tsdb/tsdbutil" + "github.com/prometheus/prometheus/tsdb/wlog" + "github.com/prometheus/prometheus/util/annotations" + "github.com/prometheus/prometheus/util/compression" + "github.com/prometheus/prometheus/util/testutil" +) + +func TestMain(m *testing.M) { + var isolationEnabled bool + flag.BoolVar(&isolationEnabled, "test.tsdb-isolation", true, "enable isolation") + flag.Parse() + defaultIsolationDisabled = !isolationEnabled + + goleak.VerifyTestMain(m, + goleak.IgnoreTopFunction("github.com/prometheus/prometheus/tsdb.(*SegmentWAL).cut.func1"), + goleak.IgnoreTopFunction("github.com/prometheus/prometheus/tsdb.(*SegmentWAL).cut.func2"), + goleak.IgnoreTopFunction("go.opencensus.io/stats/view.(*worker).start")) +} + +type testDBOptions struct { + dir string + opts *Options + rngs []int64 +} +type testDBOpt func(o *testDBOptions) + +func withDir(dir string) testDBOpt { + return func(o *testDBOptions) { + o.dir = dir + } +} + +func withOpts(opts *Options) testDBOpt { + return func(o *testDBOptions) { + o.opts = opts + } +} + +func withRngs(rngs ...int64) testDBOpt { + return func(o *testDBOptions) { + o.rngs = rngs + } +} + +func newTestDB(t testing.TB, opts ...testDBOpt) (db *DB) { + var o testDBOptions + for _, opt := range opts { + opt(&o) + } + if o.opts == nil { + o.opts = DefaultOptions() + } + if o.dir == "" { + o.dir = t.TempDir() + } + + var err error + if len(o.rngs) == 0 { + db, err = Open(o.dir, nil, nil, o.opts, nil) + } else { + o.opts, o.rngs = validateOpts(o.opts, o.rngs) + db, err = open(o.dir, nil, nil, o.opts, o.rngs, nil) + } + require.NoError(t, err) + t.Cleanup(func() { + // Always close. DB is safe for close-after-close. + require.NoError(t, db.Close()) + }) + return db +} + +func TestDBClose_AfterClose(t *testing.T) { + db := newTestDB(t) + require.NoError(t, db.Close()) + require.NoError(t, db.Close()) + + // Double check if we are closing correct DB after reuse. + db = newTestDB(t) + require.NoError(t, db.Close()) + require.NoError(t, db.Close()) +} + +// query runs a matcher query against the querier and fully expands its data. +func query(t testing.TB, q storage.Querier, matchers ...*labels.Matcher) map[string][]chunks.Sample { + ss := q.Select(context.Background(), false, nil, matchers...) + defer func() { + require.NoError(t, q.Close()) + }() + + var it chunkenc.Iterator + result := map[string][]chunks.Sample{} + for ss.Next() { + series := ss.At() + + it = series.Iterator(it) + samples, err := storage.ExpandSamples(it, newSample) + require.NoError(t, err) + require.NoError(t, it.Err()) + + if len(samples) == 0 { + continue + } + + name := series.Labels().String() + result[name] = samples + } + require.NoError(t, ss.Err()) + require.Empty(t, ss.Warnings()) + + return result +} + +// queryAndExpandChunks runs a matcher query against the querier and fully expands its data into samples. +func queryAndExpandChunks(t testing.TB, q storage.ChunkQuerier, matchers ...*labels.Matcher) map[string][][]chunks.Sample { + s := queryChunks(t, q, matchers...) + + res := make(map[string][][]chunks.Sample) + for k, v := range s { + var samples [][]chunks.Sample + for _, chk := range v { + sam, err := storage.ExpandSamples(chk.Chunk.Iterator(nil), nil) + require.NoError(t, err) + samples = append(samples, sam) + } + res[k] = samples + } + + return res +} + +// queryChunks runs a matcher query against the querier and expands its data. +func queryChunks(t testing.TB, q storage.ChunkQuerier, matchers ...*labels.Matcher) map[string][]chunks.Meta { + ss := q.Select(context.Background(), false, nil, matchers...) + defer func() { + require.NoError(t, q.Close()) + }() + + var it chunks.Iterator + result := map[string][]chunks.Meta{} + for ss.Next() { + series := ss.At() + + chks := []chunks.Meta{} + it = series.Iterator(it) + for it.Next() { + chks = append(chks, it.At()) + } + require.NoError(t, it.Err()) + + if len(chks) == 0 { + continue + } + + name := series.Labels().String() + result[name] = chks + } + require.NoError(t, ss.Err()) + require.Empty(t, ss.Warnings()) + return result +} + +// Ensure that blocks are held in memory in their time order +// and not in ULID order as they are read from the directory. +func TestDB_reloadOrder(t *testing.T) { + db := newTestDB(t) + + metas := []BlockMeta{ + {MinTime: 90, MaxTime: 100}, + {MinTime: 70, MaxTime: 80}, + {MinTime: 100, MaxTime: 110}, + } + for _, m := range metas { + createBlock(t, db.Dir(), genSeries(1, 1, m.MinTime, m.MaxTime)) + } + + require.NoError(t, db.reloadBlocks()) + blocks := db.Blocks() + require.Len(t, blocks, 3) + require.Equal(t, metas[1].MinTime, blocks[0].Meta().MinTime) + require.Equal(t, metas[1].MaxTime, blocks[0].Meta().MaxTime) + require.Equal(t, metas[0].MinTime, blocks[1].Meta().MinTime) + require.Equal(t, metas[0].MaxTime, blocks[1].Meta().MaxTime) + require.Equal(t, metas[2].MinTime, blocks[2].Meta().MinTime) + require.Equal(t, metas[2].MaxTime, blocks[2].Meta().MaxTime) +} + +func TestDataAvailableOnlyAfterCommit(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + require.NoError(t, err) + + querier, err := db.Querier(0, 1) + require.NoError(t, err) + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + require.Equal(t, map[string][]chunks.Sample{}, seriesSet) + + err = app.Commit() + require.NoError(t, err) + + querier, err = db.Querier(0, 1) + require.NoError(t, err) + defer querier.Close() + + seriesSet = query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + + require.Equal(t, map[string][]chunks.Sample{`{foo="bar"}`: {sample{t: 0, f: 0}}}, seriesSet) +} + +// TestNoPanicAfterWALCorruption ensures that querying the db after a WAL corruption doesn't cause a panic. +// https://github.com/prometheus/prometheus/issues/7548 +func TestNoPanicAfterWALCorruption(t *testing.T) { + db := newTestDB(t, withOpts(&Options{WALSegmentSize: 32 * 1024})) + + // Append until the first mmapped head chunk. + // This is to ensure that all samples can be read from the mmapped chunks when the WAL is corrupted. + var expSamples []chunks.Sample + var maxt int64 + ctx := context.Background() + { + // Appending 121 samples because on the 121st a new chunk will be created. + for range 121 { + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), maxt, 0) + expSamples = append(expSamples, sample{t: maxt, f: 0}) + require.NoError(t, err) + require.NoError(t, app.Commit()) + maxt++ + } + require.NoError(t, db.Close()) + } + + // Corrupt the WAL after the first sample of the series so that it has at least one sample and + // it is not garbage collected. + // The repair deletes all WAL records after the corrupted record and these are read from the mmapped chunk. + { + walFiles, err := os.ReadDir(path.Join(db.Dir(), "wal")) + require.NoError(t, err) + f, err := os.OpenFile(path.Join(db.Dir(), "wal", walFiles[0].Name()), os.O_RDWR, 0o666) + require.NoError(t, err) + r := wlog.NewReader(bufio.NewReader(f)) + require.True(t, r.Next(), "reading the series record") + require.True(t, r.Next(), "reading the first sample record") + // Write an invalid record header to corrupt everything after the first wal sample. + _, err = f.WriteAt([]byte{99}, r.Offset()) + require.NoError(t, err) + f.Close() + } + + // Query the data. + { + db := newTestDB(t, withDir(db.Dir())) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal), "WAL corruption count mismatch") + + querier, err := db.Querier(0, maxt) + require.NoError(t, err) + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "", "")) + // The last sample should be missing as it was after the WAL segment corruption. + require.Equal(t, map[string][]chunks.Sample{`{foo="bar"}`: expSamples[0 : len(expSamples)-1]}, seriesSet) + } +} + +func TestDataNotAvailableAfterRollback(t *testing.T) { + db := newTestDB(t) + + app := db.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("type", "float"), 0, 0) + require.NoError(t, err) + + _, err = app.AppendHistogram( + 0, labels.FromStrings("type", "histogram"), 0, + &histogram.Histogram{Count: 42, Sum: math.NaN()}, nil, + ) + require.NoError(t, err) + + _, err = app.AppendHistogram( + 0, labels.FromStrings("type", "floathistogram"), 0, + nil, &histogram.FloatHistogram{Count: 42, Sum: math.NaN()}, + ) + require.NoError(t, err) + + err = app.Rollback() + require.NoError(t, err) + + for _, typ := range []string{"float", "histogram", "floathistogram"} { + querier, err := db.Querier(0, 1) + require.NoError(t, err) + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "type", typ)) + require.Equal(t, map[string][]chunks.Sample{}, seriesSet) + } + + sr, err := wlog.NewSegmentsReader(db.head.wal.Dir()) + require.NoError(t, err) + defer func() { + require.NoError(t, sr.Close()) + }() + + // Read records from WAL and check for expected count of series and samples. + var ( + r = wlog.NewReader(sr) + dec = record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + + walSeriesCount, walSamplesCount, walHistogramCount, walFloatHistogramCount, walExemplarsCount int + ) + for r.Next() { + rec := r.Record() + switch dec.Type(rec) { + case record.Series: + var series []record.RefSeries + series, err = dec.Series(rec, series) + require.NoError(t, err) + walSeriesCount += len(series) + + case record.Samples: + var samples []record.RefSample + samples, err = dec.Samples(rec, samples) + require.NoError(t, err) + walSamplesCount += len(samples) + + case record.Exemplars: + var exemplars []record.RefExemplar + exemplars, err = dec.Exemplars(rec, exemplars) + require.NoError(t, err) + walExemplarsCount += len(exemplars) + + case record.HistogramSamples, record.CustomBucketsHistogramSamples: + var histograms []record.RefHistogramSample + histograms, err = dec.HistogramSamples(rec, histograms) + require.NoError(t, err) + walHistogramCount += len(histograms) + + case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: + var floatHistograms []record.RefFloatHistogramSample + floatHistograms, err = dec.FloatHistogramSamples(rec, floatHistograms) + require.NoError(t, err) + walFloatHistogramCount += len(floatHistograms) + + default: + } + } + + // Check that only series get stored after calling Rollback. + require.Equal(t, 3, walSeriesCount, "series should have been written to WAL") + require.Equal(t, 0, walSamplesCount, "samples should not have been written to WAL") + require.Equal(t, 0, walExemplarsCount, "exemplars should not have been written to WAL") + require.Equal(t, 0, walHistogramCount, "histograms should not have been written to WAL") + require.Equal(t, 0, walFloatHistogramCount, "float histograms should not have been written to WAL") +} + +func TestDBAppenderAddRef(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app1 := db.Appender(ctx) + + ref1, err := app1.Append(0, labels.FromStrings("a", "b"), 123, 0) + require.NoError(t, err) + + // Reference should already work before commit. + ref2, err := app1.Append(ref1, labels.EmptyLabels(), 124, 1) + require.NoError(t, err) + require.Equal(t, ref1, ref2) + + err = app1.Commit() + require.NoError(t, err) + + app2 := db.Appender(ctx) + + // first ref should already work in next transaction. + ref3, err := app2.Append(ref1, labels.EmptyLabels(), 125, 0) + require.NoError(t, err) + require.Equal(t, ref1, ref3) + + ref4, err := app2.Append(ref1, labels.FromStrings("a", "b"), 133, 1) + require.NoError(t, err) + require.Equal(t, ref1, ref4) + + // Reference must be valid to add another sample. + ref5, err := app2.Append(ref2, labels.EmptyLabels(), 143, 2) + require.NoError(t, err) + require.Equal(t, ref1, ref5) + + // Missing labels & invalid refs should fail. + _, err = app2.Append(9999999, labels.EmptyLabels(), 1, 1) + require.ErrorIs(t, err, ErrInvalidSample) + + require.NoError(t, app2.Commit()) + + q, err := db.Querier(0, 200) + require.NoError(t, err) + + res := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + require.Equal(t, map[string][]chunks.Sample{ + labels.FromStrings("a", "b").String(): { + sample{t: 123, f: 0}, + sample{t: 124, f: 1}, + sample{t: 125, f: 0}, + sample{t: 133, f: 1}, + sample{t: 143, f: 2}, + }, + }, res) +} + +func TestAppendEmptyLabelsIgnored(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app1 := db.Appender(ctx) + + ref1, err := app1.Append(0, labels.FromStrings("a", "b"), 123, 0) + require.NoError(t, err) + + // Add with empty label. + ref2, err := app1.Append(0, labels.FromStrings("a", "b", "c", ""), 124, 0) + require.NoError(t, err) + + // Should be the same series. + require.Equal(t, ref1, ref2) + + err = app1.Commit() + require.NoError(t, err) +} + +func TestDeleteSimple(t *testing.T) { + const numSamples int64 = 10 + + cases := []struct { + Intervals tombstones.Intervals + remaint []int64 + }{ + { + Intervals: tombstones.Intervals{{Mint: 0, Maxt: 3}}, + remaint: []int64{4, 5, 6, 7, 8, 9}, + }, + { + Intervals: tombstones.Intervals{{Mint: 1, Maxt: 3}}, + remaint: []int64{0, 4, 5, 6, 7, 8, 9}, + }, + { + Intervals: tombstones.Intervals{{Mint: 1, Maxt: 3}, {Mint: 4, Maxt: 7}}, + remaint: []int64{0, 8, 9}, + }, + { + Intervals: tombstones.Intervals{{Mint: 1, Maxt: 3}, {Mint: 4, Maxt: 700}}, + remaint: []int64{0}, + }, + { // This case is to ensure that labels and symbols are deleted. + Intervals: tombstones.Intervals{{Mint: 0, Maxt: 9}}, + remaint: []int64{}, + }, + } + + for _, c := range cases { + t.Run("", func(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + smpls := make([]float64, numSamples) + for i := range numSamples { + smpls[i] = rand.Float64() + app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + } + + require.NoError(t, app.Commit()) + + // TODO(gouthamve): Reset the tombstones somehow. + // Delete the ranges. + for _, r := range c.Intervals { + require.NoError(t, db.Delete(ctx, r.Mint, r.Maxt, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) + } + + // Compare the result. + q, err := db.Querier(0, numSamples) + require.NoError(t, err) + + res := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + expSamples := make([]chunks.Sample, 0, len(c.remaint)) + for _, ts := range c.remaint { + expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil}) + } + + expss := newMockSeriesSet([]storage.Series{ + storage.NewListSeries(labels.FromStrings("a", "b"), expSamples), + }) + + for { + eok, rok := expss.Next(), res.Next() + require.Equal(t, eok, rok) + + if !eok { + require.Empty(t, res.Warnings()) + break + } + sexp := expss.At() + sres := res.At() + + require.Equal(t, sexp.Labels(), sres.Labels()) + + smplExp, errExp := storage.ExpandSamples(sexp.Iterator(nil), nil) + smplRes, errRes := storage.ExpandSamples(sres.Iterator(nil), nil) + + require.Equal(t, errExp, errRes) + require.Equal(t, smplExp, smplRes) + } + }) + } +} + +func TestAmendHistogramDatapointCausesError(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + app = db.Appender(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 1) + require.ErrorIs(t, err, storage.ErrDuplicateSampleForTimestamp) + require.NoError(t, app.Rollback()) + + h := histogram.Histogram{ + Schema: 3, + Count: 52, + Sum: 2.7, + ZeroThreshold: 0.1, + ZeroCount: 42, + PositiveSpans: []histogram.Span{ + {Offset: 0, Length: 4}, + {Offset: 10, Length: 3}, + }, + PositiveBuckets: []int64{1, 2, -2, 1, -1, 0, 0}, + } + fh := h.ToFloat(nil) + + app = db.Appender(ctx) + _, err = app.AppendHistogram(0, labels.FromStrings("a", "c"), 0, h.Copy(), nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + app = db.Appender(ctx) + _, err = app.AppendHistogram(0, labels.FromStrings("a", "c"), 0, h.Copy(), nil) + require.NoError(t, err) + h.Schema = 2 + _, err = app.AppendHistogram(0, labels.FromStrings("a", "c"), 0, h.Copy(), nil) + require.Equal(t, storage.ErrDuplicateSampleForTimestamp, err) + require.NoError(t, app.Rollback()) + + // Float histogram. + app = db.Appender(ctx) + _, err = app.AppendHistogram(0, labels.FromStrings("a", "d"), 0, nil, fh.Copy()) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + app = db.Appender(ctx) + _, err = app.AppendHistogram(0, labels.FromStrings("a", "d"), 0, nil, fh.Copy()) + require.NoError(t, err) + fh.Schema = 2 + _, err = app.AppendHistogram(0, labels.FromStrings("a", "d"), 0, nil, fh.Copy()) + require.Equal(t, storage.ErrDuplicateSampleForTimestamp, err) + require.NoError(t, app.Rollback()) +} + +func TestDuplicateNaNDatapointNoAmendError(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, math.NaN()) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + app = db.Appender(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, math.NaN()) + require.NoError(t, err) +} + +func TestNonDuplicateNaNDatapointsCausesAmendError(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, math.Float64frombits(0x7ff0000000000001)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + app = db.Appender(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, math.Float64frombits(0x7ff0000000000002)) + require.ErrorIs(t, err, storage.ErrDuplicateSampleForTimestamp) +} + +func TestEmptyLabelsetCausesError(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, labels.Labels{}, 0, 0) + require.Error(t, err) + require.Equal(t, "empty labelset: invalid sample", err.Error()) +} + +func TestSkippingInvalidValuesInSameTxn(t *testing.T) { + db := newTestDB(t) + + // Append AmendedValue. + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 1) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 2) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // Make sure the right value is stored. + q, err := db.Querier(0, 10) + require.NoError(t, err) + + ssMap := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + require.Equal(t, map[string][]chunks.Sample{ + labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 1, nil, nil}}, + }, ssMap) + + // Append Out of Order Value. + app = db.Appender(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 10, 3) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("a", "b"), 7, 5) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + q, err = db.Querier(0, 10) + require.NoError(t, err) + + ssMap = query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + require.Equal(t, map[string][]chunks.Sample{ + labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 1, nil, nil}, sample{10, 3, nil, nil}}, + }, ssMap) +} + +func TestDB_Snapshot(t *testing.T) { + db := newTestDB(t) + + // append data + ctx := context.Background() + app := db.Appender(ctx) + mint := int64(1414141414000) + for i := range 1000 { + _, err := app.Append(0, labels.FromStrings("foo", "bar"), mint+int64(i), 1.0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // create snapshot + snap := t.TempDir() + require.NoError(t, db.Snapshot(snap, true)) + require.NoError(t, db.Close()) + + // reopen DB from snapshot + db = newTestDB(t, withDir(snap)) + + querier, err := db.Querier(mint, mint+1000) + require.NoError(t, err) + defer func() { require.NoError(t, querier.Close()) }() + + // sum values + seriesSet := querier.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + var series chunkenc.Iterator + sum := 0.0 + for seriesSet.Next() { + series = seriesSet.At().Iterator(series) + for series.Next() == chunkenc.ValFloat { + _, v := series.At() + sum += v + } + require.NoError(t, series.Err()) + } + require.NoError(t, seriesSet.Err()) + require.Empty(t, seriesSet.Warnings()) + require.Equal(t, 1000.0, sum) +} + +// TestDB_Snapshot_ChunksOutsideOfCompactedRange ensures that a snapshot removes chunks samples +// that are outside the set block time range. +// See https://github.com/prometheus/prometheus/issues/5105 +func TestDB_Snapshot_ChunksOutsideOfCompactedRange(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + mint := int64(1414141414000) + for i := range 1000 { + _, err := app.Append(0, labels.FromStrings("foo", "bar"), mint+int64(i), 1.0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + snap := t.TempDir() + + // Hackingly introduce "race", by having lower max time then maxTime in last chunk. + db.head.maxTime.Sub(10) + + require.NoError(t, db.Snapshot(snap, true)) + require.NoError(t, db.Close()) + + // reopen DB from snapshot + db = newTestDB(t, withDir(snap)) + + querier, err := db.Querier(mint, mint+1000) + require.NoError(t, err) + defer func() { require.NoError(t, querier.Close()) }() + + // Sum values. + seriesSet := querier.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + var series chunkenc.Iterator + sum := 0.0 + for seriesSet.Next() { + series = seriesSet.At().Iterator(series) + for series.Next() == chunkenc.ValFloat { + _, v := series.At() + sum += v + } + require.NoError(t, series.Err()) + } + require.NoError(t, seriesSet.Err()) + require.Empty(t, seriesSet.Warnings()) + + // Since we snapshotted with MaxTime - 10, so expect 10 less samples. + require.Equal(t, 1000.0-10, sum) +} + +func TestDB_SnapshotWithDelete(t *testing.T) { + const numSamples int64 = 10 + + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + smpls := make([]float64, numSamples) + for i := range numSamples { + smpls[i] = rand.Float64() + app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + } + + require.NoError(t, app.Commit()) + cases := []struct { + intervals tombstones.Intervals + remaint []int64 + }{ + { + intervals: tombstones.Intervals{{Mint: 1, Maxt: 3}, {Mint: 4, Maxt: 7}}, + remaint: []int64{0, 8, 9}, + }, + } + + for _, c := range cases { + t.Run("", func(t *testing.T) { + // TODO(gouthamve): Reset the tombstones somehow. + // Delete the ranges. + for _, r := range c.intervals { + require.NoError(t, db.Delete(ctx, r.Mint, r.Maxt, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) + } + + // create snapshot + snap := t.TempDir() + + require.NoError(t, db.Snapshot(snap, true)) + + // reopen DB from snapshot + db := newTestDB(t, withDir(snap)) + + // Compare the result. + q, err := db.Querier(0, numSamples) + require.NoError(t, err) + defer func() { require.NoError(t, q.Close()) }() + + res := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + expSamples := make([]chunks.Sample, 0, len(c.remaint)) + for _, ts := range c.remaint { + expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil}) + } + + expss := newMockSeriesSet([]storage.Series{ + storage.NewListSeries(labels.FromStrings("a", "b"), expSamples), + }) + + if len(expSamples) == 0 { + require.False(t, res.Next()) + return + } + + for { + eok, rok := expss.Next(), res.Next() + require.Equal(t, eok, rok) + + if !eok { + require.Empty(t, res.Warnings()) + break + } + sexp := expss.At() + sres := res.At() + + require.Equal(t, sexp.Labels(), sres.Labels()) + + smplExp, errExp := storage.ExpandSamples(sexp.Iterator(nil), nil) + smplRes, errRes := storage.ExpandSamples(sres.Iterator(nil), nil) + + require.Equal(t, errExp, errRes) + require.Equal(t, smplExp, smplRes) + } + }) + } +} + +func TestDB_e2e(t *testing.T) { + const ( + numDatapoints = 1000 + numRanges = 1000 + timeInterval = int64(3) + ) + // Create 8 series with 1000 data-points of different ranges and run queries. + lbls := [][]labels.Label{ + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + { + {Name: "a", Value: "b"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prometheus"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "127.0.0.1:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + { + {Name: "a", Value: "c"}, + {Name: "instance", Value: "localhost:9090"}, + {Name: "job", Value: "prom-k8s"}, + }, + } + + seriesMap := map[string][]chunks.Sample{} + for _, l := range lbls { + seriesMap[labels.New(l...).String()] = []chunks.Sample{} + } + + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + for _, l := range lbls { + lset := labels.New(l...) + series := []chunks.Sample{} + + ts := rand.Int63n(300) + for range numDatapoints { + v := rand.Float64() + + series = append(series, sample{ts, v, nil, nil}) + + _, err := app.Append(0, lset, ts, v) + require.NoError(t, err) + + ts += rand.Int63n(timeInterval) + 1 + } + + seriesMap[lset.String()] = series + } + + require.NoError(t, app.Commit()) + + // Query each selector on 1000 random time-ranges. + queries := []struct { + ms []*labels.Matcher + }{ + { + ms: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "b")}, + }, + { + ms: []*labels.Matcher{ + labels.MustNewMatcher(labels.MatchEqual, "a", "b"), + labels.MustNewMatcher(labels.MatchEqual, "job", "prom-k8s"), + }, + }, + { + ms: []*labels.Matcher{ + labels.MustNewMatcher(labels.MatchEqual, "a", "c"), + labels.MustNewMatcher(labels.MatchEqual, "instance", "localhost:9090"), + labels.MustNewMatcher(labels.MatchEqual, "job", "prometheus"), + }, + }, + // TODO: Add Regexp Matchers. + } + + for _, qry := range queries { + matched := labels.Slice{} + for _, l := range lbls { + s := labels.Selector(qry.ms) + ls := labels.New(l...) + if s.Matches(ls) { + matched = append(matched, ls) + } + } + + sort.Sort(matched) + + for range numRanges { + mint := rand.Int63n(300) + maxt := mint + rand.Int63n(timeInterval*int64(numDatapoints)) + + expected := map[string][]chunks.Sample{} + + // Build the mockSeriesSet. + for _, m := range matched { + smpls := boundedSamples(seriesMap[m.String()], mint, maxt) + if len(smpls) > 0 { + expected[m.String()] = smpls + } + } + + q, err := db.Querier(mint, maxt) + require.NoError(t, err) + + ss := q.Select(ctx, false, nil, qry.ms...) + result := map[string][]chunks.Sample{} + + for ss.Next() { + x := ss.At() + + smpls, err := storage.ExpandSamples(x.Iterator(nil), newSample) + require.NoError(t, err) + + if len(smpls) > 0 { + result[x.Labels().String()] = smpls + } + } + + require.NoError(t, ss.Err()) + require.Empty(t, ss.Warnings()) + require.Equal(t, expected, result) + + q.Close() + } + } +} + +func TestWALFlushedOnDBClose(t *testing.T) { + db := newTestDB(t) + + lbls := labels.FromStrings("labelname", "labelvalue") + + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, lbls, 0, 1) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + require.NoError(t, db.Close()) + + db = newTestDB(t, withDir(db.Dir())) + + q, err := db.Querier(0, 1) + require.NoError(t, err) + + values, ws, err := q.LabelValues(ctx, "labelname", nil) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, []string{"labelvalue"}, values) +} + +func TestWALSegmentSizeOptions(t *testing.T) { + tests := map[int]func(dbdir string, segmentSize int){ + // Default Wal Size. + 0: func(dbDir string, _ int) { + filesAndDir, err := os.ReadDir(filepath.Join(dbDir, "wal")) + require.NoError(t, err) + files := []os.FileInfo{} + for _, f := range filesAndDir { + if !f.IsDir() { + fi, err := f.Info() + require.NoError(t, err) + files = append(files, fi) + } + } + // All the full segment files (all but the last) should match the segment size option. + for _, f := range files[:len(files)-1] { + require.Equal(t, int64(DefaultOptions().WALSegmentSize), f.Size(), "WAL file size doesn't match WALSegmentSize option, filename: %v", f.Name()) + } + lastFile := files[len(files)-1] + require.Greater(t, int64(DefaultOptions().WALSegmentSize), lastFile.Size(), "last WAL file size is not smaller than the WALSegmentSize option, filename: %v", lastFile.Name()) + }, + // Custom Wal Size. + 2 * 32 * 1024: func(dbDir string, segmentSize int) { + filesAndDir, err := os.ReadDir(filepath.Join(dbDir, "wal")) + require.NoError(t, err) + files := []os.FileInfo{} + for _, f := range filesAndDir { + if !f.IsDir() { + fi, err := f.Info() + require.NoError(t, err) + files = append(files, fi) + } + } + require.NotEmpty(t, files, "current WALSegmentSize should result in more than a single WAL file.") + // All the full segment files (all but the last) should match the segment size option. + for _, f := range files[:len(files)-1] { + require.Equal(t, int64(segmentSize), f.Size(), "WAL file size doesn't match WALSegmentSize option, filename: %v", f.Name()) + } + lastFile := files[len(files)-1] + require.Greater(t, int64(segmentSize), lastFile.Size(), "last WAL file size is not smaller than the WALSegmentSize option, filename: %v", lastFile.Name()) + }, + // Wal disabled. + -1: func(dbDir string, _ int) { + // Check that WAL dir is not there. + _, err := os.Stat(filepath.Join(dbDir, "wal")) + require.Error(t, err) + // Check that there is chunks dir. + _, err = os.Stat(mmappedChunksDir(dbDir)) + require.NoError(t, err) + }, + } + for segmentSize, testFunc := range tests { + t.Run(fmt.Sprintf("WALSegmentSize %d test", segmentSize), func(t *testing.T) { + opts := DefaultOptions() + opts.WALSegmentSize = segmentSize + db := newTestDB(t, withOpts(opts)) + + for i := range int64(155) { + app := db.Appender(context.Background()) + ref, err := app.Append(0, labels.FromStrings("wal"+strconv.Itoa(int(i)), "size"), i, rand.Float64()) + require.NoError(t, err) + for j := int64(1); j <= 78; j++ { + _, err := app.Append(ref, labels.EmptyLabels(), i+j, rand.Float64()) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + require.NoError(t, db.Close()) + testFunc(db.Dir(), opts.WALSegmentSize) + }) + } +} + +// https://github.com/prometheus/prometheus/issues/9846 +// https://github.com/prometheus/prometheus/issues/9859 +func TestWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T) { + const ( + numRuns = 1 + numSamplesBeforeSeriesCreation = 1000 + ) + + // We test both with few and many samples appended after series creation. If samples are < 120 then there's no + // mmap-ed chunk, otherwise there's at least 1 mmap-ed chunk when replaying the WAL. + for _, numSamplesAfterSeriesCreation := range []int{1, 1000} { + for run := 1; run <= numRuns; run++ { + t.Run(fmt.Sprintf("samples after series creation = %d, run = %d", numSamplesAfterSeriesCreation, run), func(t *testing.T) { + testWALReplayRaceOnSamplesLoggedBeforeSeries(t, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation) + }) + } + } +} + +func testWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation int) { + const numSeries = 1000 + + db := newTestDB(t) + db.DisableCompactions() + + for seriesRef := 1; seriesRef <= numSeries; seriesRef++ { + // Log samples before the series is logged to the WAL. + var enc record.Encoder + var samples []record.RefSample + + for ts := range numSamplesBeforeSeriesCreation { + samples = append(samples, record.RefSample{ + Ref: chunks.HeadSeriesRef(uint64(seriesRef)), + T: int64(ts), + V: float64(ts), + }) + } + + err := db.Head().wal.Log(enc.Samples(samples, nil)) + require.NoError(t, err) + + // Add samples via appender so that they're logged after the series in the WAL. + app := db.Appender(context.Background()) + lbls := labels.FromStrings("series_id", strconv.Itoa(seriesRef)) + + for ts := numSamplesBeforeSeriesCreation; ts < numSamplesBeforeSeriesCreation+numSamplesAfterSeriesCreation; ts++ { + _, err := app.Append(0, lbls, int64(ts), float64(ts)) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + require.NoError(t, db.Close()) + + // Reopen the DB, replaying the WAL. + db = newTestDB(t, withDir(db.Dir())) + + // Query back chunks for all series. + q, err := db.ChunkQuerier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + set := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchRegexp, "series_id", ".+")) + actualSeries := 0 + var chunksIt chunks.Iterator + + for set.Next() { + actualSeries++ + actualChunks := 0 + + chunksIt = set.At().Iterator(chunksIt) + for chunksIt.Next() { + actualChunks++ + } + require.NoError(t, chunksIt.Err()) + + // We expect 1 chunk every 120 samples after series creation. + require.Equalf(t, (numSamplesAfterSeriesCreation/120)+1, actualChunks, "series: %s", set.At().Labels().String()) + } + + require.NoError(t, set.Err()) + require.Equal(t, numSeries, actualSeries) +} + +func TestTombstoneClean(t *testing.T) { + t.Parallel() + const numSamples int64 = 10 + + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + smpls := make([]float64, numSamples) + for i := range numSamples { + smpls[i] = rand.Float64() + app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + } + + require.NoError(t, app.Commit()) + cases := []struct { + intervals tombstones.Intervals + remaint []int64 + }{ + { + intervals: tombstones.Intervals{{Mint: 1, Maxt: 3}, {Mint: 4, Maxt: 7}}, + remaint: []int64{0, 8, 9}, + }, + } + + for _, c := range cases { + // Delete the ranges. + + // Create snapshot. + snap := t.TempDir() + require.NoError(t, db.Snapshot(snap, true)) + require.NoError(t, db.Close()) + + // Reopen DB from snapshot. + db := newTestDB(t, withDir(snap)) + + for _, r := range c.intervals { + require.NoError(t, db.Delete(ctx, r.Mint, r.Maxt, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) + } + + // All of the setup for THIS line. + require.NoError(t, db.CleanTombstones()) + + // Compare the result. + q, err := db.Querier(0, numSamples) + require.NoError(t, err) + defer q.Close() + + res := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + + expSamples := make([]chunks.Sample, 0, len(c.remaint)) + for _, ts := range c.remaint { + expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil}) + } + + expss := newMockSeriesSet([]storage.Series{ + storage.NewListSeries(labels.FromStrings("a", "b"), expSamples), + }) + + if len(expSamples) == 0 { + require.False(t, res.Next()) + continue + } + + for { + eok, rok := expss.Next(), res.Next() + require.Equal(t, eok, rok) + + if !eok { + break + } + sexp := expss.At() + sres := res.At() + + require.Equal(t, sexp.Labels(), sres.Labels()) + + smplExp, errExp := storage.ExpandSamples(sexp.Iterator(nil), nil) + smplRes, errRes := storage.ExpandSamples(sres.Iterator(nil), nil) + + require.Equal(t, errExp, errRes) + require.Equal(t, smplExp, smplRes) + } + require.Empty(t, res.Warnings()) + + for _, b := range db.Blocks() { + require.Equal(t, tombstones.NewMemTombstones(), b.tombstones) + } + } +} + +// TestTombstoneCleanResultEmptyBlock tests that a TombstoneClean that results in empty blocks (no timeseries) +// will also delete the resultant block. +func TestTombstoneCleanResultEmptyBlock(t *testing.T) { + t.Parallel() + numSamples := int64(10) + + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + smpls := make([]float64, numSamples) + for i := range numSamples { + smpls[i] = rand.Float64() + app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + } + + require.NoError(t, app.Commit()) + // Interval should cover the whole block. + intervals := tombstones.Intervals{{Mint: 0, Maxt: numSamples}} + + // Create snapshot. + snap := t.TempDir() + require.NoError(t, db.Snapshot(snap, true)) + require.NoError(t, db.Close()) + + // Reopen DB from snapshot. + db = newTestDB(t, withDir(snap)) + + // Create tombstones by deleting all samples. + for _, r := range intervals { + require.NoError(t, db.Delete(ctx, r.Mint, r.Maxt, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))) + } + + require.NoError(t, db.CleanTombstones()) + + // After cleaning tombstones that covers the entire block, no blocks should be left behind. + actualBlockDirs, err := blockDirs(db.Dir()) + require.NoError(t, err) + require.Empty(t, actualBlockDirs) +} + +// TestTombstoneCleanFail tests that a failing TombstoneClean doesn't leave any blocks behind. +// When TombstoneClean errors the original block that should be rebuilt doesn't get deleted so +// if TombstoneClean leaves any blocks behind these will overlap. +func TestTombstoneCleanFail(t *testing.T) { + t.Parallel() + db := newTestDB(t) + + var oldBlockDirs []string + + // Create some blocks pending for compaction. + // totalBlocks should be >=2 so we have enough blocks to trigger compaction failure. + totalBlocks := 2 + for i := range totalBlocks { + blockDir := createBlock(t, db.Dir(), genSeries(1, 1, int64(i), int64(i)+1)) + block, err := OpenBlock(nil, blockDir, nil, nil) + require.NoError(t, err) + // Add some fake tombstones to trigger the compaction. + tomb := tombstones.NewMemTombstones() + tomb.AddInterval(0, tombstones.Interval{Mint: int64(i), Maxt: int64(i) + 1}) + block.tombstones = tomb + + db.blocks = append(db.blocks, block) + oldBlockDirs = append(oldBlockDirs, blockDir) + } + + // Initialize the mockCompactorFailing with a room for a single compaction iteration. + // mockCompactorFailing will fail on the second iteration so we can check if the cleanup works as expected. + db.compactor = &mockCompactorFailing{ + t: t, + blocks: db.blocks, + max: totalBlocks + 1, + } + + // The compactor should trigger a failure here. + require.Error(t, db.CleanTombstones()) + + // Now check that the CleanTombstones replaced the old block even after a failure. + actualBlockDirs, err := blockDirs(db.Dir()) + require.NoError(t, err) + // Only one block should have been replaced by a new block. + require.Len(t, actualBlockDirs, len(oldBlockDirs)) + require.Len(t, intersection(oldBlockDirs, actualBlockDirs), len(actualBlockDirs)-1) +} + +func intersection(oldBlocks, actualBlocks []string) (intersection []string) { + hash := make(map[string]bool) + for _, e := range oldBlocks { + hash[e] = true + } + for _, e := range actualBlocks { + // If block present in the hashmap then append intersection list. + if hash[e] { + intersection = append(intersection, e) + } + } + return intersection +} + +// mockCompactorFailing creates a new empty block on every write and fails when reached the max allowed total. +// For CompactOOO, it always fails. +type mockCompactorFailing struct { + t *testing.T + blocks []*Block + max int +} + +func (*mockCompactorFailing) Plan(string) ([]string, error) { + return nil, nil +} + +func (c *mockCompactorFailing) Write(dest string, _ BlockReader, _, _ int64, _ *BlockMeta) ([]ulid.ULID, error) { + if len(c.blocks) >= c.max { + return []ulid.ULID{}, errors.New("the compactor already did the maximum allowed blocks so it is time to fail") + } + + block, err := OpenBlock(nil, createBlock(c.t, dest, genSeries(1, 1, 0, 1)), nil, nil) + require.NoError(c.t, err) + require.NoError(c.t, block.Close()) // Close block as we won't be using anywhere. + c.blocks = append(c.blocks, block) + + // Now check that all expected blocks are actually persisted on disk. + // This way we make sure that we have some blocks that are supposed to be removed. + var expectedBlocks []string + for _, b := range c.blocks { + expectedBlocks = append(expectedBlocks, filepath.Join(dest, b.Meta().ULID.String())) + } + actualBlockDirs, err := blockDirs(dest) + require.NoError(c.t, err) + + require.Equal(c.t, expectedBlocks, actualBlockDirs) + + return []ulid.ULID{block.Meta().ULID}, nil +} + +func (*mockCompactorFailing) Compact(string, []string, []*Block) ([]ulid.ULID, error) { + return []ulid.ULID{}, nil +} + +func (*mockCompactorFailing) CompactOOO(string, *OOOCompactionHead) (result []ulid.ULID, err error) { + return nil, errors.New("mock compaction failing CompactOOO") +} + +func TestTimeRetention(t *testing.T) { + t.Parallel() + testCases := []struct { + name string + blocks []*BlockMeta + expBlocks []*BlockMeta + retentionDuration int64 + }{ + { + name: "Block max time delta greater than retention duration", + blocks: []*BlockMeta{ + {MinTime: 500, MaxTime: 900}, // Oldest block, beyond retention + {MinTime: 1000, MaxTime: 1500}, + {MinTime: 1500, MaxTime: 2000}, // Newest block + }, + expBlocks: []*BlockMeta{ + {MinTime: 1000, MaxTime: 1500}, + {MinTime: 1500, MaxTime: 2000}, + }, + retentionDuration: 1000, + }, + { + name: "Block max time delta equal to retention duration", + blocks: []*BlockMeta{ + {MinTime: 500, MaxTime: 900}, // Oldest block + {MinTime: 1000, MaxTime: 1500}, // Coinciding exactly with the retention duration. + {MinTime: 1500, MaxTime: 2000}, // Newest block + }, + expBlocks: []*BlockMeta{ + {MinTime: 1500, MaxTime: 2000}, + }, + retentionDuration: 500, + }, + } + for _, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + db := newTestDB(t, withRngs(1000)) + + for _, m := range tc.blocks { + createBlock(t, db.Dir(), genSeries(10, 10, m.MinTime, m.MaxTime)) + } + + require.NoError(t, db.reloadBlocks()) // Reload the db to register the new blocks. + require.Len(t, db.Blocks(), len(tc.blocks)) // Ensure all blocks are registered. + + db.opts.RetentionDuration = tc.retentionDuration + // Reloading should truncate the blocks which are >= the retention duration vs the first block. + require.NoError(t, db.reloadBlocks()) + + actBlocks := db.Blocks() + + require.Equal(t, 1, int(prom_testutil.ToFloat64(db.metrics.timeRetentionCount)), "metric retention count mismatch") + require.Len(t, actBlocks, len(tc.expBlocks)) + for i, eb := range tc.expBlocks { + require.Equal(t, eb.MinTime, actBlocks[i].meta.MinTime) + require.Equal(t, eb.MaxTime, actBlocks[i].meta.MaxTime) + } + }) + } +} + +func TestRetentionDurationMetric(t *testing.T) { + db := newTestDB(t, withOpts(&Options{ + RetentionDuration: 1000, + }), withRngs(100)) + + expRetentionDuration := 1.0 + actRetentionDuration := prom_testutil.ToFloat64(db.metrics.retentionDuration) + require.Equal(t, expRetentionDuration, actRetentionDuration, "metric retention duration mismatch") +} + +func TestSizeRetention(t *testing.T) { + t.Parallel() + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 100 + db := newTestDB(t, withOpts(opts), withRngs(100)) + + blocks := []*BlockMeta{ + {MinTime: 100, MaxTime: 200}, // Oldest block + {MinTime: 200, MaxTime: 300}, + {MinTime: 300, MaxTime: 400}, + {MinTime: 400, MaxTime: 500}, + {MinTime: 500, MaxTime: 600}, // Newest Block + } + + for _, m := range blocks { + createBlock(t, db.Dir(), genSeries(100, 10, m.MinTime, m.MaxTime)) + } + + headBlocks := []*BlockMeta{ + {MinTime: 700, MaxTime: 800}, + } + + // Add some data to the WAL. + headApp := db.Head().Appender(context.Background()) + var aSeries labels.Labels + var it chunkenc.Iterator + for _, m := range headBlocks { + series := genSeries(100, 10, m.MinTime, m.MaxTime+1) + for _, s := range series { + aSeries = s.Labels() + it = s.Iterator(it) + for it.Next() == chunkenc.ValFloat { + tim, v := it.At() + _, err := headApp.Append(0, s.Labels(), tim, v) + require.NoError(t, err) + } + require.NoError(t, it.Err()) + } + } + require.NoError(t, headApp.Commit()) + db.Head().mmapHeadChunks() + + require.Eventually(t, func() bool { + return db.Head().chunkDiskMapper.IsQueueEmpty() + }, 2*time.Second, 100*time.Millisecond) + + // Test that registered size matches the actual disk size. + require.NoError(t, db.reloadBlocks()) // Reload the db to register the new db size. + require.Len(t, db.Blocks(), len(blocks)) // Ensure all blocks are registered. + blockSize := int64(prom_testutil.ToFloat64(db.metrics.blocksBytes)) // Use the actual internal metrics. + walSize, err := db.Head().wal.Size() + require.NoError(t, err) + cdmSize, err := db.Head().chunkDiskMapper.Size() + require.NoError(t, err) + require.NotZero(t, cdmSize) + // Expected size should take into account block size + WAL size + Head + // chunks size + expSize := blockSize + walSize + cdmSize + actSize, err := fileutil.DirSize(db.Dir()) + require.NoError(t, err) + require.Equal(t, expSize, actSize, "registered size doesn't match actual disk size") + + // Create a WAL checkpoint, and compare sizes. + first, last, err := wlog.Segments(db.Head().wal.Dir()) + require.NoError(t, err) + _, err = wlog.Checkpoint(promslog.NewNopLogger(), db.Head().wal, first, last-1, func(chunks.HeadSeriesRef) bool { return false }, 0) + require.NoError(t, err) + blockSize = int64(prom_testutil.ToFloat64(db.metrics.blocksBytes)) // Use the actual internal metrics. + walSize, err = db.Head().wal.Size() + require.NoError(t, err) + cdmSize, err = db.Head().chunkDiskMapper.Size() + require.NoError(t, err) + require.NotZero(t, cdmSize) + expSize = blockSize + walSize + cdmSize + actSize, err = fileutil.DirSize(db.Dir()) + require.NoError(t, err) + require.Equal(t, expSize, actSize, "registered size doesn't match actual disk size") + + // Truncate Chunk Disk Mapper and compare sizes. + require.NoError(t, db.Head().chunkDiskMapper.Truncate(900)) + cdmSize, err = db.Head().chunkDiskMapper.Size() + require.NoError(t, err) + require.NotZero(t, cdmSize) + expSize = blockSize + walSize + cdmSize + actSize, err = fileutil.DirSize(db.Dir()) + require.NoError(t, err) + require.Equal(t, expSize, actSize, "registered size doesn't match actual disk size") + + // Add some out of order samples to check the size of WBL. + headApp = db.Head().Appender(context.Background()) + for ts := int64(750); ts < 800; ts++ { + _, err := headApp.Append(0, aSeries, ts, float64(ts)) + require.NoError(t, err) + } + require.NoError(t, headApp.Commit()) + + walSize, err = db.Head().wal.Size() + require.NoError(t, err) + wblSize, err := db.Head().wbl.Size() + require.NoError(t, err) + require.NotZero(t, wblSize) + cdmSize, err = db.Head().chunkDiskMapper.Size() + require.NoError(t, err) + expSize = blockSize + walSize + wblSize + cdmSize + actSize, err = fileutil.DirSize(db.Dir()) + require.NoError(t, err) + require.Equal(t, expSize, actSize, "registered size doesn't match actual disk size") + + // Decrease the max bytes limit so that a delete is triggered. + // Check total size, total count and check that the oldest block was deleted. + firstBlockSize := db.Blocks()[0].Size() + sizeLimit := actSize - firstBlockSize + db.opts.MaxBytes = sizeLimit // Set the new db size limit one block smaller that the actual size. + require.NoError(t, db.reloadBlocks()) // Reload the db to register the new db size. + + expBlocks := blocks[1:] + actBlocks := db.Blocks() + blockSize = int64(prom_testutil.ToFloat64(db.metrics.blocksBytes)) + walSize, err = db.Head().wal.Size() + require.NoError(t, err) + cdmSize, err = db.Head().chunkDiskMapper.Size() + require.NoError(t, err) + require.NotZero(t, cdmSize) + // Expected size should take into account block size + WAL size + WBL size + expSize = blockSize + walSize + wblSize + cdmSize + actRetentionCount := int(prom_testutil.ToFloat64(db.metrics.sizeRetentionCount)) + actSize, err = fileutil.DirSize(db.Dir()) + require.NoError(t, err) + + require.Equal(t, 1, actRetentionCount, "metric retention count mismatch") + require.Equal(t, expSize, actSize, "metric db size doesn't match actual disk size") + require.LessOrEqual(t, expSize, sizeLimit, "actual size (%v) is expected to be less than or equal to limit (%v)", expSize, sizeLimit) + require.Len(t, actBlocks, len(blocks)-1, "new block count should be decreased from:%v to:%v", len(blocks), len(blocks)-1) + require.Equal(t, expBlocks[0].MaxTime, actBlocks[0].meta.MaxTime, "maxT mismatch of the first block") + require.Equal(t, expBlocks[len(expBlocks)-1].MaxTime, actBlocks[len(actBlocks)-1].meta.MaxTime, "maxT mismatch of the last block") +} + +func TestSizeRetentionMetric(t *testing.T) { + cases := []struct { + maxBytes int64 + expMaxBytes int64 + }{ + {maxBytes: 1000, expMaxBytes: 1000}, + {maxBytes: 0, expMaxBytes: 0}, + {maxBytes: -1000, expMaxBytes: 0}, + } + + for _, c := range cases { + db := newTestDB(t, withOpts(&Options{ + MaxBytes: c.maxBytes, + }), withRngs(100)) + + actMaxBytes := int64(prom_testutil.ToFloat64(db.metrics.maxBytes)) + require.Equal(t, c.expMaxBytes, actMaxBytes, "metric retention limit bytes mismatch") + } +} + +// TestRuntimeRetentionConfigChange tests that retention configuration can be +// changed at runtime via ApplyConfig and that the retention logic properly +// deletes blocks when retention is shortened. This test also ensures race-free +// concurrent access to retention settings. +func TestRuntimeRetentionConfigChange(t *testing.T) { + const ( + initialRetentionDuration = int64(10 * time.Hour / time.Millisecond) // 10 hours + shorterRetentionDuration = int64(1 * time.Hour / time.Millisecond) // 1 hour + ) + + db := newTestDB(t, withOpts(&Options{ + RetentionDuration: initialRetentionDuration, + }), withRngs(100)) + + nineHoursMs := int64(9 * time.Hour / time.Millisecond) + nineAndHalfHoursMs := int64((9*time.Hour + 30*time.Minute) / time.Millisecond) + blocks := []*BlockMeta{ + {MinTime: 0, MaxTime: 100}, // 10 hours old (beyond new retention) + {MinTime: 100, MaxTime: 200}, // 9.9 hours old (beyond new retention) + {MinTime: nineHoursMs, MaxTime: nineAndHalfHoursMs}, // 1 hour old (within new retention) + {MinTime: nineAndHalfHoursMs, MaxTime: initialRetentionDuration}, // 0.5 hours old (within new retention) + } + + for _, m := range blocks { + createBlock(t, db.Dir(), genSeriesFromSampleGenerator(10, 10, m.MinTime, m.MaxTime, int64(time.Minute/time.Millisecond), func(ts int64) chunks.Sample { + return sample{t: ts, f: rand.Float64()} + })) + } + + // Reload blocks and verify all are loaded. + require.NoError(t, db.reloadBlocks()) + require.Len(t, db.Blocks(), len(blocks), "expected all blocks to be loaded initially") + + cfg := &config.Config{ + StorageConfig: config.StorageConfig{ + TSDBConfig: &config.TSDBConfig{ + Retention: &config.TSDBRetentionConfig{ + Time: model.Duration(shorterRetentionDuration), + }, + }, + }, + } + + require.NoError(t, db.ApplyConfig(cfg), "ApplyConfig should succeed") + + actualRetention := db.getRetentionDuration() + require.Equal(t, shorterRetentionDuration, actualRetention, "retention duration should be updated") + + expectedRetentionSeconds := (time.Duration(shorterRetentionDuration) * time.Millisecond).Seconds() + actualRetentionSeconds := prom_testutil.ToFloat64(db.metrics.retentionDuration) + require.Equal(t, expectedRetentionSeconds, actualRetentionSeconds, "retention duration metric should be updated") + + require.NoError(t, db.reloadBlocks()) + + // Verify that blocks beyond the new retention were deleted. + // We expect only the last 2 blocks to remain (those within 1 hour). + actBlocks := db.Blocks() + require.Len(t, actBlocks, 2, "expected old blocks to be deleted after retention change") + + // Verify the remaining blocks are the newest ones. + require.Equal(t, nineHoursMs, actBlocks[0].meta.MinTime, "first remaining block should be within retention") + require.Equal(t, nineAndHalfHoursMs, actBlocks[1].meta.MinTime, "last remaining block should be the newest") + + require.Positive(t, int(prom_testutil.ToFloat64(db.metrics.timeRetentionCount)), "time retention count should be incremented") +} + +func TestNotMatcherSelectsLabelsUnsetSeries(t *testing.T) { + db := newTestDB(t) + + labelpairs := []labels.Labels{ + labels.FromStrings("a", "abcd", "b", "abcde"), + labels.FromStrings("labelname", "labelvalue"), + } + + ctx := context.Background() + app := db.Appender(ctx) + for _, lbls := range labelpairs { + _, err := app.Append(0, lbls, 0, 1) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + cases := []struct { + selector labels.Selector + series []labels.Labels + }{{ + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchNotEqual, "lname", "lvalue"), + }, + series: labelpairs, + }, { + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchEqual, "a", "abcd"), + labels.MustNewMatcher(labels.MatchNotEqual, "b", "abcde"), + }, + series: []labels.Labels{}, + }, { + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchEqual, "a", "abcd"), + labels.MustNewMatcher(labels.MatchNotEqual, "b", "abc"), + }, + series: []labels.Labels{labelpairs[0]}, + }, { + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchNotRegexp, "a", "abd.*"), + }, + series: labelpairs, + }, { + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchNotRegexp, "a", "abc.*"), + }, + series: labelpairs[1:], + }, { + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchNotRegexp, "c", "abd.*"), + }, + series: labelpairs, + }, { + selector: labels.Selector{ + labels.MustNewMatcher(labels.MatchNotRegexp, "labelname", "labelvalue"), + }, + series: labelpairs[:1], + }} + + q, err := db.Querier(0, 10) + require.NoError(t, err) + defer func() { require.NoError(t, q.Close()) }() + + for _, c := range cases { + ss := q.Select(ctx, false, nil, c.selector...) + lres, _, ws, err := expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, c.series, lres) + } +} + +// expandSeriesSet returns the raw labels in the order they are retrieved from +// the series set and the samples keyed by Labels().String(). +func expandSeriesSet(ss storage.SeriesSet) ([]labels.Labels, map[string][]sample, annotations.Annotations, error) { + resultLabels := []labels.Labels{} + resultSamples := map[string][]sample{} + var it chunkenc.Iterator + for ss.Next() { + series := ss.At() + samples := []sample{} + it = series.Iterator(it) + for it.Next() == chunkenc.ValFloat { + t, v := it.At() + samples = append(samples, sample{t: t, f: v}) + } + resultLabels = append(resultLabels, series.Labels()) + resultSamples[series.Labels().String()] = samples + } + return resultLabels, resultSamples, ss.Warnings(), ss.Err() +} + +func TestOverlappingBlocksDetectsAllOverlaps(t *testing.T) { + // Create 10 blocks that does not overlap (0-10, 10-20, ..., 100-110) but in reverse order to ensure our algorithm + // will handle that. + metas := make([]BlockMeta, 11) + for i := 10; i >= 0; i-- { + metas[i] = BlockMeta{MinTime: int64(i * 10), MaxTime: int64((i + 1) * 10)} + } + + require.Empty(t, OverlappingBlocks(metas), "we found unexpected overlaps") + + // Add overlapping blocks. We've to establish order again since we aren't interested + // in trivial overlaps caused by unorderedness. + add := func(ms ...BlockMeta) []BlockMeta { + repl := append(append([]BlockMeta{}, metas...), ms...) + sort.Slice(repl, func(i, j int) bool { + return repl[i].MinTime < repl[j].MinTime + }) + return repl + } + + // o1 overlaps with 10-20. + o1 := BlockMeta{MinTime: 15, MaxTime: 17} + require.Equal(t, Overlaps{ + {Min: 15, Max: 17}: {metas[1], o1}, + }, OverlappingBlocks(add(o1))) + + // o2 overlaps with 20-30 and 30-40. + o2 := BlockMeta{MinTime: 21, MaxTime: 31} + require.Equal(t, Overlaps{ + {Min: 21, Max: 30}: {metas[2], o2}, + {Min: 30, Max: 31}: {o2, metas[3]}, + }, OverlappingBlocks(add(o2))) + + // o3a and o3b overlaps with 30-40 and each other. + o3a := BlockMeta{MinTime: 33, MaxTime: 39} + o3b := BlockMeta{MinTime: 34, MaxTime: 36} + require.Equal(t, Overlaps{ + {Min: 34, Max: 36}: {metas[3], o3a, o3b}, + }, OverlappingBlocks(add(o3a, o3b))) + + // o4 is 1:1 overlap with 50-60. + o4 := BlockMeta{MinTime: 50, MaxTime: 60} + require.Equal(t, Overlaps{ + {Min: 50, Max: 60}: {metas[5], o4}, + }, OverlappingBlocks(add(o4))) + + // o5 overlaps with 60-70, 70-80 and 80-90. + o5 := BlockMeta{MinTime: 61, MaxTime: 85} + require.Equal(t, Overlaps{ + {Min: 61, Max: 70}: {metas[6], o5}, + {Min: 70, Max: 80}: {o5, metas[7]}, + {Min: 80, Max: 85}: {o5, metas[8]}, + }, OverlappingBlocks(add(o5))) + + // o6a overlaps with 90-100, 100-110 and o6b, o6b overlaps with 90-100 and o6a. + o6a := BlockMeta{MinTime: 92, MaxTime: 105} + o6b := BlockMeta{MinTime: 94, MaxTime: 99} + require.Equal(t, Overlaps{ + {Min: 94, Max: 99}: {metas[9], o6a, o6b}, + {Min: 100, Max: 105}: {o6a, metas[10]}, + }, OverlappingBlocks(add(o6a, o6b))) + + // All together. + require.Equal(t, Overlaps{ + {Min: 15, Max: 17}: {metas[1], o1}, + {Min: 21, Max: 30}: {metas[2], o2}, {Min: 30, Max: 31}: {o2, metas[3]}, + {Min: 34, Max: 36}: {metas[3], o3a, o3b}, + {Min: 50, Max: 60}: {metas[5], o4}, + {Min: 61, Max: 70}: {metas[6], o5}, {Min: 70, Max: 80}: {o5, metas[7]}, {Min: 80, Max: 85}: {o5, metas[8]}, + {Min: 94, Max: 99}: {metas[9], o6a, o6b}, {Min: 100, Max: 105}: {o6a, metas[10]}, + }, OverlappingBlocks(add(o1, o2, o3a, o3b, o4, o5, o6a, o6b))) + + // Additional case. + var nc1 []BlockMeta + nc1 = append(nc1, BlockMeta{MinTime: 1, MaxTime: 5}) + nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) + nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) + nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) + nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) + nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 6}) + nc1 = append(nc1, BlockMeta{MinTime: 3, MaxTime: 5}) + nc1 = append(nc1, BlockMeta{MinTime: 5, MaxTime: 7}) + nc1 = append(nc1, BlockMeta{MinTime: 7, MaxTime: 10}) + nc1 = append(nc1, BlockMeta{MinTime: 8, MaxTime: 9}) + require.Equal(t, Overlaps{ + {Min: 2, Max: 3}: {nc1[0], nc1[1], nc1[2], nc1[3], nc1[4], nc1[5]}, // 1-5, 2-3, 2-3, 2-3, 2-3, 2,6 + {Min: 3, Max: 5}: {nc1[0], nc1[5], nc1[6]}, // 1-5, 2-6, 3-5 + {Min: 5, Max: 6}: {nc1[5], nc1[7]}, // 2-6, 5-7 + {Min: 8, Max: 9}: {nc1[8], nc1[9]}, // 7-10, 8-9 + }, OverlappingBlocks(nc1)) +} + +// Regression test for https://github.com/prometheus/tsdb/issues/347 +func TestChunkAtBlockBoundary(t *testing.T) { + t.Parallel() + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + blockRange := db.compactor.(*LeveledCompactor).ranges[0] + label := labels.FromStrings("foo", "bar") + + for i := range int64(3) { + _, err := app.Append(0, label, i*blockRange, 0) + require.NoError(t, err) + _, err = app.Append(0, label, i*blockRange+1000, 0) + require.NoError(t, err) + } + + err := app.Commit() + require.NoError(t, err) + + err = db.Compact(ctx) + require.NoError(t, err) + + var builder labels.ScratchBuilder + + for _, block := range db.Blocks() { + r, err := block.Index() + require.NoError(t, err) + defer r.Close() + + meta := block.Meta() + + k, v := index.AllPostingsKey() + p, err := r.Postings(ctx, k, v) + require.NoError(t, err) + + var chks []chunks.Meta + + chunkCount := 0 + + for p.Next() { + err = r.Series(p.At(), &builder, &chks) + require.NoError(t, err) + for _, c := range chks { + require.True(t, meta.MinTime <= c.MinTime && c.MaxTime <= meta.MaxTime, + "chunk spans beyond block boundaries: [block.MinTime=%d, block.MaxTime=%d]; [chunk.MinTime=%d, chunk.MaxTime=%d]", + meta.MinTime, meta.MaxTime, c.MinTime, c.MaxTime) + chunkCount++ + } + } + require.Equal(t, 1, chunkCount, "expected 1 chunk in block %s, got %d", meta.ULID, chunkCount) + } +} + +func TestQuerierWithBoundaryChunks(t *testing.T) { + t.Parallel() + db := newTestDB(t) + + ctx := context.Background() + app := db.Appender(ctx) + + blockRange := db.compactor.(*LeveledCompactor).ranges[0] + label := labels.FromStrings("foo", "bar") + + for i := range int64(5) { + _, err := app.Append(0, label, i*blockRange, 0) + require.NoError(t, err) + _, err = app.Append(0, labels.FromStrings("blockID", strconv.FormatInt(i, 10)), i*blockRange, 0) + require.NoError(t, err) + } + + err := app.Commit() + require.NoError(t, err) + + err = db.Compact(ctx) + require.NoError(t, err) + + require.GreaterOrEqual(t, len(db.blocks), 3, "invalid test, less than three blocks in DB") + + q, err := db.Querier(blockRange, 2*blockRange) + require.NoError(t, err) + defer q.Close() + + // The requested interval covers 2 blocks, so the querier's label values for blockID should give us 2 values, one from each block. + b, ws, err := q.LabelValues(ctx, "blockID", nil) + require.NoError(t, err) + var nilAnnotations annotations.Annotations + require.Equal(t, nilAnnotations, ws) + require.Equal(t, []string{"1", "2"}, b) +} + +// TestInitializeHeadTimestamp ensures that the h.minTime is set properly. +// - no blocks no WAL: set to the time of the first appended sample +// - no blocks with WAL: set to the smallest sample from the WAL +// - with blocks no WAL: set to the last block maxT +// - with blocks with WAL: same as above +func TestInitializeHeadTimestamp(t *testing.T) { + t.Parallel() + t.Run("clean", func(t *testing.T) { + db := newTestDB(t) + + // Should be set to init values if no WAL or blocks exist so far. + require.Equal(t, int64(math.MaxInt64), db.head.MinTime()) + require.Equal(t, int64(math.MinInt64), db.head.MaxTime()) + require.False(t, db.head.initialized()) + + // First added sample initializes the writable range. + ctx := context.Background() + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 1000, 1) + require.NoError(t, err) + + require.Equal(t, int64(1000), db.head.MinTime()) + require.Equal(t, int64(1000), db.head.MaxTime()) + require.True(t, db.head.initialized()) + }) + t.Run("wal-only", func(t *testing.T) { + dir := t.TempDir() + + require.NoError(t, os.MkdirAll(path.Join(dir, "wal"), 0o777)) + w, err := wlog.New(nil, nil, path.Join(dir, "wal"), compression.None) + require.NoError(t, err) + + var enc record.Encoder + err = w.Log( + enc.Series([]record.RefSeries{ + {Ref: 123, Labels: labels.FromStrings("a", "1")}, + {Ref: 124, Labels: labels.FromStrings("a", "2")}, + }, nil), + enc.Samples([]record.RefSample{ + {Ref: 123, T: 5000, V: 1}, + {Ref: 124, T: 15000, V: 1}, + }, nil), + ) + require.NoError(t, err) + require.NoError(t, w.Close()) + + db := newTestDB(t, withDir(dir)) + + require.Equal(t, int64(5000), db.head.MinTime()) + require.Equal(t, int64(15000), db.head.MaxTime()) + require.True(t, db.head.initialized()) + }) + t.Run("existing-block", func(t *testing.T) { + dir := t.TempDir() + + createBlock(t, dir, genSeries(1, 1, 1000, 2000)) + + db := newTestDB(t, withDir(dir)) + + require.Equal(t, int64(2000), db.head.MinTime()) + require.Equal(t, int64(2000), db.head.MaxTime()) + require.True(t, db.head.initialized()) + }) + t.Run("existing-block-and-wal", func(t *testing.T) { + dir := t.TempDir() + + createBlock(t, dir, genSeries(1, 1, 1000, 6000)) + + require.NoError(t, os.MkdirAll(path.Join(dir, "wal"), 0o777)) + w, err := wlog.New(nil, nil, path.Join(dir, "wal"), compression.None) + require.NoError(t, err) + + var enc record.Encoder + err = w.Log( + enc.Series([]record.RefSeries{ + {Ref: 123, Labels: labels.FromStrings("a", "1")}, + {Ref: 124, Labels: labels.FromStrings("a", "2")}, + }, nil), + enc.Samples([]record.RefSample{ + {Ref: 123, T: 5000, V: 1}, + {Ref: 124, T: 15000, V: 1}, + }, nil), + ) + require.NoError(t, err) + require.NoError(t, w.Close()) + + db := newTestDB(t, withDir(dir)) + + require.Equal(t, int64(6000), db.head.MinTime()) + require.Equal(t, int64(15000), db.head.MaxTime()) + require.True(t, db.head.initialized()) + // Check that old series has been GCed. + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.series)) + }) +} + +func TestNoEmptyBlocks(t *testing.T) { + t.Parallel() + db := newTestDB(t, withRngs(100)) + ctx := context.Background() + + db.DisableCompactions() + + rangeToTriggerCompaction := db.compactor.(*LeveledCompactor).ranges[0]/2*3 - 1 + defaultLabel := labels.FromStrings("foo", "bar") + defaultMatcher := labels.MustNewMatcher(labels.MatchRegexp, "", ".*") + + t.Run("Test no blocks after compact with empty head.", func(t *testing.T) { + require.NoError(t, db.Compact(ctx)) + actBlocks, err := blockDirs(db.Dir()) + require.NoError(t, err) + require.Len(t, actBlocks, len(db.Blocks())) + require.Empty(t, actBlocks) + require.Equal(t, 0, int(prom_testutil.ToFloat64(db.compactor.(*LeveledCompactor).metrics.Ran)), "no compaction should be triggered here") + }) + + t.Run("Test no blocks after deleting all samples from head.", func(t *testing.T) { + app := db.Appender(ctx) + _, err := app.Append(0, defaultLabel, 1, 0) + require.NoError(t, err) + _, err = app.Append(0, defaultLabel, 2, 0) + require.NoError(t, err) + _, err = app.Append(0, defaultLabel, 3+rangeToTriggerCompaction, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.NoError(t, db.Delete(ctx, math.MinInt64, math.MaxInt64, defaultMatcher)) + require.NoError(t, db.Compact(ctx)) + require.Equal(t, 1, int(prom_testutil.ToFloat64(db.compactor.(*LeveledCompactor).metrics.Ran)), "compaction should have been triggered here") + + actBlocks, err := blockDirs(db.Dir()) + require.NoError(t, err) + require.Len(t, actBlocks, len(db.Blocks())) + require.Empty(t, actBlocks) + + app = db.Appender(ctx) + _, err = app.Append(0, defaultLabel, 1, 0) + require.Equal(t, storage.ErrOutOfBounds, err, "the head should be truncated so no samples in the past should be allowed") + + // Adding new blocks. + currentTime := db.Head().MaxTime() + _, err = app.Append(0, defaultLabel, currentTime, 0) + require.NoError(t, err) + _, err = app.Append(0, defaultLabel, currentTime+1, 0) + require.NoError(t, err) + _, err = app.Append(0, defaultLabel, currentTime+rangeToTriggerCompaction, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + require.NoError(t, db.Compact(ctx)) + require.Equal(t, 2, int(prom_testutil.ToFloat64(db.compactor.(*LeveledCompactor).metrics.Ran)), "compaction should have been triggered here") + actBlocks, err = blockDirs(db.Dir()) + require.NoError(t, err) + require.Len(t, actBlocks, len(db.Blocks())) + require.Len(t, actBlocks, 1, "No blocks created when compacting with >0 samples") + }) + + t.Run(`When no new block is created from head, and there are some blocks on disk + compaction should not run into infinite loop (was seen during development).`, func(t *testing.T) { + oldBlocks := db.Blocks() + app := db.Appender(ctx) + currentTime := db.Head().MaxTime() + _, err := app.Append(0, defaultLabel, currentTime, 0) + require.NoError(t, err) + _, err = app.Append(0, defaultLabel, currentTime+1, 0) + require.NoError(t, err) + _, err = app.Append(0, defaultLabel, currentTime+rangeToTriggerCompaction, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.NoError(t, db.head.Delete(ctx, math.MinInt64, math.MaxInt64, defaultMatcher)) + require.NoError(t, db.Compact(ctx)) + require.Equal(t, 3, int(prom_testutil.ToFloat64(db.compactor.(*LeveledCompactor).metrics.Ran)), "compaction should have been triggered here") + require.Equal(t, oldBlocks, db.Blocks()) + }) + + t.Run("Test no blocks remaining after deleting all samples from disk.", func(t *testing.T) { + currentTime := db.Head().MaxTime() + blocks := []*BlockMeta{ + {MinTime: currentTime, MaxTime: currentTime + db.compactor.(*LeveledCompactor).ranges[0]}, + {MinTime: currentTime + 100, MaxTime: currentTime + 100 + db.compactor.(*LeveledCompactor).ranges[0]}, + } + for _, m := range blocks { + createBlock(t, db.Dir(), genSeries(2, 2, m.MinTime, m.MaxTime)) + } + + oldBlocks := db.Blocks() + require.NoError(t, db.reloadBlocks()) // Reload the db to register the new blocks. + require.Len(t, db.Blocks(), len(blocks)+len(oldBlocks)) // Ensure all blocks are registered. + require.NoError(t, db.Delete(ctx, math.MinInt64, math.MaxInt64, defaultMatcher)) + require.NoError(t, db.Compact(ctx)) + require.Equal(t, 5, int(prom_testutil.ToFloat64(db.compactor.(*LeveledCompactor).metrics.Ran)), "compaction should have been triggered here once for each block that have tombstones") + + actBlocks, err := blockDirs(db.Dir()) + require.NoError(t, err) + require.Len(t, actBlocks, len(db.Blocks())) + require.Len(t, actBlocks, 1, "All samples are deleted. Only the most recent block should remain after compaction.") + }) +} + +func TestDB_LabelNames(t *testing.T) { + ctx := context.Background() + tests := []struct { + // Add 'sampleLabels1' -> Test Head -> Compact -> Test Disk -> + // -> Add 'sampleLabels2' -> Test Head+Disk + + sampleLabels1 [][2]string // For checking head and disk separately. + // To test Head+Disk, sampleLabels2 should have + // at least 1 unique label name which is not in sampleLabels1. + sampleLabels2 [][2]string // For checking head and disk together. + exp1 []string // after adding sampleLabels1. + exp2 []string // after adding sampleLabels1 and sampleLabels2. + }{ + { + sampleLabels1: [][2]string{ + {"name1", "1"}, + {"name3", "3"}, + {"name2", "2"}, + }, + sampleLabels2: [][2]string{ + {"name4", "4"}, + {"name1", "1"}, + }, + exp1: []string{"name1", "name2", "name3"}, + exp2: []string{"name1", "name2", "name3", "name4"}, + }, + { + sampleLabels1: [][2]string{ + {"name2", "2"}, + {"name1", "1"}, + {"name2", "2"}, + }, + sampleLabels2: [][2]string{ + {"name6", "6"}, + {"name0", "0"}, + }, + exp1: []string{"name1", "name2"}, + exp2: []string{"name0", "name1", "name2", "name6"}, + }, + } + + blockRange := int64(1000) + // Appends samples into the database. + appendSamples := func(db *DB, mint, maxt int64, sampleLabels [][2]string) { + t.Helper() + app := db.Appender(ctx) + for i := mint; i <= maxt; i++ { + for _, tuple := range sampleLabels { + label := labels.FromStrings(tuple[0], tuple[1]) + _, err := app.Append(0, label, i*blockRange, 0) + require.NoError(t, err) + } + } + err := app.Commit() + require.NoError(t, err) + } + for _, tst := range tests { + t.Run("", func(t *testing.T) { + ctx := context.Background() + db := newTestDB(t) + + appendSamples(db, 0, 4, tst.sampleLabels1) + + // Testing head. + headIndexr, err := db.head.Index() + require.NoError(t, err) + labelNames, err := headIndexr.LabelNames(ctx) + require.NoError(t, err) + require.Equal(t, tst.exp1, labelNames) + require.NoError(t, headIndexr.Close()) + + // Testing disk. + err = db.Compact(ctx) + require.NoError(t, err) + // All blocks have same label names, hence check them individually. + // No need to aggregate and check. + for _, b := range db.Blocks() { + blockIndexr, err := b.Index() + require.NoError(t, err) + labelNames, err = blockIndexr.LabelNames(ctx) + require.NoError(t, err) + require.Equal(t, tst.exp1, labelNames) + require.NoError(t, blockIndexr.Close()) + } + + // Adding more samples to head with new label names + // so that we can test (head+disk).LabelNames(ctx) (the union). + appendSamples(db, 5, 9, tst.sampleLabels2) + + // Testing DB (union). + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + var ws annotations.Annotations + labelNames, ws, err = q.LabelNames(ctx, nil) + require.NoError(t, err) + require.Empty(t, ws) + require.NoError(t, q.Close()) + require.Equal(t, tst.exp2, labelNames) + }) + } +} + +func TestCorrectNumTombstones(t *testing.T) { + t.Parallel() + db := newTestDB(t) + + blockRange := db.compactor.(*LeveledCompactor).ranges[0] + name, value := "foo", "bar" + defaultLabel := labels.FromStrings(name, value) + defaultMatcher := labels.MustNewMatcher(labels.MatchEqual, name, value) + + ctx := context.Background() + app := db.Appender(ctx) + for i := range int64(3) { + for j := range int64(15) { + _, err := app.Append(0, defaultLabel, i*blockRange+j, 0) + require.NoError(t, err) + } + } + require.NoError(t, app.Commit()) + + err := db.Compact(ctx) + require.NoError(t, err) + require.Len(t, db.blocks, 1) + + require.NoError(t, db.Delete(ctx, 0, 1, defaultMatcher)) + require.Equal(t, uint64(1), db.blocks[0].meta.Stats.NumTombstones) + + // {0, 1} and {2, 3} are merged to form 1 tombstone. + require.NoError(t, db.Delete(ctx, 2, 3, defaultMatcher)) + require.Equal(t, uint64(1), db.blocks[0].meta.Stats.NumTombstones) + + require.NoError(t, db.Delete(ctx, 5, 6, defaultMatcher)) + require.Equal(t, uint64(2), db.blocks[0].meta.Stats.NumTombstones) + + require.NoError(t, db.Delete(ctx, 9, 11, defaultMatcher)) + require.Equal(t, uint64(3), db.blocks[0].meta.Stats.NumTombstones) +} + +// TestBlockRanges checks the following use cases: +// - No samples can be added with timestamps lower than the last block maxt. +// - The compactor doesn't create overlapping blocks +// +// even when the last blocks is not within the default boundaries. +// - Lower boundary is based on the smallest sample in the head and +// +// upper boundary is rounded to the configured block range. +// +// This ensures that a snapshot that includes the head and creates a block with a custom time range +// will not overlap with the first block created by the next compaction. +func TestBlockRanges(t *testing.T) { + t.Parallel() + logger := promslog.New(&promslog.Config{}) + ctx := context.Background() + + dir := t.TempDir() + + // Test that the compactor doesn't create overlapping blocks + // when a non standard block already exists. + firstBlockMaxT := int64(3) + createBlock(t, dir, genSeries(1, 1, 0, firstBlockMaxT)) + db, err := open(dir, logger, nil, DefaultOptions(), []int64{10000}, nil) + require.NoError(t, err) + + rangeToTriggerCompaction := db.compactor.(*LeveledCompactor).ranges[0]/2*3 + 1 + + app := db.Appender(ctx) + lbl := labels.FromStrings("a", "b") + _, err = app.Append(0, lbl, firstBlockMaxT-1, rand.Float64()) + require.Error(t, err, "appending a sample with a timestamp covered by a previous block shouldn't be possible") + _, err = app.Append(0, lbl, firstBlockMaxT+1, rand.Float64()) + require.NoError(t, err) + _, err = app.Append(0, lbl, firstBlockMaxT+2, rand.Float64()) + require.NoError(t, err) + secondBlockMaxt := firstBlockMaxT + rangeToTriggerCompaction + _, err = app.Append(0, lbl, secondBlockMaxt, rand.Float64()) // Add samples to trigger a new compaction + + require.NoError(t, err) + require.NoError(t, app.Commit()) + for range 100 { + if len(db.Blocks()) == 2 { + break + } + time.Sleep(100 * time.Millisecond) + } + require.Len(t, db.Blocks(), 2, "no new block created after the set timeout") + + require.LessOrEqual(t, db.Blocks()[1].Meta().MinTime, db.Blocks()[0].Meta().MaxTime, + "new block overlaps old:%v,new:%v", db.Blocks()[0].Meta(), db.Blocks()[1].Meta()) + + // Test that wal records are skipped when an existing block covers the same time ranges + // and compaction doesn't create an overlapping block. + app = db.Appender(ctx) + db.DisableCompactions() + _, err = app.Append(0, lbl, secondBlockMaxt+1, rand.Float64()) + require.NoError(t, err) + _, err = app.Append(0, lbl, secondBlockMaxt+2, rand.Float64()) + require.NoError(t, err) + _, err = app.Append(0, lbl, secondBlockMaxt+3, rand.Float64()) + require.NoError(t, err) + _, err = app.Append(0, lbl, secondBlockMaxt+4, rand.Float64()) + require.NoError(t, err) + require.NoError(t, app.Commit()) + require.NoError(t, db.Close()) + + thirdBlockMaxt := secondBlockMaxt + 2 + createBlock(t, dir, genSeries(1, 1, secondBlockMaxt+1, thirdBlockMaxt)) + + db, err = open(dir, logger, nil, DefaultOptions(), []int64{10000}, nil) + require.NoError(t, err) + + defer db.Close() + require.Len(t, db.Blocks(), 3, "db doesn't include expected number of blocks") + require.Equal(t, db.Blocks()[2].Meta().MaxTime, thirdBlockMaxt, "unexpected maxt of the last block") + + app = db.Appender(ctx) + _, err = app.Append(0, lbl, thirdBlockMaxt+rangeToTriggerCompaction, rand.Float64()) // Trigger a compaction + require.NoError(t, err) + require.NoError(t, app.Commit()) + for range 100 { + if len(db.Blocks()) == 4 { + break + } + time.Sleep(100 * time.Millisecond) + } + + require.Len(t, db.Blocks(), 4, "no new block created after the set timeout") + + require.LessOrEqual(t, db.Blocks()[3].Meta().MinTime, db.Blocks()[2].Meta().MaxTime, + "new block overlaps old:%v,new:%v", db.Blocks()[2].Meta(), db.Blocks()[3].Meta()) +} + +// TestDBReadOnly ensures that opening a DB in readonly mode doesn't modify any files on the disk. +// It also checks that the API calls return equivalent results as a normal db.Open() mode. +func TestDBReadOnly(t *testing.T) { + t.Parallel() + var ( + dbDir = t.TempDir() + expBlocks []*Block + expBlock *Block + expSeries map[string][]chunks.Sample + expChunks map[string][][]chunks.Sample + expDBHash []byte + matchAll = labels.MustNewMatcher(labels.MatchEqual, "", "") + err error + ) + + // Bootstrap the db. + { + dbBlocks := []*BlockMeta{ + // Create three 2-sample blocks. + {MinTime: 10, MaxTime: 12}, + {MinTime: 12, MaxTime: 14}, + {MinTime: 14, MaxTime: 16}, + } + + for _, m := range dbBlocks { + _ = createBlock(t, dbDir, genSeries(1, 1, m.MinTime, m.MaxTime)) + } + + // Add head to test DBReadOnly WAL reading capabilities. + w, err := wlog.New(nil, nil, filepath.Join(dbDir, "wal"), compression.Snappy) + require.NoError(t, err) + h := createHead(t, w, genSeries(1, 1, 16, 18), dbDir) + require.NoError(t, h.Close()) + } + + // Open a normal db to use for a comparison. + { + dbWritable := newTestDB(t, withDir(dbDir)) + dbWritable.DisableCompactions() + + dbSizeBeforeAppend, err := fileutil.DirSize(dbWritable.Dir()) + require.NoError(t, err) + app := dbWritable.Appender(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), dbWritable.Head().MaxTime()+1, 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + expBlocks = dbWritable.Blocks() + expBlock = expBlocks[0] + expDbSize, err := fileutil.DirSize(dbWritable.Dir()) + require.NoError(t, err) + require.Greater(t, expDbSize, dbSizeBeforeAppend, "db size didn't increase after an append") + + q, err := dbWritable.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + expSeries = query(t, q, matchAll) + cq, err := dbWritable.ChunkQuerier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + expChunks = queryAndExpandChunks(t, cq, matchAll) + + require.NoError(t, dbWritable.Close()) // Close here to allow getting the dir hash for windows. + expDBHash = testutil.DirHash(t, dbWritable.Dir()) + } + + // Open a read only db and ensure that the API returns the same result as the normal DB. + dbReadOnly, err := OpenDBReadOnly(dbDir, "", nil) + require.NoError(t, err) + defer func() { require.NoError(t, dbReadOnly.Close()) }() + + t.Run("blocks", func(t *testing.T) { + blocks, err := dbReadOnly.Blocks() + require.NoError(t, err) + require.Len(t, blocks, len(expBlocks)) + for i, expBlock := range expBlocks { + require.Equal(t, expBlock.Meta(), blocks[i].Meta(), "block meta mismatch") + } + }) + t.Run("block", func(t *testing.T) { + blockID := expBlock.meta.ULID.String() + block, err := dbReadOnly.Block(blockID, nil) + require.NoError(t, err) + require.Equal(t, expBlock.Meta(), block.Meta(), "block meta mismatch") + }) + t.Run("invalid block ID", func(t *testing.T) { + blockID := "01GTDVZZF52NSWB5SXQF0P2PGF" + _, err := dbReadOnly.Block(blockID, nil) + require.Error(t, err) + }) + t.Run("last block ID", func(t *testing.T) { + blockID, err := dbReadOnly.LastBlockID() + require.NoError(t, err) + require.Equal(t, expBlocks[2].Meta().ULID.String(), blockID) + }) + t.Run("querier", func(t *testing.T) { + // Open a read only db and ensure that the API returns the same result as the normal DB. + q, err := dbReadOnly.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + readOnlySeries := query(t, q, matchAll) + readOnlyDBHash := testutil.DirHash(t, dbDir) + + require.Len(t, readOnlySeries, len(expSeries), "total series mismatch") + require.Equal(t, expSeries, readOnlySeries, "series mismatch") + require.Equal(t, expDBHash, readOnlyDBHash, "after all read operations the db hash should remain the same") + }) + t.Run("chunk querier", func(t *testing.T) { + cq, err := dbReadOnly.ChunkQuerier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + readOnlySeries := queryAndExpandChunks(t, cq, matchAll) + readOnlyDBHash := testutil.DirHash(t, dbDir) + + require.Len(t, readOnlySeries, len(expChunks), "total series mismatch") + require.Equal(t, expChunks, readOnlySeries, "series chunks mismatch") + require.Equal(t, expDBHash, readOnlyDBHash, "after all read operations the db hash should remain the same") + }) +} + +// TestDBReadOnlyClosing ensures that after closing the db +// all api methods return an ErrClosed. +func TestDBReadOnlyClosing(t *testing.T) { + t.Parallel() + sandboxDir := t.TempDir() + db, err := OpenDBReadOnly(t.TempDir(), sandboxDir, promslog.New(&promslog.Config{})) + require.NoError(t, err) + // The sandboxDir was there. + require.DirExists(t, db.sandboxDir) + require.NoError(t, db.Close()) + // The sandboxDir was deleted when closing. + require.NoDirExists(t, db.sandboxDir) + require.Equal(t, db.Close(), ErrClosed) + _, err = db.Blocks() + require.Equal(t, err, ErrClosed) + _, err = db.Querier(0, 1) + require.Equal(t, err, ErrClosed) +} + +func TestDBReadOnly_FlushWAL(t *testing.T) { + t.Parallel() + var ( + dbDir = t.TempDir() + err error + maxt int + ctx = context.Background() + ) + + // Bootstrap the db. + { + // Append data to the WAL. + db := newTestDB(t, withDir(dbDir)) + db.DisableCompactions() + app := db.Appender(ctx) + maxt = 1000 + for i := 0; i < maxt; i++ { + _, err := app.Append(0, labels.FromStrings(defaultLabelName, "flush"), int64(i), 1.0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + require.NoError(t, db.Close()) + } + + // Flush WAL. + db, err := OpenDBReadOnly(dbDir, "", nil) + require.NoError(t, err) + + flush := t.TempDir() + require.NoError(t, db.FlushWAL(flush)) + require.NoError(t, db.Close()) + + // Reopen the DB from the flushed WAL block. + db, err = OpenDBReadOnly(flush, "", nil) + require.NoError(t, err) + defer func() { require.NoError(t, db.Close()) }() + blocks, err := db.Blocks() + require.NoError(t, err) + require.Len(t, blocks, 1) + + querier, err := db.Querier(0, int64(maxt)-1) + require.NoError(t, err) + defer func() { require.NoError(t, querier.Close()) }() + + // Sum the values. + seriesSet := querier.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, defaultLabelName, "flush")) + var series chunkenc.Iterator + + sum := 0.0 + for seriesSet.Next() { + series = seriesSet.At().Iterator(series) + for series.Next() == chunkenc.ValFloat { + _, v := series.At() + sum += v + } + require.NoError(t, series.Err()) + } + require.NoError(t, seriesSet.Err()) + require.Empty(t, seriesSet.Warnings()) + require.Equal(t, 1000.0, sum) +} + +func TestDBReadOnly_Querier_NoAlteration(t *testing.T) { + countChunks := func(dir string) int { + files, err := os.ReadDir(mmappedChunksDir(dir)) + require.NoError(t, err) + return len(files) + } + + dirHash := func(dir string) (hash []byte) { + // Windows requires the DB to be closed: "xxx\lock: The process cannot access the file because it is being used by another process." + // But closing the DB alters the directory in this case (it'll cut a new chunk). + if runtime.GOOS != "windows" { + hash = testutil.DirHash(t, dir) + } + return hash + } + + spinUpQuerierAndCheck := func(dir, sandboxDir string, chunksCount int) { + dBDirHash := dirHash(dir) + // Bootstrap a RO db from the same dir and set up a querier. + dbReadOnly, err := OpenDBReadOnly(dir, sandboxDir, nil) + require.NoError(t, err) + require.Equal(t, chunksCount, countChunks(dir)) + q, err := dbReadOnly.Querier(math.MinInt, math.MaxInt) + require.NoError(t, err) + require.NoError(t, q.Close()) + require.NoError(t, dbReadOnly.Close()) + // The RO Head doesn't alter RW db chunks_head/. + require.Equal(t, chunksCount, countChunks(dir)) + require.Equal(t, dirHash(dir), dBDirHash) + } + + t.Run("doesn't cut chunks while replaying WAL", func(t *testing.T) { + db := newTestDB(t) + + // Append until the first mmapped head chunk. + for i := range 121 { + app := db.Appender(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), int64(i), 0) + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + + spinUpQuerierAndCheck(db.Dir(), t.TempDir(), 0) + + // The RW Head should have no problem cutting its own chunk, + // this also proves that a chunk needed to be cut. + require.NotPanics(t, func() { db.ForceHeadMMap() }) + require.Equal(t, 1, countChunks(db.Dir())) + }) + + t.Run("doesn't truncate corrupted chunks", func(t *testing.T) { + db := newTestDB(t) + require.NoError(t, db.Close()) + + // Simulate a corrupted chunk: without a header. + chunk, err := os.Create(path.Join(mmappedChunksDir(db.Dir()), "000001")) + require.NoError(t, err) + require.NoError(t, chunk.Close()) + + spinUpQuerierAndCheck(db.Dir(), t.TempDir(), 1) + + // The RW Head should have no problem truncating its corrupted file: + // this proves that the chunk needed to be truncated. + db = newTestDB(t, withDir(db.Dir())) + + require.NoError(t, err) + require.Equal(t, 0, countChunks(db.Dir())) + }) +} + +func TestDBCannotSeePartialCommits(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + db := newTestDB(t) + + stop := make(chan struct{}) + firstInsert := make(chan struct{}) + ctx := context.Background() + + // Insert data in batches. + go func() { + iter := 0 + for { + app := db.Appender(ctx) + + for j := range 100 { + _, err := app.Append(0, labels.FromStrings("foo", "bar", "a", strconv.Itoa(j)), int64(iter), float64(iter)) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + if iter == 0 { + close(firstInsert) + } + iter++ + + select { + case <-stop: + return + default: + } + } + }() + + <-firstInsert + + // This is a race condition, so do a few tests to tickle it. + // Usually most will fail. + inconsistencies := 0 + for range 10 { + func() { + querier, err := db.Querier(0, 1000000) + require.NoError(t, err) + defer querier.Close() + + ss := querier.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err := expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + + values := map[float64]struct{}{} + for _, series := range seriesSet { + values[series[len(series)-1].f] = struct{}{} + } + if len(values) != 1 { + inconsistencies++ + } + }() + } + stop <- struct{}{} + + require.Equal(t, 0, inconsistencies, "Some queries saw inconsistent results.") +} + +func TestDBQueryDoesntSeeAppendsAfterCreation(t *testing.T) { + if defaultIsolationDisabled { + t.Skip("skipping test since tsdb isolation is disabled") + } + + db := newTestDB(t) + querierBeforeAdd, err := db.Querier(0, 1000000) + require.NoError(t, err) + defer querierBeforeAdd.Close() + + ctx := context.Background() + app := db.Appender(ctx) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + require.NoError(t, err) + + querierAfterAddButBeforeCommit, err := db.Querier(0, 1000000) + require.NoError(t, err) + defer querierAfterAddButBeforeCommit.Close() + + // None of the queriers should return anything after the Add but before the commit. + ss := querierBeforeAdd.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err := expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, map[string][]sample{}, seriesSet) + + ss = querierAfterAddButBeforeCommit.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err = expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, map[string][]sample{}, seriesSet) + + // This commit is after the queriers are created, so should not be returned. + err = app.Commit() + require.NoError(t, err) + + // Nothing returned for querier created before the Add. + ss = querierBeforeAdd.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err = expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, map[string][]sample{}, seriesSet) + + // Series exists but has no samples for querier created after Add. + ss = querierAfterAddButBeforeCommit.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err = expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, map[string][]sample{`{foo="bar"}`: {}}, seriesSet) + + querierAfterCommit, err := db.Querier(0, 1000000) + require.NoError(t, err) + defer querierAfterCommit.Close() + + // Samples are returned for querier created after Commit. + ss = querierAfterCommit.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + _, seriesSet, ws, err = expandSeriesSet(ss) + require.NoError(t, err) + require.Empty(t, ws) + require.Equal(t, map[string][]sample{`{foo="bar"}`: {{t: 0, f: 0}}}, seriesSet) +} + +func assureChunkFromSamples(t *testing.T, samples []chunks.Sample) chunks.Meta { + chks, err := chunks.ChunkFromSamples(samples) + require.NoError(t, err) + return chks +} + +// TestChunkWriter_ReadAfterWrite ensures that chunk segment are cut at the set segment size and +// that the resulted segments includes the expected chunks data. +func TestChunkWriter_ReadAfterWrite(t *testing.T) { + chk1 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 1, nil, nil}}) + chk2 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 2, nil, nil}}) + chk3 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 3, nil, nil}}) + chk4 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 4, nil, nil}}) + chk5 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 5, nil, nil}}) + chunkSize := len(chk1.Chunk.Bytes()) + chunks.MaxChunkLengthFieldSize + chunks.ChunkEncodingSize + crc32.Size + + tests := []struct { + chks [][]chunks.Meta + segmentSize, + expSegmentsCount int + expSegmentSizes []int + }{ + // 0:Last chunk ends at the segment boundary so + // all chunks should fit in a single segment. + { + chks: [][]chunks.Meta{ + { + chk1, + chk2, + chk3, + }, + }, + segmentSize: 3 * chunkSize, + expSegmentSizes: []int{3 * chunkSize}, + expSegmentsCount: 1, + }, + // 1:Two chunks can fit in a single segment so the last one should result in a new segment. + { + chks: [][]chunks.Meta{ + { + chk1, + chk2, + chk3, + chk4, + chk5, + }, + }, + segmentSize: 2 * chunkSize, + expSegmentSizes: []int{2 * chunkSize, 2 * chunkSize, chunkSize}, + expSegmentsCount: 3, + }, + // 2:When the segment size is smaller than the size of 2 chunks + // the last segment should still create a new segment. + { + chks: [][]chunks.Meta{ + { + chk1, + chk2, + chk3, + }, + }, + segmentSize: 2*chunkSize - 1, + expSegmentSizes: []int{chunkSize, chunkSize, chunkSize}, + expSegmentsCount: 3, + }, + // 3:When the segment is smaller than a single chunk + // it should still be written by ignoring the max segment size. + { + chks: [][]chunks.Meta{ + { + chk1, + }, + }, + segmentSize: chunkSize - 1, + expSegmentSizes: []int{chunkSize}, + expSegmentsCount: 1, + }, + // 4:All chunks are bigger than the max segment size, but + // these should still be written even when this will result in bigger segment than the set size. + // Each segment will hold a single chunk. + { + chks: [][]chunks.Meta{ + { + chk1, + chk2, + chk3, + }, + }, + segmentSize: 1, + expSegmentSizes: []int{chunkSize, chunkSize, chunkSize}, + expSegmentsCount: 3, + }, + // 5:Adding multiple batches of chunks. + { + chks: [][]chunks.Meta{ + { + chk1, + chk2, + chk3, + }, + { + chk4, + chk5, + }, + }, + segmentSize: 3 * chunkSize, + expSegmentSizes: []int{3 * chunkSize, 2 * chunkSize}, + expSegmentsCount: 2, + }, + // 6:Adding multiple batches of chunks. + { + chks: [][]chunks.Meta{ + { + chk1, + }, + { + chk2, + chk3, + }, + { + chk4, + }, + }, + segmentSize: 2 * chunkSize, + expSegmentSizes: []int{2 * chunkSize, 2 * chunkSize}, + expSegmentsCount: 2, + }, + } + + for i, test := range tests { + t.Run(strconv.Itoa(i), func(t *testing.T) { + tempDir := t.TempDir() + + chunkw, err := chunks.NewWriter(tempDir, chunks.WithSegmentSize(chunks.SegmentHeaderSize+int64(test.segmentSize))) + require.NoError(t, err) + + for _, chks := range test.chks { + require.NoError(t, chunkw.WriteChunks(chks...)) + } + require.NoError(t, chunkw.Close()) + + files, err := os.ReadDir(tempDir) + require.NoError(t, err) + require.Len(t, files, test.expSegmentsCount, "expected segments count mismatch") + + // Verify that all data is written to the segments. + sizeExp := 0 + sizeAct := 0 + + for _, chks := range test.chks { + for _, chk := range chks { + l := make([]byte, binary.MaxVarintLen32) + sizeExp += binary.PutUvarint(l, uint64(len(chk.Chunk.Bytes()))) // The length field. + sizeExp += chunks.ChunkEncodingSize + sizeExp += len(chk.Chunk.Bytes()) // The data itself. + sizeExp += crc32.Size // The 4 bytes of crc32 + } + } + sizeExp += test.expSegmentsCount * chunks.SegmentHeaderSize // The segment header bytes. + + for i, f := range files { + fi, err := f.Info() + require.NoError(t, err) + size := int(fi.Size()) + // Verify that the segment is the same or smaller than the expected size. + require.GreaterOrEqual(t, chunks.SegmentHeaderSize+test.expSegmentSizes[i], size, "Segment:%v should NOT be bigger than:%v actual:%v", i, chunks.SegmentHeaderSize+test.expSegmentSizes[i], size) + + sizeAct += size + } + require.Equal(t, sizeExp, sizeAct) + + // Check the content of the chunks. + r, err := chunks.NewDirReader(tempDir, nil) + require.NoError(t, err) + defer func() { require.NoError(t, r.Close()) }() + + for _, chks := range test.chks { + for _, chkExp := range chks { + chkAct, iterable, err := r.ChunkOrIterable(chkExp) + require.NoError(t, err) + require.Nil(t, iterable) + require.Equal(t, chkExp.Chunk.Bytes(), chkAct.Bytes()) + } + } + }) + } +} + +func TestRangeForTimestamp(t *testing.T) { + type args struct { + t int64 + width int64 + } + tests := []struct { + args args + expected int64 + }{ + {args{0, 5}, 5}, + {args{1, 5}, 5}, + {args{5, 5}, 10}, + {args{6, 5}, 10}, + {args{13, 5}, 15}, + {args{95, 5}, 100}, + } + for _, tt := range tests { + got := rangeForTimestamp(tt.args.t, tt.args.width) + require.Equal(t, tt.expected, got) + } +} + +// TestChunkReader_ConcurrentReads checks that the chunk result can be read concurrently. +// Regression test for https://github.com/prometheus/prometheus/pull/6514. +func TestChunkReader_ConcurrentReads(t *testing.T) { + t.Parallel() + chks := []chunks.Meta{ + assureChunkFromSamples(t, []chunks.Sample{sample{1, 1, nil, nil}}), + assureChunkFromSamples(t, []chunks.Sample{sample{1, 2, nil, nil}}), + assureChunkFromSamples(t, []chunks.Sample{sample{1, 3, nil, nil}}), + assureChunkFromSamples(t, []chunks.Sample{sample{1, 4, nil, nil}}), + assureChunkFromSamples(t, []chunks.Sample{sample{1, 5, nil, nil}}), + } + + tempDir := t.TempDir() + + chunkw, err := chunks.NewWriter(tempDir) + require.NoError(t, err) + + require.NoError(t, chunkw.WriteChunks(chks...)) + require.NoError(t, chunkw.Close()) + + r, err := chunks.NewDirReader(tempDir, nil) + require.NoError(t, err) + + var wg sync.WaitGroup + for _, chk := range chks { + for range 100 { + wg.Add(1) + go func(chunk chunks.Meta) { + defer wg.Done() + + chkAct, iterable, err := r.ChunkOrIterable(chunk) + require.NoError(t, err) + require.Nil(t, iterable) + require.Equal(t, chunk.Chunk.Bytes(), chkAct.Bytes()) + }(chk) + } + wg.Wait() + } + require.NoError(t, r.Close()) +} + +// TestCompactHead ensures that the head compaction +// creates a block that is ready for loading and +// does not cause data loss. +// This test: +// * opens a storage; +// * appends values; +// * compacts the head; and +// * queries the db to ensure the samples are present from the compacted head. +func TestCompactHead(t *testing.T) { + t.Parallel() + + // Open a DB and append data to the WAL. + opts := &Options{ + RetentionDuration: int64(time.Hour * 24 * 15 / time.Millisecond), + NoLockfile: true, + MinBlockDuration: int64(time.Hour * 2 / time.Millisecond), + MaxBlockDuration: int64(time.Hour * 2 / time.Millisecond), + WALCompression: compression.Snappy, + } + db := newTestDB(t, withOpts(opts)) + ctx := context.Background() + app := db.Appender(ctx) + var expSamples []sample + maxt := 100 + for i := range maxt { + val := rand.Float64() + _, err := app.Append(0, labels.FromStrings("a", "b"), int64(i), val) + require.NoError(t, err) + expSamples = append(expSamples, sample{int64(i), val, nil, nil}) + } + require.NoError(t, app.Commit()) + + // Compact the Head to create a new block. + require.NoError(t, db.CompactHead(NewRangeHead(db.Head(), 0, int64(maxt)-1))) + require.NoError(t, db.Close()) + + // Delete everything but the new block and + // reopen the db to query it to ensure it includes the head data. + require.NoError(t, deleteNonBlocks(db.Dir())) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.Len(t, db.Blocks(), 1) + require.Equal(t, int64(maxt), db.Head().MinTime()) + defer func() { require.NoError(t, db.Close()) }() + querier, err := db.Querier(0, int64(maxt)-1) + require.NoError(t, err) + defer func() { require.NoError(t, querier.Close()) }() + + seriesSet := querier.Select(ctx, false, nil, &labels.Matcher{Type: labels.MatchEqual, Name: "a", Value: "b"}) + var series chunkenc.Iterator + var actSamples []sample + + for seriesSet.Next() { + series = seriesSet.At().Iterator(series) + for series.Next() == chunkenc.ValFloat { + time, val := series.At() + actSamples = append(actSamples, sample{time, val, nil, nil}) + } + require.NoError(t, series.Err()) + } + require.Equal(t, expSamples, actSamples) + require.NoError(t, seriesSet.Err()) +} + +// TestCompactHeadWithDeletion tests https://github.com/prometheus/prometheus/issues/11585. +func TestCompactHeadWithDeletion(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + + app := db.Appender(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 10, rand.Float64()) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + err = db.Delete(ctx, 0, 100, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) + require.NoError(t, err) + + // This recreates the bug. + require.NoError(t, db.CompactHead(NewRangeHead(db.Head(), 0, 100))) +} + +func deleteNonBlocks(dbDir string) error { + dirs, err := os.ReadDir(dbDir) + if err != nil { + return err + } + for _, dir := range dirs { + if ok := isBlockDir(dir); !ok { + if err := os.RemoveAll(filepath.Join(dbDir, dir.Name())); err != nil { + return err + } + } + } + dirs, err = os.ReadDir(dbDir) + if err != nil { + return err + } + for _, dir := range dirs { + if ok := isBlockDir(dir); !ok { + return fmt.Errorf("root folder:%v still hase non block directory:%v", dbDir, dir.Name()) + } + } + return nil +} + +func TestOpen_VariousBlockStates(t *testing.T) { + tmpDir := t.TempDir() + + var ( + expectedLoadedDirs = map[string]struct{}{} + expectedRemovedDirs = map[string]struct{}{} + expectedIgnoredDirs = map[string]struct{}{} + ) + + { + // Ok blocks; should be loaded. + expectedLoadedDirs[createBlock(t, tmpDir, genSeries(10, 2, 0, 10))] = struct{}{} + expectedLoadedDirs[createBlock(t, tmpDir, genSeries(10, 2, 10, 20))] = struct{}{} + } + { + // Block to repair; should be repaired & loaded. + dbDir := filepath.Join("testdata", "repair_index_version", "01BZJ9WJQPWHGNC2W4J9TA62KC") + outDir := filepath.Join(tmpDir, "01BZJ9WJQPWHGNC2W4J9TA62KC") + expectedLoadedDirs[outDir] = struct{}{} + + // Touch chunks dir in block. + require.NoError(t, os.MkdirAll(filepath.Join(dbDir, "chunks"), 0o777)) + defer func() { + require.NoError(t, os.RemoveAll(filepath.Join(dbDir, "chunks"))) + }() + require.NoError(t, os.Mkdir(outDir, os.ModePerm)) + require.NoError(t, fileutil.CopyDirs(dbDir, outDir)) + } + { + // Missing meta.json; should be ignored and only logged. + // TODO(bwplotka): Probably add metric. + dir := createBlock(t, tmpDir, genSeries(10, 2, 20, 30)) + expectedIgnoredDirs[dir] = struct{}{} + require.NoError(t, os.Remove(filepath.Join(dir, metaFilename))) + } + { + // Tmp blocks during creation; those should be removed on start. + dir := createBlock(t, tmpDir, genSeries(10, 2, 30, 40)) + require.NoError(t, fileutil.Replace(dir, dir+tmpForCreationBlockDirSuffix)) + expectedRemovedDirs[dir+tmpForCreationBlockDirSuffix] = struct{}{} + + // Tmp blocks during deletion; those should be removed on start. + dir = createBlock(t, tmpDir, genSeries(10, 2, 40, 50)) + require.NoError(t, fileutil.Replace(dir, dir+tmpForDeletionBlockDirSuffix)) + expectedRemovedDirs[dir+tmpForDeletionBlockDirSuffix] = struct{}{} + + // Pre-2.21 tmp blocks; those should be removed on start. + dir = createBlock(t, tmpDir, genSeries(10, 2, 50, 60)) + require.NoError(t, fileutil.Replace(dir, dir+tmpLegacy)) + expectedRemovedDirs[dir+tmpLegacy] = struct{}{} + } + { + // One ok block; but two should be replaced. + dir := createBlock(t, tmpDir, genSeries(10, 2, 50, 60)) + expectedLoadedDirs[dir] = struct{}{} + + m, _, err := readMetaFile(dir) + require.NoError(t, err) + + compacted := createBlock(t, tmpDir, genSeries(10, 2, 50, 55)) + expectedRemovedDirs[compacted] = struct{}{} + + m.Compaction.Parents = append(m.Compaction.Parents, + BlockDesc{ULID: ulid.MustParse(filepath.Base(compacted))}, + BlockDesc{ULID: ulid.MustNew(1, nil)}, + BlockDesc{ULID: ulid.MustNew(123, nil)}, + ) + + // Regression test: Already removed parent can be still in list, which was causing Open errors. + m.Compaction.Parents = append(m.Compaction.Parents, BlockDesc{ULID: ulid.MustParse(filepath.Base(compacted))}) + m.Compaction.Parents = append(m.Compaction.Parents, BlockDesc{ULID: ulid.MustParse(filepath.Base(compacted))}) + _, err = writeMetaFile(promslog.New(&promslog.Config{}), dir, m) + require.NoError(t, err) + } + tmpCheckpointDir := path.Join(tmpDir, "wal/checkpoint.00000001.tmp") + err := os.MkdirAll(tmpCheckpointDir, 0o777) + require.NoError(t, err) + tmpChunkSnapshotDir := path.Join(tmpDir, chunkSnapshotPrefix+"0000.00000001.tmp") + err = os.MkdirAll(tmpChunkSnapshotDir, 0o777) + require.NoError(t, err) + + opts := DefaultOptions() + opts.RetentionDuration = 0 + db := newTestDB(t, withDir(tmpDir), withOpts(opts)) + loadedBlocks := db.Blocks() + + var loaded int + for _, l := range loadedBlocks { + _, ok := expectedLoadedDirs[filepath.Join(tmpDir, l.meta.ULID.String())] + require.True(t, ok, "unexpected block", l.meta.ULID, "was loaded") + loaded++ + } + require.Len(t, expectedLoadedDirs, loaded) + require.NoError(t, db.Close()) + + files, err := os.ReadDir(tmpDir) + require.NoError(t, err) + + var ignored int + for _, f := range files { + _, ok := expectedRemovedDirs[filepath.Join(tmpDir, f.Name())] + require.False(t, ok, "expected", filepath.Join(tmpDir, f.Name()), "to be removed, but still exists") + if _, ok := expectedIgnoredDirs[filepath.Join(tmpDir, f.Name())]; ok { + ignored++ + } + } + require.Len(t, expectedIgnoredDirs, ignored) + _, err = os.Stat(tmpCheckpointDir) + require.True(t, os.IsNotExist(err)) + _, err = os.Stat(tmpChunkSnapshotDir) + require.True(t, os.IsNotExist(err)) +} + +func TestOneCheckpointPerCompactCall(t *testing.T) { + t.Parallel() + blockRange := int64(1000) + opts := &Options{ + RetentionDuration: blockRange * 1000, + NoLockfile: true, + MinBlockDuration: blockRange, + MaxBlockDuration: blockRange, + } + + ctx := context.Background() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + // Case 1: Lot's of uncompacted data in Head. + + lbls := labels.FromStrings("foo_d", "choco_bar") + // Append samples spanning 59 block ranges. + app := db.Appender(context.Background()) + for i := range int64(60) { + _, err := app.Append(0, lbls, blockRange*i, rand.Float64()) + require.NoError(t, err) + _, err = app.Append(0, lbls, (blockRange*i)+blockRange/2, rand.Float64()) + require.NoError(t, err) + // Rotate the WAL file so that there is >3 files for checkpoint to happen. + _, err = db.head.wal.NextSegment() + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Check the existing WAL files. + first, last, err := wlog.Segments(db.head.wal.Dir()) + require.NoError(t, err) + require.Equal(t, 0, first) + require.Equal(t, 60, last) + + require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.checkpointCreationTotal)) + require.NoError(t, db.Compact(ctx)) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.checkpointCreationTotal)) + + // As the data spans for 59 blocks, 58 go to disk and 1 remains in Head. + require.Len(t, db.Blocks(), 58) + // Though WAL was truncated only once, head should be truncated after each compaction. + require.Equal(t, 58.0, prom_testutil.ToFloat64(db.head.metrics.headTruncateTotal)) + + // The compaction should have only truncated first 2/3 of WAL (while also rotating the files). + first, last, err = wlog.Segments(db.head.wal.Dir()) + require.NoError(t, err) + require.Equal(t, 40, first) + require.Equal(t, 61, last) + + // The first checkpoint would be for first 2/3rd of WAL, hence till 39. + // That should be the last checkpoint. + _, cno, err := wlog.LastCheckpoint(db.head.wal.Dir()) + require.NoError(t, err) + require.Equal(t, 39, cno) + + // Case 2: Old blocks on disk. + // The above blocks will act as old blocks. + + // Creating a block to cover the data in the Head so that + // Head will skip the data during replay and start fresh. + blocks := db.Blocks() + newBlockMint := blocks[len(blocks)-1].Meta().MaxTime + newBlockMaxt := db.Head().MaxTime() + 1 + require.NoError(t, db.Close()) + + createBlock(t, db.Dir(), genSeries(1, 1, newBlockMint, newBlockMaxt)) + + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + db.DisableCompactions() + + // 1 block more. + require.Len(t, db.Blocks(), 59) + // No series in Head because of this new block. + require.Equal(t, 0, int(db.head.NumSeries())) + + // Adding sample way into the future. + app = db.Appender(context.Background()) + _, err = app.Append(0, lbls, blockRange*120, rand.Float64()) + require.NoError(t, err) + require.NoError(t, app.Commit()) + + // The mint of head is the last block maxt, that means the gap between mint and maxt + // of Head is too large. This will trigger many compactions. + require.Equal(t, newBlockMaxt, db.head.MinTime()) + + // Another WAL file was rotated. + first, last, err = wlog.Segments(db.head.wal.Dir()) + require.NoError(t, err) + require.Equal(t, 40, first) + require.Equal(t, 62, last) + + require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.checkpointCreationTotal)) + require.NoError(t, db.Compact(ctx)) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.checkpointCreationTotal)) + + // No new blocks should be created as there was not data in between the new samples and the blocks. + require.Len(t, db.Blocks(), 59) + + // The compaction should have only truncated first 2/3 of WAL (while also rotating the files). + first, last, err = wlog.Segments(db.head.wal.Dir()) + require.NoError(t, err) + require.Equal(t, 55, first) + require.Equal(t, 63, last) + + // The first checkpoint would be for first 2/3rd of WAL, hence till 54. + // That should be the last checkpoint. + _, cno, err = wlog.LastCheckpoint(db.head.wal.Dir()) + require.NoError(t, err) + require.Equal(t, 54, cno) +} + +func TestNoPanicOnTSDBOpenError(t *testing.T) { + tmpdir := t.TempDir() + + // Taking the lock will cause a TSDB startup error. + l, err := tsdbutil.NewDirLocker(tmpdir, "tsdb", promslog.NewNopLogger(), nil) + require.NoError(t, err) + require.NoError(t, l.Lock()) + + _, err = Open(tmpdir, nil, nil, DefaultOptions(), nil) + require.Error(t, err) + + require.NoError(t, l.Release()) +} + +func TestLockfile(t *testing.T) { + tsdbutil.TestDirLockerUsage(t, func(t *testing.T, data string, createLock bool) (*tsdbutil.DirLocker, testutil.Closer) { + opts := DefaultOptions() + opts.NoLockfile = !createLock + + // Create the DB. This should create lockfile and its metrics. + db, err := Open(data, nil, nil, opts, nil) + require.NoError(t, err) + + return db.locker, testutil.NewCallbackCloser(func() { + require.NoError(t, db.Close()) + }) + }) +} + +func TestQuerier_ShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { + t.Skip("TODO: investigate why process crash in CI") + + const numRuns = 5 + + for i := 1; i <= numRuns; i++ { + t.Run(strconv.Itoa(i), func(t *testing.T) { + testQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t) + }) + } +} + +func testQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { + const ( + numSeries = 1000 + numStressIterations = 10000 + minStressAllocationBytes = 128 * 1024 + maxStressAllocationBytes = 512 * 1024 + ) + + db := newTestDB(t) + + // Disable compactions so we can control it. + db.DisableCompactions() + + // Generate the metrics we're going to append. + metrics := make([]labels.Labels, 0, numSeries) + for i := range numSeries { + metrics = append(metrics, labels.FromStrings(labels.MetricName, fmt.Sprintf("test_%d", i))) + } + + // Push 1 sample every 15s for 2x the block duration period. + ctx := context.Background() + interval := int64(15 * time.Second / time.Millisecond) + ts := int64(0) + + for ; ts < 2*DefaultBlockDuration; ts += interval { + app := db.Appender(ctx) + + for _, metric := range metrics { + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + } + + require.NoError(t, app.Commit()) + } + + // Compact the TSDB head for the first time. We expect the head chunks file has been cut. + require.NoError(t, db.Compact(ctx)) + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) + + // Push more samples for another 1x block duration period. + for ; ts < 3*DefaultBlockDuration; ts += interval { + app := db.Appender(ctx) + + for _, metric := range metrics { + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + } + + require.NoError(t, app.Commit()) + } + + // At this point we expect 2 mmap-ed head chunks. + + // Get a querier and make sure it's closed only once the test is over. + querier, err := db.Querier(0, math.MaxInt64) + require.NoError(t, err) + defer func() { + require.NoError(t, querier.Close()) + }() + + // Query back all series. + hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} + seriesSet := querier.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchRegexp, labels.MetricName, ".+")) + + // Fetch samples iterators from all series. + var iterators []chunkenc.Iterator + actualSeries := 0 + for seriesSet.Next() { + actualSeries++ + + // Get the iterator and call Next() so that we're sure the chunk is loaded. + it := seriesSet.At().Iterator(nil) + it.Next() + it.At() + + iterators = append(iterators, it) + } + require.NoError(t, seriesSet.Err()) + require.Equal(t, numSeries, actualSeries) + + // Compact the TSDB head again. + require.NoError(t, db.Compact(ctx)) + require.Equal(t, float64(2), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) + + // At this point we expect 1 head chunk has been deleted. + + // Stress the memory and call GC. This is required to increase the chances + // the chunk memory area is released to the kernel. + var buf []byte + for i := range numStressIterations { + //nolint:staticcheck + buf = append(buf, make([]byte, minStressAllocationBytes+rand.Int31n(maxStressAllocationBytes-minStressAllocationBytes))...) + if i%1000 == 0 { + buf = nil + } + } + + // Iterate samples. Here we're summing it just to make sure no golang compiler + // optimization triggers in case we discard the result of it.At(). + var sum float64 + var firstErr error + for _, it := range iterators { + for it.Next() == chunkenc.ValFloat { + _, v := it.At() + sum += v + } + + if err := it.Err(); err != nil { + firstErr = err + } + } + + // After having iterated all samples we also want to be sure no error occurred or + // the "cannot populate chunk XXX: not found" error occurred. This error can occur + // when the iterator tries to fetch an head chunk which has been offloaded because + // of the head compaction in the meanwhile. + if firstErr != nil { + require.ErrorContains(t, firstErr, "cannot populate chunk") + } +} + +func TestChunkQuerier_ShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { + t.Skip("TODO: investigate why process crash in CI") + + const numRuns = 5 + + for i := 1; i <= numRuns; i++ { + t.Run(strconv.Itoa(i), func(t *testing.T) { + testChunkQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t) + }) + } +} + +func testChunkQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { + const ( + numSeries = 1000 + numStressIterations = 10000 + minStressAllocationBytes = 128 * 1024 + maxStressAllocationBytes = 512 * 1024 + ) + + db := newTestDB(t) + + // Disable compactions so we can control it. + db.DisableCompactions() + + // Generate the metrics we're going to append. + metrics := make([]labels.Labels, 0, numSeries) + for i := range numSeries { + metrics = append(metrics, labels.FromStrings(labels.MetricName, fmt.Sprintf("test_%d", i))) + } + + // Push 1 sample every 15s for 2x the block duration period. + ctx := context.Background() + interval := int64(15 * time.Second / time.Millisecond) + ts := int64(0) + + for ; ts < 2*DefaultBlockDuration; ts += interval { + app := db.Appender(ctx) + + for _, metric := range metrics { + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + } + + require.NoError(t, app.Commit()) + } + + // Compact the TSDB head for the first time. We expect the head chunks file has been cut. + require.NoError(t, db.Compact(ctx)) + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) + + // Push more samples for another 1x block duration period. + for ; ts < 3*DefaultBlockDuration; ts += interval { + app := db.Appender(ctx) + + for _, metric := range metrics { + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + } + + require.NoError(t, app.Commit()) + } + + // At this point we expect 2 mmap-ed head chunks. + + // Get a querier and make sure it's closed only once the test is over. + querier, err := db.ChunkQuerier(0, math.MaxInt64) + require.NoError(t, err) + defer func() { + require.NoError(t, querier.Close()) + }() + + // Query back all series. + hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} + seriesSet := querier.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchRegexp, labels.MetricName, ".+")) + + // Iterate all series and get their chunks. + var it chunks.Iterator + var chunks []chunkenc.Chunk + actualSeries := 0 + for seriesSet.Next() { + actualSeries++ + it = seriesSet.At().Iterator(it) + for it.Next() { + chunks = append(chunks, it.At().Chunk) + } + } + require.NoError(t, seriesSet.Err()) + require.Equal(t, numSeries, actualSeries) + + // Compact the TSDB head again. + require.NoError(t, db.Compact(ctx)) + require.Equal(t, float64(2), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) + + // At this point we expect 1 head chunk has been deleted. + + // Stress the memory and call GC. This is required to increase the chances + // the chunk memory area is released to the kernel. + var buf []byte + for i := range numStressIterations { + //nolint:staticcheck + buf = append(buf, make([]byte, minStressAllocationBytes+rand.Int31n(maxStressAllocationBytes-minStressAllocationBytes))...) + if i%1000 == 0 { + buf = nil + } + } + + // Iterate chunks and read their bytes slice. Here we're computing the CRC32 + // just to iterate through the bytes slice. We don't really care the reason why + // we read this data, we just need to read it to make sure the memory address + // of the []byte is still valid. + chkCRC32 := crc32.New(crc32.MakeTable(crc32.Castagnoli)) + for _, chunk := range chunks { + chkCRC32.Reset() + _, err := chkCRC32.Write(chunk.Bytes()) + require.NoError(t, err) + } +} + +func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration + db := newTestDB(t, withOpts(opts)) + + // Disable compactions so we can control it. + db.DisableCompactions() + + metric := labels.FromStrings(labels.MetricName, "test_metric") + ctx := context.Background() + interval := int64(15 * time.Second / time.Millisecond) + ts := int64(0) + samplesWritten := 0 + + // Capture the first timestamp - this will be the timestamp of the OOO sample we'll append below. + oooTS := ts + ts += interval + + // Push samples after the OOO sample we'll write below. + for ; ts < 10*interval; ts += interval { + app := db.Appender(ctx) + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + samplesWritten++ + } + + // Push a single OOO sample. + app := db.Appender(ctx) + _, err := app.Append(0, metric, oooTS, float64(ts)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + samplesWritten++ + + // Get a querier. + querierCreatedBeforeCompaction, err := db.ChunkQuerier(0, math.MaxInt64) + require.NoError(t, err) + + // Start OOO head compaction. + compactionComplete := atomic.NewBool(false) + go func() { + defer compactionComplete.Store(true) + + require.NoError(t, db.CompactOOOHead(ctx)) + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.chunksRemoved)) + }() + + // Give CompactOOOHead time to start work. + // If it does not wait for querierCreatedBeforeCompaction to be closed, then the query will return incorrect results or fail. + time.Sleep(time.Second) + require.False(t, compactionComplete.Load(), "compaction completed before reading chunks or closing querier created before compaction") + + // Get another querier. This one should only use the compacted blocks from disk and ignore the chunks that will be garbage collected. + querierCreatedAfterCompaction, err := db.ChunkQuerier(0, math.MaxInt64) + require.NoError(t, err) + + testQuerier := func(q storage.ChunkQuerier) { + // Query back the series. + hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} + seriesSet := q.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchEqual, labels.MetricName, "test_metric")) + + // Collect the iterator for the series. + var iterators []chunks.Iterator + for seriesSet.Next() { + iterators = append(iterators, seriesSet.At().Iterator(nil)) + } + require.NoError(t, seriesSet.Err()) + require.Len(t, iterators, 1) + iterator := iterators[0] + + // Check that we can still successfully read all samples. + samplesRead := 0 + for iterator.Next() { + samplesRead += iterator.At().Chunk.NumSamples() + } + + require.NoError(t, iterator.Err()) + require.Equal(t, samplesWritten, samplesRead) + } + + testQuerier(querierCreatedBeforeCompaction) + + require.False(t, compactionComplete.Load(), "compaction completed before closing querier created before compaction") + require.NoError(t, querierCreatedBeforeCompaction.Close()) + require.Eventually(t, compactionComplete.Load, time.Second, 10*time.Millisecond, "compaction should complete after querier created before compaction was closed, and not wait for querier created after compaction") + + // Use the querier created after compaction and confirm it returns the expected results (ie. from the disk block created from OOO head and in-order head) without error. + testQuerier(querierCreatedAfterCompaction) + require.NoError(t, querierCreatedAfterCompaction.Close()) +} + +func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration + db := newTestDB(t, withOpts(opts)) + + // Disable compactions so we can control it. + db.DisableCompactions() + + metric := labels.FromStrings(labels.MetricName, "test_metric") + ctx := context.Background() + interval := int64(15 * time.Second / time.Millisecond) + ts := int64(0) + samplesWritten := 0 + + // Capture the first timestamp - this will be the timestamp of the OOO sample we'll append below. + oooTS := ts + ts += interval + + // Push samples after the OOO sample we'll write below. + for ; ts < 10*interval; ts += interval { + app := db.Appender(ctx) + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + samplesWritten++ + } + + // Push a single OOO sample. + app := db.Appender(ctx) + _, err := app.Append(0, metric, oooTS, float64(ts)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + samplesWritten++ + + // Get a querier. + querier, err := db.ChunkQuerier(0, math.MaxInt64) + require.NoError(t, err) + + // Query back the series. + hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} + seriesSet := querier.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchEqual, labels.MetricName, "test_metric")) + + // Start OOO head compaction. + compactionComplete := atomic.NewBool(false) + go func() { + defer compactionComplete.Store(true) + + require.NoError(t, db.CompactOOOHead(ctx)) + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.chunksRemoved)) + }() + + // Give CompactOOOHead time to start work. + // If it does not wait for the querier to be closed, then the query will return incorrect results or fail. + time.Sleep(time.Second) + require.False(t, compactionComplete.Load(), "compaction completed before reading chunks or closing querier") + + // Collect the iterator for the series. + var iterators []chunks.Iterator + for seriesSet.Next() { + iterators = append(iterators, seriesSet.At().Iterator(nil)) + } + require.NoError(t, seriesSet.Err()) + require.Len(t, iterators, 1) + iterator := iterators[0] + + // Check that we can still successfully read all samples. + samplesRead := 0 + for iterator.Next() { + samplesRead += iterator.At().Chunk.NumSamples() + } + + require.NoError(t, iterator.Err()) + require.Equal(t, samplesWritten, samplesRead) + + require.False(t, compactionComplete.Load(), "compaction completed before closing querier") + require.NoError(t, querier.Close()) + require.Eventually(t, compactionComplete.Load, time.Second, 10*time.Millisecond, "compaction should complete after querier was closed") +} + +func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration + db := newTestDB(t, withOpts(opts)) + + // Disable compactions so we can control it. + db.DisableCompactions() + + metric := labels.FromStrings(labels.MetricName, "test_metric") + ctx := context.Background() + interval := int64(15 * time.Second / time.Millisecond) + ts := int64(0) + samplesWritten := 0 + + // Capture the first timestamp - this will be the timestamp of the OOO sample we'll append below. + oooTS := ts + ts += interval + + // Push samples after the OOO sample we'll write below. + for ; ts < 10*interval; ts += interval { + app := db.Appender(ctx) + _, err := app.Append(0, metric, ts, float64(ts)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + samplesWritten++ + } + + // Push a single OOO sample. + app := db.Appender(ctx) + _, err := app.Append(0, metric, oooTS, float64(ts)) + require.NoError(t, err) + require.NoError(t, app.Commit()) + samplesWritten++ + + // Get a querier. + querier, err := db.ChunkQuerier(0, math.MaxInt64) + require.NoError(t, err) + + // Query back the series. + hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} + seriesSet := querier.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchEqual, labels.MetricName, "test_metric")) + + // Collect the iterator for the series. + var iterators []chunks.Iterator + for seriesSet.Next() { + iterators = append(iterators, seriesSet.At().Iterator(nil)) + } + require.NoError(t, seriesSet.Err()) + require.Len(t, iterators, 1) + iterator := iterators[0] + + // Start OOO head compaction. + compactionComplete := atomic.NewBool(false) + go func() { + defer compactionComplete.Store(true) + + require.NoError(t, db.CompactOOOHead(ctx)) + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.chunksRemoved)) + }() + + // Give CompactOOOHead time to start work. + // If it does not wait for the querier to be closed, then the query will return incorrect results or fail. + time.Sleep(time.Second) + require.False(t, compactionComplete.Load(), "compaction completed before reading chunks or closing querier") + + // Check that we can still successfully read all samples. + samplesRead := 0 + for iterator.Next() { + samplesRead += iterator.At().Chunk.NumSamples() + } + + require.NoError(t, iterator.Err()) + require.Equal(t, samplesWritten, samplesRead) + + require.False(t, compactionComplete.Load(), "compaction completed before closing querier") + require.NoError(t, querier.Close()) + require.Eventually(t, compactionComplete.Load, time.Second, 10*time.Millisecond, "compaction should complete after querier was closed") +} + +func TestOOOWALWrite(t *testing.T) { + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + + s := labels.NewSymbolTable() + scratchBuilder1 := labels.NewScratchBuilderWithSymbolTable(s, 1) + scratchBuilder1.Add("l", "v1") + s1 := scratchBuilder1.Labels() + scratchBuilder2 := labels.NewScratchBuilderWithSymbolTable(s, 1) + scratchBuilder2.Add("l", "v2") + s2 := scratchBuilder2.Labels() + + scenarios := map[string]struct { + appendSample func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) + expectedOOORecords []any + expectedInORecords []any + }{ + "float": { + appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.Append(0, l, minutes(mins), float64(mins)) + require.NoError(t, err) + return seriesRef, nil + }, + expectedOOORecords: []any{ + // The MmapRef in this are not hand calculated, and instead taken from the test run. + // What is important here is the order of records, and that MmapRef increases for each record. + []record.RefMmapMarker{ + {Ref: 1}, + }, + []record.RefSample{ + {Ref: 1, T: minutes(40), V: 40}, + }, + + []record.RefMmapMarker{ + {Ref: 2}, + }, + []record.RefSample{ + {Ref: 2, T: minutes(42), V: 42}, + }, + + []record.RefSample{ + {Ref: 2, T: minutes(45), V: 45}, + {Ref: 1, T: minutes(35), V: 35}, + }, + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 8}, + }, + []record.RefSample{ + {Ref: 1, T: minutes(36), V: 36}, + {Ref: 1, T: minutes(37), V: 37}, + }, + + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 58}, + }, + []record.RefSample{ // Does not contain the in-order sample here. + {Ref: 1, T: minutes(50), V: 50}, + }, + + // Single commit but multiple OOO records. + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 107}, + }, + []record.RefSample{ + {Ref: 2, T: minutes(50), V: 50}, + {Ref: 2, T: minutes(51), V: 51}, + }, + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 156}, + }, + []record.RefSample{ + {Ref: 2, T: minutes(52), V: 52}, + {Ref: 2, T: minutes(53), V: 53}, + }, + }, + expectedInORecords: []any{ + []record.RefSeries{ + {Ref: 1, Labels: s1}, + {Ref: 2, Labels: s2}, + }, + []record.RefSample{ + {Ref: 1, T: minutes(60), V: 60}, + {Ref: 2, T: minutes(60), V: 60}, + }, + []record.RefSample{ + {Ref: 1, T: minutes(40), V: 40}, + }, + []record.RefSample{ + {Ref: 2, T: minutes(42), V: 42}, + }, + []record.RefSample{ + {Ref: 2, T: minutes(45), V: 45}, + {Ref: 1, T: minutes(35), V: 35}, + {Ref: 1, T: minutes(36), V: 36}, + {Ref: 1, T: minutes(37), V: 37}, + }, + []record.RefSample{ // Contains both in-order and ooo sample. + {Ref: 1, T: minutes(50), V: 50}, + {Ref: 2, T: minutes(65), V: 65}, + }, + []record.RefSample{ + {Ref: 2, T: minutes(50), V: 50}, + {Ref: 2, T: minutes(51), V: 51}, + {Ref: 2, T: minutes(52), V: 52}, + {Ref: 2, T: minutes(53), V: 53}, + }, + }, + }, + "integer histogram": { + appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.AppendHistogram(0, l, minutes(mins), tsdbutil.GenerateTestHistogram(mins), nil) + require.NoError(t, err) + return seriesRef, nil + }, + expectedOOORecords: []any{ + // The MmapRef in this are not hand calculated, and instead taken from the test run. + // What is important here is the order of records, and that MmapRef increases for each record. + []record.RefMmapMarker{ + {Ref: 1}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(40), H: tsdbutil.GenerateTestHistogram(40)}, + }, + + []record.RefMmapMarker{ + {Ref: 2}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(42), H: tsdbutil.GenerateTestHistogram(42)}, + }, + + []record.RefHistogramSample{ + {Ref: 2, T: minutes(45), H: tsdbutil.GenerateTestHistogram(45)}, + {Ref: 1, T: minutes(35), H: tsdbutil.GenerateTestHistogram(35)}, + }, + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 8}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(36), H: tsdbutil.GenerateTestHistogram(36)}, + {Ref: 1, T: minutes(37), H: tsdbutil.GenerateTestHistogram(37)}, + }, + + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 89}, + }, + []record.RefHistogramSample{ // Does not contain the in-order sample here. + {Ref: 1, T: minutes(50), H: tsdbutil.GenerateTestHistogram(50)}, + }, + + // Single commit but multiple OOO records. + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 172}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(50), H: tsdbutil.GenerateTestHistogram(50)}, + {Ref: 2, T: minutes(51), H: tsdbutil.GenerateTestHistogram(51)}, + }, + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 257}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(52), H: tsdbutil.GenerateTestHistogram(52)}, + {Ref: 2, T: minutes(53), H: tsdbutil.GenerateTestHistogram(53)}, + }, + }, + expectedInORecords: []any{ + []record.RefSeries{ + {Ref: 1, Labels: s1}, + {Ref: 2, Labels: s2}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(60), H: tsdbutil.GenerateTestHistogram(60)}, + {Ref: 2, T: minutes(60), H: tsdbutil.GenerateTestHistogram(60)}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(40), H: tsdbutil.GenerateTestHistogram(40)}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(42), H: tsdbutil.GenerateTestHistogram(42)}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(45), H: tsdbutil.GenerateTestHistogram(45)}, + {Ref: 1, T: minutes(35), H: tsdbutil.GenerateTestHistogram(35)}, + {Ref: 1, T: minutes(36), H: tsdbutil.GenerateTestHistogram(36)}, + {Ref: 1, T: minutes(37), H: tsdbutil.GenerateTestHistogram(37)}, + }, + []record.RefHistogramSample{ // Contains both in-order and ooo sample. + {Ref: 1, T: minutes(50), H: tsdbutil.GenerateTestHistogram(50)}, + {Ref: 2, T: minutes(65), H: tsdbutil.GenerateTestHistogram(65)}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(50), H: tsdbutil.GenerateTestHistogram(50)}, + {Ref: 2, T: minutes(51), H: tsdbutil.GenerateTestHistogram(51)}, + {Ref: 2, T: minutes(52), H: tsdbutil.GenerateTestHistogram(52)}, + {Ref: 2, T: minutes(53), H: tsdbutil.GenerateTestHistogram(53)}, + }, + }, + }, + "float histogram": { + appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.AppendHistogram(0, l, minutes(mins), nil, tsdbutil.GenerateTestFloatHistogram(mins)) + require.NoError(t, err) + return seriesRef, nil + }, + expectedOOORecords: []any{ + // The MmapRef in this are not hand calculated, and instead taken from the test run. + // What is important here is the order of records, and that MmapRef increases for each record. + []record.RefMmapMarker{ + {Ref: 1}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(40), FH: tsdbutil.GenerateTestFloatHistogram(40)}, + }, + + []record.RefMmapMarker{ + {Ref: 2}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(42), FH: tsdbutil.GenerateTestFloatHistogram(42)}, + }, + + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(45), FH: tsdbutil.GenerateTestFloatHistogram(45)}, + {Ref: 1, T: minutes(35), FH: tsdbutil.GenerateTestFloatHistogram(35)}, + }, + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 8}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(36), FH: tsdbutil.GenerateTestFloatHistogram(36)}, + {Ref: 1, T: minutes(37), FH: tsdbutil.GenerateTestFloatHistogram(37)}, + }, + + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 177}, + }, + []record.RefFloatHistogramSample{ // Does not contain the in-order sample here. + {Ref: 1, T: minutes(50), FH: tsdbutil.GenerateTestFloatHistogram(50)}, + }, + + // Single commit but multiple OOO records. + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 348}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(50), FH: tsdbutil.GenerateTestFloatHistogram(50)}, + {Ref: 2, T: minutes(51), FH: tsdbutil.GenerateTestFloatHistogram(51)}, + }, + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 521}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(52), FH: tsdbutil.GenerateTestFloatHistogram(52)}, + {Ref: 2, T: minutes(53), FH: tsdbutil.GenerateTestFloatHistogram(53)}, + }, + }, + expectedInORecords: []any{ + []record.RefSeries{ + {Ref: 1, Labels: s1}, + {Ref: 2, Labels: s2}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(60), FH: tsdbutil.GenerateTestFloatHistogram(60)}, + {Ref: 2, T: minutes(60), FH: tsdbutil.GenerateTestFloatHistogram(60)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(40), FH: tsdbutil.GenerateTestFloatHistogram(40)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(42), FH: tsdbutil.GenerateTestFloatHistogram(42)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(45), FH: tsdbutil.GenerateTestFloatHistogram(45)}, + {Ref: 1, T: minutes(35), FH: tsdbutil.GenerateTestFloatHistogram(35)}, + {Ref: 1, T: minutes(36), FH: tsdbutil.GenerateTestFloatHistogram(36)}, + {Ref: 1, T: minutes(37), FH: tsdbutil.GenerateTestFloatHistogram(37)}, + }, + []record.RefFloatHistogramSample{ // Contains both in-order and ooo sample. + {Ref: 1, T: minutes(50), FH: tsdbutil.GenerateTestFloatHistogram(50)}, + {Ref: 2, T: minutes(65), FH: tsdbutil.GenerateTestFloatHistogram(65)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(50), FH: tsdbutil.GenerateTestFloatHistogram(50)}, + {Ref: 2, T: minutes(51), FH: tsdbutil.GenerateTestFloatHistogram(51)}, + {Ref: 2, T: minutes(52), FH: tsdbutil.GenerateTestFloatHistogram(52)}, + {Ref: 2, T: minutes(53), FH: tsdbutil.GenerateTestFloatHistogram(53)}, + }, + }, + }, + "custom buckets histogram": { + appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.AppendHistogram(0, l, minutes(mins), tsdbutil.GenerateTestCustomBucketsHistogram(mins), nil) + require.NoError(t, err) + return seriesRef, nil + }, + expectedOOORecords: []any{ + // The MmapRef in this are not hand calculated, and instead taken from the test run. + // What is important here is the order of records, and that MmapRef increases for each record. + []record.RefMmapMarker{ + {Ref: 1}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(40), H: tsdbutil.GenerateTestCustomBucketsHistogram(40)}, + }, + + []record.RefMmapMarker{ + {Ref: 2}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(42), H: tsdbutil.GenerateTestCustomBucketsHistogram(42)}, + }, + + []record.RefHistogramSample{ + {Ref: 2, T: minutes(45), H: tsdbutil.GenerateTestCustomBucketsHistogram(45)}, + {Ref: 1, T: minutes(35), H: tsdbutil.GenerateTestCustomBucketsHistogram(35)}, + }, + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 8}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(36), H: tsdbutil.GenerateTestCustomBucketsHistogram(36)}, + {Ref: 1, T: minutes(37), H: tsdbutil.GenerateTestCustomBucketsHistogram(37)}, + }, + + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 82}, + }, + []record.RefHistogramSample{ // Does not contain the in-order sample here. + {Ref: 1, T: minutes(50), H: tsdbutil.GenerateTestCustomBucketsHistogram(50)}, + }, + + // Single commit but multiple OOO records. + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 160}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(50), H: tsdbutil.GenerateTestCustomBucketsHistogram(50)}, + {Ref: 2, T: minutes(51), H: tsdbutil.GenerateTestCustomBucketsHistogram(51)}, + }, + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 239}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(52), H: tsdbutil.GenerateTestCustomBucketsHistogram(52)}, + {Ref: 2, T: minutes(53), H: tsdbutil.GenerateTestCustomBucketsHistogram(53)}, + }, + }, + expectedInORecords: []any{ + []record.RefSeries{ + {Ref: 1, Labels: s1}, + {Ref: 2, Labels: s2}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(60), H: tsdbutil.GenerateTestCustomBucketsHistogram(60)}, + {Ref: 2, T: minutes(60), H: tsdbutil.GenerateTestCustomBucketsHistogram(60)}, + }, + []record.RefHistogramSample{ + {Ref: 1, T: minutes(40), H: tsdbutil.GenerateTestCustomBucketsHistogram(40)}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(42), H: tsdbutil.GenerateTestCustomBucketsHistogram(42)}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(45), H: tsdbutil.GenerateTestCustomBucketsHistogram(45)}, + {Ref: 1, T: minutes(35), H: tsdbutil.GenerateTestCustomBucketsHistogram(35)}, + {Ref: 1, T: minutes(36), H: tsdbutil.GenerateTestCustomBucketsHistogram(36)}, + {Ref: 1, T: minutes(37), H: tsdbutil.GenerateTestCustomBucketsHistogram(37)}, + }, + []record.RefHistogramSample{ // Contains both in-order and ooo sample. + {Ref: 1, T: minutes(50), H: tsdbutil.GenerateTestCustomBucketsHistogram(50)}, + {Ref: 2, T: minutes(65), H: tsdbutil.GenerateTestCustomBucketsHistogram(65)}, + }, + []record.RefHistogramSample{ + {Ref: 2, T: minutes(50), H: tsdbutil.GenerateTestCustomBucketsHistogram(50)}, + {Ref: 2, T: minutes(51), H: tsdbutil.GenerateTestCustomBucketsHistogram(51)}, + {Ref: 2, T: minutes(52), H: tsdbutil.GenerateTestCustomBucketsHistogram(52)}, + {Ref: 2, T: minutes(53), H: tsdbutil.GenerateTestCustomBucketsHistogram(53)}, + }, + }, + }, + "custom buckets float histogram": { + appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.AppendHistogram(0, l, minutes(mins), nil, tsdbutil.GenerateTestCustomBucketsFloatHistogram(mins)) + require.NoError(t, err) + return seriesRef, nil + }, + expectedOOORecords: []any{ + // The MmapRef in this are not hand calculated, and instead taken from the test run. + // What is important here is the order of records, and that MmapRef increases for each record. + []record.RefMmapMarker{ + {Ref: 1}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(40), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(40)}, + }, + + []record.RefMmapMarker{ + {Ref: 2}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(42), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(42)}, + }, + + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(45), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(45)}, + {Ref: 1, T: minutes(35), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(35)}, + }, + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 8}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(36), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(36)}, + {Ref: 1, T: minutes(37), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(37)}, + }, + + []record.RefMmapMarker{ // 3rd sample, hence m-mapped. + {Ref: 1, MmapRef: 0x100000000 + 134}, + }, + []record.RefFloatHistogramSample{ // Does not contain the in-order sample here. + {Ref: 1, T: minutes(50), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(50)}, + }, + + // Single commit but multiple OOO records. + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 263}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(50), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(50)}, + {Ref: 2, T: minutes(51), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(51)}, + }, + []record.RefMmapMarker{ + {Ref: 2, MmapRef: 0x100000000 + 393}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(52), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(52)}, + {Ref: 2, T: minutes(53), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(53)}, + }, + }, + expectedInORecords: []any{ + []record.RefSeries{ + {Ref: 1, Labels: s1}, + {Ref: 2, Labels: s2}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(60), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(60)}, + {Ref: 2, T: minutes(60), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(60)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 1, T: minutes(40), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(40)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(42), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(42)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(45), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(45)}, + {Ref: 1, T: minutes(35), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(35)}, + {Ref: 1, T: minutes(36), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(36)}, + {Ref: 1, T: minutes(37), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(37)}, + }, + []record.RefFloatHistogramSample{ // Contains both in-order and ooo sample. + {Ref: 1, T: minutes(50), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(50)}, + {Ref: 2, T: minutes(65), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(65)}, + }, + []record.RefFloatHistogramSample{ + {Ref: 2, T: minutes(50), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(50)}, + {Ref: 2, T: minutes(51), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(51)}, + {Ref: 2, T: minutes(52), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(52)}, + {Ref: 2, T: minutes(53), FH: tsdbutil.GenerateTestCustomBucketsFloatHistogram(53)}, + }, + }, + }, + } + for name, scenario := range scenarios { + t.Run(name, func(t *testing.T) { + testOOOWALWrite(t, scenario.appendSample, scenario.expectedOOORecords, scenario.expectedInORecords) + }) + } +} + +func testOOOWALWrite(t *testing.T, + appendSample func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error), + expectedOOORecords []any, + expectedInORecords []any, +) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 2 + opts.OutOfOrderTimeWindow = 30 * time.Minute.Milliseconds() + db := newTestDB(t, withOpts(opts)) + + s1, s2 := labels.FromStrings("l", "v1"), labels.FromStrings("l", "v2") + + // Ingest sample at 1h. + app := db.Appender(context.Background()) + appendSample(app, s1, 60) + appendSample(app, s2, 60) + require.NoError(t, app.Commit()) + + // OOO for s1. + app = db.Appender(context.Background()) + appendSample(app, s1, 40) + require.NoError(t, app.Commit()) + + // OOO for s2. + app = db.Appender(context.Background()) + appendSample(app, s2, 42) + require.NoError(t, app.Commit()) + + // OOO for both s1 and s2 in the same commit. + app = db.Appender(context.Background()) + appendSample(app, s2, 45) + appendSample(app, s1, 35) + appendSample(app, s1, 36) // m-maps. + appendSample(app, s1, 37) + require.NoError(t, app.Commit()) + + // OOO for s1 but not for s2 in the same commit. + app = db.Appender(context.Background()) + appendSample(app, s1, 50) // m-maps. + appendSample(app, s2, 65) + require.NoError(t, app.Commit()) + + // Single commit has 2 times m-mapping and more samples after m-map. + app = db.Appender(context.Background()) + appendSample(app, s2, 50) // m-maps. + appendSample(app, s2, 51) + appendSample(app, s2, 52) // m-maps. + appendSample(app, s2, 53) + require.NoError(t, app.Commit()) + + getRecords := func(walDir string) []any { + sr, err := wlog.NewSegmentsReader(walDir) + require.NoError(t, err) + r := wlog.NewReader(sr) + defer func() { + require.NoError(t, sr.Close()) + }() + + var records []any + dec := record.NewDecoder(nil, promslog.NewNopLogger()) + for r.Next() { + rec := r.Record() + switch typ := dec.Type(rec); typ { + case record.Series: + series, err := dec.Series(rec, nil) + require.NoError(t, err) + records = append(records, series) + case record.Samples: + samples, err := dec.Samples(rec, nil) + require.NoError(t, err) + records = append(records, samples) + case record.MmapMarkers: + markers, err := dec.MmapMarkers(rec, nil) + require.NoError(t, err) + records = append(records, markers) + case record.HistogramSamples, record.CustomBucketsHistogramSamples: + histogramSamples, err := dec.HistogramSamples(rec, nil) + require.NoError(t, err) + records = append(records, histogramSamples) + case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: + floatHistogramSamples, err := dec.FloatHistogramSamples(rec, nil) + require.NoError(t, err) + records = append(records, floatHistogramSamples) + default: + t.Fatalf("got a WAL record that is not series or samples: %v", typ) + } + } + + return records + } + + // The normal WAL. + actRecs := getRecords(path.Join(db.Dir(), "wal")) + require.Equal(t, expectedInORecords, actRecs) + + // The WBL. + actRecs = getRecords(path.Join(db.Dir(), wlog.WblDirName)) + require.Equal(t, expectedOOORecords, actRecs) +} + +// Tests https://github.com/prometheus/prometheus/issues/10291#issuecomment-1044373110. +func TestDBPanicOnMmappingHeadChunk(t *testing.T) { + var err error + ctx := context.Background() + + db := newTestDB(t) + db.DisableCompactions() + + // Choosing scrape interval of 45s to have chunk larger than 1h. + itvl := int64(45 * time.Second / time.Millisecond) + + lastTs := int64(0) + addSamples := func(numSamples int) { + app := db.Appender(context.Background()) + var ref storage.SeriesRef + lbls := labels.FromStrings("__name__", "testing", "foo", "bar") + for i := range numSamples { + ref, err = app.Append(ref, lbls, lastTs, float64(lastTs)) + require.NoError(t, err) + lastTs += itvl + if i%10 == 0 { + require.NoError(t, app.Commit()) + app = db.Appender(context.Background()) + } + } + require.NoError(t, app.Commit()) + } + + // Ingest samples upto 2h50m to make the head "about to compact". + numSamples := int(170*time.Minute/time.Millisecond) / int(itvl) + addSamples(numSamples) + + require.Empty(t, db.Blocks()) + require.NoError(t, db.Compact(ctx)) + require.Empty(t, db.Blocks()) + + // Restarting. + require.NoError(t, db.Close()) + + db = newTestDB(t, withDir(db.Dir())) + db.DisableCompactions() + + // Ingest samples upto 20m more to make the head compact. + numSamples = int(20*time.Minute/time.Millisecond) / int(itvl) + addSamples(numSamples) + + require.Empty(t, db.Blocks()) + require.NoError(t, db.Compact(ctx)) + require.Len(t, db.Blocks(), 1) + + // More samples to m-map and panic. + numSamples = int(120*time.Minute/time.Millisecond) / int(itvl) + addSamples(numSamples) + + require.NoError(t, db.Close()) +} + +func TestMetadataInWAL(t *testing.T) { + updateMetadata := func(t *testing.T, app storage.Appender, s labels.Labels, m metadata.Metadata) { + _, err := app.UpdateMetadata(0, s, m) + require.NoError(t, err) + } + + db := newTestDB(t) + ctx := context.Background() + + // Add some series so we can append metadata to them. + app := db.Appender(ctx) + s1 := labels.FromStrings("a", "b") + s2 := labels.FromStrings("c", "d") + s3 := labels.FromStrings("e", "f") + s4 := labels.FromStrings("g", "h") + + for _, s := range []labels.Labels{s1, s2, s3, s4} { + _, err := app.Append(0, s, 0, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Add a first round of metadata to the first three series. + // Re-take the Appender, as the previous Commit will have it closed. + m1 := metadata.Metadata{Type: "gauge", Unit: "unit_1", Help: "help_1"} + m2 := metadata.Metadata{Type: "gauge", Unit: "unit_2", Help: "help_2"} + m3 := metadata.Metadata{Type: "gauge", Unit: "unit_3", Help: "help_3"} + app = db.Appender(ctx) + updateMetadata(t, app, s1, m1) + updateMetadata(t, app, s2, m2) + updateMetadata(t, app, s3, m3) + require.NoError(t, app.Commit()) + + // Add a replicated metadata entry to the first series, + // a completely new metadata entry for the fourth series, + // and a changed metadata entry to the second series. + m4 := metadata.Metadata{Type: "counter", Unit: "unit_4", Help: "help_4"} + m5 := metadata.Metadata{Type: "counter", Unit: "unit_5", Help: "help_5"} + app = db.Appender(ctx) + updateMetadata(t, app, s1, m1) + updateMetadata(t, app, s4, m4) + updateMetadata(t, app, s2, m5) + require.NoError(t, app.Commit()) + + // Read the WAL to see if the disk storage format is correct. + recs := readTestWAL(t, path.Join(db.Dir(), "wal")) + var gotMetadataBlocks [][]record.RefMetadata + for _, rec := range recs { + if mr, ok := rec.([]record.RefMetadata); ok { + gotMetadataBlocks = append(gotMetadataBlocks, mr) + } + } + + expectedMetadata := []record.RefMetadata{ + {Ref: 1, Type: record.GetMetricType(m1.Type), Unit: m1.Unit, Help: m1.Help}, + {Ref: 2, Type: record.GetMetricType(m2.Type), Unit: m2.Unit, Help: m2.Help}, + {Ref: 3, Type: record.GetMetricType(m3.Type), Unit: m3.Unit, Help: m3.Help}, + {Ref: 4, Type: record.GetMetricType(m4.Type), Unit: m4.Unit, Help: m4.Help}, + {Ref: 2, Type: record.GetMetricType(m5.Type), Unit: m5.Unit, Help: m5.Help}, + } + require.Len(t, gotMetadataBlocks, 2) + require.Equal(t, expectedMetadata[:3], gotMetadataBlocks[0]) + require.Equal(t, expectedMetadata[3:], gotMetadataBlocks[1]) +} + +func TestMetadataCheckpointingOnlyKeepsLatestEntry(t *testing.T) { + updateMetadata := func(t *testing.T, app storage.Appender, s labels.Labels, m metadata.Metadata) { + _, err := app.UpdateMetadata(0, s, m) + require.NoError(t, err) + } + + ctx := context.Background() + numSamples := 10000 + hb, w := newTestHead(t, int64(numSamples)*10, compression.None, false) + + // Add some series so we can append metadata to them. + app := hb.Appender(ctx) + s1 := labels.FromStrings("a", "b") + s2 := labels.FromStrings("c", "d") + s3 := labels.FromStrings("e", "f") + s4 := labels.FromStrings("g", "h") + + for _, s := range []labels.Labels{s1, s2, s3, s4} { + _, err := app.Append(0, s, 0, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Add a first round of metadata to the first three series. + // Re-take the Appender, as the previous Commit will have it closed. + m1 := metadata.Metadata{Type: "gauge", Unit: "unit_1", Help: "help_1"} + m2 := metadata.Metadata{Type: "gauge", Unit: "unit_2", Help: "help_2"} + m3 := metadata.Metadata{Type: "gauge", Unit: "unit_3", Help: "help_3"} + m4 := metadata.Metadata{Type: "gauge", Unit: "unit_4", Help: "help_4"} + app = hb.Appender(ctx) + updateMetadata(t, app, s1, m1) + updateMetadata(t, app, s2, m2) + updateMetadata(t, app, s3, m3) + updateMetadata(t, app, s4, m4) + require.NoError(t, app.Commit()) + + // Update metadata for first series. + m5 := metadata.Metadata{Type: "counter", Unit: "unit_5", Help: "help_5"} + app = hb.Appender(ctx) + updateMetadata(t, app, s1, m5) + require.NoError(t, app.Commit()) + + // Switch back-and-forth metadata for second series. + // Since it ended on a new metadata record, we expect a single new entry. + m6 := metadata.Metadata{Type: "counter", Unit: "unit_6", Help: "help_6"} + + app = hb.Appender(ctx) + updateMetadata(t, app, s2, m6) + require.NoError(t, app.Commit()) + + app = hb.Appender(ctx) + updateMetadata(t, app, s2, m2) + require.NoError(t, app.Commit()) + + app = hb.Appender(ctx) + updateMetadata(t, app, s2, m6) + require.NoError(t, app.Commit()) + + app = hb.Appender(ctx) + updateMetadata(t, app, s2, m2) + require.NoError(t, app.Commit()) + + app = hb.Appender(ctx) + updateMetadata(t, app, s2, m6) + require.NoError(t, app.Commit()) + + // Let's create a checkpoint. + first, last, err := wlog.Segments(w.Dir()) + require.NoError(t, err) + keep := func(id chunks.HeadSeriesRef) bool { + return id != 3 + } + _, err = wlog.Checkpoint(promslog.NewNopLogger(), w, first, last-1, keep, 0) + require.NoError(t, err) + + // Confirm there's been a checkpoint. + cdir, _, err := wlog.LastCheckpoint(w.Dir()) + require.NoError(t, err) + + // Read in checkpoint and WAL. + recs := readTestWAL(t, cdir) + var gotMetadataBlocks [][]record.RefMetadata + for _, rec := range recs { + if mr, ok := rec.([]record.RefMetadata); ok { + gotMetadataBlocks = append(gotMetadataBlocks, mr) + } + } + + // There should only be 1 metadata block present, with only the latest + // metadata kept around. + wantMetadata := []record.RefMetadata{ + {Ref: 1, Type: record.GetMetricType(m5.Type), Unit: m5.Unit, Help: m5.Help}, + {Ref: 2, Type: record.GetMetricType(m6.Type), Unit: m6.Unit, Help: m6.Help}, + {Ref: 4, Type: record.GetMetricType(m4.Type), Unit: m4.Unit, Help: m4.Help}, + } + require.Len(t, gotMetadataBlocks, 1) + require.Len(t, gotMetadataBlocks[0], 3) + gotMetadataBlock := gotMetadataBlocks[0] + + sort.Slice(gotMetadataBlock, func(i, j int) bool { return gotMetadataBlock[i].Ref < gotMetadataBlock[j].Ref }) + require.Equal(t, wantMetadata, gotMetadataBlock) + require.NoError(t, hb.Close()) +} + +func TestMetadataAssertInMemoryData(t *testing.T) { + updateMetadata := func(t *testing.T, app storage.Appender, s labels.Labels, m metadata.Metadata) { + _, err := app.UpdateMetadata(0, s, m) + require.NoError(t, err) + } + + db := newTestDB(t) + ctx := context.Background() + + // Add some series so we can append metadata to them. + app := db.Appender(ctx) + s1 := labels.FromStrings("a", "b") + s2 := labels.FromStrings("c", "d") + s3 := labels.FromStrings("e", "f") + s4 := labels.FromStrings("g", "h") + + for _, s := range []labels.Labels{s1, s2, s3, s4} { + _, err := app.Append(0, s, 0, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Add a first round of metadata to the first three series. + // The in-memory data held in the db Head should hold the metadata. + m1 := metadata.Metadata{Type: "gauge", Unit: "unit_1", Help: "help_1"} + m2 := metadata.Metadata{Type: "gauge", Unit: "unit_2", Help: "help_2"} + m3 := metadata.Metadata{Type: "gauge", Unit: "unit_3", Help: "help_3"} + app = db.Appender(ctx) + updateMetadata(t, app, s1, m1) + updateMetadata(t, app, s2, m2) + updateMetadata(t, app, s3, m3) + require.NoError(t, app.Commit()) + + series1 := db.head.series.getByHash(s1.Hash(), s1) + series2 := db.head.series.getByHash(s2.Hash(), s2) + series3 := db.head.series.getByHash(s3.Hash(), s3) + series4 := db.head.series.getByHash(s4.Hash(), s4) + require.Equal(t, *series1.meta, m1) + require.Equal(t, *series2.meta, m2) + require.Equal(t, *series3.meta, m3) + require.Nil(t, series4.meta) + + // Add a replicated metadata entry to the first series, + // a changed metadata entry to the second series, + // and a completely new metadata entry for the fourth series. + // The in-memory data held in the db Head should be correctly updated. + m4 := metadata.Metadata{Type: "counter", Unit: "unit_4", Help: "help_4"} + m5 := metadata.Metadata{Type: "counter", Unit: "unit_5", Help: "help_5"} + app = db.Appender(ctx) + updateMetadata(t, app, s1, m1) + updateMetadata(t, app, s4, m4) + updateMetadata(t, app, s2, m5) + require.NoError(t, app.Commit()) + + series1 = db.head.series.getByHash(s1.Hash(), s1) + series2 = db.head.series.getByHash(s2.Hash(), s2) + series3 = db.head.series.getByHash(s3.Hash(), s3) + series4 = db.head.series.getByHash(s4.Hash(), s4) + require.Equal(t, *series1.meta, m1) + require.Equal(t, *series2.meta, m5) + require.Equal(t, *series3.meta, m3) + require.Equal(t, *series4.meta, m4) + + require.NoError(t, db.Close()) + + // Reopen the DB, replaying the WAL. The Head must have been replayed + // correctly in memory. + db = newTestDB(t, withDir(db.Dir())) + _, err := db.head.wal.Size() + require.NoError(t, err) + + require.Equal(t, *db.head.series.getByHash(s1.Hash(), s1).meta, m1) + require.Equal(t, *db.head.series.getByHash(s2.Hash(), s2).meta, m5) + require.Equal(t, *db.head.series.getByHash(s3.Hash(), s3).meta, m3) + require.Equal(t, *db.head.series.getByHash(s4.Hash(), s4).meta, m4) +} + +// TestMultipleEncodingsCommitOrder mainly serves to demonstrate when happens when committing a batch of samples for the +// same series when there are multiple encodings. With issue #15177 fixed, this now all works as expected. +func TestMultipleEncodingsCommitOrder(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + series1 := labels.FromStrings("foo", "bar1") + addSample := func(app storage.Appender, ts int64, valType chunkenc.ValueType) chunks.Sample { + if valType == chunkenc.ValFloat { + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) + require.NoError(t, err) + return sample{t: ts, f: float64(ts)} + } + if valType == chunkenc.ValHistogram { + h := tsdbutil.GenerateTestHistogram(ts) + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + require.NoError(t, err) + return sample{t: ts, h: h} + } + fh := tsdbutil.GenerateTestFloatHistogram(ts) + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + require.NoError(t, err) + return sample{t: ts, fh: fh} + } + + verifySamples := func(minT, maxT int64, expSamples []chunks.Sample, oooCount int) { + requireEqualOOOSamples(t, oooCount, db) + + // Verify samples querier. + querier, err := db.Querier(minT, maxT) + require.NoError(t, err) + defer querier.Close() + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + require.Len(t, seriesSet, 1) + gotSamples := seriesSet[series1.String()] + requireEqualSamples(t, series1.String(), expSamples, gotSamples, requireEqualSamplesIgnoreCounterResets) + + // Verify chunks querier. + chunkQuerier, err := db.ChunkQuerier(minT, maxT) + require.NoError(t, err) + defer chunkQuerier.Close() + + chks := queryChunks(t, chunkQuerier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + require.NotNil(t, chks[series1.String()]) + require.Len(t, chks, 1) + var gotChunkSamples []chunks.Sample + for _, chunk := range chks[series1.String()] { + it := chunk.Chunk.Iterator(nil) + smpls, err := storage.ExpandSamples(it, newSample) + require.NoError(t, err) + gotChunkSamples = append(gotChunkSamples, smpls...) + require.NoError(t, it.Err()) + } + requireEqualSamples(t, series1.String(), expSamples, gotChunkSamples, requireEqualSamplesIgnoreCounterResets) + } + + var expSamples []chunks.Sample + + // Append samples with different encoding types and then commit them at once. + app := db.Appender(context.Background()) + + for i := 100; i < 105; i++ { + s := addSample(app, int64(i), chunkenc.ValFloat) + expSamples = append(expSamples, s) + } + for i := 110; i < 120; i++ { + s := addSample(app, int64(i), chunkenc.ValHistogram) + expSamples = append(expSamples, s) + } + for i := 120; i < 130; i++ { + s := addSample(app, int64(i), chunkenc.ValFloatHistogram) + expSamples = append(expSamples, s) + } + for i := 140; i < 150; i++ { + s := addSample(app, int64(i), chunkenc.ValFloatHistogram) + expSamples = append(expSamples, s) + } + // These samples will be marked as out-of-order. + for i := 130; i < 135; i++ { + s := addSample(app, int64(i), chunkenc.ValFloat) + expSamples = append(expSamples, s) + } + + require.NoError(t, app.Commit()) + + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + // oooCount = 5 for the samples 130 to 134. + verifySamples(100, 150, expSamples, 5) + + // Append and commit some in-order histograms by themselves. + app = db.Appender(context.Background()) + for i := 150; i < 160; i++ { + s := addSample(app, int64(i), chunkenc.ValHistogram) + expSamples = append(expSamples, s) + } + require.NoError(t, app.Commit()) + + // oooCount remains at 5. + verifySamples(100, 160, expSamples, 5) + + // Append and commit samples for all encoding types. This time all samples will be treated as OOO because samples + // with newer timestamps have already been committed. + app = db.Appender(context.Background()) + for i := 50; i < 55; i++ { + s := addSample(app, int64(i), chunkenc.ValFloat) + expSamples = append(expSamples, s) + } + for i := 60; i < 70; i++ { + s := addSample(app, int64(i), chunkenc.ValHistogram) + expSamples = append(expSamples, s) + } + for i := 70; i < 75; i++ { + s := addSample(app, int64(i), chunkenc.ValFloat) + expSamples = append(expSamples, s) + } + for i := 80; i < 90; i++ { + s := addSample(app, int64(i), chunkenc.ValFloatHistogram) + expSamples = append(expSamples, s) + } + require.NoError(t, app.Commit()) + + // Sort samples again because OOO samples have been added. + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + // oooCount = 35 as we've added 30 more OOO samples. + verifySamples(50, 160, expSamples, 35) +} + +// TODO(codesome): test more samples incoming once compaction has started. To verify new samples after the start +// +// are not included in this compaction. +func TestOOOCompaction(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOCompaction(t, scenario, false) + }) + t.Run(name+"+extra", func(t *testing.T) { + testOOOCompaction(t, scenario, true) + }) + } +} + +func testOOOCompaction(t *testing.T, scenario sampleTypeScenario, addExtraSamples bool) { + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + series2 := labels.FromStrings("foo", "bar2") + + addSample := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, _, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSample(250, 300) + + // Verify that the in-memory ooo chunk is empty. + checkEmptyOOOChunk := func(lbls labels.Labels) { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Nil(t, ms.ooo) + } + checkEmptyOOOChunk(series1) + checkEmptyOOOChunk(series2) + + // Add ooo samples that creates multiple chunks. + // 90 to 300 spans across 3 block ranges: [0, 120), [120, 240), [240, 360) + addSample(90, 300) + // Adding same samples to create overlapping chunks. + // Since the active chunk won't start at 90 again, all the new + // chunks will have different time ranges than the previous chunks. + addSample(90, 300) + + var highest int64 = 300 + + verifyDBSamples := func() { + var series1Samples, series2Samples []chunks.Sample + for _, r := range [][2]int64{{90, 119}, {120, 239}, {240, highest}} { + fromMins, toMins := r[0], r[1] + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + series1Samples = append(series1Samples, scenario.sampleFunc(ts, ts)) + series2Samples = append(series2Samples, scenario.sampleFunc(ts, 2*ts)) + } + } + expRes := map[string][]chunks.Sample{ + series1.String(): series1Samples, + series2.String(): series2Samples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + verifyDBSamples() // Before any compaction. + + // Verify that the in-memory ooo chunk is not empty. + checkNonEmptyOOOChunk := func(lbls labels.Labels) { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Positive(t, ms.ooo.oooHeadChunk.chunk.NumSamples()) + require.Len(t, ms.ooo.oooMmappedChunks, 13) // 7 original, 6 duplicate. + } + checkNonEmptyOOOChunk(series1) + checkNonEmptyOOOChunk(series2) + + // No blocks before compaction. + require.Empty(t, db.Blocks()) + + // There is a 0th WBL file. + require.NoError(t, db.head.wbl.Sync()) // syncing to make sure wbl is flushed in windows + files, err := os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "00000000", files[0].Name()) + f, err := files[0].Info() + require.NoError(t, err) + require.Greater(t, f.Size(), int64(100)) + + if addExtraSamples { + compactOOOHeadTestingCallback = func() { + addSample(90, 120) // Back in time, to generate a new OOO chunk. + addSample(300, 330) // Now some samples after the previous highest timestamp. + addSample(300, 330) // Repeat to generate an OOO chunk at these timestamps. + } + highest = 330 + } + + // OOO compaction happens here. + require.NoError(t, db.CompactOOOHead(ctx)) + + // 3 blocks exist now. [0, 120), [120, 240), [240, 360) + require.Len(t, db.Blocks(), 3) + + verifyDBSamples() // Blocks created out of OOO head now. + + // 0th WBL file will be deleted and 1st will be the only present. + files, err = os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "00000001", files[0].Name()) + f, err = files[0].Info() + require.NoError(t, err) + + if !addExtraSamples { + require.Equal(t, int64(0), f.Size()) + // OOO stuff should not be present in the Head now. + checkEmptyOOOChunk(series1) + checkEmptyOOOChunk(series2) + } + + verifySamples := func(block *Block, fromMins, toMins int64) { + series1Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + series2Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + series1Samples = append(series1Samples, scenario.sampleFunc(ts, ts)) + series2Samples = append(series2Samples, scenario.sampleFunc(ts, 2*ts)) + } + expRes := map[string][]chunks.Sample{ + series1.String(): series1Samples, + series2.String(): series2Samples, + } + + q, err := NewBlockQuerier(block, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + // Checking for expected data in the blocks. + verifySamples(db.Blocks()[0], 90, 119) + verifySamples(db.Blocks()[1], 120, 239) + verifySamples(db.Blocks()[2], 240, 299) + + // There should be a single m-map file. + mmapDir := mmappedChunksDir(db.head.opts.ChunkDirRoot) + files, err = os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, 1) + + // Compact the in-order head and expect another block. + // Since this is a forced compaction, this block is not aligned with 2h. + err = db.CompactHead(NewRangeHead(db.head, 250*time.Minute.Milliseconds(), 350*time.Minute.Milliseconds())) + require.NoError(t, err) + require.Len(t, db.Blocks(), 4) // [0, 120), [120, 240), [240, 360), [250, 351) + verifySamples(db.Blocks()[3], 250, highest) + + verifyDBSamples() // Blocks created out of normal and OOO head now. But not merged. + + // The compaction also clears out the old m-map files. Including + // the file that has ooo chunks. + files, err = os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "000001", files[0].Name()) + + // This will merge overlapping block. + require.NoError(t, db.Compact(ctx)) + + require.Len(t, db.Blocks(), 3) // [0, 120), [120, 240), [240, 360) + verifySamples(db.Blocks()[0], 90, 119) + verifySamples(db.Blocks()[1], 120, 239) + verifySamples(db.Blocks()[2], 240, highest) // Merged block. + + verifyDBSamples() // Final state. Blocks from normal and OOO head are merged. +} + +// TestOOOCompactionWithNormalCompaction tests if OOO compaction is performed +// when the normal head's compaction is done. +func TestOOOCompactionWithNormalCompaction(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOCompactionWithNormalCompaction(t, scenario) + }) + } +} + +func testOOOCompactionWithNormalCompaction(t *testing.T, scenario sampleTypeScenario) { + t.Parallel() + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + series2 := labels.FromStrings("foo", "bar2") + + addSamples := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, _, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSamples(250, 350) + + // Add ooo samples that will result into a single block. + addSamples(90, 110) + + // Checking that ooo chunk is not empty. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Positive(t, ms.ooo.oooHeadChunk.chunk.NumSamples()) + } + + // If the normal Head is not compacted, the OOO head compaction does not take place. + require.NoError(t, db.Compact(ctx)) + require.Empty(t, db.Blocks()) + + // Add more in-order samples in future that would trigger the compaction. + addSamples(400, 450) + + // No blocks before compaction. + require.Empty(t, db.Blocks()) + + // Compacts normal and OOO head. + require.NoError(t, db.Compact(ctx)) + + // 2 blocks exist now. [0, 120), [250, 360) + require.Len(t, db.Blocks(), 2) + require.Equal(t, int64(0), db.Blocks()[0].MinTime()) + require.Equal(t, 120*time.Minute.Milliseconds(), db.Blocks()[0].MaxTime()) + require.Equal(t, 250*time.Minute.Milliseconds(), db.Blocks()[1].MinTime()) + require.Equal(t, 360*time.Minute.Milliseconds(), db.Blocks()[1].MaxTime()) + + // Checking that ooo chunk is empty. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Nil(t, ms.ooo) + } + + verifySamples := func(block *Block, fromMins, toMins int64) { + series1Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + series2Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + series1Samples = append(series1Samples, scenario.sampleFunc(ts, ts)) + series2Samples = append(series2Samples, scenario.sampleFunc(ts, 2*ts)) + } + expRes := map[string][]chunks.Sample{ + series1.String(): series1Samples, + series2.String(): series2Samples, + } + + q, err := NewBlockQuerier(block, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + // Checking for expected data in the blocks. + verifySamples(db.Blocks()[0], 90, 110) + verifySamples(db.Blocks()[1], 250, 350) +} + +// TestOOOCompactionWithDisabledWriteLog tests the scenario where the TSDB is +// configured to not have wal and wbl but its able to compact both the in-order +// and out-of-order head. +func TestOOOCompactionWithDisabledWriteLog(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOCompactionWithDisabledWriteLog(t, scenario) + }) + } +} + +func testOOOCompactionWithDisabledWriteLog(t *testing.T, scenario sampleTypeScenario) { + t.Parallel() + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + opts.WALSegmentSize = -1 // disabled WAL and WBL + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + series2 := labels.FromStrings("foo", "bar2") + + addSamples := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, _, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSamples(250, 350) + + // Add ooo samples that will result into a single block. + addSamples(90, 110) + + // Checking that ooo chunk is not empty. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Positive(t, ms.ooo.oooHeadChunk.chunk.NumSamples()) + } + + // If the normal Head is not compacted, the OOO head compaction does not take place. + require.NoError(t, db.Compact(ctx)) + require.Empty(t, db.Blocks()) + + // Add more in-order samples in future that would trigger the compaction. + addSamples(400, 450) + + // No blocks before compaction. + require.Empty(t, db.Blocks()) + + // Compacts normal and OOO head. + require.NoError(t, db.Compact(ctx)) + + // 2 blocks exist now. [0, 120), [250, 360) + require.Len(t, db.Blocks(), 2) + require.Equal(t, int64(0), db.Blocks()[0].MinTime()) + require.Equal(t, 120*time.Minute.Milliseconds(), db.Blocks()[0].MaxTime()) + require.Equal(t, 250*time.Minute.Milliseconds(), db.Blocks()[1].MinTime()) + require.Equal(t, 360*time.Minute.Milliseconds(), db.Blocks()[1].MaxTime()) + + // Checking that ooo chunk is empty. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Nil(t, ms.ooo) + } + + verifySamples := func(block *Block, fromMins, toMins int64) { + series1Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + series2Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + series1Samples = append(series1Samples, scenario.sampleFunc(ts, ts)) + series2Samples = append(series2Samples, scenario.sampleFunc(ts, 2*ts)) + } + expRes := map[string][]chunks.Sample{ + series1.String(): series1Samples, + series2.String(): series2Samples, + } + + q, err := NewBlockQuerier(block, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + // Checking for expected data in the blocks. + verifySamples(db.Blocks()[0], 90, 110) + verifySamples(db.Blocks()[1], 250, 350) +} + +// TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL tests the scenario where the WBL goes +// missing after a restart while snapshot was enabled, but the query still returns the right +// data from the mmap chunks. +func TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t, scenario) + }) + } +} + +func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sampleTypeScenario) { + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 10 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + opts.EnableMemorySnapshotOnShutdown = true + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + series2 := labels.FromStrings("foo", "bar2") + + addSamples := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, _, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSamples(250, 350) + + // Add ooo samples that will result into a single block. + addSamples(90, 110) // The sample 110 will not be in m-map chunks. + + // Checking that there are some ooo m-map chunks. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Len(t, ms.ooo.oooMmappedChunks, 2) + require.NotNil(t, ms.ooo.oooHeadChunk) + } + + // Restart DB. + require.NoError(t, db.Close()) + + // For some reason wbl goes missing. + require.NoError(t, os.RemoveAll(path.Join(db.Dir(), "wbl"))) + + db = newTestDB(t, withDir(db.Dir())) + db.DisableCompactions() // We want to manually call it. + + // Check ooo m-map chunks again. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Len(t, ms.ooo.oooMmappedChunks, 2) + require.Equal(t, 109*time.Minute.Milliseconds(), ms.ooo.oooMmappedChunks[1].maxTime) + require.Nil(t, ms.ooo.oooHeadChunk) // Because of missing wbl. + } + + verifySamples := func(fromMins, toMins int64) { + series1Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + series2Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + series1Samples = append(series1Samples, scenario.sampleFunc(ts, ts)) + series2Samples = append(series2Samples, scenario.sampleFunc(ts, ts*2)) + } + expRes := map[string][]chunks.Sample{ + series1.String(): series1Samples, + series2.String(): series2Samples, + } + + q, err := db.Querier(fromMins*time.Minute.Milliseconds(), toMins*time.Minute.Milliseconds()) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + // Checking for expected ooo data from mmap chunks. + verifySamples(90, 109) + + // Compaction should also work fine. + require.Empty(t, db.Blocks()) + require.NoError(t, db.CompactOOOHead(ctx)) + require.Len(t, db.Blocks(), 1) // One block from OOO data. + require.Equal(t, int64(0), db.Blocks()[0].MinTime()) + require.Equal(t, 120*time.Minute.Milliseconds(), db.Blocks()[0].MaxTime()) + + // Checking that ooo chunk is empty in Head. + for _, lbls := range []labels.Labels{series1, series2} { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Nil(t, ms.ooo) + } + + verifySamples(90, 109) +} + +func TestQuerierOOOQuery(t *testing.T) { + scenarios := map[string]struct { + appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) + sampleFunc func(ts int64) chunks.Sample + }{ + "float": { + appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + return app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, f: float64(ts)} + }, + }, + "integer histogram": { + appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + h := tsdbutil.GenerateTestHistogram(ts) + if counterReset { + h.CounterResetHint = histogram.CounterReset + } + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} + }, + }, + "float histogram": { + appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + fh := tsdbutil.GenerateTestFloatHistogram(ts) + if counterReset { + fh.CounterResetHint = histogram.CounterReset + } + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, fh: tsdbutil.GenerateTestFloatHistogram(ts)} + }, + }, + "integer histogram counter resets": { + // Adding counter reset to all histograms means each histogram will have its own chunk. + appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + h := tsdbutil.GenerateTestHistogram(ts) + h.CounterResetHint = histogram.CounterReset // For this scenario, ignore the counterReset argument. + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} + }, + }, + } + + for name, scenario := range scenarios { + t.Run(name, func(t *testing.T) { + testQuerierOOOQuery(t, scenario.appendFunc, scenario.sampleFunc) + }) + } +} + +func testQuerierOOOQuery(t *testing.T, + appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error), + sampleFunc func(ts int64) chunks.Sample, +) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() + + series1 := labels.FromStrings("foo", "bar1") + + type filterFunc func(t int64) bool + defaultFilterFunc := func(int64) bool { return true } + + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + addSample := func(db *DB, fromMins, toMins, queryMinT, queryMaxT int64, expSamples []chunks.Sample, filter filterFunc, counterReset bool) ([]chunks.Sample, int) { + app := db.Appender(context.Background()) + totalAppended := 0 + for m := fromMins; m <= toMins; m += time.Minute.Milliseconds() { + if !filter(m / time.Minute.Milliseconds()) { + continue + } + _, err := appendFunc(app, m, counterReset) + if m >= queryMinT && m <= queryMaxT { + expSamples = append(expSamples, sampleFunc(m)) + } + require.NoError(t, err) + totalAppended++ + } + require.NoError(t, app.Commit()) + require.Positive(t, totalAppended, 0) // Sanity check that filter is not too zealous. + return expSamples, totalAppended + } + + type sampleBatch struct { + minT int64 + maxT int64 + filter filterFunc + counterReset bool + isOOO bool + } + + tests := []struct { + name string + oooCap int64 + queryMinT int64 + queryMaxT int64 + batches []sampleBatch + }{ + { + name: "query interval covering ooomint and inordermaxt returns all ingested samples", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: defaultFilterFunc, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: defaultFilterFunc, + isOOO: true, + }, + }, + }, + { + name: "partial query interval returns only samples within interval", + oooCap: 30, + queryMinT: minutes(20), + queryMaxT: minutes(180), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: defaultFilterFunc, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: defaultFilterFunc, + isOOO: true, + }, + }, + }, + { + name: "alternating OOO batches", // In order: 100-200 normal. out of order first path: 0, 2, 4, ... 98 (no counter reset), second pass: 1, 3, 5, ... 99 (with counter reset). + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: defaultFilterFunc, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: true, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: func(t int64) bool { return t%2 == 1 }, + counterReset: true, + isOOO: true, + }, + }, + }, + { + name: "query overlapping inorder and ooo samples returns all ingested samples at the end of the interval", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(170), + maxT: minutes(180), + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + { + name: "query overlapping inorder and ooo in-memory samples returns all ingested samples at the beginning of the interval", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(100), + maxT: minutes(110), + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + { + name: "query inorder contain ooo mmapped samples returns all ingested samples at the beginning of the interval", + oooCap: 5, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(101), + maxT: minutes(101 + (5-1)*2), // Append samples to fit in a single mmapped OOO chunk and fit inside the first in-order mmapped chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + { + minT: minutes(191), + maxT: minutes(193), // Append some more OOO samples to trigger mapping the OOO chunk, but use time 151 to not overlap with in-order head chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + { + name: "query overlapping inorder and ooo mmapped samples returns all ingested samples at the beginning of the interval", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(101), + maxT: minutes(101 + (30-1)*2), // Append samples to fit in a single mmapped OOO chunk and overlap the first in-order mmapped chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + { + minT: minutes(191), + maxT: minutes(193), // Append some more OOO samples to trigger mapping the OOO chunk, but use time 151 to not overlap with in-order head chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + } + for _, tc := range tests { + t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { + opts.OutOfOrderCapMax = tc.oooCap + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + var expSamples []chunks.Sample + var oooSamples, appendedCount int + + for _, batch := range tc.batches { + expSamples, appendedCount = addSample(db, batch.minT, batch.maxT, tc.queryMinT, tc.queryMaxT, expSamples, batch.filter, batch.counterReset) + if batch.isOOO { + oooSamples += appendedCount + } + } + + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + querier, err := db.Querier(tc.queryMinT, tc.queryMaxT) + require.NoError(t, err) + defer querier.Close() + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + gotSamples := seriesSet[series1.String()] + require.NotNil(t, gotSamples) + require.Len(t, seriesSet, 1) + requireEqualSamples(t, series1.String(), expSamples, gotSamples, requireEqualSamplesIgnoreCounterResets) + requireEqualOOOSamples(t, oooSamples, db) + }) + } +} + +func TestChunkQuerierOOOQuery(t *testing.T) { + nBucketHistogram := func(n int64) *histogram.Histogram { + h := &histogram.Histogram{ + Count: uint64(n), + Sum: float64(n), + } + if n == 0 { + h.PositiveSpans = []histogram.Span{} + h.PositiveBuckets = []int64{} + return h + } + h.PositiveSpans = []histogram.Span{{Offset: 0, Length: uint32(n)}} + h.PositiveBuckets = make([]int64, n) + h.PositiveBuckets[0] = 1 + return h + } + + scenarios := map[string]struct { + appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) + sampleFunc func(ts int64) chunks.Sample + checkInUseBucket bool + }{ + "float": { + appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + return app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, f: float64(ts)} + }, + }, + "integer histogram": { + appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + h := tsdbutil.GenerateTestHistogram(ts) + if counterReset { + h.CounterResetHint = histogram.CounterReset + } + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} + }, + }, + "float histogram": { + appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + fh := tsdbutil.GenerateTestFloatHistogram(ts) + if counterReset { + fh.CounterResetHint = histogram.CounterReset + } + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, fh: tsdbutil.GenerateTestFloatHistogram(ts)} + }, + }, + "integer histogram counter resets": { + // Adding counter reset to all histograms means each histogram will have its own chunk. + appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + h := tsdbutil.GenerateTestHistogram(ts) + h.CounterResetHint = histogram.CounterReset // For this scenario, ignore the counterReset argument. + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + }, + sampleFunc: func(ts int64) chunks.Sample { + return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} + }, + }, + "integer histogram with recode": { + // Histograms have increasing number of buckets so their chunks are recoded. + appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + n := ts / time.Minute.Milliseconds() + return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nBucketHistogram(n), nil) + }, + sampleFunc: func(ts int64) chunks.Sample { + n := ts / time.Minute.Milliseconds() + return sample{t: ts, h: nBucketHistogram(n)} + }, + // Only check in-use buckets for this scenario. + // Recoding adds empty buckets. + checkInUseBucket: true, + }, + } + for name, scenario := range scenarios { + t.Run(name, func(t *testing.T) { + testChunkQuerierOOOQuery(t, scenario.appendFunc, scenario.sampleFunc, scenario.checkInUseBucket) + }) + } +} + +func testChunkQuerierOOOQuery(t *testing.T, + appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error), + sampleFunc func(ts int64) chunks.Sample, + checkInUseBuckets bool, +) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() + + series1 := labels.FromStrings("foo", "bar1") + + type filterFunc func(t int64) bool + defaultFilterFunc := func(int64) bool { return true } + + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + addSample := func(db *DB, fromMins, toMins, queryMinT, queryMaxT int64, expSamples []chunks.Sample, filter filterFunc, counterReset bool) ([]chunks.Sample, int) { + app := db.Appender(context.Background()) + totalAppended := 0 + for m := fromMins; m <= toMins; m += time.Minute.Milliseconds() { + if !filter(m / time.Minute.Milliseconds()) { + continue + } + _, err := appendFunc(app, m, counterReset) + if m >= queryMinT && m <= queryMaxT { + expSamples = append(expSamples, sampleFunc(m)) + } + require.NoError(t, err) + totalAppended++ + } + require.NoError(t, app.Commit()) + require.Positive(t, totalAppended) // Sanity check that filter is not too zealous. + return expSamples, totalAppended + } + + type sampleBatch struct { + minT int64 + maxT int64 + filter filterFunc + counterReset bool + isOOO bool + } + + tests := []struct { + name string + oooCap int64 + queryMinT int64 + queryMaxT int64 + batches []sampleBatch + }{ + { + name: "query interval covering ooomint and inordermaxt returns all ingested samples", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: defaultFilterFunc, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: defaultFilterFunc, + isOOO: true, + }, + }, + }, + { + name: "partial query interval returns only samples within interval", + oooCap: 30, + queryMinT: minutes(20), + queryMaxT: minutes(180), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: defaultFilterFunc, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: defaultFilterFunc, + isOOO: true, + }, + }, + }, + { + name: "alternating OOO batches", // In order: 100-200 normal. out of order first path: 0, 2, 4, ... 98 (no counter reset), second pass: 1, 3, 5, ... 99 (with counter reset). + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: defaultFilterFunc, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: true, + }, + { + minT: minutes(0), + maxT: minutes(99), + filter: func(t int64) bool { return t%2 == 1 }, + counterReset: true, + isOOO: true, + }, + }, + }, + { + name: "query overlapping inorder and ooo samples returns all ingested samples at the end of the interval", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(170), + maxT: minutes(180), + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + { + name: "query overlapping inorder and ooo in-memory samples returns all ingested samples at the beginning of the interval", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(100), + maxT: minutes(110), + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + { + name: "query inorder contain ooo mmapped samples returns all ingested samples at the beginning of the interval", + oooCap: 5, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(101), + maxT: minutes(101 + (5-1)*2), // Append samples to fit in a single mmapped OOO chunk and fit inside the first in-order mmapped chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + { + minT: minutes(191), + maxT: minutes(193), // Append some more OOO samples to trigger mapping the OOO chunk, but use time 151 to not overlap with in-order head chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + { + name: "query overlapping inorder and ooo mmapped samples returns all ingested samples at the beginning of the interval", + oooCap: 30, + queryMinT: minutes(0), + queryMaxT: minutes(200), + batches: []sampleBatch{ + { + minT: minutes(100), + maxT: minutes(200), + filter: func(t int64) bool { return t%2 == 0 }, + isOOO: false, + }, + { + minT: minutes(101), + maxT: minutes(101 + (30-1)*2), // Append samples to fit in a single mmapped OOO chunk and overlap the first in-order mmapped chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + { + minT: minutes(191), + maxT: minutes(193), // Append some more OOO samples to trigger mapping the OOO chunk, but use time 151 to not overlap with in-order head chunk. + filter: func(t int64) bool { return t%2 == 1 }, + isOOO: true, + }, + }, + }, + } + for _, tc := range tests { + t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { + opts.OutOfOrderCapMax = tc.oooCap + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + var expSamples []chunks.Sample + var oooSamples, appendedCount int + + for _, batch := range tc.batches { + expSamples, appendedCount = addSample(db, batch.minT, batch.maxT, tc.queryMinT, tc.queryMaxT, expSamples, batch.filter, batch.counterReset) + if batch.isOOO { + oooSamples += appendedCount + } + } + + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + querier, err := db.ChunkQuerier(tc.queryMinT, tc.queryMaxT) + require.NoError(t, err) + defer querier.Close() + + chks := queryChunks(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + require.NotNil(t, chks[series1.String()]) + require.Len(t, chks, 1) + requireEqualOOOSamples(t, oooSamples, db) + var gotSamples []chunks.Sample + for _, chunk := range chks[series1.String()] { + it := chunk.Chunk.Iterator(nil) + smpls, err := storage.ExpandSamples(it, newSample) + require.NoError(t, err) + + // Verify that no sample is outside the chunk's time range. + for i, s := range smpls { + switch i { + case 0: + require.Equal(t, chunk.MinTime, s.T(), "first sample %v not at chunk min time %v", s, chunk.MinTime) + case len(smpls) - 1: + require.Equal(t, chunk.MaxTime, s.T(), "last sample %v not at chunk max time %v", s, chunk.MaxTime) + default: + require.GreaterOrEqual(t, s.T(), chunk.MinTime, "sample %v before chunk min time %v", s, chunk.MinTime) + require.LessOrEqual(t, s.T(), chunk.MaxTime, "sample %v after chunk max time %v", s, chunk.MaxTime) + } + } + + gotSamples = append(gotSamples, smpls...) + require.NoError(t, it.Err()) + } + if checkInUseBuckets { + requireEqualSamples(t, series1.String(), expSamples, gotSamples, requireEqualSamplesIgnoreCounterResets, requireEqualSamplesInUseBucketCompare) + } else { + requireEqualSamples(t, series1.String(), expSamples, gotSamples, requireEqualSamplesIgnoreCounterResets) + } + }) + } +} + +// TestOOONativeHistogramsWithCounterResets verifies the counter reset headers for in-order and out-of-order samples +// upon ingestion. Note that when the counter reset(s) occur in OOO samples, the header is set to UnknownCounterReset +// rather than CounterReset. This is because with OOO native histogram samples, it cannot be definitely +// determined if a counter reset occurred because the samples are not consecutive, and another sample +// could potentially come in that would change the status of the header. In this case, the UnknownCounterReset +// headers would be re-checked at query time and updated as needed. However, this test is checking the counter +// reset headers at the time of storage. +func TestOOONativeHistogramsWithCounterResets(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + if name == intHistogram || name == floatHistogram { + testOOONativeHistogramsWithCounterResets(t, scenario) + } + }) + } +} + +func testOOONativeHistogramsWithCounterResets(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() + + type resetFunc func(v int64) bool + defaultResetFunc := func(int64) bool { return false } + + lbls := labels.FromStrings("foo", "bar1") + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + + type sampleBatch struct { + from int64 + until int64 + shouldReset resetFunc + expCounterResetHints []histogram.CounterResetHint + } + + tests := []struct { + name string + queryMin int64 + queryMax int64 + batches []sampleBatch + expectedSamples []chunks.Sample + }{ + { + name: "Counter reset within in-order samples", + queryMin: minutes(40), + queryMax: minutes(55), + batches: []sampleBatch{ + // In-order samples + { + from: 40, + until: 50, + shouldReset: func(v int64) bool { + return v == 45 + }, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset}, + }, + }, + }, + { + name: "Counter reset right at beginning of OOO samples", + queryMin: minutes(40), + queryMax: minutes(55), + batches: []sampleBatch{ + // In-order samples + { + from: 40, + until: 45, + shouldReset: defaultResetFunc, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset}, + }, + { + from: 50, + until: 55, + shouldReset: defaultResetFunc, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset}, + }, + // OOO samples + { + from: 45, + until: 50, + shouldReset: func(v int64) bool { + return v == 45 + }, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset}, + }, + }, + }, + { + name: "Counter resets in both in-order and OOO samples", + queryMin: minutes(40), + queryMax: minutes(55), + batches: []sampleBatch{ + // In-order samples + { + from: 40, + until: 45, + shouldReset: func(v int64) bool { + return v == 44 + }, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.UnknownCounterReset}, + }, + { + from: 50, + until: 55, + shouldReset: defaultResetFunc, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset}, + }, + // OOO samples + { + from: 45, + until: 50, + shouldReset: func(v int64) bool { + return v == 49 + }, + expCounterResetHints: []histogram.CounterResetHint{histogram.UnknownCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.NotCounterReset, histogram.UnknownCounterReset}, + }, + }, + }, + } + for _, tc := range tests { + t.Run(fmt.Sprintf("name=%s", tc.name), func(t *testing.T) { + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + app := db.Appender(context.Background()) + + expSamples := make(map[string][]chunks.Sample) + + for _, batch := range tc.batches { + j := batch.from + smplIdx := 0 + for i := batch.from; i < batch.until; i++ { + resetCount := batch.shouldReset(i) + if resetCount { + j = 0 + } + _, s, err := scenario.appendFunc(app, lbls, minutes(i), j) + require.NoError(t, err) + if s.Type() == chunkenc.ValHistogram { + s.H().CounterResetHint = batch.expCounterResetHints[smplIdx] + } else if s.Type() == chunkenc.ValFloatHistogram { + s.FH().CounterResetHint = batch.expCounterResetHints[smplIdx] + } + expSamples[lbls.String()] = append(expSamples[lbls.String()], s) + j++ + smplIdx++ + } + } + + require.NoError(t, app.Commit()) + + for k, v := range expSamples { + sort.Slice(v, func(i, j int) bool { + return v[i].T() < v[j].T() + }) + expSamples[k] = v + } + + querier, err := db.Querier(tc.queryMin, tc.queryMax) + require.NoError(t, err) + defer querier.Close() + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + require.NotNil(t, seriesSet[lbls.String()]) + require.Len(t, seriesSet, 1) + requireEqualSeries(t, expSamples, seriesSet, false) + }) + } +} + +func TestOOOInterleavedImplicitCounterResets(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOInterleavedImplicitCounterResets(t, name, scenario) + }) + } +} + +func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario sampleTypeScenario) { + var appendFunc func(app storage.Appender, ts, v int64) error + + if scenario.sampleType != sampleMetricTypeHistogram { + return + } + + switch name { + case intHistogram: + appendFunc = func(app storage.Appender, ts, v int64) error { + h := &histogram.Histogram{ + Count: uint64(v), + Sum: float64(v), + PositiveSpans: []histogram.Span{{Offset: 0, Length: 1}}, + PositiveBuckets: []int64{v}, + } + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + return err + } + case floatHistogram: + appendFunc = func(app storage.Appender, ts, v int64) error { + fh := &histogram.FloatHistogram{ + Count: float64(v), + Sum: float64(v), + PositiveSpans: []histogram.Span{{Offset: 0, Length: 1}}, + PositiveBuckets: []float64{float64(v)}, + } + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + return err + } + case customBucketsIntHistogram: + appendFunc = func(app storage.Appender, ts, v int64) error { + h := &histogram.Histogram{ + Schema: -53, + Count: uint64(v), + Sum: float64(v), + PositiveSpans: []histogram.Span{{Offset: 0, Length: 1}}, + PositiveBuckets: []int64{v}, + CustomValues: []float64{float64(1), float64(2), float64(3)}, + } + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + return err + } + case customBucketsFloatHistogram: + appendFunc = func(app storage.Appender, ts, v int64) error { + fh := &histogram.FloatHistogram{ + Schema: -53, + Count: float64(v), + Sum: float64(v), + PositiveSpans: []histogram.Span{{Offset: 0, Length: 1}}, + PositiveBuckets: []float64{float64(v)}, + CustomValues: []float64{float64(1), float64(2), float64(3)}, + } + _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + return err + } + case gaugeIntHistogram, gaugeFloatHistogram: + return + } + + // Not a sample, we're encoding an integer counter that we convert to a + // histogram with a single bucket. + type tsValue struct { + ts int64 + v int64 + } + + type expectedTsValue struct { + ts int64 + v int64 + hint histogram.CounterResetHint + } + + type expectedChunk struct { + hint histogram.CounterResetHint + size int + } + + cases := map[string]struct { + samples []tsValue + oooCap int64 + // The expected samples with counter reset. + expectedSamples []expectedTsValue + // The expected counter reset hint for each chunk. + expectedChunks []expectedChunk + }{ + "counter reset in-order cleared by in-memory OOO chunk": { + samples: []tsValue{ + {1, 40}, // New in In-order. I1. + {4, 30}, // In-order counter reset. I2. + {2, 40}, // New in OOO. O1. + {3, 10}, // OOO counter reset. O2. + }, + oooCap: 30, + // Expect all to be set to UnknownCounterReset because we switch between + // in-order and out-of-order samples. + expectedSamples: []expectedTsValue{ + {1, 40, histogram.UnknownCounterReset}, // I1. + {2, 40, histogram.UnknownCounterReset}, // O1. + {3, 10, histogram.UnknownCounterReset}, // O2. + {4, 30, histogram.UnknownCounterReset}, // I2. Counter reset cleared by iterator change. + }, + expectedChunks: []expectedChunk{ + {histogram.UnknownCounterReset, 1}, // I1. + {histogram.UnknownCounterReset, 1}, // O1. + {histogram.UnknownCounterReset, 1}, // O2. + {histogram.UnknownCounterReset, 1}, // I2. + }, + }, + "counter reset in OOO mmapped chunk cleared by in-memory ooo chunk": { + samples: []tsValue{ + {8, 30}, // In-order, new chunk. I1. + {1, 10}, // OOO, new chunk (will be mmapped). MO1. + {2, 20}, // OOO, no reset (will be mmapped). MO1. + {3, 30}, // OOO, no reset (will be mmapped). MO1. + {5, 20}, // OOO, reset (will be mmapped). MO2. + {6, 10}, // OOO, reset (will be mmapped). MO3. + {7, 20}, // OOO, no reset (will be mmapped). MO3. + {4, 10}, // OOO, inserted into memory, triggers mmap. O1. + }, + oooCap: 6, + expectedSamples: []expectedTsValue{ + {1, 10, histogram.UnknownCounterReset}, // MO1. + {2, 20, histogram.NotCounterReset}, // MO1. + {3, 30, histogram.NotCounterReset}, // MO1. + {4, 10, histogram.UnknownCounterReset}, // O1. Counter reset cleared by iterator change. + {5, 20, histogram.UnknownCounterReset}, // MO2. + {6, 10, histogram.UnknownCounterReset}, // MO3. + {7, 20, histogram.NotCounterReset}, // MO3. + {8, 30, histogram.UnknownCounterReset}, // I1. + }, + expectedChunks: []expectedChunk{ + {histogram.UnknownCounterReset, 3}, // MO1. + {histogram.UnknownCounterReset, 1}, // O1. + {histogram.UnknownCounterReset, 1}, // MO2. + {histogram.UnknownCounterReset, 2}, // MO3. + {histogram.UnknownCounterReset, 1}, // I1. + }, + }, + "counter reset in OOO mmapped chunk cleared by another OOO mmapped chunk": { + samples: []tsValue{ + {8, 100}, // In-order, new chunk. I1. + {1, 50}, // OOO, new chunk (will be mmapped). MO1. + {5, 40}, // OOO, reset (will be mmapped). MO2. + {6, 50}, // OOO, no reset (will be mmapped). MO2. + {2, 10}, // OOO, new chunk no reset (will be mmapped). MO3. + {3, 20}, // OOO, no reset (will be mmapped). MO3. + {4, 30}, // OOO, no reset (will be mmapped). MO3. + {7, 60}, // OOO, no reset in memory. O1. + }, + oooCap: 3, + expectedSamples: []expectedTsValue{ + {1, 50, histogram.UnknownCounterReset}, // MO1. + {2, 10, histogram.UnknownCounterReset}, // MO3. + {3, 20, histogram.NotCounterReset}, // MO3. + {4, 30, histogram.NotCounterReset}, // MO3. + {5, 40, histogram.UnknownCounterReset}, // MO2. + {6, 50, histogram.NotCounterReset}, // MO2. + {7, 60, histogram.UnknownCounterReset}, // O1. + {8, 100, histogram.UnknownCounterReset}, // I1. + }, + expectedChunks: []expectedChunk{ + {histogram.UnknownCounterReset, 1}, // MO1. + {histogram.UnknownCounterReset, 3}, // MO3. + {histogram.UnknownCounterReset, 2}, // MO2. + {histogram.UnknownCounterReset, 1}, // O1. + {histogram.UnknownCounterReset, 1}, // I1. + }, + }, + } + + for tcName, tc := range cases { + t.Run(tcName, func(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = tc.oooCap + opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + app := db.Appender(context.Background()) + for _, s := range tc.samples { + require.NoError(t, appendFunc(app, s.ts, s.v)) + } + require.NoError(t, app.Commit()) + + t.Run("querier", func(t *testing.T) { + querier, err := db.Querier(0, 10) + require.NoError(t, err) + defer querier.Close() + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + require.Len(t, seriesSet, 1) + samples, ok := seriesSet["{foo=\"bar1\"}"] + require.True(t, ok) + require.Len(t, samples, len(tc.samples)) + require.Len(t, samples, len(tc.expectedSamples)) + + // We expect all unknown counter resets because we clear the counter reset + // hint when we switch between in-order and out-of-order samples. + for i, s := range samples { + switch name { + case intHistogram: + require.Equal(t, tc.expectedSamples[i].hint, s.H().CounterResetHint, "sample %d", i) + require.Equal(t, tc.expectedSamples[i].v, int64(s.H().Count), "sample %d", i) + case floatHistogram: + require.Equal(t, tc.expectedSamples[i].hint, s.FH().CounterResetHint, "sample %d", i) + require.Equal(t, tc.expectedSamples[i].v, int64(s.FH().Count), "sample %d", i) + case customBucketsIntHistogram: + require.Equal(t, tc.expectedSamples[i].hint, s.H().CounterResetHint, "sample %d", i) + require.Equal(t, tc.expectedSamples[i].v, int64(s.H().Count), "sample %d", i) + case customBucketsFloatHistogram: + require.Equal(t, tc.expectedSamples[i].hint, s.FH().CounterResetHint, "sample %d", i) + require.Equal(t, tc.expectedSamples[i].v, int64(s.FH().Count), "sample %d", i) + default: + t.Fatalf("unexpected sample type %s", name) + } + } + }) + + t.Run("chunk-querier", func(t *testing.T) { + querier, err := db.ChunkQuerier(0, 10) + require.NoError(t, err) + defer querier.Close() + + chunkSet := queryAndExpandChunks(t, querier, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar1")) + require.Len(t, chunkSet, 1) + chunks, ok := chunkSet["{foo=\"bar1\"}"] + require.True(t, ok) + require.Len(t, chunks, len(tc.expectedChunks)) + idx := 0 + for i, samples := range chunks { + require.Len(t, samples, tc.expectedChunks[i].size) + for j, s := range samples { + expectHint := tc.expectedChunks[i].hint + if j > 0 { + expectHint = histogram.NotCounterReset + } + switch name { + case intHistogram: + require.Equal(t, expectHint, s.H().CounterResetHint, "sample %d", idx) + require.Equal(t, tc.expectedSamples[idx].v, int64(s.H().Count), "sample %d", idx) + case floatHistogram: + require.Equal(t, expectHint, s.FH().CounterResetHint, "sample %d", idx) + require.Equal(t, tc.expectedSamples[idx].v, int64(s.FH().Count), "sample %d", idx) + case customBucketsIntHistogram: + require.Equal(t, expectHint, s.H().CounterResetHint, "sample %d", idx) + require.Equal(t, tc.expectedSamples[idx].v, int64(s.H().Count), "sample %d", idx) + case customBucketsFloatHistogram: + require.Equal(t, expectHint, s.FH().CounterResetHint, "sample %d", idx) + require.Equal(t, tc.expectedSamples[idx].v, int64(s.FH().Count), "sample %d", idx) + default: + t.Fatalf("unexpected sample type %s", name) + } + idx++ + } + } + }) + }) + } +} + +func TestOOOAppendAndQuery(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOAppendAndQuery(t, scenario) + }) + } +} + +func testOOOAppendAndQuery(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 4 * time.Hour.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + s1 := labels.FromStrings("foo", "bar1") + s2 := labels.FromStrings("foo", "bar2") + + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + appendedSamples := make(map[string][]chunks.Sample) + totalSamples := 0 + addSample := func(lbls labels.Labels, fromMins, toMins int64, faceError bool) { + app := db.Appender(context.Background()) + key := lbls.String() + from, to := minutes(fromMins), minutes(toMins) + for m := from; m <= to; m += time.Minute.Milliseconds() { + val := rand.Intn(1000) + _, s, err := scenario.appendFunc(app, lbls, m, int64(val)) + if faceError { + require.Error(t, err) + } else { + require.NoError(t, err) + appendedSamples[key] = append(appendedSamples[key], s) + totalSamples++ + } + } + if faceError { + require.NoError(t, app.Rollback()) + } else { + require.NoError(t, app.Commit()) + } + } + + testQuery := func(from, to int64) { + querier, err := db.Querier(from, to) + require.NoError(t, err) + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.")) + + for k, v := range appendedSamples { + sort.Slice(v, func(i, j int) bool { + return v[i].T() < v[j].T() + }) + appendedSamples[k] = v + } + + expSamples := make(map[string][]chunks.Sample) + for k, samples := range appendedSamples { + for _, s := range samples { + if s.T() < from { + continue + } + if s.T() > to { + continue + } + expSamples[k] = append(expSamples[k], s) + } + } + requireEqualSeries(t, expSamples, seriesSet, true) + requireEqualOOOSamples(t, totalSamples-2, db) + } + + verifyOOOMinMaxTimes := func(expMin, expMax int64) { + require.Equal(t, minutes(expMin), db.head.MinOOOTime()) + require.Equal(t, minutes(expMax), db.head.MaxOOOTime()) + } + + // In-order samples. + addSample(s1, 300, 300, false) + addSample(s2, 290, 290, false) + require.Equal(t, float64(2), prom_testutil.ToFloat64(db.head.metrics.chunksCreated)) + testQuery(math.MinInt64, math.MaxInt64) + + // Some ooo samples. + addSample(s1, 250, 260, false) + addSample(s2, 255, 265, false) + verifyOOOMinMaxTimes(250, 265) + testQuery(math.MinInt64, math.MaxInt64) + testQuery(minutes(250), minutes(265)) // Test querying ooo data time range. + testQuery(minutes(290), minutes(300)) // Test querying in-order data time range. + testQuery(minutes(250), minutes(300)) // Test querying the entire range. + + // Out of time window. + addSample(s1, 59, 59, true) + addSample(s2, 49, 49, true) + verifyOOOMinMaxTimes(250, 265) + testQuery(math.MinInt64, math.MaxInt64) + + // At the edge of time window, also it would be "out of bound" without the ooo support. + addSample(s1, 60, 65, false) + verifyOOOMinMaxTimes(60, 265) + testQuery(math.MinInt64, math.MaxInt64) + + // This sample is not within the time window w.r.t. the head's maxt, but it is within the window + // w.r.t. the series' maxt. But we consider only head's maxt. + addSample(s2, 59, 59, true) + verifyOOOMinMaxTimes(60, 265) + testQuery(math.MinInt64, math.MaxInt64) + + // Now the sample is within time window w.r.t. the head's maxt. + addSample(s2, 60, 65, false) + verifyOOOMinMaxTimes(60, 265) + testQuery(math.MinInt64, math.MaxInt64) + + // Out of time window again. + addSample(s1, 59, 59, true) + addSample(s2, 49, 49, true) + testQuery(math.MinInt64, math.MaxInt64) + + // Generating some m-map chunks. The m-map chunks here are in such a way + // that when sorted w.r.t. mint, the last chunk's maxt is not the overall maxt + // of the merged chunk. This tests a bug fixed in https://github.com/grafana/mimir-prometheus/pull/238/. + require.Equal(t, float64(4), prom_testutil.ToFloat64(db.head.metrics.chunksCreated)) + addSample(s1, 180, 249, false) + require.Equal(t, float64(6), prom_testutil.ToFloat64(db.head.metrics.chunksCreated)) + verifyOOOMinMaxTimes(60, 265) + testQuery(math.MinInt64, math.MaxInt64) +} + +func TestOOODisabled(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOODisabled(t, scenario) + }) + } +} + +func testOOODisabled(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 0 + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + s1 := labels.FromStrings("foo", "bar1") + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + expSamples := make(map[string][]chunks.Sample) + totalSamples := 0 + failedSamples := 0 + + addSample := func(db *DB, lbls labels.Labels, fromMins, toMins int64, faceError bool) { + app := db.Appender(context.Background()) + key := lbls.String() + from, to := minutes(fromMins), minutes(toMins) + for m := from; m <= to; m += time.Minute.Milliseconds() { + _, _, err := scenario.appendFunc(app, lbls, m, m) + if faceError { + require.Error(t, err) + failedSamples++ + } else { + require.NoError(t, err) + expSamples[key] = append(expSamples[key], scenario.sampleFunc(m, m)) + totalSamples++ + } + } + if faceError { + require.NoError(t, app.Rollback()) + } else { + require.NoError(t, app.Commit()) + } + } + + addSample(db, s1, 300, 300, false) // In-order samples. + addSample(db, s1, 250, 260, true) // Some ooo samples. + addSample(db, s1, 59, 59, true) // Out of time window. + addSample(db, s1, 60, 65, true) // At the edge of time window, also it would be "out of bound" without the ooo support. + addSample(db, s1, 59, 59, true) // Out of time window again. + addSample(db, s1, 301, 310, false) // More in-order samples. + + querier, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.")) + requireEqualSeries(t, expSamples, seriesSet, true) + requireEqualOOOSamples(t, 0, db) + require.Equal(t, float64(failedSamples), + prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))+prom_testutil.ToFloat64(db.head.metrics.outOfBoundSamples.WithLabelValues(scenario.sampleType)), + "number of ooo/oob samples mismatch") + + // Verifying that no OOO artifacts were generated. + _, err = os.ReadDir(path.Join(db.Dir(), wlog.WblDirName)) + require.True(t, os.IsNotExist(err)) + + ms, created, err := db.head.getOrCreate(s1.Hash(), s1, false) + require.NoError(t, err) + require.False(t, created) + require.NotNil(t, ms) + require.Nil(t, ms.ooo) +} + +func TestWBLAndMmapReplay(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testWBLAndMmapReplay(t, scenario) + }) + } +} + +func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 4 * time.Hour.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + s1 := labels.FromStrings("foo", "bar1") + + minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } + expSamples := make(map[string][]chunks.Sample) + totalSamples := 0 + addSample := func(lbls labels.Labels, fromMins, toMins int64) { + app := db.Appender(context.Background()) + key := lbls.String() + from, to := minutes(fromMins), minutes(toMins) + for m := from; m <= to; m += time.Minute.Milliseconds() { + val := rand.Intn(1000) + _, s, err := scenario.appendFunc(app, lbls, m, int64(val)) + require.NoError(t, err) + expSamples[key] = append(expSamples[key], s) + totalSamples++ + } + require.NoError(t, app.Commit()) + } + + testQuery := func(exp map[string][]chunks.Sample) { + querier, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + seriesSet := query(t, querier, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.")) + + for k, v := range exp { + sort.Slice(v, func(i, j int) bool { + return v[i].T() < v[j].T() + }) + exp[k] = v + } + requireEqualSeries(t, exp, seriesSet, true) + } + + // In-order samples. + addSample(s1, 300, 300) + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.head.metrics.chunksCreated)) + + // Some ooo samples. + addSample(s1, 250, 260) + addSample(s1, 195, 249) // This creates some m-map chunks. + require.Equal(t, float64(4), prom_testutil.ToFloat64(db.head.metrics.chunksCreated)) + testQuery(expSamples) + oooMint, oooMaxt := minutes(195), minutes(260) + + // Collect the samples only present in the ooo m-map chunks. + ms, created, err := db.head.getOrCreate(s1.Hash(), s1, false) + require.False(t, created) + require.NoError(t, err) + var s1MmapSamples []chunks.Sample + for _, mc := range ms.ooo.oooMmappedChunks { + chk, err := db.head.chunkDiskMapper.Chunk(mc.ref) + require.NoError(t, err) + it := chk.Iterator(nil) + smpls, err := storage.ExpandSamples(it, newSample) + require.NoError(t, err) + s1MmapSamples = append(s1MmapSamples, smpls...) + } + require.NotEmpty(t, s1MmapSamples) + + require.NoError(t, db.Close()) + + // Making a copy of original state of WBL and Mmap files to use it later. + mmapDir := mmappedChunksDir(db.head.opts.ChunkDirRoot) + wblDir := db.head.wbl.Dir() + originalWblDir := filepath.Join(t.TempDir(), "original_wbl") + originalMmapDir := filepath.Join(t.TempDir(), "original_mmap") + require.NoError(t, fileutil.CopyDirs(wblDir, originalWblDir)) + require.NoError(t, fileutil.CopyDirs(mmapDir, originalMmapDir)) + resetWBLToOriginal := func() { + require.NoError(t, os.RemoveAll(wblDir)) + require.NoError(t, fileutil.CopyDirs(originalWblDir, wblDir)) + } + resetMmapToOriginal := func() { + require.NoError(t, os.RemoveAll(mmapDir)) + require.NoError(t, fileutil.CopyDirs(originalMmapDir, mmapDir)) + } + + t.Run("Restart DB with both WBL and M-map files for ooo data", func(t *testing.T) { + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.Equal(t, oooMint, db.head.MinOOOTime()) + require.Equal(t, oooMaxt, db.head.MaxOOOTime()) + testQuery(expSamples) + }) + + t.Run("Restart DB with only WBL for ooo data", func(t *testing.T) { + require.NoError(t, os.RemoveAll(mmapDir)) + + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.Equal(t, oooMint, db.head.MinOOOTime()) + require.Equal(t, oooMaxt, db.head.MaxOOOTime()) + testQuery(expSamples) + }) + + t.Run("Restart DB with only M-map files for ooo data", func(t *testing.T) { + require.NoError(t, os.RemoveAll(wblDir)) + resetMmapToOriginal() + + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.Equal(t, oooMint, db.head.MinOOOTime()) + require.Equal(t, oooMaxt, db.head.MaxOOOTime()) + inOrderSample := expSamples[s1.String()][len(expSamples[s1.String()])-1] + testQuery(map[string][]chunks.Sample{ + s1.String(): append(s1MmapSamples, inOrderSample), + }) + }) + + t.Run("Restart DB with WBL+Mmap while increasing the OOOCapMax", func(t *testing.T) { + resetWBLToOriginal() + resetMmapToOriginal() + + opts.OutOfOrderCapMax = 60 + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, oooMint, db.head.MinOOOTime()) + require.Equal(t, oooMaxt, db.head.MaxOOOTime()) + testQuery(expSamples) + }) + + t.Run("Restart DB with WBL+Mmap while decreasing the OOOCapMax", func(t *testing.T) { + resetMmapToOriginal() // We need to reset because new duplicate chunks can be written above. + + opts.OutOfOrderCapMax = 10 + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, oooMint, db.head.MinOOOTime()) + require.Equal(t, oooMaxt, db.head.MaxOOOTime()) + testQuery(expSamples) + }) + + t.Run("Restart DB with WBL+Mmap while having no m-map markers in WBL", func(t *testing.T) { + resetMmapToOriginal() // We neet to reset because new duplicate chunks can be written above. + + // Removing m-map markers in WBL by rewriting it. + newWbl, err := wlog.New(promslog.NewNopLogger(), nil, filepath.Join(t.TempDir(), "new_wbl"), compression.None) + require.NoError(t, err) + sr, err := wlog.NewSegmentsReader(originalWblDir) + require.NoError(t, err) + dec := record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + r, markers, addedRecs := wlog.NewReader(sr), 0, 0 + for r.Next() { + rec := r.Record() + if dec.Type(rec) == record.MmapMarkers { + markers++ + continue + } + addedRecs++ + require.NoError(t, newWbl.Log(rec)) + } + require.Positive(t, markers) + require.Positive(t, addedRecs) + require.NoError(t, newWbl.Close()) + require.NoError(t, sr.Close()) + require.NoError(t, os.RemoveAll(wblDir)) + require.NoError(t, os.Rename(newWbl.Dir(), wblDir)) + + opts.OutOfOrderCapMax = 30 + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, oooMint, db.head.MinOOOTime()) + require.Equal(t, oooMaxt, db.head.MaxOOOTime()) + testQuery(expSamples) + }) +} + +func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { + for _, floatHistogram := range []bool{false, true} { + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 500 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + series2 := labels.FromStrings("foo", "bar2") + + var series1ExpSamplesPreCompact, series2ExpSamplesPreCompact, series1ExpSamplesPostCompact, series2ExpSamplesPostCompact []chunks.Sample + + addSample := func(ts int64, l labels.Labels, val int, hint histogram.CounterResetHint) sample { + app := db.Appender(context.Background()) + tsMs := ts * time.Minute.Milliseconds() + if floatHistogram { + h := tsdbutil.GenerateTestFloatHistogram(int64(val)) + h.CounterResetHint = hint + _, err := app.AppendHistogram(0, l, tsMs, nil, h) + require.NoError(t, err) + require.NoError(t, app.Commit()) + return sample{t: tsMs, fh: h.Copy()} + } + + h := tsdbutil.GenerateTestHistogram(int64(val)) + h.CounterResetHint = hint + _, err := app.AppendHistogram(0, l, tsMs, h, nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + return sample{t: tsMs, h: h.Copy()} + } + + // Add an in-order sample to each series. + s := addSample(520, series1, 1000000, histogram.UnknownCounterReset) + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + + s = addSample(520, series2, 1000000, histogram.UnknownCounterReset) + series2ExpSamplesPreCompact = append(series2ExpSamplesPreCompact, s) + series2ExpSamplesPostCompact = append(series2ExpSamplesPostCompact, s) + + // Verify that the in-memory ooo chunk is empty. + checkEmptyOOOChunk := func(lbls labels.Labels) { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Nil(t, ms.ooo) + } + + checkEmptyOOOChunk(series1) + checkEmptyOOOChunk(series2) + + // Add samples for series1. There are three head chunks that will be created: + // Chunk 1 - Samples between 100 - 440. One explicit counter reset at ts 250. + // Chunk 2 - Samples between 105 - 395. Overlaps with Chunk 1. One detected counter reset at ts 165. + // Chunk 3 - Samples between 480 - 509. All within one block boundary. One detected counter reset at 490. + + // Chunk 1. + // First add 10 samples. + for i := 100; i < 200; i += 10 { + s = addSample(int64(i), series1, 100000+i, histogram.UnknownCounterReset) + // Before compaction, all the samples have UnknownCounterReset even though they've been added to the same + // chunk. This is because they overlap with the samples from chunk two and when merging two chunks on read, + // the header is set as unknown when the next sample is not in the same chunk as the previous one. + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + // After compaction, samples from multiple mmapped chunks will be merged, so there won't be any overlapping + // chunks. Therefore, most samples will have the NotCounterReset header. + // 100 is the first sample in the first chunk in the blocks, so is still set to UnknownCounterReset. + // 120 is a block boundary - after compaction, 120 will be the first sample in a chunk, so is still set to + // UnknownCounterReset. + if i > 100 && i != 120 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + } + // Explicit counter reset - the counter reset header is set to CounterReset but the value is higher + // than for the previous timestamp. Explicit counter reset headers are actually ignored though, so when reading + // the sample back you actually get unknown/not counter reset. This is as the chainSampleIterator ignores + // existing headers and sets the header as UnknownCounterReset if the next sample is not in the same chunk as + // the previous one, and counter resets always create a new chunk. + // This case has been added to document what's happening, though it might not be the ideal behavior. + s = addSample(250, series1, 100000+250, histogram.CounterReset) + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, copyWithCounterReset(s, histogram.UnknownCounterReset)) + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, copyWithCounterReset(s, histogram.NotCounterReset)) + + // Add 19 more samples to complete a chunk. + for i := 260; i < 450; i += 10 { + s = addSample(int64(i), series1, 100000+i, histogram.UnknownCounterReset) + // The samples with timestamp less than 410 overlap with the samples from chunk 2, so before compaction, + // they're all UnknownCounterReset. Samples greater than or equal to 410 don't overlap with other chunks + // so they're always detected as NotCounterReset pre and post compaction. + if i >= 410 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + // + // 360 is a block boundary, so after compaction its header is still UnknownCounterReset. + if i != 360 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + } + + // Chunk 2. + // Add six OOO samples. + for i := 105; i < 165; i += 10 { + s = addSample(int64(i), series1, 100000+i, histogram.UnknownCounterReset) + // Samples overlap with chunk 1 so before compaction all headers are UnknownCounterReset. + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, copyWithCounterReset(s, histogram.NotCounterReset)) + } + + // Add sample that will be detected as a counter reset. + s = addSample(165, series1, 100000, histogram.UnknownCounterReset) + // Before compaction, sample has an UnknownCounterReset header due to the chainSampleIterator. + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + // After compaction, the sample's counter reset is still UnknownCounterReset as we cannot trust CounterReset + // headers in chunks at the moment, so when reading the first sample in a chunk, its hint is set to + // UnknownCounterReset. + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + + // Add 23 more samples to complete a chunk. + for i := 175; i < 405; i += 10 { + s = addSample(int64(i), series1, 100000+i, histogram.UnknownCounterReset) + // Samples between 205-255 overlap with chunk 1 so before compaction those samples will have the + // UnknownCounterReset header. + if i >= 205 && i < 255 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + // 245 is the first sample >= the block boundary at 240, so it's still UnknownCounterReset after compaction. + if i != 245 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } else { + s = copyWithCounterReset(s, histogram.UnknownCounterReset) + } + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + } + + // Chunk 3. + for i := 480; i < 490; i++ { + s = addSample(int64(i), series1, 100000+i, histogram.UnknownCounterReset) + // No overlapping samples in other chunks, so all other samples will already be detected as NotCounterReset + // before compaction. + if i > 480 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + // 480 is block boundary. + if i == 480 { + s = copyWithCounterReset(s, histogram.UnknownCounterReset) + } + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + } + // Counter reset. + s = addSample(int64(490), series1, 100000, histogram.UnknownCounterReset) + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + // Add some more samples after the counter reset. + for i := 491; i < 510; i++ { + s = addSample(int64(i), series1, 100000+i, histogram.UnknownCounterReset) + s = copyWithCounterReset(s, histogram.NotCounterReset) + series1ExpSamplesPreCompact = append(series1ExpSamplesPreCompact, s) + series1ExpSamplesPostCompact = append(series1ExpSamplesPostCompact, s) + } + + // Add samples for series2 - one chunk with one detected counter reset at 300. + for i := 200; i < 300; i += 10 { + s = addSample(int64(i), series2, 100000+i, histogram.UnknownCounterReset) + if i > 200 { + s = copyWithCounterReset(s, histogram.NotCounterReset) + } + series2ExpSamplesPreCompact = append(series2ExpSamplesPreCompact, s) + if i == 240 { + s = copyWithCounterReset(s, histogram.UnknownCounterReset) + } + series2ExpSamplesPostCompact = append(series2ExpSamplesPostCompact, s) + } + // Counter reset. + s = addSample(int64(300), series2, 100000, histogram.UnknownCounterReset) + series2ExpSamplesPreCompact = append(series2ExpSamplesPreCompact, s) + series2ExpSamplesPostCompact = append(series2ExpSamplesPostCompact, s) + // Add some more samples after the counter reset. + for i := 310; i < 500; i += 10 { + s := addSample(int64(i), series2, 100000+i, histogram.UnknownCounterReset) + s = copyWithCounterReset(s, histogram.NotCounterReset) + series2ExpSamplesPreCompact = append(series2ExpSamplesPreCompact, s) + // 360 and 480 are block boundaries. + if i == 360 || i == 480 { + s = copyWithCounterReset(s, histogram.UnknownCounterReset) + } + series2ExpSamplesPostCompact = append(series2ExpSamplesPostCompact, s) + } + + // Sort samples (as OOO samples not added in time-order). + sort.Slice(series1ExpSamplesPreCompact, func(i, j int) bool { + return series1ExpSamplesPreCompact[i].T() < series1ExpSamplesPreCompact[j].T() + }) + sort.Slice(series1ExpSamplesPostCompact, func(i, j int) bool { + return series1ExpSamplesPostCompact[i].T() < series1ExpSamplesPostCompact[j].T() + }) + sort.Slice(series2ExpSamplesPreCompact, func(i, j int) bool { + return series2ExpSamplesPreCompact[i].T() < series2ExpSamplesPreCompact[j].T() + }) + sort.Slice(series2ExpSamplesPostCompact, func(i, j int) bool { + return series2ExpSamplesPostCompact[i].T() < series2ExpSamplesPostCompact[j].T() + }) + + verifyDBSamples := func(s1Samples, s2Samples []chunks.Sample) { + expRes := map[string][]chunks.Sample{ + series1.String(): s1Samples, + series2.String(): s2Samples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, false) + } + + // Verify DB samples before compaction. + verifyDBSamples(series1ExpSamplesPreCompact, series2ExpSamplesPreCompact) + + // Verify that the in-memory ooo chunk is not empty. + checkNonEmptyOOOChunk := func(lbls labels.Labels) { + ms, created, err := db.head.getOrCreate(lbls.Hash(), lbls, false) + require.NoError(t, err) + require.False(t, created) + require.Positive(t, ms.ooo.oooHeadChunk.chunk.NumSamples()) + } + + checkNonEmptyOOOChunk(series1) + checkNonEmptyOOOChunk(series2) + + // No blocks before compaction. + require.Empty(t, db.Blocks()) + + // There is a 0th WBL file. + require.NoError(t, db.head.wbl.Sync()) // syncing to make sure wbl is flushed in windows + files, err := os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "00000000", files[0].Name()) + f, err := files[0].Info() + require.NoError(t, err) + require.Greater(t, f.Size(), int64(100)) + + // OOO compaction happens here. + require.NoError(t, db.CompactOOOHead(ctx)) + + // Check that blocks are created after compaction. + require.Len(t, db.Blocks(), 5) + + // Check samples after compaction. + verifyDBSamples(series1ExpSamplesPostCompact, series2ExpSamplesPostCompact) + + // 0th WBL file will be deleted and 1st will be the only present. + files, err = os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "00000001", files[0].Name()) + f, err = files[0].Info() + require.NoError(t, err) + require.Equal(t, int64(0), f.Size()) + + // OOO stuff should not be present in the Head now. + checkEmptyOOOChunk(series1) + checkEmptyOOOChunk(series2) + + verifyBlockSamples := func(block *Block, fromMins, toMins int64) { + var series1Samples, series2Samples []chunks.Sample + + for _, s := range series1ExpSamplesPostCompact { + if s.T() >= fromMins*time.Minute.Milliseconds() { + // Samples should be sorted, so break out of loop when we reach a timestamp that's too big. + if s.T() > toMins*time.Minute.Milliseconds() { + break + } + series1Samples = append(series1Samples, s) + } + } + for _, s := range series2ExpSamplesPostCompact { + if s.T() >= fromMins*time.Minute.Milliseconds() { + // Samples should be sorted, so break out of loop when we reach a timestamp that's too big. + if s.T() > toMins*time.Minute.Milliseconds() { + break + } + series2Samples = append(series2Samples, s) + } + } + + expRes := map[string][]chunks.Sample{} + if len(series1Samples) != 0 { + expRes[series1.String()] = series1Samples + } + if len(series2Samples) != 0 { + expRes[series2.String()] = series2Samples + } + + q, err := NewBlockQuerier(block, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, false) + } + + // Checking for expected data in the blocks. + verifyBlockSamples(db.Blocks()[0], 100, 119) + verifyBlockSamples(db.Blocks()[1], 120, 239) + verifyBlockSamples(db.Blocks()[2], 240, 359) + verifyBlockSamples(db.Blocks()[3], 360, 479) + verifyBlockSamples(db.Blocks()[4], 480, 509) + + // There should be a single m-map file. + mmapDir := mmappedChunksDir(db.head.opts.ChunkDirRoot) + files, err = os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, 1) + + // Compact the in-order head and expect another block. + // Since this is a forced compaction, this block is not aligned with 2h. + err = db.CompactHead(NewRangeHead(db.head, 500*time.Minute.Milliseconds(), 550*time.Minute.Milliseconds())) + require.NoError(t, err) + require.Len(t, db.Blocks(), 6) + verifyBlockSamples(db.Blocks()[5], 520, 520) + + // Blocks created out of normal and OOO head now. But not merged. + verifyDBSamples(series1ExpSamplesPostCompact, series2ExpSamplesPostCompact) + + // The compaction also clears out the old m-map files. Including + // the file that has ooo chunks. + files, err = os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "000001", files[0].Name()) + + // This will merge overlapping block. + require.NoError(t, db.Compact(ctx)) + + require.Len(t, db.Blocks(), 5) + verifyBlockSamples(db.Blocks()[0], 100, 119) + verifyBlockSamples(db.Blocks()[1], 120, 239) + verifyBlockSamples(db.Blocks()[2], 240, 359) + verifyBlockSamples(db.Blocks()[3], 360, 479) + verifyBlockSamples(db.Blocks()[4], 480, 520) // Merged block. + + // Final state. Blocks from normal and OOO head are merged. + verifyDBSamples(series1ExpSamplesPostCompact, series2ExpSamplesPostCompact) + } +} + +func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing.T) { + for _, floatHistogram := range []bool{false, true} { + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 500 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + + addSample := func(ts int64, l labels.Labels, val int) sample { + app := db.Appender(context.Background()) + tsMs := ts + if floatHistogram { + h := tsdbutil.GenerateTestFloatHistogram(int64(val)) + _, err := app.AppendHistogram(0, l, tsMs, nil, h) + require.NoError(t, err) + require.NoError(t, app.Commit()) + return sample{t: tsMs, fh: h.Copy()} + } + + h := tsdbutil.GenerateTestHistogram(int64(val)) + _, err := app.AppendHistogram(0, l, tsMs, h, nil) + require.NoError(t, err) + require.NoError(t, app.Commit()) + return sample{t: tsMs, h: h.Copy()} + } + + var expSamples []chunks.Sample + + s := addSample(0, series1, 0) + expSamples = append(expSamples, s) + s = addSample(1, series1, 10) + expSamples = append(expSamples, copyWithCounterReset(s, histogram.NotCounterReset)) + s = addSample(3, series1, 3) + expSamples = append(expSamples, copyWithCounterReset(s, histogram.UnknownCounterReset)) + s = addSample(2, series1, 0) + expSamples = append(expSamples, copyWithCounterReset(s, histogram.UnknownCounterReset)) + + // Sort samples (as OOO samples not added in time-order). + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + verifyDBSamples := func(s1Samples []chunks.Sample) { + t.Helper() + expRes := map[string][]chunks.Sample{ + series1.String(): s1Samples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, false) + } + + // Verify DB samples before compaction. + verifyDBSamples(expSamples) + + require.NoError(t, db.CompactOOOHead(ctx)) + + // Check samples after OOO compaction. + verifyDBSamples(expSamples) + + // Checking for expected data in the blocks. + // Check that blocks are created after compaction. + require.Len(t, db.Blocks(), 1) + + // Compact the in-order head and expect another block. + // Since this is a forced compaction, this block is not aligned with 2h. + require.NoError(t, db.CompactHead(NewRangeHead(db.head, 0, 3))) + require.Len(t, db.Blocks(), 2) + + // Blocks created out of normal and OOO head now. But not merged. + verifyDBSamples(expSamples) + + // This will merge overlapping block. + require.NoError(t, db.Compact(ctx)) + + require.Len(t, db.Blocks(), 1) + + // Final state. Blocks from normal and OOO head are merged. + verifyDBSamples(expSamples) + } +} + +func copyWithCounterReset(s sample, hint histogram.CounterResetHint) sample { + if s.h != nil { + h := s.h.Copy() + h.CounterResetHint = hint + return sample{t: s.t, h: h} + } + + h := s.fh.Copy() + h.CounterResetHint = hint + return sample{t: s.t, fh: h} +} + +func TestOOOCompactionFailure(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOCompactionFailure(t, scenario) + }) + } +} + +func testOOOCompactionFailure(t *testing.T, scenario sampleTypeScenario) { + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() // We want to manually call it. + + series1 := labels.FromStrings("foo", "bar1") + + addSample := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, _, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSample(250, 350) + + // Add ooo samples that creates multiple chunks. + addSample(90, 310) + + // No blocks before compaction. + require.Empty(t, db.Blocks()) + + // There is a 0th WBL file. + verifyFirstWBLFileIs0 := func(count int) { + require.NoError(t, db.head.wbl.Sync()) // Syncing to make sure wbl is flushed in windows. + files, err := os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, count) + require.Equal(t, "00000000", files[0].Name()) + f, err := files[0].Info() + require.NoError(t, err) + require.Greater(t, f.Size(), int64(100)) + } + verifyFirstWBLFileIs0(1) + + verifyMmapFiles := func(exp ...string) { + mmapDir := mmappedChunksDir(db.head.opts.ChunkDirRoot) + files, err := os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, len(exp)) + for i, f := range files { + require.Equal(t, exp[i], f.Name()) + } + } + + verifyMmapFiles("000001") + + // OOO compaction fails 5 times. + originalCompactor := db.compactor + db.compactor = &mockCompactorFailing{t: t} + for range 5 { + require.Error(t, db.CompactOOOHead(ctx)) + } + require.Empty(t, db.Blocks()) + + // M-map files don't change after failed compaction. + verifyMmapFiles("000001") + + // Because of 5 compaction attempts, there are 6 files now. + verifyFirstWBLFileIs0(6) + + db.compactor = originalCompactor + require.NoError(t, db.CompactOOOHead(ctx)) + oldBlocks := db.Blocks() + require.Len(t, db.Blocks(), 3) + + // Check that the ooo chunks were removed. + ms, created, err := db.head.getOrCreate(series1.Hash(), series1, false) + require.NoError(t, err) + require.False(t, created) + require.Nil(t, ms.ooo) + + // The failed compaction should not have left the ooo Head corrupted. + // Hence, expect no new blocks with another OOO compaction call. + require.NoError(t, db.CompactOOOHead(ctx)) + require.Len(t, db.Blocks(), 3) + require.Equal(t, oldBlocks, db.Blocks()) + + // There should be a single m-map file. + verifyMmapFiles("000001") + + // All but last WBL file will be deleted. + // 8 files in total (starting at 0) because of 7 compaction calls. + files, err := os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 1) + require.Equal(t, "00000007", files[0].Name()) + f, err := files[0].Info() + require.NoError(t, err) + require.Equal(t, int64(0), f.Size()) + + verifySamples := func(block *Block, fromMins, toMins int64) { + series1Samples := make([]chunks.Sample, 0, toMins-fromMins+1) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + series1Samples = append(series1Samples, scenario.sampleFunc(ts, ts)) + } + expRes := map[string][]chunks.Sample{ + series1.String(): series1Samples, + } + + q, err := NewBlockQuerier(block, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + // Checking for expected data in the blocks. + verifySamples(db.Blocks()[0], 90, 119) + verifySamples(db.Blocks()[1], 120, 239) + verifySamples(db.Blocks()[2], 240, 310) + + // Compact the in-order head and expect another block. + // Since this is a forced compaction, this block is not aligned with 2h. + err = db.CompactHead(NewRangeHead(db.head, 250*time.Minute.Milliseconds(), 350*time.Minute.Milliseconds())) + require.NoError(t, err) + require.Len(t, db.Blocks(), 4) // [0, 120), [120, 240), [240, 360), [250, 351) + verifySamples(db.Blocks()[3], 250, 350) + + // The compaction also clears out the old m-map files. Including + // the file that has ooo chunks. + verifyMmapFiles("000001") +} + +func TestWBLCorruption(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 30 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + + series1 := labels.FromStrings("foo", "bar1") + var allSamples, expAfterRestart []chunks.Sample + addSamples := func(fromMins, toMins int64, afterRestart bool) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, err := app.Append(0, series1, ts, float64(ts)) + require.NoError(t, err) + allSamples = append(allSamples, sample{t: ts, f: float64(ts)}) + if afterRestart { + expAfterRestart = append(expAfterRestart, sample{t: ts, f: float64(ts)}) + } + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSamples(340, 350, true) + + // OOO samples. + addSamples(90, 99, true) + addSamples(100, 119, true) + addSamples(120, 130, true) + + // Moving onto the second file. + _, err := db.head.wbl.NextSegment() + require.NoError(t, err) + + // More OOO samples. + addSamples(200, 230, true) + addSamples(240, 255, true) + + // We corrupt WBL after the sample at 255. So everything added later + // should be deleted after replay. + + // Checking where we corrupt it. + require.NoError(t, db.head.wbl.Sync()) // Syncing to make sure wbl is flushed in windows. + files, err := os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 2) + f1, err := files[1].Info() + require.NoError(t, err) + corruptIndex := f1.Size() + corruptFilePath := path.Join(db.head.wbl.Dir(), files[1].Name()) + + // Corrupt the WBL by adding a malformed record. + require.NoError(t, db.head.wbl.Log([]byte{byte(record.Samples), 99, 9, 99, 9, 99, 9, 99})) + + // More samples after the corruption point. + addSamples(260, 280, false) + addSamples(290, 300, false) + + // Another file. + _, err = db.head.wbl.NextSegment() + require.NoError(t, err) + + addSamples(310, 320, false) + + // Verifying that we have data after corruption point. + require.NoError(t, db.head.wbl.Sync()) // Syncing to make sure wbl is flushed in windows. + files, err = os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 3) + f1, err = files[1].Info() + require.NoError(t, err) + require.Greater(t, f1.Size(), corruptIndex) + f0, err := files[0].Info() + require.NoError(t, err) + require.Greater(t, f0.Size(), int64(100)) + f2, err := files[2].Info() + require.NoError(t, err) + require.Greater(t, f2.Size(), int64(100)) + + verifySamples := func(expSamples []chunks.Sample) { + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + expRes := map[string][]chunks.Sample{ + series1.String(): expSamples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + require.Equal(t, expRes, actRes) + } + + verifySamples(allSamples) + + require.NoError(t, db.Close()) + + // We want everything to be replayed from the WBL. So we delete the m-map files. + require.NoError(t, os.RemoveAll(mmappedChunksDir(db.head.opts.ChunkDirRoot))) + + // Restart does the replay and repair. + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) + require.Less(t, len(expAfterRestart), len(allSamples)) + verifySamples(expAfterRestart) + + // Verify that it did the repair on disk. + files, err = os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 3) + f0, err = files[0].Info() + require.NoError(t, err) + require.Greater(t, f0.Size(), int64(100)) + f2, err = files[2].Info() + require.NoError(t, err) + require.Equal(t, int64(0), f2.Size()) + require.Equal(t, corruptFilePath, path.Join(db.head.wbl.Dir(), files[1].Name())) + + // Verifying that everything after the corruption point is set to 0. + b, err := os.ReadFile(corruptFilePath) + require.NoError(t, err) + sum := 0 + for _, val := range b[corruptIndex:] { + sum += int(val) + } + require.Equal(t, 0, sum) + + // Another restart, everything normal with no repair. + require.NoError(t, db.Close()) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) + verifySamples(expAfterRestart) +} + +func TestOOOMmapCorruption(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOOOMmapCorruption(t, scenario) + }) + } +} + +func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderCapMax = 10 + opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + + series1 := labels.FromStrings("foo", "bar1") + var allSamples, expInMmapChunks []chunks.Sample + addSamples := func(fromMins, toMins int64, inMmapAfterCorruption bool) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, s, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + allSamples = append(allSamples, s) + if inMmapAfterCorruption { + expInMmapChunks = append(expInMmapChunks, s) + } + } + require.NoError(t, app.Commit()) + } + + // Add an in-order samples. + addSamples(340, 350, true) + + // OOO samples. + addSamples(90, 99, true) + addSamples(100, 109, true) + // This sample m-maps a chunk. But 120 goes into a new chunk. + addSamples(120, 120, false) + + // Second m-map file. We will corrupt this file. Sample 120 goes into this new file. + db.head.chunkDiskMapper.CutNewFile() + + // More OOO samples. + addSamples(200, 230, false) + addSamples(240, 255, false) + + db.head.chunkDiskMapper.CutNewFile() + addSamples(260, 290, false) + + verifySamples := func(expSamples []chunks.Sample) { + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + expRes := map[string][]chunks.Sample{ + series1.String(): expSamples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + verifySamples(allSamples) + + // Verifying existing files. + mmapDir := mmappedChunksDir(db.head.opts.ChunkDirRoot) + files, err := os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, 3) + + // Corrupting the 2nd file. + f, err := os.OpenFile(path.Join(mmapDir, files[1].Name()), os.O_RDWR, 0o666) + require.NoError(t, err) + _, err = f.WriteAt([]byte{99, 9, 99, 9, 99}, 20) + require.NoError(t, err) + require.NoError(t, f.Close()) + firstFileName := files[0].Name() + + require.NoError(t, db.Close()) + + // Moving OOO WBL to use it later. + wblDir := db.head.wbl.Dir() + wblDirTmp := path.Join(t.TempDir(), "wbl_tmp") + require.NoError(t, os.Rename(wblDir, wblDirTmp)) + + // Restart does the replay and repair of m-map files. + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.mmapChunkCorruptionTotal)) + require.Less(t, len(expInMmapChunks), len(allSamples)) + + // Since there is no WBL, only samples from m-map chunks comes in the query. + verifySamples(expInMmapChunks) + + // Verify that it did the repair on disk. All files from the point of corruption + // should be deleted. + files, err = os.ReadDir(mmapDir) + require.NoError(t, err) + require.Len(t, files, 1) + f0, err := files[0].Info() + require.NoError(t, err) + require.Greater(t, f0.Size(), int64(100)) + require.Equal(t, firstFileName, files[0].Name()) + + // Another restart, everything normal with no repair. + require.NoError(t, db.Close()) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.mmapChunkCorruptionTotal)) + verifySamples(expInMmapChunks) + + // Restart again with the WBL, all samples should be present now. + require.NoError(t, db.Close()) + require.NoError(t, os.RemoveAll(wblDir)) + require.NoError(t, os.Rename(wblDirTmp, wblDir)) + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + require.NoError(t, err) + verifySamples(allSamples) +} + +func TestOutOfOrderRuntimeConfig(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testOutOfOrderRuntimeConfig(t, scenario) + }) + } +} + +func testOutOfOrderRuntimeConfig(t *testing.T, scenario sampleTypeScenario) { + ctx := context.Background() + + getDB := func(oooTimeWindow int64) *DB { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = oooTimeWindow + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + return db + } + + makeConfig := func(oooTimeWindow int) *config.Config { + return &config.Config{ + StorageConfig: config.StorageConfig{ + TSDBConfig: &config.TSDBConfig{ + OutOfOrderTimeWindow: int64(oooTimeWindow) * time.Minute.Milliseconds(), + }, + }, + } + } + + series1 := labels.FromStrings("foo", "bar1") + addSamples := func(t *testing.T, db *DB, fromMins, toMins int64, success bool, allSamples []chunks.Sample) []chunks.Sample { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, s, err := scenario.appendFunc(app, series1, ts, ts) + if success { + require.NoError(t, err) + allSamples = append(allSamples, s) + } else { + require.Error(t, err) + } + } + require.NoError(t, app.Commit()) + return allSamples + } + + verifySamples := func(t *testing.T, db *DB, expSamples []chunks.Sample) { + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + expRes := map[string][]chunks.Sample{ + series1.String(): expSamples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + doOOOCompaction := func(t *testing.T, db *DB) { + // WBL is not empty. + size, err := db.head.wbl.Size() + require.NoError(t, err) + require.Positive(t, size) + + require.Empty(t, db.Blocks()) + require.NoError(t, db.compactOOOHead(ctx)) + require.NotEmpty(t, db.Blocks()) + + // WBL is empty. + size, err = db.head.wbl.Size() + require.NoError(t, err) + require.Equal(t, int64(0), size) + } + + t.Run("increase time window", func(t *testing.T) { + var allSamples []chunks.Sample + db := getDB(30 * time.Minute.Milliseconds()) + + // In-order. + allSamples = addSamples(t, db, 300, 310, true, allSamples) + + // OOO upto 30m old is success. + allSamples = addSamples(t, db, 281, 290, true, allSamples) + + // OOO of 59m old fails. + s := addSamples(t, db, 251, 260, false, nil) + require.Empty(t, s) + verifySamples(t, db, allSamples) + + oldWblPtr := fmt.Sprintf("%p", db.head.wbl) + + // Increase time window and try adding again. + err := db.ApplyConfig(makeConfig(60)) + require.NoError(t, err) + allSamples = addSamples(t, db, 251, 260, true, allSamples) + + // WBL does not change. + newWblPtr := fmt.Sprintf("%p", db.head.wbl) + require.Equal(t, oldWblPtr, newWblPtr) + + doOOOCompaction(t, db) + verifySamples(t, db, allSamples) + }) + + t.Run("decrease time window and increase again", func(t *testing.T) { + var allSamples []chunks.Sample + db := getDB(60 * time.Minute.Milliseconds()) + + // In-order. + allSamples = addSamples(t, db, 300, 310, true, allSamples) + + // OOO upto 59m old is success. + allSamples = addSamples(t, db, 251, 260, true, allSamples) + + oldWblPtr := fmt.Sprintf("%p", db.head.wbl) + // Decrease time window. + err := db.ApplyConfig(makeConfig(30)) + require.NoError(t, err) + + // OOO of 49m old fails. + s := addSamples(t, db, 261, 270, false, nil) + require.Empty(t, s) + + // WBL does not change. + newWblPtr := fmt.Sprintf("%p", db.head.wbl) + require.Equal(t, oldWblPtr, newWblPtr) + + verifySamples(t, db, allSamples) + + // Increase time window again and check + err = db.ApplyConfig(makeConfig(60)) + require.NoError(t, err) + allSamples = addSamples(t, db, 261, 270, true, allSamples) + verifySamples(t, db, allSamples) + + // WBL does not change. + newWblPtr = fmt.Sprintf("%p", db.head.wbl) + require.Equal(t, oldWblPtr, newWblPtr) + + doOOOCompaction(t, db) + verifySamples(t, db, allSamples) + }) + + t.Run("disabled to enabled", func(t *testing.T) { + var allSamples []chunks.Sample + db := getDB(0) + + // In-order. + allSamples = addSamples(t, db, 300, 310, true, allSamples) + + // OOO fails. + s := addSamples(t, db, 251, 260, false, nil) + require.Empty(t, s) + verifySamples(t, db, allSamples) + + require.Nil(t, db.head.wbl) + + // Increase time window and try adding again. + err := db.ApplyConfig(makeConfig(60)) + require.NoError(t, err) + allSamples = addSamples(t, db, 251, 260, true, allSamples) + + // WBL gets created. + require.NotNil(t, db.head.wbl) + + verifySamples(t, db, allSamples) + + // OOO compaction works now. + doOOOCompaction(t, db) + verifySamples(t, db, allSamples) + }) + + t.Run("enabled to disabled", func(t *testing.T) { + var allSamples []chunks.Sample + db := getDB(60 * time.Minute.Milliseconds()) + + // In-order. + allSamples = addSamples(t, db, 300, 310, true, allSamples) + + // OOO upto 59m old is success. + allSamples = addSamples(t, db, 251, 260, true, allSamples) + + oldWblPtr := fmt.Sprintf("%p", db.head.wbl) + // Time Window to 0, hence disabled. + err := db.ApplyConfig(makeConfig(0)) + require.NoError(t, err) + + // OOO within old time window fails. + s := addSamples(t, db, 290, 309, false, nil) + require.Empty(t, s) + + // WBL does not change and is not removed. + newWblPtr := fmt.Sprintf("%p", db.head.wbl) + require.Equal(t, oldWblPtr, newWblPtr) + + verifySamples(t, db, allSamples) + + // Compaction still works after disabling with WBL cleanup. + doOOOCompaction(t, db) + verifySamples(t, db, allSamples) + }) + + t.Run("disabled to disabled", func(t *testing.T) { + var allSamples []chunks.Sample + db := getDB(0) + + // In-order. + allSamples = addSamples(t, db, 300, 310, true, allSamples) + + // OOO fails. + s := addSamples(t, db, 290, 309, false, nil) + require.Empty(t, s) + verifySamples(t, db, allSamples) + require.Nil(t, db.head.wbl) + + // Time window to 0. + err := db.ApplyConfig(makeConfig(0)) + require.NoError(t, err) + + // OOO still fails. + s = addSamples(t, db, 290, 309, false, nil) + require.Empty(t, s) + verifySamples(t, db, allSamples) + require.Nil(t, db.head.wbl) + }) +} + +func TestNoGapAfterRestartWithOOO(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testNoGapAfterRestartWithOOO(t, scenario) + }) + } +} + +func testNoGapAfterRestartWithOOO(t *testing.T, scenario sampleTypeScenario) { + series1 := labels.FromStrings("foo", "bar1") + addSamples := func(t *testing.T, db *DB, fromMins, toMins int64, success bool) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, _, err := scenario.appendFunc(app, series1, ts, ts) + if success { + require.NoError(t, err) + } else { + require.Error(t, err) + } + } + require.NoError(t, app.Commit()) + } + + verifySamples := func(t *testing.T, db *DB, fromMins, toMins int64) { + var expSamples []chunks.Sample + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + expSamples = append(expSamples, scenario.sampleFunc(ts, ts)) + } + + expRes := map[string][]chunks.Sample{ + series1.String(): expSamples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + cases := []struct { + inOrderMint, inOrderMaxt int64 + oooMint, oooMaxt int64 + // After compaction. + blockRanges [][2]int64 + headMint, headMaxt int64 + }{ + { + 300, 490, + 489, 489, + [][2]int64{{300, 360}, {480, 600}}, + 360, 490, + }, + { + 300, 490, + 479, 479, + [][2]int64{{300, 360}, {360, 480}}, + 360, 490, + }, + } + + for i, c := range cases { + t.Run(fmt.Sprintf("case=%d", i), func(t *testing.T) { + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 30 * time.Minute.Milliseconds() + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + // 3h10m=190m worth in-order data. + addSamples(t, db, c.inOrderMint, c.inOrderMaxt, true) + verifySamples(t, db, c.inOrderMint, c.inOrderMaxt) + + // One ooo samples. + addSamples(t, db, c.oooMint, c.oooMaxt, true) + verifySamples(t, db, c.inOrderMint, c.inOrderMaxt) + + // We get 2 blocks. 1 from OOO, 1 from in-order. + require.NoError(t, db.Compact(ctx)) + verifyBlockRanges := func() { + blocks := db.Blocks() + require.Len(t, blocks, len(c.blockRanges)) + for j, br := range c.blockRanges { + require.Equal(t, br[0]*time.Minute.Milliseconds(), blocks[j].MinTime()) + require.Equal(t, br[1]*time.Minute.Milliseconds(), blocks[j].MaxTime()) + } + } + verifyBlockRanges() + require.Equal(t, c.headMint*time.Minute.Milliseconds(), db.head.MinTime()) + require.Equal(t, c.headMaxt*time.Minute.Milliseconds(), db.head.MaxTime()) + + // Restart and expect all samples to be present. + require.NoError(t, db.Close()) + + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + db.DisableCompactions() + + verifyBlockRanges() + require.Equal(t, c.headMint*time.Minute.Milliseconds(), db.head.MinTime()) + require.Equal(t, c.headMaxt*time.Minute.Milliseconds(), db.head.MaxTime()) + verifySamples(t, db, c.inOrderMint, c.inOrderMaxt) + }) + } +} + +func TestWblReplayAfterOOODisableAndRestart(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testWblReplayAfterOOODisableAndRestart(t, scenario) + }) + } +} + +func testWblReplayAfterOOODisableAndRestart(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + + series1 := labels.FromStrings("foo", "bar1") + var allSamples []chunks.Sample + addSamples := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, s, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + allSamples = append(allSamples, s) + } + require.NoError(t, app.Commit()) + } + + // In-order samples. + addSamples(290, 300) + // OOO samples. + addSamples(250, 260) + + verifySamples := func(expSamples []chunks.Sample) { + sort.Slice(expSamples, func(i, j int) bool { + return expSamples[i].T() < expSamples[j].T() + }) + + expRes := map[string][]chunks.Sample{ + series1.String(): expSamples, + } + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + + actRes := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "foo", "bar.*")) + requireEqualSeries(t, expRes, actRes, true) + } + + verifySamples(allSamples) + + // Restart DB with OOO disabled. + require.NoError(t, db.Close()) + + opts.OutOfOrderTimeWindow = 0 + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + + // We can still query OOO samples when OOO is disabled. + verifySamples(allSamples) +} + +func TestPanicOnApplyConfig(t *testing.T) { + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testPanicOnApplyConfig(t, scenario) + }) + } +} + +func testPanicOnApplyConfig(t *testing.T, scenario sampleTypeScenario) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + + series1 := labels.FromStrings("foo", "bar1") + var allSamples []chunks.Sample + addSamples := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, s, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + allSamples = append(allSamples, s) + } + require.NoError(t, app.Commit()) + } + + // In-order samples. + addSamples(290, 300) + // OOO samples. + addSamples(250, 260) + + // Restart DB with OOO disabled. + require.NoError(t, db.Close()) + + opts.OutOfOrderTimeWindow = 0 + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + + // ApplyConfig with OOO enabled and expect no panic. + err := db.ApplyConfig(&config.Config{ + StorageConfig: config.StorageConfig{ + TSDBConfig: &config.TSDBConfig{ + OutOfOrderTimeWindow: 60 * time.Minute.Milliseconds(), + }, + }, + }) + require.NoError(t, err) +} + +func TestDiskFillingUpAfterDisablingOOO(t *testing.T) { + t.Parallel() + for name, scenario := range sampleTypeScenarios { + t.Run(name, func(t *testing.T) { + testDiskFillingUpAfterDisablingOOO(t, scenario) + }) + } +} + +func testDiskFillingUpAfterDisablingOOO(t *testing.T, scenario sampleTypeScenario) { + t.Parallel() + ctx := context.Background() + + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() + + db := newTestDB(t, withOpts(opts)) + db.DisableCompactions() + + series1 := labels.FromStrings("foo", "bar1") + var allSamples []chunks.Sample + addSamples := func(fromMins, toMins int64) { + app := db.Appender(context.Background()) + for m := fromMins; m <= toMins; m++ { + ts := m * time.Minute.Milliseconds() + _, s, err := scenario.appendFunc(app, series1, ts, ts) + require.NoError(t, err) + allSamples = append(allSamples, s) + } + require.NoError(t, app.Commit()) + } + + // In-order samples. + addSamples(290, 300) + // OOO samples. + addSamples(250, 299) + + // Restart DB with OOO disabled. + require.NoError(t, db.Close()) + + opts.OutOfOrderTimeWindow = 0 + db = newTestDB(t, withDir(db.Dir()), withOpts(opts)) + db.DisableCompactions() + + ms := db.head.series.getByHash(series1.Hash(), series1) + require.NotEmpty(t, ms.ooo.oooMmappedChunks, "OOO mmap chunk was not replayed") + + checkMmapFileContents := func(contains, notContains []string) { + mmapDir := mmappedChunksDir(db.head.opts.ChunkDirRoot) + files, err := os.ReadDir(mmapDir) + require.NoError(t, err) + + fnames := make([]string, 0, len(files)) + for _, f := range files { + fnames = append(fnames, f.Name()) + } + + for _, f := range contains { + require.Contains(t, fnames, f) + } + for _, f := range notContains { + require.NotContains(t, fnames, f) + } + } + + // Add in-order samples until ready for compaction.. + addSamples(301, 500) + + // Check that m-map files gets deleted properly after compactions. + + db.head.mmapHeadChunks() + checkMmapFileContents([]string{"000001", "000002"}, nil) + require.NoError(t, db.Compact(ctx)) + checkMmapFileContents([]string{"000002"}, []string{"000001"}) + require.Nil(t, ms.ooo, "OOO mmap chunk was not compacted") + + addSamples(501, 650) + db.head.mmapHeadChunks() + checkMmapFileContents([]string{"000002", "000003"}, []string{"000001"}) + require.NoError(t, db.Compact(ctx)) + checkMmapFileContents(nil, []string{"000001", "000002", "000003"}) + + // Verify that WBL is empty. + files, err := os.ReadDir(db.head.wbl.Dir()) + require.NoError(t, err) + require.Len(t, files, 1) // Last empty file after compaction. + finfo, err := files[0].Info() + require.NoError(t, err) + require.Equal(t, int64(0), finfo.Size()) +} + +func TestHistogramAppendAndQuery(t *testing.T) { + t.Run("integer histograms", func(t *testing.T) { + testHistogramAppendAndQueryHelper(t, false) + }) + t.Run("float histograms", func(t *testing.T) { + testHistogramAppendAndQueryHelper(t, true) + }) +} + +func testHistogramAppendAndQueryHelper(t *testing.T, floatHistogram bool) { + t.Helper() + db := newTestDB(t) + minute := func(m int) int64 { return int64(m) * time.Minute.Milliseconds() } + + ctx := context.Background() + appendHistogram := func(t *testing.T, + lbls labels.Labels, tsMinute int, h *histogram.Histogram, + exp *[]chunks.Sample, expCRH histogram.CounterResetHint, + ) { + t.Helper() + var err error + app := db.Appender(ctx) + if floatHistogram { + _, err = app.AppendHistogram(0, lbls, minute(tsMinute), nil, h.ToFloat(nil)) + efh := h.ToFloat(nil) + efh.CounterResetHint = expCRH + *exp = append(*exp, sample{t: minute(tsMinute), fh: efh}) + } else { + _, err = app.AppendHistogram(0, lbls, minute(tsMinute), h.Copy(), nil) + eh := h.Copy() + eh.CounterResetHint = expCRH + *exp = append(*exp, sample{t: minute(tsMinute), h: eh}) + } + require.NoError(t, err) + require.NoError(t, app.Commit()) + } + appendFloat := func(t *testing.T, lbls labels.Labels, tsMinute int, val float64, exp *[]chunks.Sample) { + t.Helper() + app := db.Appender(ctx) + _, err := app.Append(0, lbls, minute(tsMinute), val) + require.NoError(t, err) + require.NoError(t, app.Commit()) + *exp = append(*exp, sample{t: minute(tsMinute), f: val}) + } + + testQuery := func(t *testing.T, name, value string, exp map[string][]chunks.Sample) { + t.Helper() + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + act := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, name, value)) + require.Equal(t, exp, act) + } + + baseH := &histogram.Histogram{ + Count: 15, + ZeroCount: 4, + ZeroThreshold: 0.001, + Sum: 35.5, + Schema: 1, + PositiveSpans: []histogram.Span{ + {Offset: 0, Length: 2}, + {Offset: 2, Length: 2}, + }, + PositiveBuckets: []int64{1, 1, -1, 0}, + NegativeSpans: []histogram.Span{ + {Offset: 0, Length: 1}, + {Offset: 1, Length: 2}, + }, + NegativeBuckets: []int64{1, 2, -1}, + } + + var ( + series1 = labels.FromStrings("foo", "bar1") + series2 = labels.FromStrings("foo", "bar2") + series3 = labels.FromStrings("foo", "bar3") + series4 = labels.FromStrings("foo", "bar4") + exp1, exp2, exp3, exp4 []chunks.Sample + ) + + // TODO(codesome): test everything for negative buckets as well. + t.Run("series with only histograms", func(t *testing.T) { + h := baseH.Copy() // This is shared across all sub tests. + + appendHistogram(t, series1, 100, h, &exp1, histogram.UnknownCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + + h.PositiveBuckets[0]++ + h.NegativeBuckets[0] += 2 + h.Count += 10 + appendHistogram(t, series1, 101, h, &exp1, histogram.NotCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + + t.Run("changing schema", func(t *testing.T) { + h.Schema = 2 + appendHistogram(t, series1, 102, h, &exp1, histogram.UnknownCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + + // Schema back to old. + h.Schema = 1 + appendHistogram(t, series1, 103, h, &exp1, histogram.UnknownCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + }) + + t.Run("new buckets incoming", func(t *testing.T) { + // In the previous unit test, during the last histogram append, we + // changed the schema and that caused a new chunk creation. Because + // of the next append the layout of the last histogram will change + // because the chunk will be re-encoded. So this forces us to modify + // the last histogram in exp1 so when we query we get the expected + // results. + if floatHistogram { + lh := exp1[len(exp1)-1].FH().Copy() + lh.PositiveSpans[1].Length++ + lh.PositiveBuckets = append(lh.PositiveBuckets, 0) + exp1[len(exp1)-1] = sample{t: exp1[len(exp1)-1].T(), fh: lh} + } else { + lh := exp1[len(exp1)-1].H().Copy() + lh.PositiveSpans[1].Length++ + lh.PositiveBuckets = append(lh.PositiveBuckets, -2) // -2 makes the last bucket 0. + exp1[len(exp1)-1] = sample{t: exp1[len(exp1)-1].T(), h: lh} + } + + // This histogram with new bucket at the end causes the re-encoding of the previous histogram. + // Hence the previous histogram is recoded into this new layout. + // But the query returns the histogram from the in-memory buffer, hence we don't see the recode here yet. + h.PositiveSpans[1].Length++ + h.PositiveBuckets = append(h.PositiveBuckets, 1) + h.Count += 3 + appendHistogram(t, series1, 104, h, &exp1, histogram.NotCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + + // Because of the previous two histograms being on the active chunk, + // and the next append is only adding a new bucket, the active chunk + // will be re-encoded to the new layout. + if floatHistogram { + lh := exp1[len(exp1)-2].FH().Copy() + lh.PositiveSpans[0].Length++ + lh.PositiveSpans[1].Offset-- + lh.PositiveBuckets = []float64{2, 3, 0, 2, 2, 0} + exp1[len(exp1)-2] = sample{t: exp1[len(exp1)-2].T(), fh: lh} + + lh = exp1[len(exp1)-1].FH().Copy() + lh.PositiveSpans[0].Length++ + lh.PositiveSpans[1].Offset-- + lh.PositiveBuckets = []float64{2, 3, 0, 2, 2, 3} + exp1[len(exp1)-1] = sample{t: exp1[len(exp1)-1].T(), fh: lh} + } else { + lh := exp1[len(exp1)-2].H().Copy() + lh.PositiveSpans[0].Length++ + lh.PositiveSpans[1].Offset-- + lh.PositiveBuckets = []int64{2, 1, -3, 2, 0, -2} + exp1[len(exp1)-2] = sample{t: exp1[len(exp1)-2].T(), h: lh} + + lh = exp1[len(exp1)-1].H().Copy() + lh.PositiveSpans[0].Length++ + lh.PositiveSpans[1].Offset-- + lh.PositiveBuckets = []int64{2, 1, -3, 2, 0, 1} + exp1[len(exp1)-1] = sample{t: exp1[len(exp1)-1].T(), h: lh} + } + + // Now we add the new buckets in between. Empty bucket is again not present for the old histogram. + h.PositiveSpans[0].Length++ + h.PositiveSpans[1].Offset-- + h.Count += 3 + // {2, 1, -1, 0, 1} -> {2, 1, 0, -1, 0, 1} + h.PositiveBuckets = append(h.PositiveBuckets[:2], append([]int64{0}, h.PositiveBuckets[2:]...)...) + appendHistogram(t, series1, 105, h, &exp1, histogram.NotCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + + // We add 4 more histograms to clear out the buffer and see the re-encoded histograms. + appendHistogram(t, series1, 106, h, &exp1, histogram.NotCounterReset) + appendHistogram(t, series1, 107, h, &exp1, histogram.NotCounterReset) + appendHistogram(t, series1, 108, h, &exp1, histogram.NotCounterReset) + appendHistogram(t, series1, 109, h, &exp1, histogram.NotCounterReset) + + // Update the expected histograms to reflect the re-encoding. + if floatHistogram { + l := len(exp1) + h7 := exp1[l-7].FH() + h7.PositiveSpans = exp1[l-1].FH().PositiveSpans + h7.PositiveBuckets = []float64{2, 3, 0, 2, 2, 0} + exp1[l-7] = sample{t: exp1[l-7].T(), fh: h7} + + h6 := exp1[l-6].FH() + h6.PositiveSpans = exp1[l-1].FH().PositiveSpans + h6.PositiveBuckets = []float64{2, 3, 0, 2, 2, 3} + exp1[l-6] = sample{t: exp1[l-6].T(), fh: h6} + } else { + l := len(exp1) + h7 := exp1[l-7].H() + h7.PositiveSpans = exp1[l-1].H().PositiveSpans + h7.PositiveBuckets = []int64{2, 1, -3, 2, 0, -2} // -3 and -2 are the empty buckets. + exp1[l-7] = sample{t: exp1[l-7].T(), h: h7} + + h6 := exp1[l-6].H() + h6.PositiveSpans = exp1[l-1].H().PositiveSpans + h6.PositiveBuckets = []int64{2, 1, -3, 2, 0, 1} // -3 is the empty bucket. + exp1[l-6] = sample{t: exp1[l-6].T(), h: h6} + } + + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + }) + + t.Run("buckets disappearing", func(t *testing.T) { + h.PositiveSpans[1].Length-- + h.PositiveBuckets = h.PositiveBuckets[:len(h.PositiveBuckets)-1] + h.Count -= 3 + appendHistogram(t, series1, 110, h, &exp1, histogram.UnknownCounterReset) + testQuery(t, "foo", "bar1", map[string][]chunks.Sample{series1.String(): exp1}) + }) + }) + + t.Run("series starting with float and then getting histograms", func(t *testing.T) { + appendFloat(t, series2, 100, 100, &exp2) + appendFloat(t, series2, 101, 101, &exp2) + appendFloat(t, series2, 102, 102, &exp2) + testQuery(t, "foo", "bar2", map[string][]chunks.Sample{series2.String(): exp2}) + + h := baseH.Copy() + appendHistogram(t, series2, 103, h, &exp2, histogram.UnknownCounterReset) + appendHistogram(t, series2, 104, h, &exp2, histogram.NotCounterReset) + appendHistogram(t, series2, 105, h, &exp2, histogram.NotCounterReset) + testQuery(t, "foo", "bar2", map[string][]chunks.Sample{series2.String(): exp2}) + + // Switching between float and histograms again. + appendFloat(t, series2, 106, 106, &exp2) + appendFloat(t, series2, 107, 107, &exp2) + testQuery(t, "foo", "bar2", map[string][]chunks.Sample{series2.String(): exp2}) + + appendHistogram(t, series2, 108, h, &exp2, histogram.UnknownCounterReset) + appendHistogram(t, series2, 109, h, &exp2, histogram.NotCounterReset) + testQuery(t, "foo", "bar2", map[string][]chunks.Sample{series2.String(): exp2}) + }) + + t.Run("series starting with histogram and then getting float", func(t *testing.T) { + h := baseH.Copy() + appendHistogram(t, series3, 101, h, &exp3, histogram.UnknownCounterReset) + appendHistogram(t, series3, 102, h, &exp3, histogram.NotCounterReset) + appendHistogram(t, series3, 103, h, &exp3, histogram.NotCounterReset) + testQuery(t, "foo", "bar3", map[string][]chunks.Sample{series3.String(): exp3}) + + appendFloat(t, series3, 104, 100, &exp3) + appendFloat(t, series3, 105, 101, &exp3) + appendFloat(t, series3, 106, 102, &exp3) + testQuery(t, "foo", "bar3", map[string][]chunks.Sample{series3.String(): exp3}) + + // Switching between histogram and float again. + appendHistogram(t, series3, 107, h, &exp3, histogram.UnknownCounterReset) + appendHistogram(t, series3, 108, h, &exp3, histogram.NotCounterReset) + testQuery(t, "foo", "bar3", map[string][]chunks.Sample{series3.String(): exp3}) + + appendFloat(t, series3, 109, 106, &exp3) + appendFloat(t, series3, 110, 107, &exp3) + testQuery(t, "foo", "bar3", map[string][]chunks.Sample{series3.String(): exp3}) + }) + + t.Run("query mix of histogram and float series", func(t *testing.T) { + // A float only series. + appendFloat(t, series4, 100, 100, &exp4) + appendFloat(t, series4, 101, 101, &exp4) + appendFloat(t, series4, 102, 102, &exp4) + + testQuery(t, "foo", "bar.*", map[string][]chunks.Sample{ + series1.String(): exp1, + series2.String(): exp2, + series3.String(): exp3, + series4.String(): exp4, + }) + }) +} + +func TestQueryHistogramFromBlocksWithCompaction(t *testing.T) { + t.Parallel() + minute := func(m int) int64 { return int64(m) * time.Minute.Milliseconds() } + + testBlockQuerying := func(t *testing.T, blockSeries ...[]storage.Series) { + t.Helper() + + opts := DefaultOptions() + db := newTestDB(t, withOpts(opts)) + + var it chunkenc.Iterator + exp := make(map[string][]chunks.Sample) + for _, series := range blockSeries { + createBlock(t, db.Dir(), series) + + for _, s := range series { + lbls := s.Labels().String() + slice := exp[lbls] + it = s.Iterator(it) + smpls, err := storage.ExpandSamples(it, nil) + require.NoError(t, err) + slice = append(slice, smpls...) + sort.Slice(slice, func(i, j int) bool { + return slice[i].T() < slice[j].T() + }) + exp[lbls] = slice + } + } + + require.Empty(t, db.Blocks()) + require.NoError(t, db.reload()) + require.Len(t, db.Blocks(), len(blockSeries)) + + q, err := db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + res := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*")) + compareSeries(t, exp, res) + + // Compact all the blocks together and query again. + blocks := db.Blocks() + blockDirs := make([]string, 0, len(blocks)) + for _, b := range blocks { + blockDirs = append(blockDirs, b.Dir()) + } + ids, err := db.compactor.Compact(db.Dir(), blockDirs, blocks) + require.NoError(t, err) + require.Len(t, ids, 1) + require.NoError(t, db.reload()) + require.Len(t, db.Blocks(), 1) + + q, err = db.Querier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + res = query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*")) + + // After compaction, we do not require "unknown" counter resets + // due to origin from different overlapping chunks anymore. + for _, ss := range exp { + for i, s := range ss[1:] { + if s.Type() == chunkenc.ValHistogram && ss[i].Type() == chunkenc.ValHistogram && s.H().CounterResetHint == histogram.UnknownCounterReset { + s.H().CounterResetHint = histogram.NotCounterReset + } + if s.Type() == chunkenc.ValFloatHistogram && ss[i].Type() == chunkenc.ValFloatHistogram && s.FH().CounterResetHint == histogram.UnknownCounterReset { + s.FH().CounterResetHint = histogram.NotCounterReset + } + } + } + compareSeries(t, exp, res) + } + + for _, floatHistogram := range []bool{false, true} { + t.Run(fmt.Sprintf("floatHistogram=%t", floatHistogram), func(t *testing.T) { + t.Run("serial blocks with only histograms", func(t *testing.T) { + testBlockQuerying(t, + genHistogramSeries(10, 5, minute(0), minute(119), minute(1), floatHistogram), + genHistogramSeries(10, 5, minute(120), minute(239), minute(1), floatHistogram), + genHistogramSeries(10, 5, minute(240), minute(359), minute(1), floatHistogram), + ) + }) + + t.Run("serial blocks with either histograms or floats in a block and not both", func(t *testing.T) { + testBlockQuerying(t, + genHistogramSeries(10, 5, minute(0), minute(119), minute(1), floatHistogram), + genSeriesFromSampleGenerator(10, 5, minute(120), minute(239), minute(1), func(ts int64) chunks.Sample { + return sample{t: ts, f: rand.Float64()} + }), + genHistogramSeries(10, 5, minute(240), minute(359), minute(1), floatHistogram), + ) + }) + + t.Run("serial blocks with mix of histograms and float64", func(t *testing.T) { + testBlockQuerying(t, + genHistogramAndFloatSeries(10, 5, minute(0), minute(60), minute(1), floatHistogram), + genHistogramSeries(10, 5, minute(61), minute(120), minute(1), floatHistogram), + genHistogramAndFloatSeries(10, 5, minute(121), minute(180), minute(1), floatHistogram), + genSeriesFromSampleGenerator(10, 5, minute(181), minute(240), minute(1), func(ts int64) chunks.Sample { + return sample{t: ts, f: rand.Float64()} + }), + ) + }) + + t.Run("overlapping blocks with only histograms", func(t *testing.T) { + testBlockQuerying(t, + genHistogramSeries(10, 5, minute(0), minute(120), minute(3), floatHistogram), + genHistogramSeries(10, 5, minute(1), minute(120), minute(3), floatHistogram), + genHistogramSeries(10, 5, minute(2), minute(120), minute(3), floatHistogram), + ) + }) + + t.Run("overlapping blocks with only histograms and only float in a series", func(t *testing.T) { + testBlockQuerying(t, + genHistogramSeries(10, 5, minute(0), minute(120), minute(3), floatHistogram), + genSeriesFromSampleGenerator(10, 5, minute(1), minute(120), minute(3), func(ts int64) chunks.Sample { + return sample{t: ts, f: rand.Float64()} + }), + genHistogramSeries(10, 5, minute(2), minute(120), minute(3), floatHistogram), + ) + }) + + t.Run("overlapping blocks with mix of histograms and float64", func(t *testing.T) { + testBlockQuerying(t, + genHistogramAndFloatSeries(10, 5, minute(0), minute(60), minute(3), floatHistogram), + genHistogramSeries(10, 5, minute(46), minute(100), minute(3), floatHistogram), + genHistogramAndFloatSeries(10, 5, minute(89), minute(140), minute(3), floatHistogram), + genSeriesFromSampleGenerator(10, 5, minute(126), minute(200), minute(3), func(ts int64) chunks.Sample { + return sample{t: ts, f: rand.Float64()} + }), + ) + }) + }) + } +} + +func TestOOONativeHistogramsSettings(t *testing.T) { + h := &histogram.Histogram{ + Count: 9, + ZeroCount: 4, + ZeroThreshold: 0.001, + Sum: 35.5, + Schema: 1, + PositiveSpans: []histogram.Span{ + {Offset: 0, Length: 2}, + {Offset: 2, Length: 2}, + }, + PositiveBuckets: []int64{1, 1, -1, 0}, + } + + l := labels.FromStrings("foo", "bar") + + t.Run("Test OOO native histograms if OOO is disabled and Native Histograms is enabled", func(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 0 + db := newTestDB(t, withOpts(opts), withRngs(100)) + + app := db.Appender(context.Background()) + _, err := app.AppendHistogram(0, l, 100, h, nil) + require.NoError(t, err) + + _, err = app.AppendHistogram(0, l, 50, h, nil) + require.NoError(t, err) // The OOO sample is not detected until it is committed, so no error is returned + + require.NoError(t, app.Commit()) + + q, err := db.Querier(math.MinInt, math.MaxInt64) + require.NoError(t, err) + act := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + require.Equal(t, map[string][]chunks.Sample{ + l.String(): {sample{t: 100, h: h}}, + }, act) + }) + t.Run("Test OOO native histograms when both OOO and Native Histograms are enabled", func(t *testing.T) { + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 100 + db := newTestDB(t, withOpts(opts), withRngs(100)) + + // Add in-order samples + app := db.Appender(context.Background()) + _, err := app.AppendHistogram(0, l, 200, h, nil) + require.NoError(t, err) + + // Add OOO samples + _, err = app.AppendHistogram(0, l, 100, h, nil) + require.NoError(t, err) + _, err = app.AppendHistogram(0, l, 150, h, nil) + require.NoError(t, err) + + require.NoError(t, app.Commit()) + + q, err := db.Querier(math.MinInt, math.MaxInt64) + require.NoError(t, err) + act := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + requireEqualSeries(t, map[string][]chunks.Sample{ + l.String(): {sample{t: 100, h: h}, sample{t: 150, h: h}, sample{t: 200, h: h}}, + }, act, true) + }) +} + +// compareSeries essentially replaces `require.Equal(t, expected, actual)` in +// situations where the actual series might contain more counter reset hints +// "unknown" than the expected series. This can easily happen for long series +// that trigger new chunks. This function therefore tolerates counter reset +// hints "CounterReset" and "NotCounterReset" in an expected series where the +// actual series contains a counter reset hint "UnknownCounterReset". +// "GaugeType" hints are still strictly checked, and any "UnknownCounterReset" +// in an expected series has to be matched precisely by the actual series. +func compareSeries(t require.TestingT, expected, actual map[string][]chunks.Sample) { + if len(expected) != len(actual) { + // The reason for the difference is not the counter reset hints + // (alone), so let's use the pretty diffing by the require + // package. + require.Equal(t, expected, actual, "number of series differs") + } + for key, expSamples := range expected { + actSamples, ok := actual[key] + if !ok { + require.Equal(t, expected, actual, "expected series %q not found", key) + } + if len(expSamples) != len(actSamples) { + require.Equal(t, expSamples, actSamples, "number of samples for series %q differs", key) + } + + for i, eS := range expSamples { + aS := actSamples[i] + + // Must use the interface as Equal does not work when actual types differ + // not only does the type differ, but chunk.Sample.FH() interface may auto convert from chunk.Sample.H()! + require.Equal(t, eS.T(), aS.T(), "timestamp of sample %d in series %q differs", i, key) + + require.Equal(t, eS.Type(), aS.Type(), "type of sample %d in series %q differs", i, key) + + switch eS.Type() { + case chunkenc.ValFloat: + require.Equal(t, eS.F(), aS.F(), "sample %d in series %q differs", i, key) + case chunkenc.ValHistogram: + eH, aH := eS.H(), aS.H() + if aH.CounterResetHint == histogram.UnknownCounterReset { + eH = eH.Copy() + // It is always safe to set the counter reset hint to UnknownCounterReset + eH.CounterResetHint = histogram.UnknownCounterReset + eS = sample{t: eS.T(), h: eH} + } + require.Equal(t, eH, aH, "histogram sample %d in series %q differs", i, key) + + case chunkenc.ValFloatHistogram: + eFH, aFH := eS.FH(), aS.FH() + if aFH.CounterResetHint == histogram.UnknownCounterReset { + eFH = eFH.Copy() + // It is always safe to set the counter reset hint to UnknownCounterReset + eFH.CounterResetHint = histogram.UnknownCounterReset + eS = sample{t: eS.T(), fh: eFH} + } + require.Equal(t, eFH, aFH, "float histogram sample %d in series %q differs", i, key) + } + } + } +} + +// TestChunkQuerierReadWriteRace looks for any possible race between appending +// samples and reading chunks because the head chunk that is being appended to +// can be read in parallel and we should be able to make a copy of the chunk without +// worrying about the parallel write. +func TestChunkQuerierReadWriteRace(t *testing.T) { + t.Parallel() + db := newTestDB(t) + + lbls := labels.FromStrings("foo", "bar") + + writer := func() error { + <-time.After(5 * time.Millisecond) // Initial pause while readers start. + ts := 0 + for range 500 { + app := db.Appender(context.Background()) + for range 10 { + ts++ + _, err := app.Append(0, lbls, int64(ts), float64(ts*100)) + if err != nil { + return err + } + } + err := app.Commit() + if err != nil { + return err + } + <-time.After(time.Millisecond) + } + return nil + } + + reader := func() { + querier, err := db.ChunkQuerier(math.MinInt64, math.MaxInt64) + require.NoError(t, err) + defer func(q storage.ChunkQuerier) { + require.NoError(t, q.Close()) + }(querier) + ss := querier.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "foo", "bar")) + for ss.Next() { + cs := ss.At() + it := cs.Iterator(nil) + for it.Next() { + m := it.At() + b := m.Chunk.Bytes() + bb := make([]byte, len(b)) + copy(bb, b) // This copying of chunk bytes detects any race. + } + } + require.NoError(t, ss.Err()) + } + + ch := make(chan struct{}) + var writerErr error + go func() { + defer close(ch) + writerErr = writer() + }() + +Outer: + for { + reader() + select { + case <-ch: + break Outer + default: + } + } + + require.NoError(t, writerErr) +} + +type mockCompactorFn struct { + planFn func() ([]string, error) + compactFn func() ([]ulid.ULID, error) + writeFn func() ([]ulid.ULID, error) +} + +func (c *mockCompactorFn) Plan(string) ([]string, error) { + return c.planFn() +} + +func (c *mockCompactorFn) Compact(string, []string, []*Block) ([]ulid.ULID, error) { + return c.compactFn() +} + +func (c *mockCompactorFn) Write(string, BlockReader, int64, int64, *BlockMeta) ([]ulid.ULID, error) { + return c.writeFn() +} + +// Regression test for https://github.com/prometheus/prometheus/pull/13754 +func TestAbortBlockCompactions(t *testing.T) { + // Create a test DB + db := newTestDB(t) + // It should NOT be compactable at the beginning of the test + require.False(t, db.head.compactable(), "head should NOT be compactable") + + // Track the number of compactions run inside db.compactBlocks() + var compactions int + + // Use a mock compactor with custom Plan() implementation + db.compactor = &mockCompactorFn{ + planFn: func() ([]string, error) { + // On every Plan() run increment compactions. After 4 compactions + // update HEAD to make it compactable to force an exit from db.compactBlocks() loop. + compactions++ + if compactions > 3 { + chunkRange := db.head.chunkRange.Load() + db.head.minTime.Store(0) + db.head.maxTime.Store(chunkRange * 2) + require.True(t, db.head.compactable(), "head should be compactable") + } + // Our custom Plan() will always return something to compact. + return []string{"1", "2", "3"}, nil + }, + compactFn: func() ([]ulid.ULID, error) { + return []ulid.ULID{}, nil + }, + writeFn: func() ([]ulid.ULID, error) { + return []ulid.ULID{}, nil + }, + } + + err := db.Compact(context.Background()) + require.NoError(t, err) + require.True(t, db.head.compactable(), "head should be compactable") + require.Equal(t, 4, compactions, "expected 4 compactions to be completed") +} + +func TestNewCompactorFunc(t *testing.T) { + opts := DefaultOptions() + block1 := ulid.MustNew(1, nil) + block2 := ulid.MustNew(2, nil) + opts.NewCompactorFunc = func(context.Context, prometheus.Registerer, *slog.Logger, []int64, chunkenc.Pool, *Options) (Compactor, error) { + return &mockCompactorFn{ + planFn: func() ([]string, error) { + return []string{block1.String(), block2.String()}, nil + }, + compactFn: func() ([]ulid.ULID, error) { + return []ulid.ULID{block1}, nil + }, + writeFn: func() ([]ulid.ULID, error) { + return []ulid.ULID{block2}, nil + }, + }, nil + } + db := newTestDB(t, withOpts(opts)) + + plans, err := db.compactor.Plan("") + require.NoError(t, err) + require.Equal(t, []string{block1.String(), block2.String()}, plans) + ulids, err := db.compactor.Compact("", nil, nil) + require.NoError(t, err) + require.Len(t, ulids, 1) + require.Equal(t, block1, ulids[0]) + ulids, err = db.compactor.Write("", nil, 0, 1, nil) + require.NoError(t, err) + require.Len(t, ulids, 1) + require.Equal(t, block2, ulids[0]) +} + +func TestBlockQuerierAndBlockChunkQuerier(t *testing.T) { + opts := DefaultOptions() + opts.BlockQuerierFunc = func(b BlockReader, mint, maxt int64) (storage.Querier, error) { + // Only block with hints can be queried. + if len(b.Meta().Compaction.Hints) > 0 { + return NewBlockQuerier(b, mint, maxt) + } + return storage.NoopQuerier(), nil + } + opts.BlockChunkQuerierFunc = func(b BlockReader, mint, maxt int64) (storage.ChunkQuerier, error) { + // Only level 4 compaction block can be queried. + if b.Meta().Compaction.Level == 4 { + return NewBlockChunkQuerier(b, mint, maxt) + } + return storage.NoopChunkedQuerier(), nil + } + + db := newTestDB(t, withOpts(opts)) + + metas := []BlockMeta{ + {Compaction: BlockMetaCompaction{Hints: []string{"test-hint"}}}, + {Compaction: BlockMetaCompaction{Level: 4}}, + } + for i := range metas { + // Include blockID into series to identify which block got touched. + serieses := []storage.Series{storage.NewListSeries(labels.FromMap(map[string]string{"block": fmt.Sprintf("block-%d", i), labels.MetricName: "test_metric"}), []chunks.Sample{sample{t: 0, f: 1}})} + blockDir := createBlock(t, db.Dir(), serieses) + b, err := OpenBlock(db.logger, blockDir, db.chunkPool, nil) + require.NoError(t, err) + + // Overwrite meta.json with compaction section for testing purpose. + b.meta.Compaction = metas[i].Compaction + _, err = writeMetaFile(db.logger, blockDir, &b.meta) + require.NoError(t, err) + require.NoError(t, b.Close()) + } + require.NoError(t, db.reloadBlocks()) + require.Len(t, db.Blocks(), 2) + + querier, err := db.Querier(0, 500) + require.NoError(t, err) + defer querier.Close() + matcher := labels.MustNewMatcher(labels.MatchEqual, labels.MetricName, "test_metric") + seriesSet := querier.Select(context.Background(), false, nil, matcher) + count := 0 + var lbls labels.Labels + for seriesSet.Next() { + count++ + lbls = seriesSet.At().Labels() + } + require.NoError(t, seriesSet.Err()) + require.Equal(t, 1, count) + // Make sure only block-0 is queried. + require.Equal(t, "block-0", lbls.Get("block")) + + chunkQuerier, err := db.ChunkQuerier(0, 500) + require.NoError(t, err) + defer chunkQuerier.Close() + css := chunkQuerier.Select(context.Background(), false, nil, matcher) + count = 0 + // Reset lbls variable. + lbls = labels.EmptyLabels() + for css.Next() { + count++ + lbls = css.At().Labels() + } + require.NoError(t, css.Err()) + require.Equal(t, 1, count) + // Make sure only block-1 is queried. + require.Equal(t, "block-1", lbls.Get("block")) +} + +func TestGenerateCompactionDelay(t *testing.T) { + assertDelay := func(delay time.Duration, expectedMaxPercentDelay int) { + t.Helper() + require.GreaterOrEqual(t, delay, time.Duration(0)) + // Expect to generate a delay up to MaxPercentDelay of the head chunk range + require.LessOrEqual(t, delay, (time.Duration(60000*expectedMaxPercentDelay/100) * time.Millisecond)) + } + + opts := DefaultOptions() + cases := []struct { + compactionDelayPercent int + }{ + { + compactionDelayPercent: 1, + }, + { + compactionDelayPercent: 10, + }, + { + compactionDelayPercent: 60, + }, + { + compactionDelayPercent: 100, + }, + } + + opts.EnableDelayedCompaction = true + + for _, c := range cases { + opts.CompactionDelayMaxPercent = c.compactionDelayPercent + db := newTestDB(t, withOpts(opts), withRngs(60000)) + + // The offset is generated and changed while opening. + assertDelay(db.opts.CompactionDelay, c.compactionDelayPercent) + + for range 1000 { + assertDelay(db.generateCompactionDelay(), c.compactionDelayPercent) + } + } +} + +type blockedResponseRecorder struct { + r *httptest.ResponseRecorder + + // writeBlocked is used to block writing until the test wants it to resume. + writeBlocked chan struct{} + // writeStarted is closed by blockedResponseRecorder to signal that writing has started. + writeStarted chan struct{} +} + +func (br *blockedResponseRecorder) Write(buf []byte) (int, error) { + select { + case <-br.writeStarted: + default: + close(br.writeStarted) + } + + <-br.writeBlocked + return br.r.Write(buf) +} + +func (br *blockedResponseRecorder) Header() http.Header { return br.r.Header() } + +func (br *blockedResponseRecorder) WriteHeader(code int) { br.r.WriteHeader(code) } + +func (br *blockedResponseRecorder) Flush() { br.r.Flush() } + +// TestBlockClosingBlockedDuringRemoteRead ensures that a TSDB Block is not closed while it is being queried +// through remote read. This is a regression test for https://github.com/prometheus/prometheus/issues/14422. +// TODO: Ideally, this should reside in storage/remote/read_handler_test.go once the necessary TSDB utils are accessible there. +func TestBlockClosingBlockedDuringRemoteRead(t *testing.T) { + dir := t.TempDir() + + createBlock(t, dir, genSeries(2, 1, 0, 10)) + + // Not using newTestDB as db.Close is expected to return error. + db, err := Open(dir, nil, nil, nil, nil) + require.NoError(t, err) + defer db.Close() + + readAPI := remote.NewReadHandler( + nil, nil, db, + func() config.Config { + return config.Config{} + }, 0, 1, 0, + ) + + matcher, err := labels.NewMatcher(labels.MatchRegexp, "__name__", ".*") + require.NoError(t, err) + + query, err := remote.ToQuery(0, 10, []*labels.Matcher{matcher}, nil) + require.NoError(t, err) + + req := &prompb.ReadRequest{ + Queries: []*prompb.Query{query}, + AcceptedResponseTypes: []prompb.ReadRequest_ResponseType{prompb.ReadRequest_STREAMED_XOR_CHUNKS}, + } + data, err := proto.Marshal(req) + require.NoError(t, err) + + request, err := http.NewRequest(http.MethodPost, "", bytes.NewBuffer(snappy.Encode(nil, data))) + require.NoError(t, err) + + blockedRecorder := &blockedResponseRecorder{ + r: httptest.NewRecorder(), + writeBlocked: make(chan struct{}), + writeStarted: make(chan struct{}), + } + + readDone := make(chan struct{}) + go func() { + readAPI.ServeHTTP(blockedRecorder, request) + require.Equal(t, http.StatusOK, blockedRecorder.r.Code) + close(readDone) + }() + + // Wait for the read API to start streaming data. + <-blockedRecorder.writeStarted + + // Try to close the queried block. + blockClosed := make(chan struct{}) + go func() { + for _, block := range db.Blocks() { + block.Close() + } + close(blockClosed) + }() + + // Closing the queried block should block. + // Wait a little bit to make sure of that. + select { + case <-time.After(100 * time.Millisecond): + case <-readDone: + require.Fail(t, "read API should still be streaming data.") + case <-blockClosed: + require.Fail(t, "Block shouldn't get closed while being queried.") + } + + // Resume the read API data streaming. + close(blockedRecorder.writeBlocked) + <-readDone + + // The block should be no longer needed and closing it should end. + select { + case <-time.After(10 * time.Millisecond): + require.Fail(t, "Closing the block timed out.") + case <-blockClosed: + } +} From 0b70a0757263930f035e48290826ac3d63b3d5ec Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 28 Nov 2025 12:43:12 +0000 Subject: [PATCH 118/439] refactor(appenderV2): add TSDB AppenderV2 implementation Signed-off-by: bwplotka tmp Signed-off-by: bwplotka --- model/metadata/metadata.go | 26 +- model/metadata/metadata_test.go | 116 + storage/interface.go | 1 + tsdb/db.go | 37 +- tsdb/head.go | 14 + tsdb/head_append.go | 55 +- tsdb/head_append_v2.go | 2391 +++------------------ tsdb/head_append_v2_test.go | 3532 ++++--------------------------- tsdb/head_bench_test.go | 303 ++- tsdb/head_bench_v2_test.go | 173 -- tsdb/head_test.go | 48 +- tsdb/testutil.go | 16 +- 12 files changed, 1168 insertions(+), 5544 deletions(-) create mode 100644 model/metadata/metadata_test.go delete mode 100644 tsdb/head_bench_v2_test.go diff --git a/model/metadata/metadata.go b/model/metadata/metadata.go index 1b7e63e0f3..d2a91bb560 100644 --- a/model/metadata/metadata.go +++ b/model/metadata/metadata.go @@ -1,4 +1,4 @@ -// Copyright 2022 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -13,7 +13,11 @@ package metadata -import "github.com/prometheus/common/model" +import ( + "strings" + + "github.com/prometheus/common/model" +) // Metadata stores a series' metadata information. type Metadata struct { @@ -21,3 +25,21 @@ type Metadata struct { Unit string `json:"unit"` Help string `json:"help"` } + +// IsEmpty returns true if metadata structure is empty, including unknown type case. +func (m Metadata) IsEmpty() bool { + return (m.Type == "" || m.Type == model.MetricTypeUnknown) && m.Unit == "" && m.Help == "" +} + +// Equals returns true if m is semantically the same as other metadata. +func (m Metadata) Equals(other Metadata) bool { + if strings.Compare(m.Unit, other.Unit) != 0 || strings.Compare(m.Help, other.Help) != 0 { + return false + } + + // Unknown means the same as empty string. + if m.Type == "" || m.Type == model.MetricTypeUnknown { + return other.Type == "" || other.Type == model.MetricTypeUnknown + } + return m.Type == other.Type +} diff --git a/model/metadata/metadata_test.go b/model/metadata/metadata_test.go new file mode 100644 index 0000000000..169cd60c2e --- /dev/null +++ b/model/metadata/metadata_test.go @@ -0,0 +1,116 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package metadata + +import ( + "testing" + + "github.com/prometheus/common/model" + "github.com/stretchr/testify/require" +) + +func TestMetadata_IsEmpty(t *testing.T) { + for _, tt := range []struct { + name string + m Metadata + expected bool + }{ + { + name: "empty struct", expected: true, + }, + { + name: "unknown type with empty fields", expected: true, + m: Metadata{Type: model.MetricTypeUnknown}, + }, + { + name: "type", expected: false, + m: Metadata{Type: model.MetricTypeCounter}, + }, + { + name: "unit", expected: false, + m: Metadata{Unit: "seconds"}, + }, + { + name: "help", expected: false, + m: Metadata{Help: "help text"}, + }, + { + name: "unknown type with help", expected: false, + m: Metadata{Type: model.MetricTypeUnknown, Help: "help text"}, + }, + } { + t.Run(tt.name, func(t *testing.T) { + require.Equal(t, tt.expected, tt.m.IsEmpty()) + }) + } +} + +func TestMetadata_Equals(t *testing.T) { + for _, tt := range []struct { + name string + m Metadata + other Metadata + expected bool + }{ + { + name: "same empty", expected: true, + }, + { + name: "same", expected: true, + m: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + }, + { + name: "same unknown type", expected: true, + m: Metadata{Type: model.MetricTypeUnknown, Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeUnknown, Unit: "s", Help: "doc"}, + }, + { + name: "same mixed unknown type", expected: true, + m: Metadata{Type: "", Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeUnknown, Unit: "s", Help: "doc"}, + }, + { + name: "different unit", expected: false, + m: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeCounter, Unit: "bytes", Help: "doc"}, + }, + { + name: "different help", expected: false, + m: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "other doc"}, + }, + { + name: "different type", expected: false, + m: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeGauge, Unit: "s", Help: "doc"}, + }, + { + name: "different type with unknown", expected: false, + m: Metadata{Type: model.MetricTypeUnknown, Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + }, + { + name: "different type with empty", expected: false, + m: Metadata{Type: "", Unit: "s", Help: "doc"}, + other: Metadata{Type: model.MetricTypeCounter, Unit: "s", Help: "doc"}, + }, + } { + t.Run(tt.name, func(t *testing.T) { + if got := tt.m.Equals(tt.other); got != tt.expected { + t.Errorf("Metadata.Equals() = %v, expected %v", got, tt.expected) + } + }) + } +} diff --git a/storage/interface.go b/storage/interface.go index fe9b3fa6e8..f7d7953de4 100644 --- a/storage/interface.go +++ b/storage/interface.go @@ -49,6 +49,7 @@ var ( // NOTE(bwplotka): This can be both an instrumentation failure or commonly expected // behaviour, and we currently don't have a way to determine this. As a result // it's recommended to ignore this error for now. + // TODO(bwplotka): Remove with appender v1 flow; not used in v2. ErrOutOfOrderST = errors.New("start timestamp out of order, ignoring") ErrSTNewerThanSample = errors.New("ST is newer or the same as sample's timestamp, ignoring") ) diff --git a/tsdb/db.go b/tsdb/db.go index dac5689b09..c4f29c225f 100644 --- a/tsdb/db.go +++ b/tsdb/db.go @@ -1136,11 +1136,16 @@ func (db *DB) run(ctx context.Context) { } } -// Appender opens a new appender against the database. +// Appender opens a new Appender against the database. func (db *DB) Appender(ctx context.Context) storage.Appender { return dbAppender{db: db, Appender: db.head.Appender(ctx)} } +// AppenderV2 opens a new AppenderV2 against the database. +func (db *DB) AppenderV2(ctx context.Context) storage.AppenderV2 { + return dbAppenderV2{db: db, AppenderV2: db.head.AppenderV2(ctx)} +} + // ApplyConfig applies a new config to the DB. // Behaviour of 'OutOfOrderTimeWindow' is as follows: // OOO enabled = oooTimeWindow > 0. OOO disabled = oooTimeWindow is 0. @@ -1254,6 +1259,36 @@ func (a dbAppender) Commit() error { return err } +// dbAppenderV2 wraps the DB's head appender and triggers compactions on commit +// if necessary. +type dbAppenderV2 struct { + storage.AppenderV2 + db *DB +} + +var _ storage.GetRef = dbAppenderV2{} + +func (a dbAppenderV2) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { + if g, ok := a.AppenderV2.(storage.GetRef); ok { + return g.GetRef(lset, hash) + } + return 0, labels.EmptyLabels() +} + +func (a dbAppenderV2) Commit() error { + err := a.AppenderV2.Commit() + + // We could just run this check every few minutes practically. But for benchmarks + // and high frequency use cases this is the safer way. + if a.db.head.compactable() { + select { + case a.db.compactc <- struct{}{}: + default: + } + } + return err +} + // waitingForCompactionDelay returns true if the DB is waiting for the Head compaction delay. // This doesn't guarantee that the Head is really compactable. func (db *DB) waitingForCompactionDelay() bool { diff --git a/tsdb/head.go b/tsdb/head.go index cf773e82b0..25a1b88cec 100644 --- a/tsdb/head.go +++ b/tsdb/head.go @@ -187,6 +187,20 @@ type HeadOptions struct { // EnableSharding enables ShardedPostings() support in the Head. EnableSharding bool + + // EnableSTAsZeroSample represents 'created-timestamp-zero-ingestion' feature flag. + // If true, ST, if non-empty and earlier than sample timestamp, will be stored + // as a zero sample before the actual sample. + // + // The zero sample is best-effort, only debug log on failure is emitted. + // NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 + // is implemented. + EnableSTAsZeroSample bool + + // EnableMetadataWALRecords represents 'metadata-wal-records' feature flag. + // NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 + // is implemented. + EnableMetadataWALRecords bool } const ( diff --git a/tsdb/head_append.go b/tsdb/head_append.go index 942c3ce974..356d1c453f 100644 --- a/tsdb/head_append.go +++ b/tsdb/head_append.go @@ -165,17 +165,19 @@ func (h *Head) appender() *headAppender { minValidTime := h.appendableMinValidTime() appendID, cleanupAppendIDsBelow := h.iso.newAppendID(minValidTime) // Every appender gets an ID that is cleared upon commit/rollback. return &headAppender{ - head: h, - minValidTime: minValidTime, - mint: math.MaxInt64, - maxt: math.MinInt64, - headMaxt: h.MaxTime(), - oooTimeWindow: h.opts.OutOfOrderTimeWindow.Load(), - seriesRefs: h.getRefSeriesBuffer(), - series: h.getSeriesBuffer(), - typesInBatch: h.getTypeMap(), - appendID: appendID, - cleanupAppendIDsBelow: cleanupAppendIDsBelow, + headAppenderBase: headAppenderBase{ + head: h, + minValidTime: minValidTime, + mint: math.MaxInt64, + maxt: math.MinInt64, + headMaxt: h.MaxTime(), + oooTimeWindow: h.opts.OutOfOrderTimeWindow.Load(), + seriesRefs: h.getRefSeriesBuffer(), + series: h.getSeriesBuffer(), + typesInBatch: h.getTypeMap(), + appendID: appendID, + cleanupAppendIDsBelow: cleanupAppendIDsBelow, + }, } } @@ -382,7 +384,7 @@ func (b *appendBatch) close(h *Head) { b.exemplars = nil } -type headAppender struct { +type headAppenderBase struct { head *Head minValidTime int64 // No samples below this timestamp are allowed. mint, maxt int64 @@ -397,7 +399,10 @@ type headAppender struct { appendID, cleanupAppendIDsBelow uint64 closed bool - hints *storage.AppendOptions +} +type headAppender struct { + headAppenderBase + hints *storage.AppendOptions } func (a *headAppender) SetOptions(opts *storage.AppendOptions) { @@ -525,7 +530,7 @@ func (a *headAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Lab return storage.SeriesRef(s.ref), nil } -func (a *headAppender) getOrCreate(lset labels.Labels) (s *memSeries, created bool, err error) { +func (a *headAppenderBase) getOrCreate(lset labels.Labels) (s *memSeries, created bool, err error) { // Ensure no empty labels have gotten through. lset = lset.WithoutEmpty() if lset.IsEmpty() { @@ -550,7 +555,7 @@ func (a *headAppender) getOrCreate(lset labels.Labels) (s *memSeries, created bo // getCurrentBatch returns the current batch if it fits the provided sampleType // for the provided series. Otherwise, it adds a new batch and returns it. -func (a *headAppender) getCurrentBatch(st sampleType, s chunks.HeadSeriesRef) *appendBatch { +func (a *headAppenderBase) getCurrentBatch(st sampleType, s chunks.HeadSeriesRef) *appendBatch { h := a.head newBatch := func() *appendBatch { @@ -1043,7 +1048,7 @@ func (a *headAppender) UpdateMetadata(ref storage.SeriesRef, lset labels.Labels, var _ storage.GetRef = &headAppender{} -func (a *headAppender) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { +func (a *headAppenderBase) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { s := a.head.series.getByHash(hash, lset) if s == nil { return 0, labels.EmptyLabels() @@ -1053,7 +1058,7 @@ func (a *headAppender) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRe } // log writes all headAppender's data to the WAL. -func (a *headAppender) log() error { +func (a *headAppenderBase) log() error { if a.head.wal == nil { return nil } @@ -1185,7 +1190,7 @@ type appenderCommitContext struct { } // commitExemplars adds all exemplars from the provided batch to the head's exemplar storage. -func (a *headAppender) commitExemplars(b *appendBatch) { +func (a *headAppenderBase) commitExemplars(b *appendBatch) { // No errors logging to WAL, so pass the exemplars along to the in memory storage. for _, e := range b.exemplars { s := a.head.series.getByID(chunks.HeadSeriesRef(e.ref)) @@ -1205,7 +1210,7 @@ func (a *headAppender) commitExemplars(b *appendBatch) { } } -func (acc *appenderCommitContext) collectOOORecords(a *headAppender) { +func (acc *appenderCommitContext) collectOOORecords(a *headAppenderBase) { if a.head.wbl == nil { // WBL is not enabled. So no need to collect. acc.wblSamples = nil @@ -1310,7 +1315,7 @@ func handleAppendableError(err error, appended, oooRejected, oobRejected, tooOld // operations on the series after appending the samples. // // There are also specific functions to commit histograms and float histograms. -func (a *headAppender) commitFloats(b *appendBatch, acc *appenderCommitContext) { +func (a *headAppenderBase) commitFloats(b *appendBatch, acc *appenderCommitContext) { var ok, chunkCreated bool var series *memSeries @@ -1466,7 +1471,7 @@ func (a *headAppender) commitFloats(b *appendBatch, acc *appenderCommitContext) } // For details on the commitHistograms function, see the commitFloats docs. -func (a *headAppender) commitHistograms(b *appendBatch, acc *appenderCommitContext) { +func (a *headAppenderBase) commitHistograms(b *appendBatch, acc *appenderCommitContext) { var ok, chunkCreated bool var series *memSeries @@ -1575,7 +1580,7 @@ func (a *headAppender) commitHistograms(b *appendBatch, acc *appenderCommitConte } // For details on the commitFloatHistograms function, see the commitFloats docs. -func (a *headAppender) commitFloatHistograms(b *appendBatch, acc *appenderCommitContext) { +func (a *headAppenderBase) commitFloatHistograms(b *appendBatch, acc *appenderCommitContext) { var ok, chunkCreated bool var series *memSeries @@ -1697,7 +1702,7 @@ func commitMetadata(b *appendBatch) { } } -func (a *headAppender) unmarkCreatedSeriesAsPendingCommit() { +func (a *headAppenderBase) unmarkCreatedSeriesAsPendingCommit() { for _, s := range a.series { s.Lock() s.pendingCommit = false @@ -1707,7 +1712,7 @@ func (a *headAppender) unmarkCreatedSeriesAsPendingCommit() { // Commit writes to the WAL and adds the data to the Head. // TODO(codesome): Refactor this method to reduce indentation and make it more readable. -func (a *headAppender) Commit() (err error) { +func (a *headAppenderBase) Commit() (err error) { if a.closed { return ErrAppenderClosed } @@ -2238,7 +2243,7 @@ func handleChunkWriteError(err error) { } // Rollback removes the samples and exemplars from headAppender and writes any series to WAL. -func (a *headAppender) Rollback() (err error) { +func (a *headAppenderBase) Rollback() (err error) { if a.closed { return ErrAppenderClosed } diff --git a/tsdb/head_append_v2.go b/tsdb/head_append_v2.go index 942c3ce974..c5ed9898e9 100644 --- a/tsdb/head_append_v2.go +++ b/tsdb/head_append_v2.go @@ -1,4 +1,4 @@ -// Copyright 2021 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -17,121 +17,42 @@ import ( "context" "errors" "fmt" - "log/slog" "math" "github.com/prometheus/prometheus/model/exemplar" "github.com/prometheus/prometheus/model/histogram" "github.com/prometheus/prometheus/model/labels" - "github.com/prometheus/prometheus/model/metadata" "github.com/prometheus/prometheus/model/value" "github.com/prometheus/prometheus/storage" - "github.com/prometheus/prometheus/tsdb/chunkenc" "github.com/prometheus/prometheus/tsdb/chunks" "github.com/prometheus/prometheus/tsdb/record" ) -// initAppender is a helper to initialize the time bounds of the head +// initAppenderV2 is a helper to initialize the time bounds of the head // upon the first sample it receives. -type initAppender struct { - app storage.Appender +type initAppenderV2 struct { + app storage.AppenderV2 head *Head } -var _ storage.GetRef = &initAppender{} +var _ storage.GetRef = &initAppenderV2{} -func (a *initAppender) SetOptions(opts *storage.AppendOptions) { - if a.app != nil { - a.app.SetOptions(opts) +func (a *initAppenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) { + if a.app == nil { + a.head.initTime(t) + a.app = a.head.appenderV2() } + return a.app.Append(ref, ls, st, t, v, h, fh, opts) } -func (a *initAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64, v float64) (storage.SeriesRef, error) { - if a.app != nil { - return a.app.Append(ref, lset, t, v) - } - - a.head.initTime(t) - a.app = a.head.appender() - return a.app.Append(ref, lset, t, v) -} - -func (a *initAppender) AppendExemplar(ref storage.SeriesRef, l labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { - // Check if exemplar storage is enabled. - if !a.head.opts.EnableExemplarStorage || a.head.opts.MaxExemplars.Load() <= 0 { - return 0, nil - } - - if a.app != nil { - return a.app.AppendExemplar(ref, l, e) - } - // We should never reach here given we would call Append before AppendExemplar - // and we probably want to always base head/WAL min time on sample times. - a.head.initTime(e.Ts) - a.app = a.head.appender() - - return a.app.AppendExemplar(ref, l, e) -} - -func (a *initAppender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - if a.app != nil { - return a.app.AppendHistogram(ref, l, t, h, fh) - } - a.head.initTime(t) - a.app = a.head.appender() - - return a.app.AppendHistogram(ref, l, t, h, fh) -} - -func (a *initAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - if a.app != nil { - return a.app.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) - } - a.head.initTime(t) - a.app = a.head.appender() - - return a.app.AppendHistogramSTZeroSample(ref, l, t, st, h, fh) -} - -func (a *initAppender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) { - if a.app != nil { - return a.app.UpdateMetadata(ref, l, m) - } - - a.app = a.head.appender() - return a.app.UpdateMetadata(ref, l, m) -} - -func (a *initAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64) (storage.SeriesRef, error) { - if a.app != nil { - return a.app.AppendSTZeroSample(ref, lset, t, st) - } - - a.head.initTime(t) - a.app = a.head.appender() - - return a.app.AppendSTZeroSample(ref, lset, t, st) -} - -// initTime initializes a head with the first timestamp. This only needs to be called -// for a completely fresh head with an empty WAL. -func (h *Head) initTime(t int64) { - if !h.minTime.CompareAndSwap(math.MaxInt64, t) { - return - } - // Ensure that max time is initialized to at least the min time we just set. - // Concurrent appenders may already have set it to a higher value. - h.maxTime.CompareAndSwap(math.MinInt64, t) -} - -func (a *initAppender) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { +func (a *initAppenderV2) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { if g, ok := a.app.(storage.GetRef); ok { return g.GetRef(lset, hash) } return 0, labels.EmptyLabels() } -func (a *initAppender) Commit() error { +func (a *initAppenderV2) Commit() error { if a.app == nil { a.head.metrics.activeAppenders.Dec() return nil @@ -139,7 +60,7 @@ func (a *initAppender) Commit() error { return a.app.Commit() } -func (a *initAppender) Rollback() error { +func (a *initAppenderV2) Rollback() error { if a.app == nil { a.head.metrics.activeAppenders.Dec() return nil @@ -147,323 +68,129 @@ func (a *initAppender) Rollback() error { return a.app.Rollback() } -// Appender returns a new Appender on the database. -func (h *Head) Appender(context.Context) storage.Appender { +// AppenderV2 returns a new AppenderV2 on the database. +func (h *Head) AppenderV2(context.Context) storage.AppenderV2 { h.metrics.activeAppenders.Inc() // The head cache might not have a starting point yet. The init appender // picks up the first appended timestamp as the base. if !h.initialized() { - return &initAppender{ + return &initAppenderV2{ head: h, } } - return h.appender() + return h.appenderV2() } -func (h *Head) appender() *headAppender { +func (h *Head) appenderV2() *headAppenderV2 { minValidTime := h.appendableMinValidTime() appendID, cleanupAppendIDsBelow := h.iso.newAppendID(minValidTime) // Every appender gets an ID that is cleared upon commit/rollback. - return &headAppender{ - head: h, - minValidTime: minValidTime, - mint: math.MaxInt64, - maxt: math.MinInt64, - headMaxt: h.MaxTime(), - oooTimeWindow: h.opts.OutOfOrderTimeWindow.Load(), - seriesRefs: h.getRefSeriesBuffer(), - series: h.getSeriesBuffer(), - typesInBatch: h.getTypeMap(), - appendID: appendID, - cleanupAppendIDsBelow: cleanupAppendIDsBelow, + return &headAppenderV2{ + headAppenderBase: headAppenderBase{ + head: h, + minValidTime: minValidTime, + mint: math.MaxInt64, + maxt: math.MinInt64, + headMaxt: h.MaxTime(), + oooTimeWindow: h.opts.OutOfOrderTimeWindow.Load(), + seriesRefs: h.getRefSeriesBuffer(), + series: h.getSeriesBuffer(), + typesInBatch: h.getTypeMap(), + appendID: appendID, + cleanupAppendIDsBelow: cleanupAppendIDsBelow, + }, } } -// appendableMinValidTime returns the minimum valid timestamp for appends, -// such that samples stay ahead of prior blocks and the head compaction window. -func (h *Head) appendableMinValidTime() int64 { - // This boundary ensures that no samples will be added to the compaction window. - // This allows race-free, concurrent appending and compaction. - cwEnd := h.MaxTime() - h.chunkRange.Load()/2 - - // This boundary ensures that we avoid overlapping timeframes from one block to the next. - // While not necessary for correctness, it means we're not required to use vertical compaction. - minValid := h.minValidTime.Load() - - return max(cwEnd, minValid) +type headAppenderV2 struct { + headAppenderBase } -// AppendableMinValidTime returns the minimum valid time for samples to be appended to the Head. -// Returns false if Head hasn't been initialized yet and the minimum time isn't known yet. -func (h *Head) AppendableMinValidTime() (int64, bool) { - if !h.initialized() { - return 0, false +func (a *headAppenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) { + var ( + // Avoid shadowing err variables for reliability. + valErr, appErr, partialErr error + sampleMetricType = sampleMetricTypeFloat + isStale bool + ) + // Fail fast on incorrect histograms. + + switch { + case fh != nil: + sampleMetricType = sampleMetricTypeHistogram + valErr = fh.Validate() + case h != nil: + sampleMetricType = sampleMetricTypeHistogram + valErr = h.Validate() + } + if valErr != nil { + return 0, valErr } - return h.appendableMinValidTime(), true -} - -func (h *Head) getRefSeriesBuffer() []record.RefSeries { - b := h.refSeriesPool.Get() - if b == nil { - return make([]record.RefSeries, 0, 512) - } - return b -} - -func (h *Head) putRefSeriesBuffer(b []record.RefSeries) { - h.refSeriesPool.Put(b[:0]) -} - -func (h *Head) getFloatBuffer() []record.RefSample { - b := h.floatsPool.Get() - if b == nil { - return make([]record.RefSample, 0, 512) - } - return b -} - -func (h *Head) putFloatBuffer(b []record.RefSample) { - h.floatsPool.Put(b[:0]) -} - -func (h *Head) getExemplarBuffer() []exemplarWithSeriesRef { - b := h.exemplarsPool.Get() - if b == nil { - return make([]exemplarWithSeriesRef, 0, 512) - } - return b -} - -func (h *Head) putExemplarBuffer(b []exemplarWithSeriesRef) { - if b == nil { - return - } - for i := range b { // Zero out to avoid retaining label data. - b[i].exemplar.Labels = labels.EmptyLabels() - } - - h.exemplarsPool.Put(b[:0]) -} - -func (h *Head) getHistogramBuffer() []record.RefHistogramSample { - b := h.histogramsPool.Get() - if b == nil { - return make([]record.RefHistogramSample, 0, 512) - } - return b -} - -func (h *Head) putHistogramBuffer(b []record.RefHistogramSample) { - h.histogramsPool.Put(b[:0]) -} - -func (h *Head) getFloatHistogramBuffer() []record.RefFloatHistogramSample { - b := h.floatHistogramsPool.Get() - if b == nil { - return make([]record.RefFloatHistogramSample, 0, 512) - } - return b -} - -func (h *Head) putFloatHistogramBuffer(b []record.RefFloatHistogramSample) { - h.floatHistogramsPool.Put(b[:0]) -} - -func (h *Head) getMetadataBuffer() []record.RefMetadata { - b := h.metadataPool.Get() - if b == nil { - return make([]record.RefMetadata, 0, 512) - } - return b -} - -func (h *Head) putMetadataBuffer(b []record.RefMetadata) { - h.metadataPool.Put(b[:0]) -} - -func (h *Head) getSeriesBuffer() []*memSeries { - b := h.seriesPool.Get() - if b == nil { - return make([]*memSeries, 0, 512) - } - return b -} - -func (h *Head) putSeriesBuffer(b []*memSeries) { - for i := range b { // Zero out to avoid retaining data. - b[i] = nil - } - h.seriesPool.Put(b[:0]) -} - -func (h *Head) getTypeMap() map[chunks.HeadSeriesRef]sampleType { - b := h.typeMapPool.Get() - if b == nil { - return make(map[chunks.HeadSeriesRef]sampleType) - } - return b -} - -func (h *Head) putTypeMap(b map[chunks.HeadSeriesRef]sampleType) { - clear(b) - h.typeMapPool.Put(b) -} - -func (h *Head) getBytesBuffer() []byte { - b := h.bytesPool.Get() - if b == nil { - return make([]byte, 0, 1024) - } - return b -} - -func (h *Head) putBytesBuffer(b []byte) { - h.bytesPool.Put(b[:0]) -} - -type exemplarWithSeriesRef struct { - ref storage.SeriesRef - exemplar exemplar.Exemplar -} - -// sampleType describes sample types we need to distinguish for append batching. -// We need separate types for everything that goes into a different WAL record -// type or into a different chunk encoding. -type sampleType byte - -const ( - stNone sampleType = iota // To mark that the sample type does not matter. - stFloat // All simple floats (counters, gauges, untyped). Goes to `floats`. - stHistogram // Native integer histograms with a standard exponential schema. Goes to `histograms`. - stCustomBucketHistogram // Native integer histograms with custom bucket boundaries. Goes to `histograms`. - stFloatHistogram // Native float histograms. Goes to `floatHistograms`. - stCustomBucketFloatHistogram // Native float histograms with custom bucket boundaries. Goes to `floatHistograms`. -) - -// appendBatch is used to partition all the appended data into batches that are -// "type clean", i.e. every series receives only samples of one type within the -// batch. Types in this regard are defined by the sampleType enum above. -// TODO(beorn7): The same concept could be extended to make sure every series in -// the batch has at most one metadata record. This is currently not implemented -// because it is unclear if it is needed at all. (Maybe we will remove metadata -// records altogether, see issue #15911.) -type appendBatch struct { - floats []record.RefSample // New float samples held by this appender. - floatSeries []*memSeries // Float series corresponding to the samples held by this appender (using corresponding slice indices - same series may appear more than once). - histograms []record.RefHistogramSample // New histogram samples held by this appender. - histogramSeries []*memSeries // HistogramSamples series corresponding to the samples held by this appender (using corresponding slice indices - same series may appear more than once). - floatHistograms []record.RefFloatHistogramSample // New float histogram samples held by this appender. - floatHistogramSeries []*memSeries // FloatHistogramSamples series corresponding to the samples held by this appender (using corresponding slice indices - same series may appear more than once). - metadata []record.RefMetadata // New metadata held by this appender. - metadataSeries []*memSeries // Series corresponding to the metadata held by this appender. - exemplars []exemplarWithSeriesRef // New exemplars held by this appender. -} - -// close returns all the slices to the pools in Head and nil's them. -func (b *appendBatch) close(h *Head) { - h.putFloatBuffer(b.floats) - b.floats = nil - h.putSeriesBuffer(b.floatSeries) - b.floatSeries = nil - h.putHistogramBuffer(b.histograms) - b.histograms = nil - h.putSeriesBuffer(b.histogramSeries) - b.histogramSeries = nil - h.putFloatHistogramBuffer(b.floatHistograms) - b.floatHistograms = nil - h.putSeriesBuffer(b.floatHistogramSeries) - b.floatHistogramSeries = nil - h.putMetadataBuffer(b.metadata) - b.metadata = nil - h.putSeriesBuffer(b.metadataSeries) - b.metadataSeries = nil - h.putExemplarBuffer(b.exemplars) - b.exemplars = nil -} - -type headAppender struct { - head *Head - minValidTime int64 // No samples below this timestamp are allowed. - mint, maxt int64 - headMaxt int64 // We track it here to not take the lock for every sample appended. - oooTimeWindow int64 // Use the same for the entire append, and don't load the atomic for each sample. - - seriesRefs []record.RefSeries // New series records held by this appender. - series []*memSeries // New series held by this appender (using corresponding slices indexes from seriesRefs) - batches []*appendBatch // Holds all the other data to append. (In regular cases, there should be only one of these.) - - typesInBatch map[chunks.HeadSeriesRef]sampleType // Which (one) sample type each series holds in the most recent batch. - - appendID, cleanupAppendIDsBelow uint64 - closed bool - hints *storage.AppendOptions -} - -func (a *headAppender) SetOptions(opts *storage.AppendOptions) { - a.hints = opts -} - -func (a *headAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64, v float64) (storage.SeriesRef, error) { // Fail fast if OOO is disabled and the sample is out of bounds. - // Otherwise a full check will be done later to decide if the sample is in-order or out-of-order. + // Otherwise, a full check will be done later to decide if the sample is in-order or out-of-order. if a.oooTimeWindow == 0 && t < a.minValidTime { - a.head.metrics.outOfBoundSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + a.head.metrics.outOfBoundSamples.WithLabelValues(sampleMetricType).Inc() return 0, storage.ErrOutOfBounds } s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) if s == nil { var err error - s, _, err = a.getOrCreate(lset) + s, _, err = a.getOrCreate(ls) if err != nil { return 0, err } } - if value.IsStaleNaN(v) { - // If we have added a sample before with this same appender, we - // can check the previously used type and turn a stale float - // sample into a stale histogram sample or stale float histogram - // sample as appropriate. This prevents an unnecessary creation - // of a new batch. However, since other appenders might append - // to the same series concurrently, this is not perfect but just - // an optimization for the more likely case. - switch a.typesInBatch[s.ref] { - case stHistogram, stCustomBucketHistogram: - return a.AppendHistogram(ref, lset, t, &histogram.Histogram{Sum: v}, nil) - case stFloatHistogram, stCustomBucketFloatHistogram: - return a.AppendHistogram(ref, lset, t, nil, &histogram.FloatHistogram{Sum: v}) - } - // Note that a series reference not yet in the map will come out - // as stNone, but since we do not handle that case separately, - // we do not need to check for the difference between "unknown - // series" and "known series with stNone". + // TODO(bwplotka): Handle ST natively (as per PROM-60). + if a.head.opts.EnableSTAsZeroSample && st != 0 { + a.bestEffortAppendSTZeroSample(s, st, t, h, fh) } - s.Lock() - defer s.Unlock() - // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise - // to skip that sample from the WAL and write only in the WBL. - isOOO, delta, err := s.appendable(t, v, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err == nil { - if isOOO && a.hints != nil && a.hints.DiscardOutOfOrder { - a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeFloat).Inc() - return 0, storage.ErrOutOfOrderSample + switch { + case fh != nil: + isStale = value.IsStaleNaN(fh.Sum) + appErr = a.appendFloatHistogram(s, t, fh, opts.RejectOutOfOrder) + case h != nil: + isStale = value.IsStaleNaN(h.Sum) + appErr = a.appendHistogram(s, t, h, opts.RejectOutOfOrder) + default: + isStale = value.IsStaleNaN(v) + if isStale { + // If we have added a sample before with this same appender, we + // can check the previously used type and turn a stale float + // sample into a stale histogram sample or stale float histogram + // sample as appropriate. This prevents an unnecessary creation + // of a new batch. However, since other appenders might append + // to the same series concurrently, this is not perfect but just + // an optimization for the more likely case. + switch a.typesInBatch[s.ref] { + case stHistogram, stCustomBucketHistogram: + return a.Append(ref, ls, st, t, 0, &histogram.Histogram{Sum: v}, nil, storage.AOptions{ + RejectOutOfOrder: opts.RejectOutOfOrder, + }) + case stFloatHistogram, stCustomBucketFloatHistogram: + return a.Append(ref, ls, st, t, 0, nil, &histogram.FloatHistogram{Sum: v}, storage.AOptions{ + RejectOutOfOrder: opts.RejectOutOfOrder, + }) + } + // Note that a series reference not yet in the map will come out + // as stNone, but since we do not handle that case separately, + // we do not need to check for the difference between "unknown + // series" and "known series with stNone". } - s.pendingCommit = true + appErr = a.appendFloat(s, t, v, opts.RejectOutOfOrder) } - if delta > 0 { - a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) - } - if err != nil { + // Handle append error, if any. + if appErr != nil { switch { - case errors.Is(err, storage.ErrOutOfOrderSample): - a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeFloat).Inc() - case errors.Is(err, storage.ErrTooOldSample): - a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + case errors.Is(appErr, storage.ErrOutOfOrderSample): + a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricType).Inc() + case errors.Is(appErr, storage.ErrTooOldSample): + a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricType).Inc() } - return 0, err + return 0, appErr } if t < a.mint { @@ -473,492 +200,161 @@ func (a *headAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64 a.maxt = t } - b := a.getCurrentBatch(stFloat, s.ref) - b.floats = append(b.floats, record.RefSample{ - Ref: s.ref, - T: t, - V: v, - }) - b.floatSeries = append(b.floatSeries, s) - return storage.SeriesRef(s.ref), nil -} - -// AppendSTZeroSample appends synthetic zero sample for st timestamp. It returns -// error when sample can't be appended. See -// storage.StartTimestampAppender.AppendSTZeroSample for further documentation. -func (a *headAppender) AppendSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64) (storage.SeriesRef, error) { - if st >= t { - return 0, storage.ErrSTNewerThanSample + if isStale { + // For stale values we never attempt to process metadata/exemplars, claim the success. + return ref, nil } - s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) - if s == nil { - var err error - s, _, err = a.getOrCreate(lset) - if err != nil { - return 0, err + // Append exemplars if any and if storage was configured for it. + if len(opts.Exemplars) > 0 && a.head.opts.EnableExemplarStorage && a.head.opts.MaxExemplars.Load() > 0 { + // Currently only exemplars can return partial errors. + partialErr = a.appendExemplars(s, opts.Exemplars) + } + + // TODO(bwplotka): Move/reuse metadata tests from scrape, once scrape adopts AppenderV2. + // Currently tsdb package does not test metadata. + if a.head.opts.EnableMetadataWALRecords && !opts.Metadata.IsEmpty() { + s.Lock() + metaChanged := s.meta == nil || !s.meta.Equals(opts.Metadata) + s.Unlock() + if metaChanged { + b := a.getCurrentBatch(stNone, s.ref) + b.metadata = append(b.metadata, record.RefMetadata{ + Ref: s.ref, + Type: record.GetMetricType(opts.Metadata.Type), + Unit: opts.Metadata.Unit, + Help: opts.Metadata.Help, + }) + b.metadataSeries = append(b.metadataSeries, s) } } + return storage.SeriesRef(s.ref), partialErr +} - // Check if ST wouldn't be OOO vs samples we already might have for this series. - // NOTE(bwplotka): This will be often hit as it's expected for long living - // counters to share the same ST. +func (a *headAppenderV2) appendFloat(s *memSeries, t int64, v float64, fastRejectOOO bool) error { s.Lock() - isOOO, _, err := s.appendable(st, 0, a.headMaxt, a.minValidTime, a.oooTimeWindow) + // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise + // to skip that sample from the WAL and write only in the WBL. + isOOO, delta, err := s.appendable(t, v, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if isOOO && fastRejectOOO { + s.Unlock() + return storage.ErrOutOfOrderSample + } if err == nil { s.pendingCommit = true } s.Unlock() - if err != nil { - return 0, err + if delta > 0 { + a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) } - if isOOO { - return storage.SeriesRef(s.ref), storage.ErrOutOfOrderST + if err != nil { + return err } - if st > a.maxt { - a.maxt = st - } b := a.getCurrentBatch(stFloat, s.ref) - b.floats = append(b.floats, record.RefSample{Ref: s.ref, T: st, V: 0.0}) + b.floats = append(b.floats, record.RefSample{Ref: s.ref, T: t, V: v}) b.floatSeries = append(b.floatSeries, s) - return storage.SeriesRef(s.ref), nil + return nil } -func (a *headAppender) getOrCreate(lset labels.Labels) (s *memSeries, created bool, err error) { - // Ensure no empty labels have gotten through. - lset = lset.WithoutEmpty() - if lset.IsEmpty() { - return nil, false, fmt.Errorf("empty labelset: %w", ErrInvalidSample) - } - if l, dup := lset.HasDuplicateLabelNames(); dup { - return nil, false, fmt.Errorf(`label name "%s" is not unique: %w`, l, ErrInvalidSample) - } - s, created, err = a.head.getOrCreate(lset.Hash(), lset, true) - if err != nil { - return nil, false, err - } - if created { - a.seriesRefs = append(a.seriesRefs, record.RefSeries{ - Ref: s.ref, - Labels: lset, - }) - a.series = append(a.series, s) - } - return s, created, nil -} - -// getCurrentBatch returns the current batch if it fits the provided sampleType -// for the provided series. Otherwise, it adds a new batch and returns it. -func (a *headAppender) getCurrentBatch(st sampleType, s chunks.HeadSeriesRef) *appendBatch { - h := a.head - - newBatch := func() *appendBatch { - b := appendBatch{ - floats: h.getFloatBuffer(), - floatSeries: h.getSeriesBuffer(), - histograms: h.getHistogramBuffer(), - histogramSeries: h.getSeriesBuffer(), - floatHistograms: h.getFloatHistogramBuffer(), - floatHistogramSeries: h.getSeriesBuffer(), - metadata: h.getMetadataBuffer(), - metadataSeries: h.getSeriesBuffer(), - } - - // Allocate the exemplars buffer only if exemplars are enabled. - if h.opts.EnableExemplarStorage { - b.exemplars = h.getExemplarBuffer() - } - clear(a.typesInBatch) - switch st { - case stHistogram, stFloatHistogram, stCustomBucketHistogram, stCustomBucketFloatHistogram: - // We only record histogram sample types in the map. - // Floats are implicit. - a.typesInBatch[s] = st - } - a.batches = append(a.batches, &b) - return &b - } - - // First batch ever. Create it. - if len(a.batches) == 0 { - return newBatch() - } - - // TODO(beorn7): If we ever see that the a.typesInBatch map grows so - // large that it matters for total memory consumption, we could limit - // the batch size here, i.e. cut a new batch even without a type change. - // Something like: - // if len(a.typesInBatch > limit) { - // return newBatch() - // } - - lastBatch := a.batches[len(a.batches)-1] - if st == stNone { - // Type doesn't matter, last batch will always do. - return lastBatch - } - prevST, ok := a.typesInBatch[s] - switch { - case prevST == st: - // An old series of some histogram type with the same type being appended. - // Continue the batch. - return lastBatch - case !ok && st == stFloat: - // A new float series, or an old float series that gets floats appended. - // Note that we do not track stFloat in typesInBatch. - // Continue the batch. - return lastBatch - case st == stFloat: - // A float being appended to a histogram series. - // Start a new batch. - return newBatch() - case !ok: - // A new series of some histogram type, or some histogram type - // being appended to on old float series. Even in the latter - // case, we don't need to start a new batch because histograms - // after floats are fine. - // Add new sample type to the map and continue batch. - a.typesInBatch[s] = st - return lastBatch - default: - // One histogram type changed to another. - // Start a new batch. - return newBatch() - } -} - -// appendable checks whether the given sample is valid for appending to the series. -// If the sample is valid and in-order, it returns false with no error. -// If the sample belongs to the out-of-order chunk, it returns true with no error. -// If the sample cannot be handled, it returns an error. -func (s *memSeries) appendable(t int64, v float64, headMaxt, minValidTime, oooTimeWindow int64) (isOOO bool, oooDelta int64, err error) { - // Check if we can append in the in-order chunk. - if t >= minValidTime { - if s.headChunks == nil { - // The series has no sample and was freshly created. - return false, 0, nil - } - msMaxt := s.maxTime() - if t > msMaxt { - return false, 0, nil - } - if t == msMaxt { - // We are allowing exact duplicates as we can encounter them in valid cases - // like federation and erroring out at that time would be extremely noisy. - // This only checks against the latest in-order sample. - // The OOO headchunk has its own method to detect these duplicates. - if s.lastHistogramValue != nil || s.lastFloatHistogramValue != nil { - return false, 0, storage.NewDuplicateHistogramToFloatErr(t, v) - } - if math.Float64bits(s.lastValue) != math.Float64bits(v) { - return false, 0, storage.NewDuplicateFloatErr(t, s.lastValue, v) - } - // Sample is identical (ts + value) with most current (highest ts) sample in sampleBuf. - return false, 0, nil - } - } - - // The sample cannot go in the in-order chunk. Check if it can go in the out-of-order chunk. - if oooTimeWindow > 0 && t >= headMaxt-oooTimeWindow { - return true, headMaxt - t, nil - } - - // The sample cannot go in both in-order and out-of-order chunk. - if oooTimeWindow > 0 { - return true, headMaxt - t, storage.ErrTooOldSample - } - if t < minValidTime { - return false, headMaxt - t, storage.ErrOutOfBounds - } - return false, headMaxt - t, storage.ErrOutOfOrderSample -} - -// appendableHistogram checks whether the given histogram sample is valid for appending to the series. (if we return false and no error) -// The sample belongs to the out of order chunk if we return true and no error. -// An error signifies the sample cannot be handled. -func (s *memSeries) appendableHistogram(t int64, h *histogram.Histogram, headMaxt, minValidTime, oooTimeWindow int64) (isOOO bool, oooDelta int64, err error) { - // Check if we can append in the in-order chunk. - if t >= minValidTime { - if s.headChunks == nil { - // The series has no sample and was freshly created. - return false, 0, nil - } - msMaxt := s.maxTime() - if t > msMaxt { - return false, 0, nil - } - if t == msMaxt { - // We are allowing exact duplicates as we can encounter them in valid cases - // like federation and erroring out at that time would be extremely noisy. - // This only checks against the latest in-order sample. - // The OOO headchunk has its own method to detect these duplicates. - if !h.Equals(s.lastHistogramValue) { - return false, 0, storage.ErrDuplicateSampleForTimestamp - } - // Sample is identical (ts + value) with most current (highest ts) sample in sampleBuf. - return false, 0, nil - } - } - - // The sample cannot go in the in-order chunk. Check if it can go in the out-of-order chunk. - if oooTimeWindow > 0 && t >= headMaxt-oooTimeWindow { - return true, headMaxt - t, nil - } - - // The sample cannot go in both in-order and out-of-order chunk. - if oooTimeWindow > 0 { - return true, headMaxt - t, storage.ErrTooOldSample - } - if t < minValidTime { - return false, headMaxt - t, storage.ErrOutOfBounds - } - return false, headMaxt - t, storage.ErrOutOfOrderSample -} - -// appendableFloatHistogram checks whether the given float histogram sample is valid for appending to the series. (if we return false and no error) -// The sample belongs to the out of order chunk if we return true and no error. -// An error signifies the sample cannot be handled. -func (s *memSeries) appendableFloatHistogram(t int64, fh *histogram.FloatHistogram, headMaxt, minValidTime, oooTimeWindow int64) (isOOO bool, oooDelta int64, err error) { - // Check if we can append in the in-order chunk. - if t >= minValidTime { - if s.headChunks == nil { - // The series has no sample and was freshly created. - return false, 0, nil - } - msMaxt := s.maxTime() - if t > msMaxt { - return false, 0, nil - } - if t == msMaxt { - // We are allowing exact duplicates as we can encounter them in valid cases - // like federation and erroring out at that time would be extremely noisy. - // This only checks against the latest in-order sample. - // The OOO headchunk has its own method to detect these duplicates. - if !fh.Equals(s.lastFloatHistogramValue) { - return false, 0, storage.ErrDuplicateSampleForTimestamp - } - // Sample is identical (ts + value) with most current (highest ts) sample in sampleBuf. - return false, 0, nil - } - } - - // The sample cannot go in the in-order chunk. Check if it can go in the out-of-order chunk. - if oooTimeWindow > 0 && t >= headMaxt-oooTimeWindow { - return true, headMaxt - t, nil - } - - // The sample cannot go in both in-order and out-of-order chunk. - if oooTimeWindow > 0 { - return true, headMaxt - t, storage.ErrTooOldSample - } - if t < minValidTime { - return false, headMaxt - t, storage.ErrOutOfBounds - } - return false, headMaxt - t, storage.ErrOutOfOrderSample -} - -// AppendExemplar for headAppender assumes the series ref already exists, and so it doesn't -// use getOrCreate or make any of the lset validity checks that Append does. -func (a *headAppender) AppendExemplar(ref storage.SeriesRef, lset labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { - // Check if exemplar storage is enabled. - if !a.head.opts.EnableExemplarStorage || a.head.opts.MaxExemplars.Load() <= 0 { - return 0, nil - } - - // Get Series - s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) - if s == nil { - s = a.head.series.getByHash(lset.Hash(), lset) - if s != nil { - ref = storage.SeriesRef(s.ref) - } - } - if s == nil { - return 0, fmt.Errorf("unknown HeadSeriesRef when trying to add exemplar: %d", ref) - } - - // Ensure no empty labels have gotten through. - e.Labels = e.Labels.WithoutEmpty() - - err := a.head.exemplars.ValidateExemplar(s.labels(), e) - if err != nil { - if errors.Is(err, storage.ErrDuplicateExemplar) || errors.Is(err, storage.ErrExemplarsDisabled) { - // Duplicate, don't return an error but don't accept the exemplar. - return 0, nil - } - return 0, err - } - - b := a.getCurrentBatch(stNone, chunks.HeadSeriesRef(ref)) - b.exemplars = append(b.exemplars, exemplarWithSeriesRef{ref, e}) - - return storage.SeriesRef(s.ref), nil -} - -func (a *headAppender) AppendHistogram(ref storage.SeriesRef, lset labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - // Fail fast if OOO is disabled and the sample is out of bounds. - // Otherwise a full check will be done later to decide if the sample is in-order or out-of-order. - if a.oooTimeWindow == 0 && t < a.minValidTime { - a.head.metrics.outOfBoundSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - return 0, storage.ErrOutOfBounds - } - - if h != nil { - if err := h.Validate(); err != nil { - return 0, err - } - } - - if fh != nil { - if err := fh.Validate(); err != nil { - return 0, err - } - } - - s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) - if s == nil { - var err error - s, _, err = a.getOrCreate(lset) - if err != nil { - return 0, err - } - } - - switch { - case h != nil: - s.Lock() - // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise - // to skip that sample from the WAL and write only in the WBL. - _, delta, err := s.appendableHistogram(t, h, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err != nil { - s.pendingCommit = true - } +func (a *headAppenderV2) appendHistogram(s *memSeries, t int64, h *histogram.Histogram, fastRejectOOO bool) error { + s.Lock() + // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise + // to skip that sample from the WAL and write only in the WBL. + isOOO, delta, err := s.appendableHistogram(t, h, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if isOOO && fastRejectOOO { s.Unlock() - if delta > 0 { - a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) - } - if err != nil { - switch { - case errors.Is(err, storage.ErrOutOfOrderSample): - a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - case errors.Is(err, storage.ErrTooOldSample): - a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - } - return 0, err - } - st := stHistogram - if h.UsesCustomBuckets() { - st = stCustomBucketHistogram - } - b := a.getCurrentBatch(st, s.ref) - b.histograms = append(b.histograms, record.RefHistogramSample{ - Ref: s.ref, - T: t, - H: h, - }) - b.histogramSeries = append(b.histogramSeries, s) - case fh != nil: - s.Lock() - // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise - // to skip that sample from the WAL and write only in the WBL. - _, delta, err := s.appendableFloatHistogram(t, fh, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err == nil { - s.pendingCommit = true - } - s.Unlock() - if delta > 0 { - a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) - } - if err != nil { - switch { - case errors.Is(err, storage.ErrOutOfOrderSample): - a.head.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - case errors.Is(err, storage.ErrTooOldSample): - a.head.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - } - return 0, err - } - st := stFloatHistogram - if fh.UsesCustomBuckets() { - st = stCustomBucketFloatHistogram - } - b := a.getCurrentBatch(st, s.ref) - b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ - Ref: s.ref, - T: t, - FH: fh, - }) - b.floatHistogramSeries = append(b.floatHistogramSeries, s) + return storage.ErrOutOfOrderSample } - - if t < a.mint { - a.mint = t - } - if t > a.maxt { - a.maxt = t - } - - return storage.SeriesRef(s.ref), nil -} - -func (a *headAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, lset labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - if st >= t { - return 0, storage.ErrSTNewerThanSample - } - - s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) - if s == nil { - var err error - s, _, err = a.getOrCreate(lset) - if err != nil { - return 0, err - } - } - - switch { - case h != nil: - zeroHistogram := &histogram.Histogram{ - // The STZeroSample represents a counter reset by definition. - CounterResetHint: histogram.CounterReset, - // Replicate other fields to avoid needless chunk creation. - Schema: h.Schema, - ZeroThreshold: h.ZeroThreshold, - CustomValues: h.CustomValues, - } - s.Lock() - // For STZeroSamples OOO is not allowed. - // We set it to true to make this implementation as close as possible to the float implementation. - isOOO, _, err := s.appendableHistogram(st, zeroHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err != nil { - s.Unlock() - if errors.Is(err, storage.ErrOutOfOrderSample) { - return 0, storage.ErrOutOfOrderST - } - - return 0, err - } - - // OOO is not allowed because after the first scrape, ST will be the same for most (if not all) future samples. - // This is to prevent the injected zero from being marked as OOO forever. - if isOOO { - s.Unlock() - return 0, storage.ErrOutOfOrderST - } - + if err == nil { s.pendingCommit = true + } + s.Unlock() + if delta > 0 { + a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) + } + if err != nil { + return err + } + st := stHistogram + if h.UsesCustomBuckets() { + st = stCustomBucketHistogram + } + b := a.getCurrentBatch(st, s.ref) + b.histograms = append(b.histograms, record.RefHistogramSample{Ref: s.ref, T: t, H: h}) + b.histogramSeries = append(b.histogramSeries, s) + return nil +} + +func (a *headAppenderV2) appendFloatHistogram(s *memSeries, t int64, fh *histogram.FloatHistogram, fastRejectOOO bool) error { + s.Lock() + // TODO(codesome): If we definitely know at this point that the sample is ooo, then optimise + // to skip that sample from the WAL and write only in the WBL. + isOOO, delta, err := s.appendableFloatHistogram(t, fh, a.headMaxt, a.minValidTime, a.oooTimeWindow) + if isOOO && fastRejectOOO { s.Unlock() - sTyp := stHistogram - if h.UsesCustomBuckets() { - sTyp = stCustomBucketHistogram + return storage.ErrOutOfOrderSample + } + if err == nil { + s.pendingCommit = true + } + s.Unlock() + if delta > 0 { + a.head.metrics.oooHistogram.Observe(float64(delta) / 1000) + } + if err != nil { + return err + } + st := stFloatHistogram + if fh.UsesCustomBuckets() { + st = stCustomBucketFloatHistogram + } + b := a.getCurrentBatch(st, s.ref) + b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{Ref: s.ref, T: t, FH: fh}) + b.floatHistogramSeries = append(b.floatHistogramSeries, s) + return nil +} + +func (a *headAppenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemplar) error { + var errs []error + for _, e := range exemplar { + // Ensure no empty labels have gotten through. + e.Labels = e.Labels.WithoutEmpty() + if err := a.head.exemplars.ValidateExemplar(s.labels(), e); err != nil { + if !errors.Is(err, storage.ErrDuplicateExemplar) && !errors.Is(err, storage.ErrExemplarsDisabled) { + // Except duplicates, return partial errors. + errs = append(errs, err) + } + if !errors.Is(err, storage.ErrOutOfOrderExemplar) { + a.head.logger.Debug("Error while adding an exemplar on AppendSample", "exemplars", fmt.Sprintf("%+v", e), "err", e) + } + continue } - b := a.getCurrentBatch(sTyp, s.ref) - b.histograms = append(b.histograms, record.RefHistogramSample{ - Ref: s.ref, - T: st, - H: zeroHistogram, - }) - b.histogramSeries = append(b.histogramSeries, s) + b := a.getCurrentBatch(stNone, s.ref) + b.exemplars = append(b.exemplars, exemplarWithSeriesRef{storage.SeriesRef(s.ref), e}) + } + if len(errs) > 0 { + return &storage.AppendPartialError{ExemplarErrors: errs} + } + return nil +} + +// NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 +// is implemented. +// +// ST is an experimental feature, we don't fail the append on errors, just debug log. +func (a *headAppenderV2) bestEffortAppendSTZeroSample(s *memSeries, st, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) { + if st >= t { + a.head.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrSTNewerThanSample) + return + } + if st < a.minValidTime { + a.head.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrOutOfBounds) + return + } + + var err error + switch { case fh != nil: zeroFloatHistogram := &histogram.FloatHistogram{ // The STZeroSample represents a counter reset by definition. @@ -968,1318 +364,33 @@ func (a *headAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, lset l ZeroThreshold: fh.ZeroThreshold, CustomValues: fh.CustomValues, } - s.Lock() - // We set it to true to make this implementation as close as possible to the float implementation. - isOOO, _, err := s.appendableFloatHistogram(st, zeroFloatHistogram, a.headMaxt, a.minValidTime, a.oooTimeWindow) // OOO is not allowed for STZeroSamples. - if err != nil { - s.Unlock() - if errors.Is(err, storage.ErrOutOfOrderSample) { - return 0, storage.ErrOutOfOrderST - } - - return 0, err + err = a.appendFloatHistogram(s, st, zeroFloatHistogram, true) + case h != nil: + zeroHistogram := &histogram.Histogram{ + // The STZeroSample represents a counter reset by definition. + CounterResetHint: histogram.CounterReset, + // Replicate other fields to avoid needless chunk creation. + Schema: h.Schema, + ZeroThreshold: h.ZeroThreshold, + CustomValues: h.CustomValues, } + err = a.appendHistogram(s, st, zeroHistogram, true) + default: + err = a.appendFloat(s, st, 0, true) + } - // OOO is not allowed because after the first scrape, ST will be the same for most (if not all) future samples. - // This is to prevent the injected zero from being marked as OOO forever. - if isOOO { - s.Unlock() - return 0, storage.ErrOutOfOrderST + if err != nil { + if errors.Is(err, storage.ErrOutOfOrderSample) { + // OOO errors are common and expected (cumulative). Explicitly ignored. + return } - - s.pendingCommit = true - s.Unlock() - sTyp := stFloatHistogram - if fh.UsesCustomBuckets() { - sTyp = stCustomBucketFloatHistogram - } - b := a.getCurrentBatch(sTyp, s.ref) - b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ - Ref: s.ref, - T: st, - FH: zeroFloatHistogram, - }) - b.floatHistogramSeries = append(b.floatHistogramSeries, s) + a.head.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", err) + return } if st > a.maxt { a.maxt = st } - - return storage.SeriesRef(s.ref), nil } -// UpdateMetadata for headAppender assumes the series ref already exists, and so it doesn't -// use getOrCreate or make any of the lset sanity checks that Append does. -func (a *headAppender) UpdateMetadata(ref storage.SeriesRef, lset labels.Labels, meta metadata.Metadata) (storage.SeriesRef, error) { - s := a.head.series.getByID(chunks.HeadSeriesRef(ref)) - if s == nil { - s = a.head.series.getByHash(lset.Hash(), lset) - if s != nil { - ref = storage.SeriesRef(s.ref) - } - } - if s == nil { - return 0, fmt.Errorf("unknown series when trying to add metadata with HeadSeriesRef: %d and labels: %s", ref, lset) - } - - s.Lock() - hasNewMetadata := s.meta == nil || *s.meta != meta - s.Unlock() - - if hasNewMetadata { - b := a.getCurrentBatch(stNone, s.ref) - b.metadata = append(b.metadata, record.RefMetadata{ - Ref: s.ref, - Type: record.GetMetricType(meta.Type), - Unit: meta.Unit, - Help: meta.Help, - }) - b.metadataSeries = append(b.metadataSeries, s) - } - - return ref, nil -} - -var _ storage.GetRef = &headAppender{} - -func (a *headAppender) GetRef(lset labels.Labels, hash uint64) (storage.SeriesRef, labels.Labels) { - s := a.head.series.getByHash(hash, lset) - if s == nil { - return 0, labels.EmptyLabels() - } - // returned labels must be suitable to pass to Append() - return storage.SeriesRef(s.ref), s.labels() -} - -// log writes all headAppender's data to the WAL. -func (a *headAppender) log() error { - if a.head.wal == nil { - return nil - } - - buf := a.head.getBytesBuffer() - defer func() { a.head.putBytesBuffer(buf) }() - - var rec []byte - var enc record.Encoder - - if len(a.seriesRefs) > 0 { - rec = enc.Series(a.seriesRefs, buf) - buf = rec[:0] - - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log series: %w", err) - } - } - for _, b := range a.batches { - if len(b.metadata) > 0 { - rec = enc.Metadata(b.metadata, buf) - buf = rec[:0] - - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log metadata: %w", err) - } - } - // It's important to do (float) Samples before histogram samples - // to end up with the correct order. - if len(b.floats) > 0 { - rec = enc.Samples(b.floats, buf) - buf = rec[:0] - - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log samples: %w", err) - } - } - if len(b.histograms) > 0 { - var customBucketsHistograms []record.RefHistogramSample - rec, customBucketsHistograms = enc.HistogramSamples(b.histograms, buf) - buf = rec[:0] - if len(rec) > 0 { - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log histograms: %w", err) - } - } - - if len(customBucketsHistograms) > 0 { - rec = enc.CustomBucketsHistogramSamples(customBucketsHistograms, buf) - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log custom buckets histograms: %w", err) - } - } - } - if len(b.floatHistograms) > 0 { - var customBucketsFloatHistograms []record.RefFloatHistogramSample - rec, customBucketsFloatHistograms = enc.FloatHistogramSamples(b.floatHistograms, buf) - buf = rec[:0] - if len(rec) > 0 { - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log float histograms: %w", err) - } - } - - if len(customBucketsFloatHistograms) > 0 { - rec = enc.CustomBucketsFloatHistogramSamples(customBucketsFloatHistograms, buf) - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log custom buckets float histograms: %w", err) - } - } - } - // Exemplars should be logged after samples (float/native histogram/etc), - // otherwise it might happen that we send the exemplars in a remote write - // batch before the samples, which in turn means the exemplar is rejected - // for missing series, since series are created due to samples. - if len(b.exemplars) > 0 { - rec = enc.Exemplars(exemplarsForEncoding(b.exemplars), buf) - buf = rec[:0] - - if err := a.head.wal.Log(rec); err != nil { - return fmt.Errorf("log exemplars: %w", err) - } - } - } - return nil -} - -func exemplarsForEncoding(es []exemplarWithSeriesRef) []record.RefExemplar { - ret := make([]record.RefExemplar, 0, len(es)) - for _, e := range es { - ret = append(ret, record.RefExemplar{ - Ref: chunks.HeadSeriesRef(e.ref), - T: e.exemplar.Ts, - V: e.exemplar.Value, - Labels: e.exemplar.Labels, - }) - } - return ret -} - -type appenderCommitContext struct { - floatsAppended int - histogramsAppended int - // Number of samples out of order but accepted: with ooo enabled and within time window. - oooFloatsAccepted int - oooHistogramAccepted int - // Number of samples rejected due to: out of order but OOO support disabled. - floatOOORejected int - histoOOORejected int - // Number of samples rejected due to: out of order but too old (OOO support enabled, but outside time window). - floatTooOldRejected int - histoTooOldRejected int - // Number of samples rejected due to: out of bounds: with t < minValidTime (OOO support disabled). - floatOOBRejected int - histoOOBRejected int - inOrderMint int64 - inOrderMaxt int64 - oooMinT int64 - oooMaxT int64 - wblSamples []record.RefSample - wblHistograms []record.RefHistogramSample - wblFloatHistograms []record.RefFloatHistogramSample - oooMmapMarkers map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef - oooMmapMarkersCount int - oooRecords [][]byte - oooCapMax int64 - appendChunkOpts chunkOpts - enc record.Encoder -} - -// commitExemplars adds all exemplars from the provided batch to the head's exemplar storage. -func (a *headAppender) commitExemplars(b *appendBatch) { - // No errors logging to WAL, so pass the exemplars along to the in memory storage. - for _, e := range b.exemplars { - s := a.head.series.getByID(chunks.HeadSeriesRef(e.ref)) - if s == nil { - // This is very unlikely to happen, but we have seen it in the wild. - // It means that the series was truncated between AppendExemplar and Commit. - // See TestHeadCompactionWhileAppendAndCommitExemplar. - continue - } - // We don't instrument exemplar appends here, all is instrumented by storage. - if err := a.head.exemplars.AddExemplar(s.labels(), e.exemplar); err != nil { - if errors.Is(err, storage.ErrOutOfOrderExemplar) { - continue - } - a.head.logger.Debug("Unknown error while adding exemplar", "err", err) - } - } -} - -func (acc *appenderCommitContext) collectOOORecords(a *headAppender) { - if a.head.wbl == nil { - // WBL is not enabled. So no need to collect. - acc.wblSamples = nil - acc.wblHistograms = nil - acc.wblFloatHistograms = nil - acc.oooMmapMarkers = nil - acc.oooMmapMarkersCount = 0 - return - } - - // The m-map happens before adding a new sample. So we collect - // the m-map markers first, and then samples. - // WBL Graphically: - // WBL Before this Commit(): [old samples before this commit for chunk 1] - // WBL After this Commit(): [old samples before this commit for chunk 1][new samples in this commit for chunk 1]mmapmarker1[samples for chunk 2]mmapmarker2[samples for chunk 3] - if acc.oooMmapMarkers != nil { - markers := make([]record.RefMmapMarker, 0, acc.oooMmapMarkersCount) - for ref, mmapRefs := range acc.oooMmapMarkers { - for _, mmapRef := range mmapRefs { - markers = append(markers, record.RefMmapMarker{ - Ref: ref, - MmapRef: mmapRef, - }) - } - } - r := acc.enc.MmapMarkers(markers, a.head.getBytesBuffer()) - acc.oooRecords = append(acc.oooRecords, r) - } - - if len(acc.wblSamples) > 0 { - r := acc.enc.Samples(acc.wblSamples, a.head.getBytesBuffer()) - acc.oooRecords = append(acc.oooRecords, r) - } - if len(acc.wblHistograms) > 0 { - r, customBucketsHistograms := acc.enc.HistogramSamples(acc.wblHistograms, a.head.getBytesBuffer()) - if len(r) > 0 { - acc.oooRecords = append(acc.oooRecords, r) - } - if len(customBucketsHistograms) > 0 { - r := acc.enc.CustomBucketsHistogramSamples(customBucketsHistograms, a.head.getBytesBuffer()) - acc.oooRecords = append(acc.oooRecords, r) - } - } - if len(acc.wblFloatHistograms) > 0 { - r, customBucketsFloatHistograms := acc.enc.FloatHistogramSamples(acc.wblFloatHistograms, a.head.getBytesBuffer()) - if len(r) > 0 { - acc.oooRecords = append(acc.oooRecords, r) - } - if len(customBucketsFloatHistograms) > 0 { - r := acc.enc.CustomBucketsFloatHistogramSamples(customBucketsFloatHistograms, a.head.getBytesBuffer()) - acc.oooRecords = append(acc.oooRecords, r) - } - } - - acc.wblSamples = nil - acc.wblHistograms = nil - acc.wblFloatHistograms = nil - acc.oooMmapMarkers = nil -} - -// handleAppendableError processes errors encountered during sample appending and updates -// the provided counters accordingly. -// -// Parameters: -// - err: The error encountered during appending. -// - appended: Pointer to the counter tracking the number of successfully appended samples. -// - oooRejected: Pointer to the counter tracking the number of out-of-order samples rejected. -// - oobRejected: Pointer to the counter tracking the number of out-of-bounds samples rejected. -// - tooOldRejected: Pointer to the counter tracking the number of too-old samples rejected. -func handleAppendableError(err error, appended, oooRejected, oobRejected, tooOldRejected *int) { - switch { - case errors.Is(err, storage.ErrOutOfOrderSample): - *appended-- - *oooRejected++ - case errors.Is(err, storage.ErrOutOfBounds): - *appended-- - *oobRejected++ - case errors.Is(err, storage.ErrTooOldSample): - *appended-- - *tooOldRejected++ - default: - *appended-- - } -} - -// commitFloats processes and commits the samples in the provided batch to the -// series. It handles both in-order and out-of-order samples, updating the -// appenderCommitContext with the results of the append operations. -// -// The function iterates over the samples in the headAppender and attempts to append each sample -// to its corresponding series. It handles various error cases such as out-of-order samples, -// out-of-bounds samples, and too-old samples, updating the appenderCommitContext accordingly. -// -// For out-of-order samples, it checks if the sample can be inserted into the series and updates -// the out-of-order mmap markers if necessary. It also updates the write-ahead log (WBL) samples -// and the minimum and maximum timestamps for out-of-order samples. -// -// For in-order samples, it attempts to append the sample to the series and updates the minimum -// and maximum timestamps for in-order samples. -// -// The function also increments the chunk metrics if a new chunk is created and performs cleanup -// operations on the series after appending the samples. -// -// There are also specific functions to commit histograms and float histograms. -func (a *headAppender) commitFloats(b *appendBatch, acc *appenderCommitContext) { - var ok, chunkCreated bool - var series *memSeries - - for i, s := range b.floats { - series = b.floatSeries[i] - series.Lock() - - if value.IsStaleNaN(s.V) { - // If a float staleness marker had been appended for a - // series that got a histogram or float histogram - // appended before via this same appender, it would not - // show up here because we had already converted it. We - // end up here for two reasons: (1) This is the very - // first sample for this series appended via this - // appender. (2) A float sample was appended to this - // series before via this same appender. - // - // In either case, we need to check the previous sample - // in the memSeries to append the appropriately typed - // staleness marker. This is obviously so in case (1). - // In case (2), we would usually expect a float sample - // as the previous sample, but there might be concurrent - // appends that have added a histogram sample in the - // meantime. (This will probably lead to OOO shenanigans - // anyway, but that's a different story.) - // - // If the last sample in the memSeries is indeed a - // float, we don't have to do anything special here and - // just go on with the normal commit for a float sample. - // However, if the last sample in the memSeries is a - // histogram or float histogram, we have to convert the - // staleness marker to a histogram (or float histogram, - // respectively), and just add it at the end of the - // histograms (or float histograms) in the same batch, - // to be committed later in commitHistograms (or - // commitFloatHistograms). The latter is fine because we - // know there is no other histogram (or float histogram) - // sample for this same series in this same batch - // (because any such sample would have triggered a new - // batch). - switch { - case series.lastHistogramValue != nil: - b.histograms = append(b.histograms, record.RefHistogramSample{ - Ref: series.ref, - T: s.T, - H: &histogram.Histogram{Sum: s.V}, - }) - b.histogramSeries = append(b.histogramSeries, series) - // This sample was counted as a float but is now a histogram. - acc.floatsAppended-- - acc.histogramsAppended++ - series.Unlock() - continue - case series.lastFloatHistogramValue != nil: - b.floatHistograms = append(b.floatHistograms, record.RefFloatHistogramSample{ - Ref: series.ref, - T: s.T, - FH: &histogram.FloatHistogram{Sum: s.V}, - }) - b.floatHistogramSeries = append(b.floatHistogramSeries, series) - // This sample was counted as a float but is now a float histogram. - acc.floatsAppended-- - acc.histogramsAppended++ - series.Unlock() - continue - } - } - oooSample, _, err := series.appendable(s.T, s.V, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err != nil { - handleAppendableError(err, &acc.floatsAppended, &acc.floatOOORejected, &acc.floatOOBRejected, &acc.floatTooOldRejected) - } - - switch { - case err != nil: - // Do nothing here. - case oooSample: - // Sample is OOO and OOO handling is enabled - // and the delta is within the OOO tolerance. - var mmapRefs []chunks.ChunkDiskMapperRef - ok, chunkCreated, mmapRefs = series.insert(s.T, s.V, nil, nil, a.head.chunkDiskMapper, acc.oooCapMax, a.head.logger) - if chunkCreated { - r, ok := acc.oooMmapMarkers[series.ref] - if !ok || r != nil { - // !ok means there are no markers collected for these samples yet. So we first flush the samples - // before setting this m-map marker. - - // r != nil means we have already m-mapped a chunk for this series in the same Commit(). - // Hence, before we m-map again, we should add the samples and m-map markers - // seen till now to the WBL records. - acc.collectOOORecords(a) - } - - if acc.oooMmapMarkers == nil { - acc.oooMmapMarkers = make(map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef) - } - if len(mmapRefs) > 0 { - acc.oooMmapMarkers[series.ref] = mmapRefs - acc.oooMmapMarkersCount += len(mmapRefs) - } else { - // No chunk was written to disk, so we need to set an initial marker for this series. - acc.oooMmapMarkers[series.ref] = []chunks.ChunkDiskMapperRef{0} - acc.oooMmapMarkersCount++ - } - } - if ok { - acc.wblSamples = append(acc.wblSamples, s) - if s.T < acc.oooMinT { - acc.oooMinT = s.T - } - if s.T > acc.oooMaxT { - acc.oooMaxT = s.T - } - acc.oooFloatsAccepted++ - } else { - // Sample is an exact duplicate of the last sample. - // NOTE: We can only detect updates if they clash with a sample in the OOOHeadChunk, - // not with samples in already flushed OOO chunks. - // TODO(codesome): Add error reporting? It depends on addressing https://github.com/prometheus/prometheus/discussions/10305. - acc.floatsAppended-- - } - default: - newlyStale := !value.IsStaleNaN(series.lastValue) && value.IsStaleNaN(s.V) - staleToNonStale := value.IsStaleNaN(series.lastValue) && !value.IsStaleNaN(s.V) - ok, chunkCreated = series.append(s.T, s.V, a.appendID, acc.appendChunkOpts) - if ok { - if s.T < acc.inOrderMint { - acc.inOrderMint = s.T - } - if s.T > acc.inOrderMaxt { - acc.inOrderMaxt = s.T - } - if newlyStale { - a.head.numStaleSeries.Inc() - } - if staleToNonStale { - a.head.numStaleSeries.Dec() - } - } else { - // The sample is an exact duplicate, and should be silently dropped. - acc.floatsAppended-- - } - } - - if chunkCreated { - a.head.metrics.chunks.Inc() - a.head.metrics.chunksCreated.Inc() - } - - series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) - series.pendingCommit = false - series.Unlock() - } -} - -// For details on the commitHistograms function, see the commitFloats docs. -func (a *headAppender) commitHistograms(b *appendBatch, acc *appenderCommitContext) { - var ok, chunkCreated bool - var series *memSeries - - for i, s := range b.histograms { - series = b.histogramSeries[i] - series.Lock() - - // At this point, we could encounter a histogram staleness - // marker that should better be a float staleness marker or a - // float histogram staleness marker. This can only happen with - // concurrent appenders appending to the same series _and_ doing - // so in a mixed-type scenario. This case is expected to be very - // rare, so we do not bother here to convert the staleness - // marker. The worst case is that we need to cut a new chunk - // just for the staleness marker. - - oooSample, _, err := series.appendableHistogram(s.T, s.H, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err != nil { - handleAppendableError(err, &acc.histogramsAppended, &acc.histoOOORejected, &acc.histoOOBRejected, &acc.histoTooOldRejected) - } - - switch { - case err != nil: - // Do nothing here. - case oooSample: - // Sample is OOO and OOO handling is enabled - // and the delta is within the OOO tolerance. - var mmapRefs []chunks.ChunkDiskMapperRef - ok, chunkCreated, mmapRefs = series.insert(s.T, 0, s.H, nil, a.head.chunkDiskMapper, acc.oooCapMax, a.head.logger) - if chunkCreated { - r, ok := acc.oooMmapMarkers[series.ref] - if !ok || r != nil { - // !ok means there are no markers collected for these samples yet. So we first flush the samples - // before setting this m-map marker. - - // r != 0 means we have already m-mapped a chunk for this series in the same Commit(). - // Hence, before we m-map again, we should add the samples and m-map markers - // seen till now to the WBL records. - acc.collectOOORecords(a) - } - - if acc.oooMmapMarkers == nil { - acc.oooMmapMarkers = make(map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef) - } - if len(mmapRefs) > 0 { - acc.oooMmapMarkers[series.ref] = mmapRefs - acc.oooMmapMarkersCount += len(mmapRefs) - } else { - // No chunk was written to disk, so we need to set an initial marker for this series. - acc.oooMmapMarkers[series.ref] = []chunks.ChunkDiskMapperRef{0} - acc.oooMmapMarkersCount++ - } - } - if ok { - acc.wblHistograms = append(acc.wblHistograms, s) - if s.T < acc.oooMinT { - acc.oooMinT = s.T - } - if s.T > acc.oooMaxT { - acc.oooMaxT = s.T - } - acc.oooHistogramAccepted++ - } else { - // Sample is an exact duplicate of the last sample. - // NOTE: We can only detect updates if they clash with a sample in the OOOHeadChunk, - // not with samples in already flushed OOO chunks. - // TODO(codesome): Add error reporting? It depends on addressing https://github.com/prometheus/prometheus/discussions/10305. - acc.histogramsAppended-- - } - default: - newlyStale := value.IsStaleNaN(s.H.Sum) - staleToNonStale := false - if series.lastHistogramValue != nil { - newlyStale = newlyStale && !value.IsStaleNaN(series.lastHistogramValue.Sum) - staleToNonStale = value.IsStaleNaN(series.lastHistogramValue.Sum) && !value.IsStaleNaN(s.H.Sum) - } - ok, chunkCreated = series.appendHistogram(s.T, s.H, a.appendID, acc.appendChunkOpts) - if ok { - if s.T < acc.inOrderMint { - acc.inOrderMint = s.T - } - if s.T > acc.inOrderMaxt { - acc.inOrderMaxt = s.T - } - if newlyStale { - a.head.numStaleSeries.Inc() - } - if staleToNonStale { - a.head.numStaleSeries.Dec() - } - } else { - acc.histogramsAppended-- - acc.histoOOORejected++ - } - } - - if chunkCreated { - a.head.metrics.chunks.Inc() - a.head.metrics.chunksCreated.Inc() - } - - series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) - series.pendingCommit = false - series.Unlock() - } -} - -// For details on the commitFloatHistograms function, see the commitFloats docs. -func (a *headAppender) commitFloatHistograms(b *appendBatch, acc *appenderCommitContext) { - var ok, chunkCreated bool - var series *memSeries - - for i, s := range b.floatHistograms { - series = b.floatHistogramSeries[i] - series.Lock() - - // At this point, we could encounter a float histogram staleness - // marker that should better be a float staleness marker or an - // integer histogram staleness marker. This can only happen with - // concurrent appenders appending to the same series _and_ doing - // so in a mixed-type scenario. This case is expected to be very - // rare, so we do not bother here to convert the staleness - // marker. The worst case is that we need to cut a new chunk - // just for the staleness marker. - - oooSample, _, err := series.appendableFloatHistogram(s.T, s.FH, a.headMaxt, a.minValidTime, a.oooTimeWindow) - if err != nil { - handleAppendableError(err, &acc.histogramsAppended, &acc.histoOOORejected, &acc.histoOOBRejected, &acc.histoTooOldRejected) - } - - switch { - case err != nil: - // Do nothing here. - case oooSample: - // Sample is OOO and OOO handling is enabled - // and the delta is within the OOO tolerance. - var mmapRefs []chunks.ChunkDiskMapperRef - ok, chunkCreated, mmapRefs = series.insert(s.T, 0, nil, s.FH, a.head.chunkDiskMapper, acc.oooCapMax, a.head.logger) - if chunkCreated { - r, ok := acc.oooMmapMarkers[series.ref] - if !ok || r != nil { - // !ok means there are no markers collected for these samples yet. So we first flush the samples - // before setting this m-map marker. - - // r != 0 means we have already m-mapped a chunk for this series in the same Commit(). - // Hence, before we m-map again, we should add the samples and m-map markers - // seen till now to the WBL records. - acc.collectOOORecords(a) - } - - if acc.oooMmapMarkers == nil { - acc.oooMmapMarkers = make(map[chunks.HeadSeriesRef][]chunks.ChunkDiskMapperRef) - } - if len(mmapRefs) > 0 { - acc.oooMmapMarkers[series.ref] = mmapRefs - acc.oooMmapMarkersCount += len(mmapRefs) - } else { - // No chunk was written to disk, so we need to set an initial marker for this series. - acc.oooMmapMarkers[series.ref] = []chunks.ChunkDiskMapperRef{0} - acc.oooMmapMarkersCount++ - } - } - if ok { - acc.wblFloatHistograms = append(acc.wblFloatHistograms, s) - if s.T < acc.oooMinT { - acc.oooMinT = s.T - } - if s.T > acc.oooMaxT { - acc.oooMaxT = s.T - } - acc.oooHistogramAccepted++ - } else { - // Sample is an exact duplicate of the last sample. - // NOTE: We can only detect updates if they clash with a sample in the OOOHeadChunk, - // not with samples in already flushed OOO chunks. - // TODO(codesome): Add error reporting? It depends on addressing https://github.com/prometheus/prometheus/discussions/10305. - acc.histogramsAppended-- - } - default: - newlyStale := value.IsStaleNaN(s.FH.Sum) - staleToNonStale := false - if series.lastFloatHistogramValue != nil { - newlyStale = newlyStale && !value.IsStaleNaN(series.lastFloatHistogramValue.Sum) - staleToNonStale = value.IsStaleNaN(series.lastFloatHistogramValue.Sum) && !value.IsStaleNaN(s.FH.Sum) - } - ok, chunkCreated = series.appendFloatHistogram(s.T, s.FH, a.appendID, acc.appendChunkOpts) - if ok { - if s.T < acc.inOrderMint { - acc.inOrderMint = s.T - } - if s.T > acc.inOrderMaxt { - acc.inOrderMaxt = s.T - } - if newlyStale { - a.head.numStaleSeries.Inc() - } - if staleToNonStale { - a.head.numStaleSeries.Dec() - } - } else { - acc.histogramsAppended-- - acc.histoOOORejected++ - } - } - - if chunkCreated { - a.head.metrics.chunks.Inc() - a.head.metrics.chunksCreated.Inc() - } - - series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) - series.pendingCommit = false - series.Unlock() - } -} - -// commitMetadata commits the metadata for each series in the provided batch. -// It iterates over the metadata slice and updates the corresponding series -// with the new metadata information. The series is locked during the update -// to ensure thread safety. -func commitMetadata(b *appendBatch) { - var series *memSeries - for i, m := range b.metadata { - series = b.metadataSeries[i] - series.Lock() - series.meta = &metadata.Metadata{Type: record.ToMetricType(m.Type), Unit: m.Unit, Help: m.Help} - series.Unlock() - } -} - -func (a *headAppender) unmarkCreatedSeriesAsPendingCommit() { - for _, s := range a.series { - s.Lock() - s.pendingCommit = false - s.Unlock() - } -} - -// Commit writes to the WAL and adds the data to the Head. -// TODO(codesome): Refactor this method to reduce indentation and make it more readable. -func (a *headAppender) Commit() (err error) { - if a.closed { - return ErrAppenderClosed - } - - h := a.head - - defer func() { - if a.closed { - // Don't double-close in case Rollback() was called. - return - } - h.putRefSeriesBuffer(a.seriesRefs) - h.putSeriesBuffer(a.series) - h.putTypeMap(a.typesInBatch) - a.closed = true - }() - - if err := a.log(); err != nil { - _ = a.Rollback() // Most likely the same error will happen again. - return fmt.Errorf("write to WAL: %w", err) - } - - if h.writeNotified != nil { - h.writeNotified.Notify() - } - - acc := &appenderCommitContext{ - inOrderMint: math.MaxInt64, - inOrderMaxt: math.MinInt64, - oooMinT: math.MaxInt64, - oooMaxT: math.MinInt64, - oooCapMax: h.opts.OutOfOrderCapMax.Load(), - appendChunkOpts: chunkOpts{ - chunkDiskMapper: h.chunkDiskMapper, - chunkRange: h.chunkRange.Load(), - samplesPerChunk: h.opts.SamplesPerChunk, - }, - } - - for _, b := range a.batches { - acc.floatsAppended += len(b.floats) - acc.histogramsAppended += len(b.histograms) + len(b.floatHistograms) - a.commitExemplars(b) - defer b.close(h) - } - defer h.metrics.activeAppenders.Dec() - defer h.iso.closeAppend(a.appendID) - - defer func() { - for i := range acc.oooRecords { - h.putBytesBuffer(acc.oooRecords[i][:0]) - } - }() - - for _, b := range a.batches { - // Do not change the order of these calls. We depend on it for - // correct commit order of samples and for the staleness marker - // handling. - a.commitFloats(b, acc) - a.commitHistograms(b, acc) - a.commitFloatHistograms(b, acc) - commitMetadata(b) - } - // Unmark all series as pending commit after all samples have been committed. - a.unmarkCreatedSeriesAsPendingCommit() - - h.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatOOORejected)) - h.metrics.outOfOrderSamples.WithLabelValues(sampleMetricTypeHistogram).Add(float64(acc.histoOOORejected)) - h.metrics.outOfBoundSamples.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatOOBRejected)) - h.metrics.tooOldSamples.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatTooOldRejected)) - h.metrics.samplesAppended.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.floatsAppended)) - h.metrics.samplesAppended.WithLabelValues(sampleMetricTypeHistogram).Add(float64(acc.histogramsAppended)) - h.metrics.outOfOrderSamplesAppended.WithLabelValues(sampleMetricTypeFloat).Add(float64(acc.oooFloatsAccepted)) - h.metrics.outOfOrderSamplesAppended.WithLabelValues(sampleMetricTypeHistogram).Add(float64(acc.oooHistogramAccepted)) - h.updateMinMaxTime(acc.inOrderMint, acc.inOrderMaxt) - h.updateMinOOOMaxOOOTime(acc.oooMinT, acc.oooMaxT) - - acc.collectOOORecords(a) - if h.wbl != nil { - if err := h.wbl.Log(acc.oooRecords...); err != nil { - // TODO(codesome): Currently WBL logging of ooo samples is best effort here since we cannot try logging - // until we have found what samples become OOO. We can try having a metric for this failure. - // Returning the error here is not correct because we have already put the samples into the memory, - // hence the append/insert was a success. - h.logger.Error("Failed to log out of order samples into the WAL", "err", err) - } - } - return nil -} - -// insert is like append, except it inserts. Used for OOO samples. -func (s *memSeries) insert(t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, chunkDiskMapper *chunks.ChunkDiskMapper, oooCapMax int64, logger *slog.Logger) (inserted, chunkCreated bool, mmapRefs []chunks.ChunkDiskMapperRef) { - if s.ooo == nil { - s.ooo = &memSeriesOOOFields{} - } - c := s.ooo.oooHeadChunk - if c == nil || c.chunk.NumSamples() == int(oooCapMax) { - // Note: If no new samples come in then we rely on compaction to clean up stale in-memory OOO chunks. - c, mmapRefs = s.cutNewOOOHeadChunk(t, chunkDiskMapper, logger) - chunkCreated = true - } - - ok := c.chunk.Insert(t, v, h, fh) - if ok { - if chunkCreated || t < c.minTime { - c.minTime = t - } - if chunkCreated || t > c.maxTime { - c.maxTime = t - } - } - return ok, chunkCreated, mmapRefs -} - -// chunkOpts are chunk-level options that are passed when appending to a memSeries. -type chunkOpts struct { - chunkDiskMapper *chunks.ChunkDiskMapper - chunkRange int64 - samplesPerChunk int -} - -// append adds the sample (t, v) to the series. The caller also has to provide -// the appendID for isolation. (The appendID can be zero, which results in no -// isolation for this append.) -// Series lock must be held when calling. -func (s *memSeries) append(t int64, v float64, appendID uint64, o chunkOpts) (sampleInOrder, chunkCreated bool) { - c, sampleInOrder, chunkCreated := s.appendPreprocessor(t, chunkenc.EncXOR, o) - if !sampleInOrder { - return sampleInOrder, chunkCreated - } - s.app.Append(t, v) - - c.maxTime = t - - s.lastValue = v - s.lastHistogramValue = nil - s.lastFloatHistogramValue = nil - - if appendID > 0 { - s.txs.add(appendID) - } - - return true, chunkCreated -} - -// appendHistogram adds the histogram. -// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. -// In case of recoding the existing chunk, a new chunk is allocated and the old chunk is dropped. -// To keep the meaning of prometheus_tsdb_head_chunks and prometheus_tsdb_head_chunks_created_total -// consistent, we return chunkCreated=false in this case. -func (s *memSeries) appendHistogram(t int64, h *histogram.Histogram, appendID uint64, o chunkOpts) (sampleInOrder, chunkCreated bool) { - // Head controls the execution of recoding, so that we own the proper - // chunk reference afterwards and mmap used up chunks. - - // Ignoring ok is ok, since we don't want to compare to the wrong previous appender anyway. - prevApp, _ := s.app.(*chunkenc.HistogramAppender) - - c, sampleInOrder, chunkCreated := s.histogramsAppendPreprocessor(t, chunkenc.EncHistogram, o) - if !sampleInOrder { - return sampleInOrder, chunkCreated - } - - var ( - newChunk chunkenc.Chunk - recoded bool - ) - - if !chunkCreated { - // Ignore the previous appender if we continue the current chunk. - prevApp = nil - } - - newChunk, recoded, s.app, _ = s.app.AppendHistogram(prevApp, t, h, false) // false=request a new chunk if needed - - s.lastHistogramValue = h - s.lastFloatHistogramValue = nil - - if appendID > 0 { - s.txs.add(appendID) - } - - if newChunk == nil { // Sample was appended to existing chunk or is the first sample in a new chunk. - c.maxTime = t - return true, chunkCreated - } - - if recoded { // The appender needed to recode the chunk. - c.maxTime = t - c.chunk = newChunk - return true, false - } - - s.headChunks = &memChunk{ - chunk: newChunk, - minTime: t, - maxTime: t, - prev: s.headChunks, - } - s.nextAt = rangeForTimestamp(t, o.chunkRange) - return true, true -} - -// appendFloatHistogram adds the float histogram. -// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. -// In case of recoding the existing chunk, a new chunk is allocated and the old chunk is dropped. -// To keep the meaning of prometheus_tsdb_head_chunks and prometheus_tsdb_head_chunks_created_total -// consistent, we return chunkCreated=false in this case. -func (s *memSeries) appendFloatHistogram(t int64, fh *histogram.FloatHistogram, appendID uint64, o chunkOpts) (sampleInOrder, chunkCreated bool) { - // Head controls the execution of recoding, so that we own the proper - // chunk reference afterwards and mmap used up chunks. - - // Ignoring ok is ok, since we don't want to compare to the wrong previous appender anyway. - prevApp, _ := s.app.(*chunkenc.FloatHistogramAppender) - - c, sampleInOrder, chunkCreated := s.histogramsAppendPreprocessor(t, chunkenc.EncFloatHistogram, o) - if !sampleInOrder { - return sampleInOrder, chunkCreated - } - - var ( - newChunk chunkenc.Chunk - recoded bool - ) - - if !chunkCreated { - // Ignore the previous appender if we continue the current chunk. - prevApp = nil - } - - newChunk, recoded, s.app, _ = s.app.AppendFloatHistogram(prevApp, t, fh, false) // False means request a new chunk if needed. - - s.lastHistogramValue = nil - s.lastFloatHistogramValue = fh - - if appendID > 0 { - s.txs.add(appendID) - } - - if newChunk == nil { // Sample was appended to existing chunk or is the first sample in a new chunk. - c.maxTime = t - return true, chunkCreated - } - - if recoded { // The appender needed to recode the chunk. - c.maxTime = t - c.chunk = newChunk - return true, false - } - - s.headChunks = &memChunk{ - chunk: newChunk, - minTime: t, - maxTime: t, - prev: s.headChunks, - } - s.nextAt = rangeForTimestamp(t, o.chunkRange) - return true, true -} - -// appendPreprocessor takes care of cutting new XOR chunks and m-mapping old ones. XOR chunks are cut based on the -// number of samples they contain with a soft cap in bytes. -// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. -// This should be called only when appending data. -func (s *memSeries) appendPreprocessor(t int64, e chunkenc.Encoding, o chunkOpts) (c *memChunk, sampleInOrder, chunkCreated bool) { - // We target chunkenc.MaxBytesPerXORChunk as a hard for the size of an XOR chunk. We must determine whether to cut - // a new head chunk without knowing the size of the next sample, however, so we assume the next sample will be a - // maximally-sized sample (19 bytes). - const maxBytesPerXORChunk = chunkenc.MaxBytesPerXORChunk - 19 - - c = s.headChunks - - if c == nil { - if len(s.mmappedChunks) > 0 && s.mmappedChunks[len(s.mmappedChunks)-1].maxTime >= t { - // Out of order sample. Sample timestamp is already in the mmapped chunks, so ignore it. - return c, false, false - } - // There is no head chunk in this series yet, create the first chunk for the sample. - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - // Out of order sample. - if c.maxTime >= t { - return c, false, chunkCreated - } - - // Check the chunk size, unless we just created it and if the chunk is too large, cut a new one. - if !chunkCreated && len(c.chunk.Bytes()) > maxBytesPerXORChunk { - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - if c.chunk.Encoding() != e { - // The chunk encoding expected by this append is different than the head chunk's - // encoding. So we cut a new chunk with the expected encoding. - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - numSamples := c.chunk.NumSamples() - if numSamples == 0 { - // It could be the new chunk created after reading the chunk snapshot, - // hence we fix the minTime of the chunk here. - c.minTime = t - s.nextAt = rangeForTimestamp(c.minTime, o.chunkRange) - } - - // If we reach 25% of a chunk's desired sample count, predict an end time - // for this chunk that will try to make samples equally distributed within - // the remaining chunks in the current chunk range. - // At latest it must happen at the timestamp set when the chunk was cut. - if numSamples == o.samplesPerChunk/4 { - s.nextAt = computeChunkEndTime(c.minTime, c.maxTime, s.nextAt, 4) - } - // If numSamples > samplesPerChunk*2 then our previous prediction was invalid, - // most likely because samples rate has changed and now they are arriving more frequently. - // Since we assume that the rate is higher, we're being conservative and cutting at 2*samplesPerChunk - // as we expect more chunks to come. - // Note that next chunk will have its nextAt recalculated for the new rate. - if t >= s.nextAt || numSamples >= o.samplesPerChunk*2 { - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - return c, true, chunkCreated -} - -// histogramsAppendPreprocessor takes care of cutting new histogram chunks and m-mapping old ones. Histogram chunks are -// cut based on their size in bytes. -// It is unsafe to call this concurrently with s.iterator(...) without holding the series lock. -// This should be called only when appending data. -func (s *memSeries) histogramsAppendPreprocessor(t int64, e chunkenc.Encoding, o chunkOpts) (c *memChunk, sampleInOrder, chunkCreated bool) { - c = s.headChunks - - if c == nil { - if len(s.mmappedChunks) > 0 && s.mmappedChunks[len(s.mmappedChunks)-1].maxTime >= t { - // Out of order sample. Sample timestamp is already in the mmapped chunks, so ignore it. - return c, false, false - } - // There is no head chunk in this series yet, create the first chunk for the sample. - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - // Out of order sample. - if c.maxTime >= t { - return c, false, chunkCreated - } - - if c.chunk.Encoding() != e { - // The chunk encoding expected by this append is different than the head chunk's - // encoding. So we cut a new chunk with the expected encoding. - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - numSamples := c.chunk.NumSamples() - targetBytes := chunkenc.TargetBytesPerHistogramChunk - numBytes := len(c.chunk.Bytes()) - - if numSamples == 0 { - // It could be the new chunk created after reading the chunk snapshot, - // hence we fix the minTime of the chunk here. - c.minTime = t - s.nextAt = rangeForTimestamp(c.minTime, o.chunkRange) - } - - // Below, we will enforce chunkenc.MinSamplesPerHistogramChunk. There are, however, two cases that supersede it: - // - The current chunk range is ending before chunkenc.MinSamplesPerHistogramChunk will be satisfied. - // - s.nextAt was set while loading a chunk snapshot with the intent that a new chunk be cut on the next append. - var nextChunkRangeStart int64 - if s.histogramChunkHasComputedEndTime { - nextChunkRangeStart = rangeForTimestamp(c.minTime, o.chunkRange) - } else { - // If we haven't yet computed an end time yet, s.nextAt is either set to - // rangeForTimestamp(c.minTime, o.chunkRange) or was set while loading a chunk snapshot. Either way, we want to - // skip enforcing chunkenc.MinSamplesPerHistogramChunk. - nextChunkRangeStart = s.nextAt - } - - // If we reach 25% of a chunk's desired maximum size, predict an end time - // for this chunk that will try to make samples equally distributed within - // the remaining chunks in the current chunk range. - // At the latest it must happen at the timestamp set when the chunk was cut. - if !s.histogramChunkHasComputedEndTime && numBytes >= targetBytes/4 { - ratioToFull := float64(targetBytes) / float64(numBytes) - s.nextAt = computeChunkEndTime(c.minTime, c.maxTime, s.nextAt, ratioToFull) - s.histogramChunkHasComputedEndTime = true - } - // If numBytes > targetBytes*2 then our previous prediction was invalid. This could happen if the sample rate has - // increased or if the bucket/span count has increased. - // Note that next chunk will have its nextAt recalculated for the new rate. - if (t >= s.nextAt || numBytes >= targetBytes*2) && (numSamples >= chunkenc.MinSamplesPerHistogramChunk || t >= nextChunkRangeStart) { - c = s.cutNewHeadChunk(t, e, o.chunkRange) - chunkCreated = true - } - - // The new chunk will also need a new computed end time. - if chunkCreated { - s.histogramChunkHasComputedEndTime = false - } - - return c, true, chunkCreated -} - -// computeChunkEndTime estimates the end timestamp based the beginning of a -// chunk, its current timestamp and the upper bound up to which we insert data. -// It assumes that the time range is 1/ratioToFull full. -// Assuming that the samples will keep arriving at the same rate, it will make the -// remaining n chunks within this chunk range (before max) equally sized. -func computeChunkEndTime(start, cur, maxT int64, ratioToFull float64) int64 { - n := float64(maxT-start) / (float64(cur-start+1) * ratioToFull) - if n <= 1 { - return maxT - } - return int64(float64(start) + float64(maxT-start)/math.Floor(n)) -} - -func (s *memSeries) cutNewHeadChunk(mint int64, e chunkenc.Encoding, chunkRange int64) *memChunk { - // When cutting a new head chunk we create a new memChunk instance with .prev - // pointing at the current .headChunks, so it forms a linked list. - // All but first headChunks list elements will be m-mapped as soon as possible - // so this is a single element list most of the time. - s.headChunks = &memChunk{ - minTime: mint, - maxTime: math.MinInt64, - prev: s.headChunks, - } - - if chunkenc.IsValidEncoding(e) { - var err error - s.headChunks.chunk, err = chunkenc.NewEmptyChunk(e) - if err != nil { - panic(err) // This should never happen. - } - } else { - s.headChunks.chunk = chunkenc.NewXORChunk() - } - - // Set upper bound on when the next chunk must be started. An earlier timestamp - // may be chosen dynamically at a later point. - s.nextAt = rangeForTimestamp(mint, chunkRange) - - app, err := s.headChunks.chunk.Appender() - if err != nil { - panic(err) - } - s.app = app - return s.headChunks -} - -// cutNewOOOHeadChunk cuts a new OOO chunk and m-maps the old chunk. -// The caller must ensure that s is locked and s.ooo is not nil. -func (s *memSeries) cutNewOOOHeadChunk(mint int64, chunkDiskMapper *chunks.ChunkDiskMapper, logger *slog.Logger) (*oooHeadChunk, []chunks.ChunkDiskMapperRef) { - ref := s.mmapCurrentOOOHeadChunk(chunkDiskMapper, logger) - - s.ooo.oooHeadChunk = &oooHeadChunk{ - chunk: NewOOOChunk(), - minTime: mint, - maxTime: math.MinInt64, - } - - return s.ooo.oooHeadChunk, ref -} - -// s must be locked when calling. -func (s *memSeries) mmapCurrentOOOHeadChunk(chunkDiskMapper *chunks.ChunkDiskMapper, logger *slog.Logger) []chunks.ChunkDiskMapperRef { - if s.ooo == nil || s.ooo.oooHeadChunk == nil { - // OOO is not enabled or there is no head chunk, so nothing to m-map here. - return nil - } - chks, err := s.ooo.oooHeadChunk.chunk.ToEncodedChunks(math.MinInt64, math.MaxInt64) - if err != nil { - handleChunkWriteError(err) - return nil - } - chunkRefs := make([]chunks.ChunkDiskMapperRef, 0, len(chks)) - for _, memchunk := range chks { - if len(s.ooo.oooMmappedChunks) >= (oooChunkIDMask - 1) { - logger.Error("Too many OOO chunks, dropping data", "series", s.lset.String()) - break - } - chunkRef := chunkDiskMapper.WriteChunk(s.ref, memchunk.minTime, memchunk.maxTime, memchunk.chunk, true, handleChunkWriteError) - chunkRefs = append(chunkRefs, chunkRef) - s.ooo.oooMmappedChunks = append(s.ooo.oooMmappedChunks, &mmappedChunk{ - ref: chunkRef, - numSamples: uint16(memchunk.chunk.NumSamples()), - minTime: memchunk.minTime, - maxTime: memchunk.maxTime, - }) - } - s.ooo.oooHeadChunk = nil - return chunkRefs -} - -// mmapChunks will m-map all but first chunk on s.headChunks list. -func (s *memSeries) mmapChunks(chunkDiskMapper *chunks.ChunkDiskMapper) (count int) { - if s.headChunks == nil || s.headChunks.prev == nil { - // There is none or only one head chunk, so nothing to m-map here. - return count - } - - // Write chunks starting from the oldest one and stop before we get to current s.headChunks. - // If we have this chain: s.headChunks{t4} -> t3 -> t2 -> t1 -> t0 - // then we need to write chunks t0 to t3, but skip s.headChunks. - for i := s.headChunks.len() - 1; i > 0; i-- { - chk := s.headChunks.atOffset(i) - chunkRef := chunkDiskMapper.WriteChunk(s.ref, chk.minTime, chk.maxTime, chk.chunk, false, handleChunkWriteError) - s.mmappedChunks = append(s.mmappedChunks, &mmappedChunk{ - ref: chunkRef, - numSamples: uint16(chk.chunk.NumSamples()), - minTime: chk.minTime, - maxTime: chk.maxTime, - }) - count++ - } - - // Once we've written out all chunks except s.headChunks we need to unlink these from s.headChunk. - s.headChunks.prev = nil - - return count -} - -func handleChunkWriteError(err error) { - if err != nil && !errors.Is(err, chunks.ErrChunkDiskMapperClosed) { - panic(err) - } -} - -// Rollback removes the samples and exemplars from headAppender and writes any series to WAL. -func (a *headAppender) Rollback() (err error) { - if a.closed { - return ErrAppenderClosed - } - h := a.head - defer func() { - a.unmarkCreatedSeriesAsPendingCommit() - h.iso.closeAppend(a.appendID) - h.metrics.activeAppenders.Dec() - a.closed = true - h.putRefSeriesBuffer(a.seriesRefs) - h.putSeriesBuffer(a.series) - h.putTypeMap(a.typesInBatch) - }() - - var series *memSeries - for _, b := range a.batches { - for i := range b.floats { - series = b.floatSeries[i] - series.Lock() - series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) - series.pendingCommit = false - series.Unlock() - } - for i := range b.histograms { - series = b.histogramSeries[i] - series.Lock() - series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) - series.pendingCommit = false - series.Unlock() - } - for i := range b.floatHistograms { - series = b.floatHistogramSeries[i] - series.Lock() - series.cleanupAppendIDsBelow(a.cleanupAppendIDsBelow) - series.pendingCommit = false - series.Unlock() - } - b.close(h) - } - a.batches = a.batches[:0] - // Series are created in the head memory regardless of rollback. Thus we have - // to log them to the WAL in any case. - return a.log() -} +var _ storage.GetRef = &headAppenderV2{} diff --git a/tsdb/head_append_v2_test.go b/tsdb/head_append_v2_test.go index 552db13d07..33bc3aec38 100644 --- a/tsdb/head_append_v2_test.go +++ b/tsdb/head_append_v2_test.go @@ -1,4 +1,4 @@ -// Copyright 2017 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -35,7 +35,6 @@ import ( "github.com/prometheus/client_golang/prometheus" prom_testutil "github.com/prometheus/client_golang/prometheus/testutil" dto "github.com/prometheus/client_model/go" - "github.com/prometheus/common/promslog" "github.com/stretchr/testify/require" "go.uber.org/atomic" "golang.org/x/sync/errgroup" @@ -48,8 +47,6 @@ import ( "github.com/prometheus/prometheus/storage" "github.com/prometheus/prometheus/tsdb/chunkenc" "github.com/prometheus/prometheus/tsdb/chunks" - "github.com/prometheus/prometheus/tsdb/fileutil" - "github.com/prometheus/prometheus/tsdb/index" "github.com/prometheus/prometheus/tsdb/record" "github.com/prometheus/prometheus/tsdb/tombstones" "github.com/prometheus/prometheus/tsdb/tsdbutil" @@ -58,463 +55,19 @@ import ( "github.com/prometheus/prometheus/util/testutil" ) -// newTestHeadDefaultOptions returns the HeadOptions that should be used by default in unit tests. -func newTestHeadDefaultOptions(chunkRange int64, oooEnabled bool) *HeadOptions { - opts := DefaultHeadOptions() - opts.ChunkRange = chunkRange - opts.EnableExemplarStorage = true - opts.MaxExemplars.Store(config.DefaultExemplarsConfig.MaxExemplars) - if oooEnabled { - opts.OutOfOrderTimeWindow.Store(10 * time.Minute.Milliseconds()) - } - return opts -} +// TODO(bwplotka): Ensure non-ported tests are not deleted from db_test.go when removing AppenderV1 flow (#17632), +// for example: +// * TestChunkNotFoundHeadGCRace +// * TestHeadSeriesChunkRace +// * TestHeadLabelValuesWithMatchers +// * TestHeadLabelNamesWithMatchers +// * TestHeadShardedPostings -func newTestHead(t testing.TB, chunkRange int64, compressWAL compression.Type, oooEnabled bool) (*Head, *wlog.WL) { - return newTestHeadWithOptions(t, compressWAL, newTestHeadDefaultOptions(chunkRange, oooEnabled)) -} - -func newTestHeadWithOptions(t testing.TB, compressWAL compression.Type, opts *HeadOptions) (*Head, *wlog.WL) { - dir := t.TempDir() - wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compressWAL) - require.NoError(t, err) - - // Override the chunks dir with the testing one. - opts.ChunkDirRoot = dir - - h, err := NewHead(nil, nil, wal, nil, opts, nil) - require.NoError(t, err) - - require.NoError(t, h.chunkDiskMapper.IterateAllChunks(func(chunks.HeadSeriesRef, chunks.ChunkDiskMapperRef, int64, int64, uint16, chunkenc.Encoding, bool) error { - return nil - })) - - return h, wal -} - -func BenchmarkCreateSeries(b *testing.B) { - series := genSeries(b.N, 10, 0, 0) - h, _ := newTestHead(b, 10000, compression.None, false) - b.Cleanup(func() { - require.NoError(b, h.Close()) - }) - - b.ReportAllocs() - b.ResetTimer() - - for _, s := range series { - h.getOrCreate(s.Labels().Hash(), s.Labels(), false) - } -} - -func BenchmarkHeadAppender_Append_Commit_ExistingSeries(b *testing.B) { - seriesCounts := []int{100, 1000, 10000} - series := genSeries(10000, 10, 0, 0) - - for _, seriesCount := range seriesCounts { - b.Run(fmt.Sprintf("%d series", seriesCount), func(b *testing.B) { - for _, samplesPerAppend := range []int64{1, 2, 5, 100} { - b.Run(fmt.Sprintf("%d samples per append", samplesPerAppend), func(b *testing.B) { - h, _ := newTestHead(b, 10000, compression.None, false) - b.Cleanup(func() { require.NoError(b, h.Close()) }) - - ts := int64(1000) - appendSamples := func() error { - var err error - app := h.Appender(context.Background()) - for _, s := range series[:seriesCount] { - var ref storage.SeriesRef - for sampleIndex := range samplesPerAppend { - ref, err = app.Append(ref, s.Labels(), ts+sampleIndex, float64(ts+sampleIndex)) - if err != nil { - return err - } - } - } - ts += 1000 // should increment more than highest samplesPerAppend - return app.Commit() - } - - // Init series, that's not what we're benchmarking here. - require.NoError(b, appendSamples()) - - b.ReportAllocs() - b.ResetTimer() - - for b.Loop() { - require.NoError(b, appendSamples()) - } - }) - } - }) - } -} - -func populateTestWL(t testing.TB, w *wlog.WL, recs []any, buf []byte) []byte { - var enc record.Encoder - for _, r := range recs { - buf = buf[:0] - switch v := r.(type) { - case []record.RefSeries: - buf = enc.Series(v, buf) - case []record.RefSample: - buf = enc.Samples(v, buf) - case []tombstones.Stone: - buf = enc.Tombstones(v, buf) - case []record.RefExemplar: - buf = enc.Exemplars(v, buf) - case []record.RefHistogramSample: - buf, _ = enc.HistogramSamples(v, buf) - case []record.RefFloatHistogramSample: - buf, _ = enc.FloatHistogramSamples(v, buf) - case []record.RefMmapMarker: - buf = enc.MmapMarkers(v, buf) - case []record.RefMetadata: - buf = enc.Metadata(v, buf) - default: - continue - } - require.NoError(t, w.Log(buf)) - } - return buf -} - -func readTestWAL(t testing.TB, dir string) (recs []any) { - sr, err := wlog.NewSegmentsReader(dir) - require.NoError(t, err) - defer func() { - require.NoError(t, sr.Close()) - }() - - dec := record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) - r := wlog.NewReader(sr) - - for r.Next() { - rec := r.Record() - - switch dec.Type(rec) { - case record.Series: - series, err := dec.Series(rec, nil) - require.NoError(t, err) - recs = append(recs, series) - case record.Samples: - samples, err := dec.Samples(rec, nil) - require.NoError(t, err) - recs = append(recs, samples) - case record.HistogramSamples, record.CustomBucketsHistogramSamples: - samples, err := dec.HistogramSamples(rec, nil) - require.NoError(t, err) - recs = append(recs, samples) - case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: - samples, err := dec.FloatHistogramSamples(rec, nil) - require.NoError(t, err) - recs = append(recs, samples) - case record.Tombstones: - tstones, err := dec.Tombstones(rec, nil) - require.NoError(t, err) - recs = append(recs, tstones) - case record.Metadata: - meta, err := dec.Metadata(rec, nil) - require.NoError(t, err) - recs = append(recs, meta) - case record.Exemplars: - exemplars, err := dec.Exemplars(rec, nil) - require.NoError(t, err) - recs = append(recs, exemplars) - default: - require.Fail(t, "unknown record type") - } - } - require.NoError(t, r.Err()) - return recs -} - -func BenchmarkLoadWLs(b *testing.B) { - cases := []struct { - // Total series is (batches*seriesPerBatch). - batches int - seriesPerBatch int - samplesPerSeries int - mmappedChunkT int64 - // The first oooSeriesPct*seriesPerBatch series in a batch are selected as "OOO" series. - oooSeriesPct float64 - // The first oooSamplesPct*samplesPerSeries samples in an OOO series are written as OOO samples. - oooSamplesPct float64 - oooCapMax int64 - }{ - { // Less series and more samples. 2 hour WAL with 1 second scrape interval. - batches: 10, - seriesPerBatch: 100, - samplesPerSeries: 7200, - }, - { // More series and less samples. - batches: 10, - seriesPerBatch: 10000, - samplesPerSeries: 50, - }, - { // In between. - batches: 10, - seriesPerBatch: 1000, - samplesPerSeries: 480, - }, - { // 2 hour WAL with 15 second scrape interval, and mmapped chunks up to last 100 samples. - batches: 100, - seriesPerBatch: 1000, - samplesPerSeries: 480, - mmappedChunkT: 3800, - }, - { // A lot of OOO samples (50% series with 50% of samples being OOO). - batches: 10, - seriesPerBatch: 1000, - samplesPerSeries: 480, - oooSeriesPct: 0.5, - oooSamplesPct: 0.5, - oooCapMax: DefaultOutOfOrderCapMax, - }, - { // Fewer OOO samples (10% of series with 10% of samples being OOO). - batches: 10, - seriesPerBatch: 1000, - samplesPerSeries: 480, - oooSeriesPct: 0.1, - oooSamplesPct: 0.1, - }, - { // 2 hour WAL with 15 second scrape interval, and mmapped chunks up to last 100 samples. - // Four mmap markers per OOO series: 480 * 0.3 = 144, 144 / 32 (DefaultOutOfOrderCapMax) = 4. - batches: 100, - seriesPerBatch: 1000, - samplesPerSeries: 480, - mmappedChunkT: 3800, - oooSeriesPct: 0.2, - oooSamplesPct: 0.3, - oooCapMax: DefaultOutOfOrderCapMax, - }, - } - - labelsPerSeries := 5 - // Rough estimates of most common % of samples that have an exemplar for each scrape. - exemplarsPercentages := []float64{0, 0.5, 1, 5} - lastExemplarsPerSeries := -1 - for _, c := range cases { - missingSeriesPercentages := []float64{0, 0.1} - for _, missingSeriesPct := range missingSeriesPercentages { - for _, p := range exemplarsPercentages { - exemplarsPerSeries := int(math.RoundToEven(float64(c.samplesPerSeries) * p / 100)) - // For tests with low samplesPerSeries we could end up testing with 0 exemplarsPerSeries - // multiple times without this check. - if exemplarsPerSeries == lastExemplarsPerSeries { - continue - } - lastExemplarsPerSeries = exemplarsPerSeries - b.Run(fmt.Sprintf("batches=%d,seriesPerBatch=%d,samplesPerSeries=%d,exemplarsPerSeries=%d,mmappedChunkT=%d,oooSeriesPct=%.3f,oooSamplesPct=%.3f,oooCapMax=%d,missingSeriesPct=%.3f", c.batches, c.seriesPerBatch, c.samplesPerSeries, exemplarsPerSeries, c.mmappedChunkT, c.oooSeriesPct, c.oooSamplesPct, c.oooCapMax, missingSeriesPct), - func(b *testing.B) { - dir := b.TempDir() - - wal, err := wlog.New(nil, nil, dir, compression.None) - require.NoError(b, err) - var wbl *wlog.WL - if c.oooSeriesPct != 0 { - wbl, err = wlog.New(nil, nil, dir, compression.None) - require.NoError(b, err) - } - - // Write series. - refSeries := make([]record.RefSeries, 0, c.seriesPerBatch) - var buf []byte - builder := labels.NewBuilder(labels.EmptyLabels()) - for j := 1; j < labelsPerSeries; j++ { - builder.Set(defaultLabelName+strconv.Itoa(j), defaultLabelValue+strconv.Itoa(j)) - } - for k := 0; k < c.batches; k++ { - refSeries = refSeries[:0] - for i := k * c.seriesPerBatch; i < (k+1)*c.seriesPerBatch; i++ { - builder.Set(defaultLabelName, strconv.Itoa(i)) - refSeries = append(refSeries, record.RefSeries{Ref: chunks.HeadSeriesRef(i) * 101, Labels: builder.Labels()}) - } - - writeSeries := refSeries - if missingSeriesPct > 0 { - newWriteSeries := make([]record.RefSeries, 0, int(float64(len(refSeries))*(1.0-missingSeriesPct))) - keepRatio := 1.0 - missingSeriesPct - // Keep approximately every 1/keepRatio series. - for i, s := range refSeries { - if int(float64(i)*keepRatio) != int(float64(i+1)*keepRatio) { - newWriteSeries = append(newWriteSeries, s) - } - } - writeSeries = newWriteSeries - } - - buf = populateTestWL(b, wal, []any{writeSeries}, buf) - } - - // Write samples. - refSamples := make([]record.RefSample, 0, c.seriesPerBatch) - - oooSeriesPerBatch := int(float64(c.seriesPerBatch) * c.oooSeriesPct) - oooSamplesPerSeries := int(float64(c.samplesPerSeries) * c.oooSamplesPct) - - for i := 0; i < c.samplesPerSeries; i++ { - for j := 0; j < c.batches; j++ { - refSamples = refSamples[:0] - - k := j * c.seriesPerBatch - // Skip appending the first oooSamplesPerSeries samples for the series in the batch that - // should have OOO samples. OOO samples are appended after all the in-order samples. - if i < oooSamplesPerSeries { - k += oooSeriesPerBatch - } - for ; k < (j+1)*c.seriesPerBatch; k++ { - refSamples = append(refSamples, record.RefSample{ - Ref: chunks.HeadSeriesRef(k) * 101, - T: int64(i) * 10, - V: float64(i) * 100, - }) - } - buf = populateTestWL(b, wal, []any{refSamples}, buf) - } - } - - // Write mmapped chunks. - if c.mmappedChunkT != 0 { - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, mmappedChunksDir(dir), chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(b, err) - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: c.mmappedChunkT, - samplesPerChunk: DefaultSamplesPerChunk, - } - for k := 0; k < c.batches*c.seriesPerBatch; k++ { - // Create one mmapped chunk per series, with one sample at the given time. - s := newMemSeries(labels.Labels{}, chunks.HeadSeriesRef(k)*101, 0, defaultIsolationDisabled, false) - s.append(c.mmappedChunkT, 42, 0, cOpts) - // There's only one head chunk because only a single sample is appended. mmapChunks() - // ignores the latest chunk, so we need to cut a new head chunk to guarantee the chunk with - // the sample at c.mmappedChunkT is mmapped. - s.cutNewHeadChunk(c.mmappedChunkT, chunkenc.EncXOR, c.mmappedChunkT) - s.mmapChunks(chunkDiskMapper) - } - require.NoError(b, chunkDiskMapper.Close()) - } - - // Write exemplars. - refExemplars := make([]record.RefExemplar, 0, c.seriesPerBatch) - for i := range exemplarsPerSeries { - for j := 0; j < c.batches; j++ { - refExemplars = refExemplars[:0] - for k := j * c.seriesPerBatch; k < (j+1)*c.seriesPerBatch; k++ { - refExemplars = append(refExemplars, record.RefExemplar{ - Ref: chunks.HeadSeriesRef(k) * 101, - T: int64(i) * 10, - V: float64(i) * 100, - Labels: labels.FromStrings("trace_id", fmt.Sprintf("trace-%d", i)), - }) - } - buf = populateTestWL(b, wal, []any{refExemplars}, buf) - } - } - - // Write OOO samples and mmap markers. - refMarkers := make([]record.RefMmapMarker, 0, oooSeriesPerBatch) - refSamples = make([]record.RefSample, 0, oooSeriesPerBatch) - for i := range oooSamplesPerSeries { - shouldAddMarkers := c.oooCapMax != 0 && i != 0 && int64(i)%c.oooCapMax == 0 - - for j := 0; j < c.batches; j++ { - refSamples = refSamples[:0] - if shouldAddMarkers { - refMarkers = refMarkers[:0] - } - for k := j * c.seriesPerBatch; k < (j*c.seriesPerBatch)+oooSeriesPerBatch; k++ { - ref := chunks.HeadSeriesRef(k) * 101 - if shouldAddMarkers { - // loadWBL() checks that the marker's MmapRef is less than or equal to the ref - // for the last mmap chunk. Setting MmapRef to 0 to always pass that check. - refMarkers = append(refMarkers, record.RefMmapMarker{Ref: ref, MmapRef: 0}) - } - refSamples = append(refSamples, record.RefSample{ - Ref: ref, - T: int64(i) * 10, - V: float64(i) * 100, - }) - } - if shouldAddMarkers { - populateTestWL(b, wbl, []any{refMarkers}, buf) - } - buf = populateTestWL(b, wal, []any{refSamples}, buf) - buf = populateTestWL(b, wbl, []any{refSamples}, buf) - } - } - - b.ResetTimer() - - // Load the WAL. - for b.Loop() { - opts := DefaultHeadOptions() - opts.ChunkRange = 1000 - opts.ChunkDirRoot = dir - if c.oooCapMax > 0 { - opts.OutOfOrderCapMax.Store(c.oooCapMax) - } - h, err := NewHead(nil, nil, wal, wbl, opts, nil) - require.NoError(b, err) - h.Init(0) - } - b.StopTimer() - wal.Close() - if wbl != nil { - wbl.Close() - } - }) - } - } - } -} - -// BenchmarkLoadRealWLs will be skipped unless the BENCHMARK_LOAD_REAL_WLS_DIR environment variable is set. -// BENCHMARK_LOAD_REAL_WLS_DIR should be the folder where `wal` and `chunks_head` are located. -// -// Using an absolute path for BENCHMARK_LOAD_REAL_WLS_DIR is recommended. -// -// Because WLs loading may alter BENCHMARK_LOAD_REAL_WLS_DIR which can affect benchmark results and to ensure consistency, -// a copy of BENCHMARK_LOAD_REAL_WLS_DIR is made for each iteration and deleted at the end. -// Make sure there is sufficient disk space for that. -func BenchmarkLoadRealWLs(b *testing.B) { - srcDir := os.Getenv("BENCHMARK_LOAD_REAL_WLS_DIR") - if srcDir == "" { - b.SkipNow() - } - - // Load the WAL. - for b.Loop() { - b.StopTimer() - dir := b.TempDir() - require.NoError(b, fileutil.CopyDirs(srcDir, dir)) - - wal, err := wlog.New(nil, nil, filepath.Join(dir, "wal"), compression.None) - require.NoError(b, err) - b.Cleanup(func() { wal.Close() }) - - wbl, err := wlog.New(nil, nil, filepath.Join(dir, "wbl"), compression.None) - require.NoError(b, err) - b.Cleanup(func() { wbl.Close() }) - b.StartTimer() - - opts := DefaultHeadOptions() - opts.ChunkDirRoot = dir - h, err := NewHead(nil, nil, wal, wbl, opts, nil) - require.NoError(b, err) - require.NoError(b, h.Init(0)) - - b.StopTimer() - require.NoError(b, os.RemoveAll(dir)) - } -} - -// TestHead_HighConcurrencyReadAndWrite generates 1000 series with a step of 15s and fills a whole block with samples, +// TestHeadAppenderV2_HighConcurrencyReadAndWrite generates 1000 series with a step of 15s and fills a whole block with samples, // this means in total it generates 4000 chunks because with a step of 15s there are 4 chunks per block per series. // While appending the samples to the head it concurrently queries them from multiple go routines and verifies that the // returned results are correct. -func TestHead_HighConcurrencyReadAndWrite(t *testing.T) { +func TestHeadAppenderV2_HighConcurrencyReadAndWrite(t *testing.T) { head, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) defer func() { require.NoError(t, head.Close()) @@ -579,10 +132,10 @@ func TestHead_HighConcurrencyReadAndWrite(t *testing.T) { return false, nil } - app := head.Appender(ctx) + app := head.AppenderV2(ctx) for i := range workerLabelSets { // We also use the timestamp as the sample value. - _, err := app.Append(0, workerLabelSets[i], int64(ts), float64(ts)) + _, err := app.Append(0, workerLabelSets[i], 0, int64(ts), float64(ts), nil, nil, storage.AOptions{}) if err != nil { return false, fmt.Errorf("Error when appending to head: %w", err) } @@ -704,160 +257,35 @@ func TestHead_HighConcurrencyReadAndWrite(t *testing.T) { require.NoError(t, g.Wait()) } -func TestHead_ReadWAL(t *testing.T) { - for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { - t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { - entries := []any{ - []record.RefSeries{ - {Ref: 10, Labels: labels.FromStrings("a", "1")}, - {Ref: 11, Labels: labels.FromStrings("a", "2")}, - {Ref: 100, Labels: labels.FromStrings("a", "3")}, - }, - []record.RefSample{ - {Ref: 0, T: 99, V: 1}, - {Ref: 10, T: 100, V: 2}, - {Ref: 100, T: 100, V: 3}, - }, - []record.RefSeries{ - {Ref: 50, Labels: labels.FromStrings("a", "4")}, - // This series has two refs pointing to it. - {Ref: 101, Labels: labels.FromStrings("a", "3")}, - }, - []record.RefSample{ - {Ref: 10, T: 101, V: 5}, - {Ref: 50, T: 101, V: 6}, - // Sample for duplicate series record. - {Ref: 101, T: 101, V: 7}, - }, - []tombstones.Stone{ - {Ref: 0, Intervals: []tombstones.Interval{{Mint: 99, Maxt: 101}}}, - // Tombstone for duplicate series record. - {Ref: 101, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 100}}}, - }, - []record.RefExemplar{ - {Ref: 10, T: 100, V: 1, Labels: labels.FromStrings("trace_id", "asdf")}, - // Exemplar for duplicate series record. - {Ref: 101, T: 101, V: 7, Labels: labels.FromStrings("trace_id", "zxcv")}, - }, - []record.RefMetadata{ - // Metadata for duplicate series record. - {Ref: 101, Type: uint8(record.Counter), Unit: "foo", Help: "total foo"}, - }, - } - - head, w := newTestHead(t, 1000, compress, false) - defer func() { - require.NoError(t, head.Close()) - }() - - populateTestWL(t, w, entries, nil) - - require.NoError(t, head.Init(math.MinInt64)) - require.Equal(t, uint64(101), head.lastSeriesID.Load()) - - s10 := head.series.getByID(10) - s11 := head.series.getByID(11) - s50 := head.series.getByID(50) - s100 := head.series.getByID(100) - s101 := head.series.getByID(101) - - testutil.RequireEqual(t, labels.FromStrings("a", "1"), s10.lset) - require.Nil(t, s11) // Series without samples should be garbage collected at head.Init(). - testutil.RequireEqual(t, labels.FromStrings("a", "4"), s50.lset) - testutil.RequireEqual(t, labels.FromStrings("a", "3"), s100.lset) - - // Duplicate series record should not be written to the head. - require.Nil(t, s101) - // But it should have a WAL expiry set. - keepUntil, ok := head.getWALExpiry(101) - require.True(t, ok) - require.Equal(t, int64(101), keepUntil) - // Only the duplicate series record should have a WAL expiry set. - _, ok = head.getWALExpiry(50) - require.False(t, ok) - - expandChunk := func(c chunkenc.Iterator) (x []sample) { - for c.Next() == chunkenc.ValFloat { - t, v := c.At() - x = append(x, sample{t: t, f: v}) - } - require.NoError(t, c.Err()) - return x - } - - // Verify samples and exemplar for series 10. - c, _, _, err := s10.chunk(0, head.chunkDiskMapper, &head.memChunkPool) - require.NoError(t, err) - require.Equal(t, []sample{{100, 2, nil, nil}, {101, 5, nil, nil}}, expandChunk(c.chunk.Iterator(nil))) - - q, err := head.ExemplarQuerier(context.Background()) - require.NoError(t, err) - e, err := q.Select(0, 1000, []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "1")}) - require.NoError(t, err) - require.NotEmpty(t, e) - require.NotEmpty(t, e[0].Exemplars) - require.True(t, exemplar.Exemplar{Ts: 100, Value: 1, Labels: labels.FromStrings("trace_id", "asdf")}.Equals(e[0].Exemplars[0])) - - // Verify samples for series 50 - c, _, _, err = s50.chunk(0, head.chunkDiskMapper, &head.memChunkPool) - require.NoError(t, err) - require.Equal(t, []sample{{101, 6, nil, nil}}, expandChunk(c.chunk.Iterator(nil))) - - // Verify records for series 100 and its duplicate, series 101. - // The samples before the new series record should be discarded since a duplicate record - // is only possible when old samples were compacted. - c, _, _, err = s100.chunk(0, head.chunkDiskMapper, &head.memChunkPool) - require.NoError(t, err) - require.Equal(t, []sample{{101, 7, nil, nil}}, expandChunk(c.chunk.Iterator(nil))) - - q, err = head.ExemplarQuerier(context.Background()) - require.NoError(t, err) - e, err = q.Select(0, 1000, []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "3")}) - require.NoError(t, err) - require.NotEmpty(t, e) - require.NotEmpty(t, e[0].Exemplars) - require.True(t, exemplar.Exemplar{Ts: 101, Value: 7, Labels: labels.FromStrings("trace_id", "zxcv")}.Equals(e[0].Exemplars[0])) - - require.NotNil(t, s100.meta) - require.Equal(t, "foo", s100.meta.Unit) - require.Equal(t, "total foo", s100.meta.Help) - - intervals, err := head.tombstones.Get(storage.SeriesRef(s100.ref)) - require.NoError(t, err) - require.Equal(t, tombstones.Intervals{{Mint: 0, Maxt: 100}}, intervals) - }) - } -} - -func TestHead_WALMultiRef(t *testing.T) { +func TestHeadAppenderV2_WALMultiRef(t *testing.T) { head, w := newTestHead(t, 1000, compression.None, false) require.NoError(t, head.Init(0)) - app := head.Appender(context.Background()) - ref1, err := app.Append(0, labels.FromStrings("foo", "bar"), 100, 1) + app := head.AppenderV2(context.Background()) + ref1, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 100, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) // Add another sample outside chunk range to mmap a chunk. - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 1500, 2) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 1500, 2, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 2.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) require.NoError(t, head.Truncate(1600)) - app = head.Appender(context.Background()) - ref2, err := app.Append(0, labels.FromStrings("foo", "bar"), 1700, 3) + app = head.AppenderV2(context.Background()) + ref2, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 1700, 3, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 3.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) // Add another sample outside chunk range to mmap a chunk. - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 2000, 4) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 2000, 4, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 4.0, prom_testutil.ToFloat64(head.metrics.chunksCreated)) @@ -889,344 +317,40 @@ func TestHead_WALMultiRef(t *testing.T) { }}, series) } -func TestHead_WALCheckpointMultiRef(t *testing.T) { - cases := []struct { - name string - walEntries []any - expectedWalExpiry int64 - walTruncateMinT int64 - expectedWalEntries []any - }{ - { - name: "Samples only; keep needed duplicate series record", - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefSample{ - {Ref: 1, T: 100, V: 1}, - {Ref: 2, T: 200, V: 2}, - {Ref: 2, T: 500, V: 3}, - }, - }, - expectedWalExpiry: 500, - walTruncateMinT: 500, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefSample{ - {Ref: 2, T: 500, V: 3}, - }, - }, - }, - { - name: "Tombstones only; keep needed duplicate series record", - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []tombstones.Stone{ - {Ref: 1, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 100}}}, - {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 200}}}, - {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 500}}}, - }, - }, - expectedWalExpiry: 500, - walTruncateMinT: 500, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []tombstones.Stone{ - {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 500}}}, - }, - }, - }, - { - name: "Exemplars only; keep needed duplicate series record", - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefExemplar{ - {Ref: 1, T: 100, V: 1, Labels: labels.FromStrings("trace_id", "asdf")}, - {Ref: 2, T: 200, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, - {Ref: 2, T: 500, V: 3, Labels: labels.FromStrings("trace_id", "asdf")}, - }, - }, - expectedWalExpiry: 500, - walTruncateMinT: 500, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefExemplar{ - {Ref: 2, T: 500, V: 3, Labels: labels.FromStrings("trace_id", "asdf")}, - }, - }, - }, - { - name: "Histograms only; keep needed duplicate series record", - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefHistogramSample{ - {Ref: 1, T: 100, H: &histogram.Histogram{}}, - {Ref: 2, T: 200, H: &histogram.Histogram{}}, - {Ref: 2, T: 500, H: &histogram.Histogram{}}, - }, - }, - expectedWalExpiry: 500, - walTruncateMinT: 500, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefHistogramSample{ - {Ref: 2, T: 500, H: &histogram.Histogram{}}, - }, - }, - }, - { - name: "Float histograms only; keep needed duplicate series record", - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefFloatHistogramSample{ - {Ref: 1, T: 100, FH: &histogram.FloatHistogram{}}, - {Ref: 2, T: 200, FH: &histogram.FloatHistogram{}}, - {Ref: 2, T: 500, FH: &histogram.FloatHistogram{}}, - }, - }, - expectedWalExpiry: 500, - walTruncateMinT: 500, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefFloatHistogramSample{ - {Ref: 2, T: 500, FH: &histogram.FloatHistogram{}}, - }, - }, - }, - { - name: "All record types; keep needed duplicate series record until last record", - // Series with 2 refs and samples for both - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefSample{ - {Ref: 2, T: 500, V: 3}, - }, - []tombstones.Stone{ - {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 500}}}, - }, - []record.RefExemplar{ - {Ref: 2, T: 800, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, - }, - []record.RefHistogramSample{ - {Ref: 2, T: 500, H: &histogram.Histogram{}}, - }, - []record.RefFloatHistogramSample{ - {Ref: 2, T: 500, FH: &histogram.FloatHistogram{}}, - }, - }, - expectedWalExpiry: 800, - walTruncateMinT: 700, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefExemplar{ - {Ref: 2, T: 800, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, - }, - }, - }, - { - name: "All record types; drop expired duplicate series record", - // Series with 2 refs and samples for both - walEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - {Ref: 2, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefSample{ - {Ref: 2, T: 500, V: 2}, - {Ref: 1, T: 900, V: 3}, - }, - []tombstones.Stone{ - {Ref: 2, Intervals: []tombstones.Interval{{Mint: 0, Maxt: 750}}}, - }, - []record.RefExemplar{ - {Ref: 2, T: 800, V: 2, Labels: labels.FromStrings("trace_id", "asdf")}, - }, - []record.RefHistogramSample{ - {Ref: 2, T: 600, H: &histogram.Histogram{}}, - }, - []record.RefFloatHistogramSample{ - {Ref: 2, T: 700, FH: &histogram.FloatHistogram{}}, - }, - }, - expectedWalExpiry: 800, - walTruncateMinT: 900, - expectedWalEntries: []any{ - []record.RefSeries{ - {Ref: 1, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefSample{ - {Ref: 1, T: 900, V: 3}, - }, - }, - }, - } - - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - h, w := newTestHead(t, 1000, compression.None, false) - t.Cleanup(func() { - require.NoError(t, h.Close()) - }) - - populateTestWL(t, w, tc.walEntries, nil) - first, _, err := wlog.Segments(w.Dir()) - require.NoError(t, err) - - require.NoError(t, h.Init(0)) - - keepUntil, ok := h.getWALExpiry(2) - require.True(t, ok) - require.Equal(t, tc.expectedWalExpiry, keepUntil) - - // Each truncation creates a new segment, so attempt truncations until a checkpoint is created - for { - h.lastWALTruncationTime.Store(0) // Reset so that it's always time to truncate the WAL - err := h.truncateWAL(tc.walTruncateMinT) - require.NoError(t, err) - f, _, err := wlog.Segments(w.Dir()) - require.NoError(t, err) - if f > first { - break - } - } - - // Read test WAL , checkpoint first - checkpointDir, _, err := wlog.LastCheckpoint(w.Dir()) - require.NoError(t, err) - cprecs := readTestWAL(t, checkpointDir) - recs := readTestWAL(t, w.Dir()) - recs = append(cprecs, recs...) - - // Use testutil.RequireEqual which handles labels properly with dedupelabels - testutil.RequireEqual(t, tc.expectedWalEntries, recs) - }) - } -} - -func TestHead_KeepSeriesInWALCheckpoint(t *testing.T) { - existingRef := 1 - existingLbls := labels.FromStrings("foo", "bar") - keepUntil := int64(10) - - cases := []struct { - name string - prepare func(t *testing.T, h *Head) - mint int64 - expected bool - }{ - { - name: "keep series still in the head", - prepare: func(t *testing.T, h *Head) { - _, _, err := h.getOrCreateWithOptionalID(chunks.HeadSeriesRef(existingRef), existingLbls.Hash(), existingLbls, false) - require.NoError(t, err) - }, - expected: true, - }, - { - name: "keep series with keepUntil > mint", - mint: keepUntil - 1, - expected: true, - }, - { - name: "keep series with keepUntil = mint", - mint: keepUntil, - expected: true, - }, - { - name: "drop series with keepUntil < mint", - mint: keepUntil + 1, - expected: false, - }, - } - - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - h, _ := newTestHead(t, 1000, compression.None, false) - t.Cleanup(func() { - require.NoError(t, h.Close()) - }) - - if tc.prepare != nil { - tc.prepare(t, h) - } else { - h.updateWALExpiry(chunks.HeadSeriesRef(existingRef), keepUntil) - } - - keep := h.keepSeriesInWALCheckpointFn(tc.mint) - require.Equal(t, tc.expected, keep(chunks.HeadSeriesRef(existingRef))) - }) - } -} - -func TestHead_ActiveAppenders(t *testing.T) { +func TestHeadAppenderV2_ActiveAppenders(t *testing.T) { head, _ := newTestHead(t, 1000, compression.None, false) defer head.Close() require.NoError(t, head.Init(0)) // First rollback with no samples. - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) require.NoError(t, app.Rollback()) require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) // Then commit with no samples. - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) require.NoError(t, app.Commit()) require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) // Now rollback with one sample. - app = head.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("foo", "bar"), 100, 1) + app = head.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 100, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) require.NoError(t, app.Rollback()) require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) // Now commit with one sample. - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 100, 1) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 100, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.activeAppenders)) } -func TestHead_RaceBetweenSeriesCreationAndGC(t *testing.T) { +func TestHeadAppenderV2_RaceBetweenSeriesCreationAndGC(t *testing.T) { head, _ := newTestHead(t, 1000, compression.None, false) t.Cleanup(func() { _ = head.Close() }) require.NoError(t, head.Init(0)) @@ -1240,14 +364,14 @@ func TestHead_RaceBetweenSeriesCreationAndGC(t *testing.T) { go func() { defer done.Store(true) - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) defer func() { if err := app.Commit(); err != nil { t.Errorf("Failed to commit: %v", err) } }() for i := range totalSeries { - _, err := app.Append(0, series[i], 100, 1) + _, err := app.Append(0, series[i], 0, 100, 1, nil, nil, storage.AOptions{}) if err != nil { t.Errorf("Failed to append: %v", err) return @@ -1263,10 +387,10 @@ func TestHead_RaceBetweenSeriesCreationAndGC(t *testing.T) { require.Equal(t, totalSeries, int(head.NumSeries())) } -func TestHead_CanGarbagecollectSeriesCreatedWithoutSamples(t *testing.T) { - for op, finishTxn := range map[string]func(app storage.Appender) error{ - "after commit": func(app storage.Appender) error { return app.Commit() }, - "after rollback": func(app storage.Appender) error { return app.Rollback() }, +func TestHeadAppenderV2_CanGCSeriesCreatedWithoutSamples(t *testing.T) { + for op, finishTxn := range map[string]func(app storage.AppenderTransaction) error{ + "after commit": func(app storage.AppenderTransaction) error { return app.Commit() }, + "after rollback": func(app storage.AppenderTransaction) error { return app.Rollback() }, } { t.Run(op, func(t *testing.T) { chunkRange := time.Hour.Milliseconds() @@ -1278,8 +402,8 @@ func TestHead_CanGarbagecollectSeriesCreatedWithoutSamples(t *testing.T) { firstSampleTime := 10 * chunkRange { // Append first sample, it should init head max time to firstSampleTime. - app := head.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("lbl", "ok"), firstSampleTime, 1) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("lbl", "ok"), 0, firstSampleTime, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 1, int(head.NumSeries())) @@ -1287,9 +411,9 @@ func TestHead_CanGarbagecollectSeriesCreatedWithoutSamples(t *testing.T) { // Append a sample in a time range that is not covered by the chunk range, // We would create series first and then append no sample. - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) invalidSampleTime := firstSampleTime - chunkRange - _, err := app.Append(0, labels.FromStrings("foo", "bar"), invalidSampleTime, 2) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, invalidSampleTime, 2, nil, nil, storage.AOptions{}) require.Error(t, err) // These are our assumptions: we're not testing them, we're just checking them to make debugging a failed // test easier if someone refactors the code and breaks these assumptions. @@ -1306,428 +430,7 @@ func TestHead_CanGarbagecollectSeriesCreatedWithoutSamples(t *testing.T) { } } -func TestHead_UnknownWALRecord(t *testing.T) { - head, w := newTestHead(t, 1000, compression.None, false) - w.Log([]byte{255, 42}) - require.NoError(t, head.Init(0)) - require.NoError(t, head.Close()) -} - -// BenchmarkHead_Truncate is quite heavy, so consider running it with -// -benchtime=10x or similar to get more stable and comparable results. -func BenchmarkHead_Truncate(b *testing.B) { - const total = 1e6 - - prepare := func(b *testing.B, churn int) *Head { - h, _ := newTestHead(b, 1000, compression.None, false) - b.Cleanup(func() { - require.NoError(b, h.Close()) - }) - - h.initTime(0) - - internedItoa := map[int]string{} - var mtx sync.RWMutex - itoa := func(i int) string { - mtx.RLock() - s, ok := internedItoa[i] - mtx.RUnlock() - if ok { - return s - } - mtx.Lock() - s = strconv.Itoa(i) - internedItoa[i] = s - mtx.Unlock() - return s - } - - allSeries := [total]labels.Labels{} - nameValues := make([]string, 0, 100) - for i := range int(total) { - nameValues = nameValues[:0] - - // A thousand labels like lbl_x_of_1000, each with total/1000 values - thousand := "lbl_" + itoa(i%1000) + "_of_1000" - nameValues = append(nameValues, thousand, itoa(i/1000)) - // A hundred labels like lbl_x_of_100, each with total/100 values. - hundred := "lbl_" + itoa(i%100) + "_of_100" - nameValues = append(nameValues, hundred, itoa(i/100)) - - if i%13 == 0 { - ten := "lbl_" + itoa(i%10) + "_of_10" - nameValues = append(nameValues, ten, itoa(i%10)) - } - - allSeries[i] = labels.FromStrings(append(nameValues, "first", "a", "second", "a", "third", "a")...) - s, _, _ := h.getOrCreate(allSeries[i].Hash(), allSeries[i], false) - s.mmappedChunks = []*mmappedChunk{ - {minTime: 1000 * int64(i/churn), maxTime: 999 + 1000*int64(i/churn)}, - } - } - - return h - } - - for _, churn := range []int{10, 100, 1000} { - b.Run(fmt.Sprintf("churn=%d", churn), func(b *testing.B) { - if b.N > total/churn { - // Just to make sure that benchmark still makes sense. - panic("benchmark not prepared") - } - h := prepare(b, churn) - b.ResetTimer() - - for i := 0; b.Loop(); i++ { - require.NoError(b, h.Truncate(1000*int64(i))) - // Make sure the benchmark is meaningful and it's actually truncating the expected amount of series. - require.Equal(b, total-churn*i, int(h.NumSeries())) - } - }) - } -} - -func TestHead_Truncate(t *testing.T) { - h, _ := newTestHead(t, 1000, compression.None, false) - defer func() { - require.NoError(t, h.Close()) - }() - - h.initTime(0) - - ctx := context.Background() - - s1, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1", "b", "1"), false) - s2, _, _ := h.getOrCreate(2, labels.FromStrings("a", "2", "b", "1"), false) - s3, _, _ := h.getOrCreate(3, labels.FromStrings("a", "1", "b", "2"), false) - s4, _, _ := h.getOrCreate(4, labels.FromStrings("a", "2", "b", "2", "c", "1"), false) - - s1.mmappedChunks = []*mmappedChunk{ - {minTime: 0, maxTime: 999}, - {minTime: 1000, maxTime: 1999}, - {minTime: 2000, maxTime: 2999}, - } - s2.mmappedChunks = []*mmappedChunk{ - {minTime: 1000, maxTime: 1999}, - {minTime: 2000, maxTime: 2999}, - {minTime: 3000, maxTime: 3999}, - } - s3.mmappedChunks = []*mmappedChunk{ - {minTime: 0, maxTime: 999}, - {minTime: 1000, maxTime: 1999}, - } - s4.mmappedChunks = []*mmappedChunk{} - - // Truncation need not be aligned. - require.NoError(t, h.Truncate(1)) - - require.NoError(t, h.Truncate(2000)) - - require.Equal(t, []*mmappedChunk{ - {minTime: 2000, maxTime: 2999}, - }, h.series.getByID(s1.ref).mmappedChunks) - - require.Equal(t, []*mmappedChunk{ - {minTime: 2000, maxTime: 2999}, - {minTime: 3000, maxTime: 3999}, - }, h.series.getByID(s2.ref).mmappedChunks) - - require.Nil(t, h.series.getByID(s3.ref)) - require.Nil(t, h.series.getByID(s4.ref)) - - postingsA1, _ := index.ExpandPostings(h.postings.Postings(ctx, "a", "1")) - postingsA2, _ := index.ExpandPostings(h.postings.Postings(ctx, "a", "2")) - postingsB1, _ := index.ExpandPostings(h.postings.Postings(ctx, "b", "1")) - postingsB2, _ := index.ExpandPostings(h.postings.Postings(ctx, "b", "2")) - postingsC1, _ := index.ExpandPostings(h.postings.Postings(ctx, "c", "1")) - postingsAll, _ := index.ExpandPostings(h.postings.Postings(ctx, "", "")) - - require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s1.ref)}, postingsA1) - require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s2.ref)}, postingsA2) - require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s1.ref), storage.SeriesRef(s2.ref)}, postingsB1) - require.Equal(t, []storage.SeriesRef{storage.SeriesRef(s1.ref), storage.SeriesRef(s2.ref)}, postingsAll) - require.Nil(t, postingsB2) - require.Nil(t, postingsC1) - - iter := h.postings.Symbols() - symbols := []string{} - for iter.Next() { - symbols = append(symbols, iter.At()) - } - require.Equal(t, - []string{"" /* from 'all' postings list */, "1", "2", "a", "b"}, - symbols) - - values := map[string]map[string]struct{}{} - for _, name := range h.postings.LabelNames() { - ss, ok := values[name] - if !ok { - ss = map[string]struct{}{} - values[name] = ss - } - for _, value := range h.postings.LabelValues(ctx, name, nil) { - ss[value] = struct{}{} - } - } - require.Equal(t, map[string]map[string]struct{}{ - "a": {"1": struct{}{}, "2": struct{}{}}, - "b": {"1": struct{}{}}, - }, values) -} - -// Validate various behaviors brought on by firstChunkID accounting for -// garbage collected chunks. -func TestMemSeries_truncateChunks(t *testing.T) { - dir := t.TempDir() - // This is usually taken from the Head, but passing manually here. - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(t, err) - defer func() { - require.NoError(t, chunkDiskMapper.Close()) - }() - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: 2000, - samplesPerChunk: DefaultSamplesPerChunk, - } - - memChunkPool := sync.Pool{ - New: func() any { - return &memChunk{} - }, - } - - s := newMemSeries(labels.FromStrings("a", "b"), 1, 0, defaultIsolationDisabled, false) - - for i := 0; i < 4000; i += 5 { - ok, _ := s.append(int64(i), float64(i), 0, cOpts) - require.True(t, ok, "sample append failed") - } - s.mmapChunks(chunkDiskMapper) - - // Check that truncate removes half of the chunks and afterwards - // that the ID of the last chunk still gives us the same chunk afterwards. - countBefore := len(s.mmappedChunks) + 1 // +1 for the head chunk. - lastID := s.headChunkID(countBefore - 1) - lastChunk, _, _, err := s.chunk(lastID, chunkDiskMapper, &memChunkPool) - require.NoError(t, err) - require.NotNil(t, lastChunk) - - chk, _, _, err := s.chunk(0, chunkDiskMapper, &memChunkPool) - require.NotNil(t, chk) - require.NoError(t, err) - - s.truncateChunksBefore(2000, 0) - - require.Equal(t, int64(2000), s.mmappedChunks[0].minTime) - _, _, _, err = s.chunk(0, chunkDiskMapper, &memChunkPool) - require.Equal(t, storage.ErrNotFound, err, "first chunks not gone") - require.Equal(t, countBefore/2, len(s.mmappedChunks)+1) // +1 for the head chunk. - chk, _, _, err = s.chunk(lastID, chunkDiskMapper, &memChunkPool) - require.NoError(t, err) - require.Equal(t, lastChunk, chk) -} - -func TestMemSeries_truncateChunks_scenarios(t *testing.T) { - const chunkRange = 100 - const chunkStep = 5 - - tests := []struct { - name string - headChunks int // the number of head chunks to create on memSeries by appending enough samples - mmappedChunks int // the number of mmapped chunks to create on memSeries by appending enough samples - truncateBefore int64 // the mint to pass to truncateChunksBefore() - expectedTruncated int // the number of chunks that we're expecting be truncated and returned by truncateChunksBefore() - expectedHead int // the expected number of head chunks after truncation - expectedMmap int // the expected number of mmapped chunks after truncation - expectedFirstChunkID chunks.HeadChunkID // the expected series.firstChunkID after truncation - }{ - { - name: "empty memSeries", - truncateBefore: chunkRange * 10, - }, - { - name: "single head chunk, not truncated", - headChunks: 1, - expectedHead: 1, - }, - { - name: "single head chunk, truncated", - headChunks: 1, - truncateBefore: chunkRange, - expectedTruncated: 1, - expectedHead: 0, - expectedFirstChunkID: 1, - }, - { - name: "2 head chunks, not truncated", - headChunks: 2, - expectedHead: 2, - }, - { - name: "2 head chunks, first truncated", - headChunks: 2, - truncateBefore: chunkRange, - expectedTruncated: 1, - expectedHead: 1, - expectedFirstChunkID: 1, - }, - { - name: "2 head chunks, everything truncated", - headChunks: 2, - truncateBefore: chunkRange * 2, - expectedTruncated: 2, - expectedHead: 0, - expectedFirstChunkID: 2, - }, - { - name: "no head chunks, 3 mmap chunks, second mmap truncated", - headChunks: 0, - mmappedChunks: 3, - truncateBefore: chunkRange * 2, - expectedTruncated: 2, - expectedHead: 0, - expectedMmap: 1, - expectedFirstChunkID: 2, - }, - { - name: "single head chunk, single mmap chunk, not truncated", - headChunks: 1, - mmappedChunks: 1, - expectedHead: 1, - expectedMmap: 1, - }, - { - name: "single head chunk, single mmap chunk, mmap truncated", - headChunks: 1, - mmappedChunks: 1, - truncateBefore: chunkRange, - expectedTruncated: 1, - expectedHead: 1, - expectedMmap: 0, - expectedFirstChunkID: 1, - }, - { - name: "5 head chunk, 5 mmap chunk, third head truncated", - headChunks: 5, - mmappedChunks: 5, - truncateBefore: chunkRange * 7, - expectedTruncated: 7, - expectedHead: 3, - expectedMmap: 0, - expectedFirstChunkID: 7, - }, - { - name: "2 head chunks, 3 mmap chunks, second mmap truncated", - headChunks: 2, - mmappedChunks: 3, - truncateBefore: chunkRange * 2, - expectedTruncated: 2, - expectedHead: 2, - expectedMmap: 1, - expectedFirstChunkID: 2, - }, - } - - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - dir := t.TempDir() - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(t, err) - defer func() { - require.NoError(t, chunkDiskMapper.Close()) - }() - - series := newMemSeries(labels.EmptyLabels(), 1, 0, true, false) - - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: chunkRange, - samplesPerChunk: DefaultSamplesPerChunk, - } - - var headStart int - if tc.mmappedChunks > 0 { - headStart = (tc.mmappedChunks + 1) * chunkRange - for i := 0; i < (tc.mmappedChunks+1)*chunkRange; i += chunkStep { - ok, _ := series.append(int64(i), float64(i), 0, cOpts) - require.True(t, ok, "sample append failed") - } - series.mmapChunks(chunkDiskMapper) - } - - if tc.headChunks == 0 { - series.headChunks = nil - } else { - for i := headStart; i < chunkRange*(tc.mmappedChunks+tc.headChunks); i += chunkStep { - ok, _ := series.append(int64(i), float64(i), 0, cOpts) - require.True(t, ok, "sample append failed: %d", i) - } - } - - if tc.headChunks > 0 { - require.NotNil(t, series.headChunks, "head chunk is missing") - require.Equal(t, tc.headChunks, series.headChunks.len(), "wrong number of head chunks") - } else { - require.Nil(t, series.headChunks, "head chunk is present") - } - require.Len(t, series.mmappedChunks, tc.mmappedChunks, "wrong number of mmapped chunks") - - truncated := series.truncateChunksBefore(tc.truncateBefore, 0) - require.Equal(t, tc.expectedTruncated, truncated, "wrong number of truncated chunks returned") - - require.Len(t, series.mmappedChunks, tc.expectedMmap, "wrong number of mmappedChunks after truncation") - - if tc.expectedHead > 0 { - require.NotNil(t, series.headChunks, "headChunks should is nil after truncation") - require.Equal(t, tc.expectedHead, series.headChunks.len(), "wrong number of head chunks after truncation") - require.Nil(t, series.headChunks.oldest().prev, "last head chunk cannot have any next chunk set") - } else { - require.Nil(t, series.headChunks, "headChunks should is non-nil after truncation") - } - - if series.headChunks != nil || len(series.mmappedChunks) > 0 { - require.GreaterOrEqual(t, series.maxTime(), tc.truncateBefore, "wrong value of series.maxTime() after truncation") - } else { - require.Equal(t, int64(math.MinInt64), series.maxTime(), "wrong value of series.maxTime() after truncation") - } - - require.Equal(t, tc.expectedFirstChunkID, series.firstChunkID, "wrong firstChunkID after truncation") - }) - } -} - -func TestHeadDeleteSeriesWithoutSamples(t *testing.T) { - for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { - t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { - entries := []any{ - []record.RefSeries{ - {Ref: 10, Labels: labels.FromStrings("a", "1")}, - }, - []record.RefSample{}, - []record.RefSeries{ - {Ref: 50, Labels: labels.FromStrings("a", "2")}, - }, - []record.RefSample{ - {Ref: 50, T: 80, V: 1}, - {Ref: 50, T: 90, V: 1}, - }, - } - head, w := newTestHead(t, 1000, compress, false) - defer func() { - require.NoError(t, head.Close()) - }() - - populateTestWL(t, w, entries, nil) - - require.NoError(t, head.Init(math.MinInt64)) - - require.NoError(t, head.Delete(context.Background(), 0, 100, labels.MustNewMatcher(labels.MatchEqual, "a", "1"))) - }) - } -} - -func TestHeadDeleteSimple(t *testing.T) { +func TestHeadAppenderV2_DeleteSimple(t *testing.T) { buildSmpls := func(s []int64) []sample { ss := make([]sample, 0, len(s)) for _, t := range s { @@ -1784,9 +487,9 @@ func TestHeadDeleteSimple(t *testing.T) { head, w := newTestHead(t, 1000, compress, false) require.NoError(t, head.Init(0)) - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) for _, smpl := range smplsAll { - _, err := app.Append(0, lblsDefault, smpl.t, smpl.f) + _, err := app.Append(0, lblsDefault, 0, smpl.t, smpl.f, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -1797,9 +500,9 @@ func TestHeadDeleteSimple(t *testing.T) { } // Add more samples. - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) for _, smpl := range c.addSamples { - _, err := app.Append(0, lblsDefault, smpl.t, smpl.f) + _, err := app.Append(0, lblsDefault, 0, smpl.t, smpl.f, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -1859,18 +562,18 @@ func TestHeadDeleteSimple(t *testing.T) { } } -func TestDeleteUntilCurMax(t *testing.T) { +func TestHeadAppenderV2_DeleteUntilCurrMax(t *testing.T) { hb, _ := newTestHead(t, 1000000, compression.None, false) defer func() { require.NoError(t, hb.Close()) }() numSamples := int64(10) - app := hb.Appender(context.Background()) + app := hb.AppenderV2(context.Background()) smpls := make([]float64, numSamples) for i := range numSamples { smpls[i] = rand.Float64() - _, err := app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, i, smpls[i], nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -1890,8 +593,8 @@ func TestDeleteUntilCurMax(t *testing.T) { require.Empty(t, res.Warnings()) // Add again and test for presence. - app = hb.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("a", "b"), 11, 1) + app = hb.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 11, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) q, err = NewBlockQuerier(hb, 0, 100000) @@ -1909,15 +612,15 @@ func TestDeleteUntilCurMax(t *testing.T) { require.Empty(t, res.Warnings()) } -func TestDeletedSamplesAndSeriesStillInWALAfterCheckpoint(t *testing.T) { +func TestHeadAppenderV2_DeleteSamplesAndSeriesStillInWALAfterCheckpoint(t *testing.T) { numSamples := 10000 // Enough samples to cause a checkpoint. hb, w := newTestHead(t, int64(numSamples)*10, compression.None, false) for i := range numSamples { - app := hb.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("a", "b"), int64(i), 0) + app := hb.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, int64(i), 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -1953,7 +656,7 @@ func TestDeletedSamplesAndSeriesStillInWALAfterCheckpoint(t *testing.T) { require.Equal(t, 0, metadata) } -func TestDelete_e2e(t *testing.T) { +func TestHeadAppenderV2_Delete_e2e(t *testing.T) { numDatapoints := 1000 numRanges := 1000 timeInterval := int64(2) @@ -2010,14 +713,14 @@ func TestDelete_e2e(t *testing.T) { require.NoError(t, hb.Close()) }() - app := hb.Appender(context.Background()) + app := hb.AppenderV2(context.Background()) for _, l := range lbls { ls := labels.New(l...) series := []chunks.Sample{} ts := rand.Int63n(300) for range numDatapoints { v := rand.Float64() - _, err := app.Append(0, ls, ts, v) + _, err := app.Append(0, ls, 0, ts, v, nil, nil, storage.AOptions{}) require.NoError(t, err) series = append(series, sample{ts, v, nil, nil}) ts += rand.Int63n(timeInterval) + 1 @@ -2112,385 +815,7 @@ func TestDelete_e2e(t *testing.T) { } } -func boundedSamples(full []chunks.Sample, mint, maxt int64) []chunks.Sample { - for len(full) > 0 { - if full[0].T() >= mint { - break - } - full = full[1:] - } - for i, s := range full { - // labels.Labelinate on the first sample larger than maxt. - if s.T() > maxt { - return full[:i] - } - } - // maxt is after highest sample. - return full -} - -func deletedSamples(full []chunks.Sample, dranges tombstones.Intervals) []chunks.Sample { - ds := make([]chunks.Sample, 0, len(full)) -Outer: - for _, s := range full { - for _, r := range dranges { - if r.InBounds(s.T()) { - continue Outer - } - } - ds = append(ds, s) - } - - return ds -} - -func TestComputeChunkEndTime(t *testing.T) { - cases := map[string]struct { - start, cur, max int64 - ratioToFull float64 - res int64 - }{ - "exactly 1/4 full, even increment": { - start: 0, - cur: 250, - max: 1000, - ratioToFull: 4, - res: 1000, - }, - "exactly 1/4 full, uneven increment": { - start: 100, - cur: 200, - max: 1000, - ratioToFull: 4, - res: 550, - }, - "decimal ratio to full": { - start: 5000, - cur: 5110, - max: 10000, - ratioToFull: 4.2, - res: 5500, - }, - // Case where we fit floored 0 chunks. Must catch division by 0 - // and default to maximum time. - "fit floored 0 chunks": { - start: 0, - cur: 500, - max: 1000, - ratioToFull: 4, - res: 1000, - }, - // Catch division by zero for cur == start. Strictly not a possible case. - "cur == start": { - start: 100, - cur: 100, - max: 1000, - ratioToFull: 4, - res: 104, - }, - } - - for testName, tc := range cases { - t.Run(testName, func(t *testing.T) { - got := computeChunkEndTime(tc.start, tc.cur, tc.max, tc.ratioToFull) - require.Equal(t, tc.res, got, "(start: %d, cur: %d, max: %d)", tc.start, tc.cur, tc.max) - }) - } -} - -func TestMemSeries_append(t *testing.T) { - dir := t.TempDir() - // This is usually taken from the Head, but passing manually here. - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(t, err) - defer func() { - require.NoError(t, chunkDiskMapper.Close()) - }() - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: 500, - samplesPerChunk: DefaultSamplesPerChunk, - } - - s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) - - // Add first two samples at the very end of a chunk range and the next two - // on and after it. - // New chunk must correctly be cut at 1000. - ok, chunkCreated := s.append(998, 1, 0, cOpts) - require.True(t, ok, "append failed") - require.True(t, chunkCreated, "first sample created chunk") - - ok, chunkCreated = s.append(999, 2, 0, cOpts) - require.True(t, ok, "append failed") - require.False(t, chunkCreated, "second sample should use same chunk") - s.mmapChunks(chunkDiskMapper) - - ok, chunkCreated = s.append(1000, 3, 0, cOpts) - require.True(t, ok, "append failed") - require.True(t, chunkCreated, "expected new chunk on boundary") - - ok, chunkCreated = s.append(1001, 4, 0, cOpts) - require.True(t, ok, "append failed") - require.False(t, chunkCreated, "second sample should use same chunk") - - s.mmapChunks(chunkDiskMapper) - require.Len(t, s.mmappedChunks, 1, "there should be only 1 mmapped chunk") - require.Equal(t, int64(998), s.mmappedChunks[0].minTime, "wrong chunk range") - require.Equal(t, int64(999), s.mmappedChunks[0].maxTime, "wrong chunk range") - require.Equal(t, int64(1000), s.headChunks.minTime, "wrong chunk range") - require.Equal(t, int64(1001), s.headChunks.maxTime, "wrong chunk range") - - // Fill the range [1000,2000) with many samples. Intermediate chunks should be cut - // at approximately 120 samples per chunk. - for i := 1; i < 1000; i++ { - ok, _ := s.append(1001+int64(i), float64(i), 0, cOpts) - require.True(t, ok, "append failed") - } - s.mmapChunks(chunkDiskMapper) - - require.Greater(t, len(s.mmappedChunks)+1, 7, "expected intermediate chunks") - - // All chunks but the first and last should now be moderately full. - for i, c := range s.mmappedChunks[1:] { - chk, err := chunkDiskMapper.Chunk(c.ref) - require.NoError(t, err) - require.Greater(t, chk.NumSamples(), 100, "unexpected small chunk %d of length %d", i, chk.NumSamples()) - } -} - -func TestMemSeries_appendHistogram(t *testing.T) { - dir := t.TempDir() - // This is usually taken from the Head, but passing manually here. - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(t, err) - defer func() { - require.NoError(t, chunkDiskMapper.Close()) - }() - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: int64(1000), - samplesPerChunk: DefaultSamplesPerChunk, - } - - s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) - - histograms := tsdbutil.GenerateTestHistograms(4) - histogramWithOneMoreBucket := histograms[3].Copy() - histogramWithOneMoreBucket.Count++ - histogramWithOneMoreBucket.Sum += 1.23 - histogramWithOneMoreBucket.PositiveSpans[1].Length = 3 - histogramWithOneMoreBucket.PositiveBuckets = append(histogramWithOneMoreBucket.PositiveBuckets, 1) - - // Add first two samples at the very end of a chunk range and the next two - // on and after it. - // New chunk must correctly be cut at 1000. - ok, chunkCreated := s.appendHistogram(998, histograms[0], 0, cOpts) - require.True(t, ok, "append failed") - require.True(t, chunkCreated, "first sample created chunk") - - ok, chunkCreated = s.appendHistogram(999, histograms[1], 0, cOpts) - require.True(t, ok, "append failed") - require.False(t, chunkCreated, "second sample should use same chunk") - - ok, chunkCreated = s.appendHistogram(1000, histograms[2], 0, cOpts) - require.True(t, ok, "append failed") - require.True(t, chunkCreated, "expected new chunk on boundary") - - ok, chunkCreated = s.appendHistogram(1001, histograms[3], 0, cOpts) - require.True(t, ok, "append failed") - require.False(t, chunkCreated, "second sample should use same chunk") - - s.mmapChunks(chunkDiskMapper) - require.Len(t, s.mmappedChunks, 1, "there should be only 1 mmapped chunk") - require.Equal(t, int64(998), s.mmappedChunks[0].minTime, "wrong chunk range") - require.Equal(t, int64(999), s.mmappedChunks[0].maxTime, "wrong chunk range") - require.Equal(t, int64(1000), s.headChunks.minTime, "wrong chunk range") - require.Equal(t, int64(1001), s.headChunks.maxTime, "wrong chunk range") - - ok, chunkCreated = s.appendHistogram(1002, histogramWithOneMoreBucket, 0, cOpts) - require.True(t, ok, "append failed") - require.False(t, chunkCreated, "third sample should trigger a re-encoded chunk") - - s.mmapChunks(chunkDiskMapper) - require.Len(t, s.mmappedChunks, 1, "there should be only 1 mmapped chunk") - require.Equal(t, int64(998), s.mmappedChunks[0].minTime, "wrong chunk range") - require.Equal(t, int64(999), s.mmappedChunks[0].maxTime, "wrong chunk range") - require.Equal(t, int64(1000), s.headChunks.minTime, "wrong chunk range") - require.Equal(t, int64(1002), s.headChunks.maxTime, "wrong chunk range") -} - -func TestMemSeries_append_atVariableRate(t *testing.T) { - const samplesPerChunk = 120 - dir := t.TempDir() - // This is usually taken from the Head, but passing manually here. - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(t, err) - t.Cleanup(func() { - require.NoError(t, chunkDiskMapper.Close()) - }) - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: DefaultBlockDuration, - samplesPerChunk: samplesPerChunk, - } - - s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) - - // At this slow rate, we will fill the chunk in two block durations. - slowRate := (DefaultBlockDuration * 2) / samplesPerChunk - - var nextTs int64 - var totalAppendedSamples int - for i := range samplesPerChunk / 4 { - ok, _ := s.append(nextTs, float64(i), 0, cOpts) - require.Truef(t, ok, "slow sample %d was not appended", i) - nextTs += slowRate - totalAppendedSamples++ - } - require.Equal(t, DefaultBlockDuration, s.nextAt, "after appending a samplesPerChunk/4 samples at a slow rate, we should aim to cut a new block at the default block duration %d, but it's set to %d", DefaultBlockDuration, s.nextAt) - - // Suddenly, the rate increases and we receive a sample every millisecond. - for i := range math.MaxUint16 { - ok, _ := s.append(nextTs, float64(i), 0, cOpts) - require.Truef(t, ok, "quick sample %d was not appended", i) - nextTs++ - totalAppendedSamples++ - } - ok, chunkCreated := s.append(DefaultBlockDuration, float64(0), 0, cOpts) - require.True(t, ok, "new chunk sample was not appended") - require.True(t, chunkCreated, "sample at block duration timestamp should create a new chunk") - - s.mmapChunks(chunkDiskMapper) - var totalSamplesInChunks int - for i, c := range s.mmappedChunks { - totalSamplesInChunks += int(c.numSamples) - require.LessOrEqualf(t, c.numSamples, uint16(2*samplesPerChunk), "mmapped chunk %d has more than %d samples", i, 2*samplesPerChunk) - } - require.Equal(t, totalAppendedSamples, totalSamplesInChunks, "wrong number of samples in %d mmapped chunks", len(s.mmappedChunks)) -} - -func TestGCChunkAccess(t *testing.T) { - // Put a chunk, select it. GC it and then access it. - const chunkRange = 1000 - h, _ := newTestHead(t, chunkRange, compression.None, false) - defer func() { - require.NoError(t, h.Close()) - }() - - cOpts := chunkOpts{ - chunkDiskMapper: h.chunkDiskMapper, - chunkRange: chunkRange, - samplesPerChunk: DefaultSamplesPerChunk, - } - - h.initTime(0) - - s, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) - - // Appending 2 samples for the first chunk. - ok, chunkCreated := s.append(0, 0, 0, cOpts) - require.True(t, ok, "series append failed") - require.True(t, chunkCreated, "chunks was not created") - ok, chunkCreated = s.append(999, 999, 0, cOpts) - require.True(t, ok, "series append failed") - require.False(t, chunkCreated, "chunks was created") - - // A new chunks should be created here as it's beyond the chunk range. - ok, chunkCreated = s.append(1000, 1000, 0, cOpts) - require.True(t, ok, "series append failed") - require.True(t, chunkCreated, "chunks was not created") - ok, chunkCreated = s.append(1999, 1999, 0, cOpts) - require.True(t, ok, "series append failed") - require.False(t, chunkCreated, "chunks was created") - - idx := h.indexRange(0, 1500) - var ( - chunks []chunks.Meta - builder labels.ScratchBuilder - ) - require.NoError(t, idx.Series(1, &builder, &chunks)) - - require.Equal(t, labels.FromStrings("a", "1"), builder.Labels()) - require.Len(t, chunks, 2) - - cr, err := h.chunksRange(0, 1500, nil) - require.NoError(t, err) - _, _, err = cr.ChunkOrIterable(chunks[0]) - require.NoError(t, err) - _, _, err = cr.ChunkOrIterable(chunks[1]) - require.NoError(t, err) - - require.NoError(t, h.Truncate(1500)) // Remove a chunk. - - _, _, err = cr.ChunkOrIterable(chunks[0]) - require.Equal(t, storage.ErrNotFound, err) - _, _, err = cr.ChunkOrIterable(chunks[1]) - require.NoError(t, err) -} - -func TestGCSeriesAccess(t *testing.T) { - // Put a series, select it. GC it and then access it. - const chunkRange = 1000 - h, _ := newTestHead(t, chunkRange, compression.None, false) - defer func() { - require.NoError(t, h.Close()) - }() - - cOpts := chunkOpts{ - chunkDiskMapper: h.chunkDiskMapper, - chunkRange: chunkRange, - samplesPerChunk: DefaultSamplesPerChunk, - } - - h.initTime(0) - - s, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) - - // Appending 2 samples for the first chunk. - ok, chunkCreated := s.append(0, 0, 0, cOpts) - require.True(t, ok, "series append failed") - require.True(t, chunkCreated, "chunks was not created") - ok, chunkCreated = s.append(999, 999, 0, cOpts) - require.True(t, ok, "series append failed") - require.False(t, chunkCreated, "chunks was created") - - // A new chunks should be created here as it's beyond the chunk range. - ok, chunkCreated = s.append(1000, 1000, 0, cOpts) - require.True(t, ok, "series append failed") - require.True(t, chunkCreated, "chunks was not created") - ok, chunkCreated = s.append(1999, 1999, 0, cOpts) - require.True(t, ok, "series append failed") - require.False(t, chunkCreated, "chunks was created") - - idx := h.indexRange(0, 2000) - var ( - chunks []chunks.Meta - builder labels.ScratchBuilder - ) - require.NoError(t, idx.Series(1, &builder, &chunks)) - - require.Equal(t, labels.FromStrings("a", "1"), builder.Labels()) - require.Len(t, chunks, 2) - - cr, err := h.chunksRange(0, 2000, nil) - require.NoError(t, err) - _, _, err = cr.ChunkOrIterable(chunks[0]) - require.NoError(t, err) - _, _, err = cr.ChunkOrIterable(chunks[1]) - require.NoError(t, err) - - require.NoError(t, h.Truncate(2000)) // Remove the series. - - require.Equal(t, (*memSeries)(nil), h.series.getByID(1)) - - _, _, err = cr.ChunkOrIterable(chunks[0]) - require.Equal(t, storage.ErrNotFound, err) - _, _, err = cr.ChunkOrIterable(chunks[1]) - require.Equal(t, storage.ErrNotFound, err) -} - -func TestUncommittedSamplesNotLostOnTruncate(t *testing.T) { +func TestHeadAppenderV2_UncommittedSamplesNotLostOnTruncate(t *testing.T) { h, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, h.Close()) @@ -2498,9 +823,9 @@ func TestUncommittedSamplesNotLostOnTruncate(t *testing.T) { h.initTime(0) - app := h.appender() + app := h.appenderV2() lset := labels.FromStrings("a", "1") - _, err := app.Append(0, lset, 2100, 1) + _, err := app.Append(0, lset, 0, 2100, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, h.Truncate(2000)) @@ -2520,7 +845,7 @@ func TestUncommittedSamplesNotLostOnTruncate(t *testing.T) { require.Empty(t, ss.Warnings()) } -func TestRemoveSeriesAfterRollbackAndTruncate(t *testing.T) { +func TestHeadAppenderV2_TestRemoveSeriesAfterRollbackAndTruncate(t *testing.T) { h, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, h.Close()) @@ -2528,9 +853,9 @@ func TestRemoveSeriesAfterRollbackAndTruncate(t *testing.T) { h.initTime(0) - app := h.appender() + app := h.appenderV2() lset := labels.FromStrings("a", "1") - _, err := app.Append(0, lset, 2100, 1) + _, err := app.Append(0, lset, 0, 2100, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, h.Truncate(2000)) @@ -2551,7 +876,7 @@ func TestRemoveSeriesAfterRollbackAndTruncate(t *testing.T) { require.Equal(t, (*memSeries)(nil), h.series.getByHash(lset.Hash(), lset)) } -func TestHead_LogRollback(t *testing.T) { +func TestHeadAppenderV2_LogRollback(t *testing.T) { for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { t.Run(fmt.Sprintf("compress=%s", compress), func(t *testing.T) { h, w := newTestHead(t, 1000, compress, false) @@ -2559,8 +884,8 @@ func TestHead_LogRollback(t *testing.T) { require.NoError(t, h.Close()) }() - app := h.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("a", "b"), 1, 2) + app := h.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 1, 2, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Rollback()) @@ -2575,7 +900,7 @@ func TestHead_LogRollback(t *testing.T) { } } -func TestHead_ReturnsSortedLabelValues(t *testing.T) { +func TestHeadAppenderV2_ReturnsSortedLabelValues(t *testing.T) { h, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, h.Close()) @@ -2583,14 +908,14 @@ func TestHead_ReturnsSortedLabelValues(t *testing.T) { h.initTime(0) - app := h.appender() + app := h.appenderV2() for i := 100; i > 0; i-- { for j := range 10 { lset := labels.FromStrings( "__name__", fmt.Sprintf("metric_%d", i), "label", fmt.Sprintf("value_%d", j), ) - _, err := app.Append(0, lset, 2100, 1) + _, err := app.Append(0, lset, 0, 2100, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) } } @@ -2605,257 +930,14 @@ func TestHead_ReturnsSortedLabelValues(t *testing.T) { require.NoError(t, q.Close()) } -// TestWalRepair_DecodingError ensures that a repair is run for an error -// when decoding a record. -func TestWalRepair_DecodingError(t *testing.T) { - var enc record.Encoder - for name, test := range map[string]struct { - corrFunc func(rec []byte) []byte // Func that applies the corruption to a record. - rec []byte - totalRecs int - expRecs int - }{ - "decode_series": { - func(rec []byte) []byte { - return rec[:3] - }, - enc.Series([]record.RefSeries{{Ref: 1, Labels: labels.FromStrings("a", "b")}}, []byte{}), - 9, - 5, - }, - "decode_samples": { - func(rec []byte) []byte { - return rec[:3] - }, - enc.Samples([]record.RefSample{{Ref: 0, T: 99, V: 1}}, []byte{}), - 9, - 5, - }, - "decode_tombstone": { - func(rec []byte) []byte { - return rec[:3] - }, - enc.Tombstones([]tombstones.Stone{{Ref: 1, Intervals: tombstones.Intervals{}}}, []byte{}), - 9, - 5, - }, - } { - for _, compress := range []compression.Type{compression.None, compression.Snappy, compression.Zstd} { - t.Run(fmt.Sprintf("%s,compress=%s", name, compress), func(t *testing.T) { - dir := t.TempDir() - - // Fill the wal and corrupt it. - { - w, err := wlog.New(nil, nil, filepath.Join(dir, "wal"), compress) - require.NoError(t, err) - - for i := 1; i <= test.totalRecs; i++ { - // At this point insert a corrupted record. - if i-1 == test.expRecs { - require.NoError(t, w.Log(test.corrFunc(test.rec))) - continue - } - require.NoError(t, w.Log(test.rec)) - } - - opts := DefaultHeadOptions() - opts.ChunkRange = 1 - opts.ChunkDirRoot = w.Dir() - h, err := NewHead(nil, nil, w, nil, opts, nil) - require.NoError(t, err) - require.Equal(t, 0.0, prom_testutil.ToFloat64(h.metrics.walCorruptionsTotal)) - initErr := h.Init(math.MinInt64) - - var cerr *wlog.CorruptionErr - require.ErrorAs(t, initErr, &cerr, "reading the wal didn't return corruption error") - require.NoError(t, h.Close()) // Head will close the wal as well. - } - - // Open the db to trigger a repair. - { - db, err := Open(dir, nil, nil, DefaultOptions(), nil) - require.NoError(t, err) - defer func() { - require.NoError(t, db.Close()) - }() - require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) - } - - // Read the wal content after the repair. - { - sr, err := wlog.NewSegmentsReader(filepath.Join(dir, "wal")) - require.NoError(t, err) - defer sr.Close() - r := wlog.NewReader(sr) - - var actRec int - for r.Next() { - actRec++ - } - require.NoError(t, r.Err()) - require.Equal(t, test.expRecs, actRec, "Wrong number of intact records") - } - }) - } - } -} - -// TestWblRepair_DecodingError ensures that a repair is run for an error -// when decoding a record. -func TestWblRepair_DecodingError(t *testing.T) { - var enc record.Encoder - corrFunc := func(rec []byte) []byte { - return rec[:3] - } - rec := enc.Samples([]record.RefSample{{Ref: 0, T: 99, V: 1}}, []byte{}) - totalRecs := 9 - expRecs := 5 - dir := t.TempDir() - - // Fill the wbl and corrupt it. - { - wal, err := wlog.New(nil, nil, filepath.Join(dir, "wal"), compression.None) - require.NoError(t, err) - wbl, err := wlog.New(nil, nil, filepath.Join(dir, "wbl"), compression.None) - require.NoError(t, err) - - for i := 1; i <= totalRecs; i++ { - // At this point insert a corrupted record. - if i-1 == expRecs { - require.NoError(t, wbl.Log(corrFunc(rec))) - continue - } - require.NoError(t, wbl.Log(rec)) - } - - opts := DefaultHeadOptions() - opts.ChunkRange = 1 - opts.ChunkDirRoot = wal.Dir() - opts.OutOfOrderCapMax.Store(30) - opts.OutOfOrderTimeWindow.Store(1000 * time.Minute.Milliseconds()) - h, err := NewHead(nil, nil, wal, wbl, opts, nil) - require.NoError(t, err) - require.Equal(t, 0.0, prom_testutil.ToFloat64(h.metrics.walCorruptionsTotal)) - initErr := h.Init(math.MinInt64) - - var elb *errLoadWbl - require.ErrorAs(t, initErr, &elb) // Wbl errors are wrapped into errLoadWbl, make sure we can unwrap it. - - var cerr *wlog.CorruptionErr - require.ErrorAs(t, initErr, &cerr, "reading the wal didn't return corruption error") - require.NoError(t, h.Close()) // Head will close the wal as well. - } - - // Open the db to trigger a repair. - { - db, err := Open(dir, nil, nil, DefaultOptions(), nil) - require.NoError(t, err) - defer func() { - require.NoError(t, db.Close()) - }() - require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.walCorruptionsTotal)) - } - - // Read the wbl content after the repair. - { - sr, err := wlog.NewSegmentsReader(filepath.Join(dir, "wbl")) - require.NoError(t, err) - defer sr.Close() - r := wlog.NewReader(sr) - - var actRec int - for r.Next() { - actRec++ - } - require.NoError(t, r.Err()) - require.Equal(t, expRecs, actRec, "Wrong number of intact records") - } -} - -func TestHeadReadWriterRepair(t *testing.T) { - dir := t.TempDir() - - const chunkRange = 1000 - - walDir := filepath.Join(dir, "wal") - // Fill the chunk segments and corrupt it. - { - w, err := wlog.New(nil, nil, walDir, compression.None) - require.NoError(t, err) - - opts := DefaultHeadOptions() - opts.ChunkRange = chunkRange - opts.ChunkDirRoot = dir - opts.ChunkWriteQueueSize = 1 // We need to set this option so that we use the async queue. Upstream prometheus uses the queue directly. - h, err := NewHead(nil, nil, w, nil, opts, nil) - require.NoError(t, err) - require.Equal(t, 0.0, prom_testutil.ToFloat64(h.metrics.mmapChunkCorruptionTotal)) - require.NoError(t, h.Init(math.MinInt64)) - - cOpts := chunkOpts{ - chunkDiskMapper: h.chunkDiskMapper, - chunkRange: chunkRange, - samplesPerChunk: DefaultSamplesPerChunk, - } - - s, created, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) - require.True(t, created, "series was not created") - - for i := range 7 { - ok, chunkCreated := s.append(int64(i*chunkRange), float64(i*chunkRange), 0, cOpts) - require.True(t, ok, "series append failed") - require.True(t, chunkCreated, "chunk was not created") - ok, chunkCreated = s.append(int64(i*chunkRange)+chunkRange-1, float64(i*chunkRange), 0, cOpts) - require.True(t, ok, "series append failed") - require.False(t, chunkCreated, "chunk was created") - h.chunkDiskMapper.CutNewFile() - s.mmapChunks(h.chunkDiskMapper) - } - require.NoError(t, h.Close()) - - // Verify that there are 6 segment files. - // It should only be 6 because the last call to .CutNewFile() won't - // take effect without another chunk being written. - files, err := os.ReadDir(mmappedChunksDir(dir)) - require.NoError(t, err) - require.Len(t, files, 6) - - // Corrupt the 4th file by writing a random byte to series ref. - f, err := os.OpenFile(filepath.Join(mmappedChunksDir(dir), files[3].Name()), os.O_WRONLY, 0o666) - require.NoError(t, err) - n, err := f.WriteAt([]byte{67, 88}, chunks.HeadChunkFileHeaderSize+2) - require.NoError(t, err) - require.Equal(t, 2, n) - require.NoError(t, f.Close()) - } - - // Open the db to trigger a repair. - { - db, err := Open(dir, nil, nil, DefaultOptions(), nil) - require.NoError(t, err) - defer func() { - require.NoError(t, db.Close()) - }() - require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.mmapChunkCorruptionTotal)) - } - - // Verify that there are 3 segment files after the repair. - // The segments from the corrupt segment should be removed. - { - files, err := os.ReadDir(mmappedChunksDir(dir)) - require.NoError(t, err) - require.Len(t, files, 3) - } -} - -func TestNewWalSegmentOnTruncate(t *testing.T) { +func TestHeadAppenderV2_NewWalSegmentOnTruncate(t *testing.T) { h, wal := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, h.Close()) }() add := func(ts int64) { - app := h.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("a", "b"), ts, 0) + app := h.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, ts, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -2878,15 +960,15 @@ func TestNewWalSegmentOnTruncate(t *testing.T) { require.Equal(t, 2, last) } -func TestAddDuplicateLabelName(t *testing.T) { +func TestHeadAppenderV2_Append_DuplicateLabelName(t *testing.T) { h, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, h.Close()) }() add := func(labels labels.Labels, labelName string) { - app := h.Appender(context.Background()) - _, err := app.Append(0, labels, 0, 0) + app := h.AppenderV2(context.Background()) + _, err := app.Append(0, labels, 0, 0, 0, nil, nil, storage.AOptions{}) require.EqualError(t, err, fmt.Sprintf(`label name "%s" is not unique: invalid sample`, labelName)) } @@ -2895,7 +977,7 @@ func TestAddDuplicateLabelName(t *testing.T) { add(labels.FromStrings("__name__", "up", "job", "prometheus", "le", "500", "le", "400", "unit", "s"), "le") } -func TestMemSeriesIsolation(t *testing.T) { +func TestHeadAppenderV2_MemSeriesIsolation(t *testing.T) { if defaultIsolationDisabled { t.Skip("skipping test since tsdb isolation is disabled") } @@ -2940,17 +1022,17 @@ func TestMemSeriesIsolation(t *testing.T) { addSamples := func(h *Head) int { i := 1 for ; i <= 1000; i++ { - var app storage.Appender + var app storage.AppenderV2 // To initialize bounds. if h.MinTime() == math.MaxInt64 { - app = &initAppender{head: h} + app = &initAppenderV2{head: h} } else { - a := h.appender() + a := h.appenderV2() a.cleanupAppendIDsBelow = 0 app = a } - _, err := app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) h.mmapHeadChunks() @@ -2977,9 +1059,9 @@ func TestMemSeriesIsolation(t *testing.T) { require.Equal(t, 999, lastValue(hb, 999)) // Cleanup appendIDs below 500. - app := hb.appender() + app := hb.appenderV2() app.cleanupAppendIDsBelow = 500 - _, err := app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) i++ @@ -2996,9 +1078,9 @@ func TestMemSeriesIsolation(t *testing.T) { // Cleanup appendIDs below 1000, which means the sample buffer is // the only thing with appendIDs. - app = hb.appender() + app = hb.appenderV2() app.cleanupAppendIDsBelow = 1000 - _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, 999, lastValue(hb, 998)) @@ -3010,9 +1092,9 @@ func TestMemSeriesIsolation(t *testing.T) { i++ // Cleanup appendIDs below 1001, but with a rollback. - app = hb.appender() + app = hb.appenderV2() app.cleanupAppendIDsBelow = 1001 - _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Rollback()) require.Equal(t, 1000, lastValue(hb, 999)) @@ -3047,8 +1129,8 @@ func TestMemSeriesIsolation(t *testing.T) { // Cleanup appendIDs below 1000, which means the sample buffer is // the only thing with appendIDs. - app = hb.appender() - _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + app = hb.appenderV2() + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), float64(i), nil, nil, storage.AOptions{}) i++ require.NoError(t, err) require.NoError(t, app.Commit()) @@ -3060,8 +1142,8 @@ func TestMemSeriesIsolation(t *testing.T) { require.Equal(t, 1001, lastValue(hb, 1003)) // Cleanup appendIDs below 1002, but with a rollback. - app = hb.appender() - _, err = app.Append(0, labels.FromStrings("foo", "bar"), int64(i), float64(i)) + app = hb.appenderV2() + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Rollback()) require.Equal(t, 1001, lastValue(hb, 999)) @@ -3071,7 +1153,7 @@ func TestMemSeriesIsolation(t *testing.T) { require.Equal(t, 1001, lastValue(hb, 1003)) } -func TestIsolationRollback(t *testing.T) { +func TestHeadAppenderV2_IsolationRollback(t *testing.T) { if defaultIsolationDisabled { t.Skip("skipping test since tsdb isolation is disabled") } @@ -3082,28 +1164,28 @@ func TestIsolationRollback(t *testing.T) { require.NoError(t, hb.Close()) }() - app := hb.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + app := hb.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, uint64(1), hb.iso.lowWatermark()) - app = hb.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 1, 1) + app = hb.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 1, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("foo", "bar", "foo", "baz"), 2, 2) + _, err = app.Append(0, labels.FromStrings("foo", "bar", "foo", "baz"), 0, 2, 2, nil, nil, storage.AOptions{}) require.Error(t, err) require.NoError(t, app.Rollback()) require.Equal(t, uint64(2), hb.iso.lowWatermark()) - app = hb.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 3, 3) + app = hb.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 3, 3, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, uint64(3), hb.iso.lowWatermark(), "Low watermark should proceed to 3 even if append #2 was rolled back.") } -func TestIsolationLowWatermarkMonotonous(t *testing.T) { +func TestHeadAppenderV2_IsolationLowWatermarkMonotonous(t *testing.T) { if defaultIsolationDisabled { t.Skip("skipping test since tsdb isolation is disabled") } @@ -3113,19 +1195,19 @@ func TestIsolationLowWatermarkMonotonous(t *testing.T) { require.NoError(t, hb.Close()) }() - app1 := hb.Appender(context.Background()) - _, err := app1.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + app1 := hb.AppenderV2(context.Background()) + _, err := app1.Append(0, labels.FromStrings("foo", "bar"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app1.Commit()) require.Equal(t, uint64(1), hb.iso.lowWatermark(), "Low watermark should by 1 after 1st append.") - app1 = hb.Appender(context.Background()) - _, err = app1.Append(0, labels.FromStrings("foo", "bar"), 1, 1) + app1 = hb.AppenderV2(context.Background()) + _, err = app1.Append(0, labels.FromStrings("foo", "bar"), 0, 1, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, uint64(2), hb.iso.lowWatermark(), "Low watermark should be two, even if append is not committed yet.") - app2 := hb.Appender(context.Background()) - _, err = app2.Append(0, labels.FromStrings("foo", "baz"), 1, 1) + app2 := hb.AppenderV2(context.Background()) + _, err = app2.Append(0, labels.FromStrings("foo", "baz"), 0, 1, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app2.Commit()) require.Equal(t, uint64(2), hb.iso.lowWatermark(), "Low watermark should stay two because app1 is not committed yet.") @@ -3140,39 +1222,7 @@ func TestIsolationLowWatermarkMonotonous(t *testing.T) { require.Equal(t, uint64(3), hb.iso.lowWatermark(), "After read has finished (iso state closed), low watermark should jump to three.") } -func TestIsolationAppendIDZeroIsNoop(t *testing.T) { - if defaultIsolationDisabled { - t.Skip("skipping test since tsdb isolation is disabled") - } - - h, _ := newTestHead(t, 1000, compression.None, false) - defer func() { - require.NoError(t, h.Close()) - }() - - h.initTime(0) - - cOpts := chunkOpts{ - chunkDiskMapper: h.chunkDiskMapper, - chunkRange: h.chunkRange.Load(), - samplesPerChunk: DefaultSamplesPerChunk, - } - - s, _, _ := h.getOrCreate(1, labels.FromStrings("a", "1"), false) - - ok, _ := s.append(0, 0, 0, cOpts) - require.True(t, ok, "Series append failed.") - require.Equal(t, 0, int(s.txs.txIDCount), "Series should not have an appendID after append with appendID=0.") -} - -func TestHeadSeriesChunkRace(t *testing.T) { - t.Parallel() - for range 100 { - testHeadSeriesChunkRace(t) - } -} - -func TestIsolationWithoutAdd(t *testing.T) { +func TestHeadAppenderV2_IsolationWithoutAdd(t *testing.T) { if defaultIsolationDisabled { t.Skip("skipping test since tsdb isolation is disabled") } @@ -3182,28 +1232,28 @@ func TestIsolationWithoutAdd(t *testing.T) { require.NoError(t, hb.Close()) }() - app := hb.Appender(context.Background()) + app := hb.AppenderV2(context.Background()) require.NoError(t, app.Commit()) - app = hb.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("foo", "baz"), 1, 1) + app = hb.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "baz"), 0, 1, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Equal(t, hb.iso.lastAppendID(), hb.iso.lowWatermark(), "High watermark should be equal to the low watermark") } -func TestOutOfOrderSamplesMetric(t *testing.T) { +func TestHeadAppenderV2_Append_OutOfOrderSamplesMetric(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { options := DefaultOptions() - testOutOfOrderSamplesMetric(t, scenario, options, storage.ErrOutOfOrderSample) + testHeadAppenderV2OutOfOrderSamplesMetric(t, scenario, options, storage.ErrOutOfOrderSample) }) } } -func TestOutOfOrderSamplesMetricNativeHistogramOOODisabled(t *testing.T) { +func TestHeadAppenderV2_Append_OutOfOrderSamplesMetricNativeHistogramOOODisabled(t *testing.T) { for name, scenario := range sampleTypeScenarios { if scenario.sampleType != "histogram" { continue @@ -3211,12 +1261,12 @@ func TestOutOfOrderSamplesMetricNativeHistogramOOODisabled(t *testing.T) { t.Run(name, func(t *testing.T) { options := DefaultOptions() options.OutOfOrderTimeWindow = 0 - testOutOfOrderSamplesMetric(t, scenario, options, storage.ErrOutOfOrderSample) + testHeadAppenderV2OutOfOrderSamplesMetric(t, scenario, options, storage.ErrOutOfOrderSample) }) } } -func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, options *Options, expectOutOfOrderError error) { +func testHeadAppenderV2OutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, options *Options, expectOutOfOrderError error) { dir := t.TempDir() db, err := Open(dir, nil, nil, options, nil) require.NoError(t, err) @@ -3225,13 +1275,14 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti }() db.DisableCompactions() - appendSample := func(appender storage.Appender, ts int64) (storage.SeriesRef, error) { - ref, _, err := scenario.appendFunc(appender, labels.FromStrings("a", "b"), ts, 99) + appendSample := func(app storage.AppenderV2, ts int64) (storage.SeriesRef, error) { + // TODO(bwplotka): Migrate to V2 natively. + ref, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), labels.FromStrings("a", "b"), ts, 99) return ref, err } ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) for i := 1; i <= 5; i++ { _, err = appendSample(app, int64(i)) require.NoError(t, err) @@ -3240,7 +1291,7 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti // Test out of order metric. require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) - app = db.Appender(ctx) + app = db.AppenderV2(ctx) _, err = appendSample(app, 2) require.Equal(t, expectOutOfOrderError, err) require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) @@ -3255,7 +1306,7 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti require.NoError(t, app.Commit()) // Compact Head to test out of bound metric. - app = db.Appender(ctx) + app = db.AppenderV2(ctx) _, err = appendSample(app, DefaultBlockDuration*2) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -3263,8 +1314,9 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti require.Equal(t, int64(math.MinInt64), db.head.minValidTime.Load()) require.NoError(t, db.Compact(ctx)) require.Positive(t, db.head.minValidTime.Load()) + require.Equal(t, 0.0, prom_testutil.ToFloat64(db.head.metrics.outOfBoundSamples.WithLabelValues(scenario.sampleType))) - app = db.Appender(ctx) + app = db.AppenderV2(ctx) _, err = appendSample(app, db.head.minValidTime.Load()-2) require.Equal(t, storage.ErrOutOfBounds, err) require.Equal(t, 1.0, prom_testutil.ToFloat64(db.head.metrics.outOfBoundSamples.WithLabelValues(scenario.sampleType))) @@ -3275,7 +1327,7 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti require.NoError(t, app.Commit()) // Some more valid samples for out of order. - app = db.Appender(ctx) + app = db.AppenderV2(ctx) for i := 1; i <= 5; i++ { _, err = appendSample(app, db.head.minValidTime.Load()+DefaultBlockDuration+int64(i)) require.NoError(t, err) @@ -3283,7 +1335,7 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti require.NoError(t, app.Commit()) // Test out of order metric. - app = db.Appender(ctx) + app = db.AppenderV2(ctx) _, err = appendSample(app, db.head.minValidTime.Load()+DefaultBlockDuration+2) require.Equal(t, expectOutOfOrderError, err) require.Equal(t, 4.0, prom_testutil.ToFloat64(db.head.metrics.outOfOrderSamples.WithLabelValues(scenario.sampleType))) @@ -3298,42 +1350,7 @@ func testOutOfOrderSamplesMetric(t *testing.T, scenario sampleTypeScenario, opti require.NoError(t, app.Commit()) } -func testHeadSeriesChunkRace(t *testing.T) { - h, _ := newTestHead(t, 1000, compression.None, false) - defer func() { - require.NoError(t, h.Close()) - }() - require.NoError(t, h.Init(0)) - app := h.Appender(context.Background()) - - s2, err := app.Append(0, labels.FromStrings("foo2", "bar"), 5, 0) - require.NoError(t, err) - for ts := int64(6); ts < 11; ts++ { - _, err = app.Append(s2, labels.EmptyLabels(), ts, 0) - require.NoError(t, err) - } - require.NoError(t, app.Commit()) - - matcher := labels.MustNewMatcher(labels.MatchEqual, "", "") - q, err := NewBlockQuerier(h, 18, 22) - require.NoError(t, err) - defer q.Close() - - var wg sync.WaitGroup - wg.Add(1) - go func() { - defer wg.Done() - h.updateMinMaxTime(20, 25) - h.gc() - }() - ss := q.Select(context.Background(), false, nil, matcher) - for ss.Next() { - } - require.NoError(t, ss.Err()) - wg.Wait() -} - -func TestHeadLabelNamesValuesWithMinMaxRange(t *testing.T) { +func TestHeadLabelNamesValuesWithMinMaxRange_AppenderV2(t *testing.T) { head, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, head.Close()) @@ -3355,9 +1372,9 @@ func TestHeadLabelNamesValuesWithMinMaxRange(t *testing.T) { ctx = context.Background() ) - app := head.Appender(ctx) + app := head.AppenderV2(ctx) for i, name := range expectedLabelNames { - _, err := app.Append(0, labels.FromStrings(name, expectedLabelValues[i]), seriesTimestamps[i], 0) + _, err := app.Append(0, labels.FromStrings(name, expectedLabelValues[i]), 0, seriesTimestamps[i], 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -3394,260 +1411,51 @@ func TestHeadLabelNamesValuesWithMinMaxRange(t *testing.T) { } } -func TestHeadLabelValuesWithMatchers(t *testing.T) { - head, _ := newTestHead(t, 1000, compression.None, false) - t.Cleanup(func() { require.NoError(t, head.Close()) }) - - ctx := context.Background() - - app := head.Appender(context.Background()) - for i := range 100 { - _, err := app.Append(0, labels.FromStrings( - "tens", fmt.Sprintf("value%d", i/10), - "unique", fmt.Sprintf("value%d", i), - ), 100, 0) - require.NoError(t, err) - } - require.NoError(t, app.Commit()) - - var uniqueWithout30s []string - for i := range 100 { - if i/10 != 3 { - uniqueWithout30s = append(uniqueWithout30s, fmt.Sprintf("value%d", i)) - } - } - sort.Strings(uniqueWithout30s) - testCases := []struct { - name string - labelName string - matchers []*labels.Matcher - expectedValues []string - }{ - { - name: "get tens based on unique id", - labelName: "tens", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "unique", "value35")}, - expectedValues: []string{"value3"}, - }, { - name: "get unique ids based on a ten", - labelName: "unique", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "tens", "value1")}, - expectedValues: []string{"value10", "value11", "value12", "value13", "value14", "value15", "value16", "value17", "value18", "value19"}, - }, { - name: "get tens by pattern matching on unique id", - labelName: "tens", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "unique", "value[5-7]5")}, - expectedValues: []string{"value5", "value6", "value7"}, - }, { - name: "get tens by matching for presence of unique label", - labelName: "tens", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchNotEqual, "unique", "")}, - expectedValues: []string{"value0", "value1", "value2", "value3", "value4", "value5", "value6", "value7", "value8", "value9"}, - }, { - name: "get unique IDs based on tens not being equal to a certain value, while not empty", - labelName: "unique", - matchers: []*labels.Matcher{ - labels.MustNewMatcher(labels.MatchNotEqual, "tens", "value3"), - labels.MustNewMatcher(labels.MatchNotEqual, "tens", ""), - }, - expectedValues: uniqueWithout30s, - }, - } - - for _, tt := range testCases { - t.Run(tt.name, func(t *testing.T) { - headIdxReader := head.indexRange(0, 200) - - actualValues, err := headIdxReader.SortedLabelValues(ctx, tt.labelName, nil, tt.matchers...) - require.NoError(t, err) - require.Equal(t, tt.expectedValues, actualValues) - - actualValues, err = headIdxReader.LabelValues(ctx, tt.labelName, nil, tt.matchers...) - sort.Strings(actualValues) - require.NoError(t, err) - require.Equal(t, tt.expectedValues, actualValues) - }) - } -} - -func TestHeadLabelNamesWithMatchers(t *testing.T) { +func TestHeadAppenderV2_ErrReuse(t *testing.T) { head, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, head.Close()) }() - app := head.Appender(context.Background()) - for i := range 100 { - _, err := app.Append(0, labels.FromStrings( - "unique", fmt.Sprintf("value%d", i), - ), 100, 0) - require.NoError(t, err) - - if i%10 == 0 { - _, err := app.Append(0, labels.FromStrings( - "tens", fmt.Sprintf("value%d", i/10), - "unique", fmt.Sprintf("value%d", i), - ), 100, 0) - require.NoError(t, err) - } - - if i%20 == 0 { - _, err := app.Append(0, labels.FromStrings( - "tens", fmt.Sprintf("value%d", i/10), - "twenties", fmt.Sprintf("value%d", i/20), - "unique", fmt.Sprintf("value%d", i), - ), 100, 0) - require.NoError(t, err) - } - } - require.NoError(t, app.Commit()) - - testCases := []struct { - name string - labelName string - matchers []*labels.Matcher - expectedNames []string - }{ - { - name: "get with non-empty unique: all", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchNotEqual, "unique", "")}, - expectedNames: []string{"tens", "twenties", "unique"}, - }, { - name: "get with unique ending in 1: only unique", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "unique", "value.*1")}, - expectedNames: []string{"unique"}, - }, { - name: "get with unique = value20: all", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "unique", "value20")}, - expectedNames: []string{"tens", "twenties", "unique"}, - }, { - name: "get tens = 1: unique & tens", - matchers: []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "tens", "value1")}, - expectedNames: []string{"tens", "unique"}, - }, - } - - for _, tt := range testCases { - t.Run(tt.name, func(t *testing.T) { - headIdxReader := head.indexRange(0, 200) - - actualNames, err := headIdxReader.LabelNames(context.Background(), tt.matchers...) - require.NoError(t, err) - require.Equal(t, tt.expectedNames, actualNames) - }) - } -} - -func TestHeadShardedPostings(t *testing.T) { - headOpts := newTestHeadDefaultOptions(1000, false) - headOpts.EnableSharding = true - head, _ := newTestHeadWithOptions(t, compression.None, headOpts) - defer func() { - require.NoError(t, head.Close()) - }() - - ctx := context.Background() - - // Append some series. - app := head.Appender(ctx) - for i := range 100 { - _, err := app.Append(0, labels.FromStrings("unique", fmt.Sprintf("value%d", i), "const", "1"), 100, 0) - require.NoError(t, err) - } - require.NoError(t, app.Commit()) - - ir := head.indexRange(0, 200) - - // List all postings for a given label value. This is what we expect to get - // in output from all shards. - p, err := ir.Postings(ctx, "const", "1") - require.NoError(t, err) - - var expected []storage.SeriesRef - for p.Next() { - expected = append(expected, p.At()) - } - require.NoError(t, p.Err()) - require.NotEmpty(t, expected) - - // Query the same postings for each shard. - const shardCount = uint64(4) - actualShards := make(map[uint64][]storage.SeriesRef) - actualPostings := make([]storage.SeriesRef, 0, len(expected)) - - for shardIndex := range shardCount { - p, err = ir.Postings(ctx, "const", "1") - require.NoError(t, err) - - p = ir.ShardedPostings(p, shardIndex, shardCount) - for p.Next() { - ref := p.At() - - actualShards[shardIndex] = append(actualShards[shardIndex], ref) - actualPostings = append(actualPostings, ref) - } - require.NoError(t, p.Err()) - } - - // We expect the postings merged out of shards is the exact same of the non sharded ones. - require.ElementsMatch(t, expected, actualPostings) - - // We expect the series in each shard are the expected ones. - for shardIndex, ids := range actualShards { - for _, id := range ids { - var lbls labels.ScratchBuilder - - require.NoError(t, ir.Series(id, &lbls, nil)) - require.Equal(t, shardIndex, labels.StableHash(lbls.Labels())%shardCount) - } - } -} - -func TestErrReuseAppender(t *testing.T) { - head, _ := newTestHead(t, 1000, compression.None, false) - defer func() { - require.NoError(t, head.Close()) - }() - - app := head.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("test", "test"), 0, 0) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("test", "test"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Error(t, app.Commit()) require.Error(t, app.Rollback()) - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("test", "test"), 1, 0) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("test", "test"), 0, 1, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Rollback()) require.Error(t, app.Rollback()) require.Error(t, app.Commit()) - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("test", "test"), 2, 0) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("test", "test"), 0, 2, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.Error(t, app.Rollback()) require.Error(t, app.Commit()) - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("test", "test"), 3, 0) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("test", "test"), 0, 3, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Rollback()) require.Error(t, app.Commit()) require.Error(t, app.Rollback()) } -func TestHeadMintAfterTruncation(t *testing.T) { +func TestHeadAppenderV2_MinTimeAfterTruncation(t *testing.T) { chunkRange := int64(2000) head, _ := newTestHead(t, chunkRange, compression.None, false) - app := head.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("a", "b"), 100, 100) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 100, 100, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("a", "b"), 4000, 200) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 4000, 200, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("a", "b"), 8000, 300) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 8000, 300, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -3672,202 +1480,33 @@ func TestHeadMintAfterTruncation(t *testing.T) { require.NoError(t, head.Close()) } -func TestHeadExemplars(t *testing.T) { +func TestHeadAppenderV2_AppendExemplars(t *testing.T) { chunkRange := int64(2000) head, _ := newTestHead(t, chunkRange, compression.None, false) - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) l := labels.FromStrings("trace_id", "123") + // It is perfectly valid to add Exemplars before the current start time - // histogram buckets that haven't been update in a while could still be // exported exemplars from an hour ago. - ref, err := app.Append(0, labels.FromStrings("a", "b"), 100, 100) - require.NoError(t, err) - _, err = app.AppendExemplar(ref, l, exemplar.Exemplar{ - Labels: l, - HasTs: true, - Ts: -1000, - Value: 1, + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 100, 100, nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{{Labels: l, HasTs: true, Ts: -1000, Value: 1}}, }) require.NoError(t, err) require.NoError(t, app.Commit()) require.NoError(t, head.Close()) } -func BenchmarkHeadLabelValuesWithMatchers(b *testing.B) { - chunkRange := int64(2000) - head, _ := newTestHead(b, chunkRange, compression.None, false) - b.Cleanup(func() { require.NoError(b, head.Close()) }) - - ctx := context.Background() - - app := head.Appender(context.Background()) - - metricCount := 1000000 - for i := range metricCount { - _, err := app.Append(0, labels.FromStrings( - "a_unique", fmt.Sprintf("value%d", i), - "b_tens", fmt.Sprintf("value%d", i/(metricCount/10)), - "c_ninety", fmt.Sprintf("value%d", i/(metricCount/10)/9), // "0" for the first 90%, then "1" - ), 100, 0) - require.NoError(b, err) - } - require.NoError(b, app.Commit()) - - headIdxReader := head.indexRange(0, 200) - matchers := []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "c_ninety", "value0")} - - b.ReportAllocs() - - for b.Loop() { - actualValues, err := headIdxReader.LabelValues(ctx, "b_tens", nil, matchers...) - require.NoError(b, err) - require.Len(b, actualValues, 9) - } -} - -func TestIteratorSeekIntoBuffer(t *testing.T) { - dir := t.TempDir() - // This is usually taken from the Head, but passing manually here. - chunkDiskMapper, err := chunks.NewChunkDiskMapper(nil, dir, chunkenc.NewPool(), chunks.DefaultWriteBufferSize, chunks.DefaultWriteQueueSize) - require.NoError(t, err) - defer func() { - require.NoError(t, chunkDiskMapper.Close()) - }() - cOpts := chunkOpts{ - chunkDiskMapper: chunkDiskMapper, - chunkRange: 500, - samplesPerChunk: DefaultSamplesPerChunk, - } - - s := newMemSeries(labels.Labels{}, 1, 0, defaultIsolationDisabled, false) - - for i := range 7 { - ok, _ := s.append(int64(i), float64(i), 0, cOpts) - require.True(t, ok, "sample append failed") - } - - c, _, _, err := s.chunk(0, chunkDiskMapper, &sync.Pool{ - New: func() any { - return &memChunk{} - }, - }) - require.NoError(t, err) - it := c.chunk.Iterator(nil) - - // First point. - require.Equal(t, chunkenc.ValFloat, it.Seek(0)) - ts, val := it.At() - require.Equal(t, int64(0), ts) - require.Equal(t, float64(0), val) - - // Advance one point. - require.Equal(t, chunkenc.ValFloat, it.Next()) - ts, val = it.At() - require.Equal(t, int64(1), ts) - require.Equal(t, float64(1), val) - - // Seeking an older timestamp shouldn't cause the iterator to go backwards. - require.Equal(t, chunkenc.ValFloat, it.Seek(0)) - ts, val = it.At() - require.Equal(t, int64(1), ts) - require.Equal(t, float64(1), val) - - // Seek into the buffer. - require.Equal(t, chunkenc.ValFloat, it.Seek(3)) - ts, val = it.At() - require.Equal(t, int64(3), ts) - require.Equal(t, float64(3), val) - - // Iterate through the rest of the buffer. - for i := 4; i < 7; i++ { - require.Equal(t, chunkenc.ValFloat, it.Next()) - ts, val = it.At() - require.Equal(t, int64(i), ts) - require.Equal(t, float64(i), val) - } - - // Run out of elements in the iterator. - require.Equal(t, chunkenc.ValNone, it.Next()) - require.Equal(t, chunkenc.ValNone, it.Seek(7)) -} - -// Tests https://github.com/prometheus/prometheus/issues/8221. -func TestChunkNotFoundHeadGCRace(t *testing.T) { - t.Parallel() - db := newTestDB(t) - db.DisableCompactions() - ctx := context.Background() - - var ( - app = db.Appender(context.Background()) - ref = storage.SeriesRef(0) - mint, maxt = int64(0), int64(0) - err error - ) - - // Appends samples to span over 1.5 block ranges. - // 7 chunks with 15s scrape interval. - for i := int64(0); i <= 120*7; i++ { - ts := i * DefaultBlockDuration / (4 * 120) - ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i)) - require.NoError(t, err) - maxt = ts - } - require.NoError(t, app.Commit()) - - // Get a querier before compaction (or when compaction is about to begin). - q, err := db.Querier(mint, maxt) - require.NoError(t, err) - - // Query the compacted range and get the first series before compaction. - ss := q.Select(context.Background(), true, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) - require.True(t, ss.Next()) - s := ss.At() - - var wg sync.WaitGroup - wg.Add(1) - go func() { - defer wg.Done() - // Compacting head while the querier spans the compaction time. - require.NoError(t, db.Compact(ctx)) - require.NotEmpty(t, db.Blocks()) - }() - - // Give enough time for compaction to finish. - // We expect it to be blocked until querier is closed. - <-time.After(3 * time.Second) - - // Now consume after compaction when it's gone. - it := s.Iterator(nil) - for it.Next() == chunkenc.ValFloat { - _, _ = it.At() - } - // It should error here without any fix for the mentioned issue. - require.NoError(t, it.Err()) - for ss.Next() { - s = ss.At() - it = s.Iterator(it) - for it.Next() == chunkenc.ValFloat { - _, _ = it.At() - } - require.NoError(t, it.Err()) - } - require.NoError(t, ss.Err()) - - require.NoError(t, q.Close()) - wg.Wait() -} - // Tests https://github.com/prometheus/prometheus/issues/9079. -func TestDataMissingOnQueryDuringCompaction(t *testing.T) { +func TestDataMissingOnQueryDuringCompaction_AppenderV2(t *testing.T) { t.Parallel() db := newTestDB(t) db.DisableCompactions() ctx := context.Background() var ( - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) ref = storage.SeriesRef(0) mint, maxt = int64(0), int64(0) err error @@ -3878,7 +1517,7 @@ func TestDataMissingOnQueryDuringCompaction(t *testing.T) { // 7 chunks with 15s scrape interval. for i := int64(0); i <= 120*7; i++ { ts := i * DefaultBlockDuration / (4 * 120) - ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i)) + ref, err = app.Append(ref, labels.FromStrings("a", "b"), 0, ts, float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) maxt = ts expSamples = append(expSamples, sample{ts, float64(i), nil, nil}) @@ -3909,18 +1548,18 @@ func TestDataMissingOnQueryDuringCompaction(t *testing.T) { wg.Wait() } -func TestIsQuerierCollidingWithTruncation(t *testing.T) { +func TestIsQuerierCollidingWithTruncation_AppenderV2(t *testing.T) { db := newTestDB(t) db.DisableCompactions() var ( - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) ref = storage.SeriesRef(0) err error ) for i := int64(0); i <= 3000; i++ { - ref, err = app.Append(ref, labels.FromStrings("a", "b"), i, float64(i)) + ref, err = app.Append(ref, labels.FromStrings("a", "b"), 0, i, float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -3954,7 +1593,7 @@ func TestIsQuerierCollidingWithTruncation(t *testing.T) { } } -func TestWaitForPendingReadersInTimeRange(t *testing.T) { +func TestWaitForPendingReadersInTimeRange_AppenderV2(t *testing.T) { t.Parallel() db := newTestDB(t) db.DisableCompactions() @@ -3962,14 +1601,14 @@ func TestWaitForPendingReadersInTimeRange(t *testing.T) { sampleTs := func(i int64) int64 { return i * DefaultBlockDuration / (4 * 120) } var ( - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) ref = storage.SeriesRef(0) err error ) for i := int64(0); i <= 3000; i++ { ts := sampleTs(i) - ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i)) + ref, err = app.Append(ref, labels.FromStrings("a", "b"), 0, ts, float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -4012,31 +1651,8 @@ func TestWaitForPendingReadersInTimeRange(t *testing.T) { } } -func TestQueryOOOHeadDuringTruncate(t *testing.T) { - testQueryOOOHeadDuringTruncate(t, - func(db *DB, minT, maxT int64) (storage.LabelQuerier, error) { - return db.Querier(minT, maxT) - }, - func(t *testing.T, lq storage.LabelQuerier, minT, _ int64) { - // Samples - q, ok := lq.(storage.Querier) - require.True(t, ok) - ss := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "a", "b")) - require.True(t, ss.Next()) - s := ss.At() - require.False(t, ss.Next()) // One series. - it := s.Iterator(nil) - require.NotEqual(t, chunkenc.ValNone, it.Next()) // Has some data. - require.Equal(t, minT, it.AtT()) // It is an in-order sample. - require.NotEqual(t, chunkenc.ValNone, it.Next()) // Has some data. - require.Equal(t, minT+50, it.AtT()) // it is an out-of-order sample. - require.NoError(t, it.Err()) - }, - ) -} - -func TestChunkQueryOOOHeadDuringTruncate(t *testing.T) { - testQueryOOOHeadDuringTruncate(t, +func TestChunkQueryOOOHeadDuringTruncate_AppenderV2(t *testing.T) { + testQueryOOOHeadDuringTruncateAppenderV2(t, func(db *DB, minT, maxT int64) (storage.LabelQuerier, error) { return db.ChunkQuerier(minT, maxT) }, @@ -4062,7 +1678,7 @@ func TestChunkQueryOOOHeadDuringTruncate(t *testing.T) { ) } -func testQueryOOOHeadDuringTruncate(t *testing.T, makeQuerier func(db *DB, minT, maxT int64) (storage.LabelQuerier, error), verify func(t *testing.T, q storage.LabelQuerier, minT, maxT int64)) { +func testQueryOOOHeadDuringTruncateAppenderV2(t *testing.T, makeQuerier func(db *DB, minT, maxT int64) (storage.LabelQuerier, error), verify func(t *testing.T, q storage.LabelQuerier, minT, maxT int64)) { const maxT int64 = 6000 dir := t.TempDir() @@ -4079,16 +1695,16 @@ func testQueryOOOHeadDuringTruncate(t *testing.T, makeQuerier func(db *DB, minT, var ( ref = storage.SeriesRef(0) - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) ) // Add in-order samples at every 100ms starting at 0ms. for i := int64(0); i < maxT; i += 100 { - _, err := app.Append(ref, labels.FromStrings("a", "b"), i, 0) + _, err := app.Append(ref, labels.FromStrings("a", "b"), 0, i, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } // Add out-of-order samples at every 100ms starting at 50ms. for i := int64(50); i < maxT; i += 100 { - _, err := app.Append(ref, labels.FromStrings("a", "b"), i, 0) + _, err := app.Append(ref, labels.FromStrings("a", "b"), 0, i, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -4138,7 +1754,7 @@ func testQueryOOOHeadDuringTruncate(t *testing.T, makeQuerier func(db *DB, minT, <-compactionFinished // Wait for compaction otherwise Go test finds stray goroutines. } -func TestAppendHistogram(t *testing.T) { +func TestHeadAppenderV2_Append_Histogram(t *testing.T) { l := labels.FromStrings("a", "b") for _, numHistograms := range []int{1, 10, 150, 200, 250, 300} { t.Run(strconv.Itoa(numHistograms), func(t *testing.T) { @@ -4149,31 +1765,31 @@ func TestAppendHistogram(t *testing.T) { require.NoError(t, head.Init(0)) ingestTs := int64(0) - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) expHistograms := make([]chunks.Sample, 0, 2*numHistograms) // Counter integer histograms. for _, h := range tsdbutil.GenerateTestHistograms(numHistograms) { - _, err := app.AppendHistogram(0, l, ingestTs, h, nil) + _, err := app.Append(0, l, 0, ingestTs, 0, h, nil, storage.AOptions{}) require.NoError(t, err) expHistograms = append(expHistograms, sample{t: ingestTs, h: h}) ingestTs++ if ingestTs%50 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } // Gauge integer histograms. for _, h := range tsdbutil.GenerateTestGaugeHistograms(numHistograms) { - _, err := app.AppendHistogram(0, l, ingestTs, h, nil) + _, err := app.Append(0, l, 0, ingestTs, 0, h, nil, storage.AOptions{}) require.NoError(t, err) expHistograms = append(expHistograms, sample{t: ingestTs, h: h}) ingestTs++ if ingestTs%50 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } @@ -4181,25 +1797,25 @@ func TestAppendHistogram(t *testing.T) { // Counter float histograms. for _, fh := range tsdbutil.GenerateTestFloatHistograms(numHistograms) { - _, err := app.AppendHistogram(0, l, ingestTs, nil, fh) + _, err := app.Append(0, l, 0, ingestTs, 0, nil, fh, storage.AOptions{}) require.NoError(t, err) expFloatHistograms = append(expFloatHistograms, sample{t: ingestTs, fh: fh}) ingestTs++ if ingestTs%50 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } // Gauge float histograms. for _, fh := range tsdbutil.GenerateTestGaugeFloatHistograms(numHistograms) { - _, err := app.AppendHistogram(0, l, ingestTs, nil, fh) + _, err := app.Append(0, l, 0, ingestTs, 0, nil, fh, storage.AOptions{}) require.NoError(t, err) expFloatHistograms = append(expFloatHistograms, sample{t: ingestTs, fh: fh}) ingestTs++ if ingestTs%50 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } @@ -4245,7 +1861,7 @@ func TestAppendHistogram(t *testing.T) { } } -func TestHistogramInWALAndMmapChunk(t *testing.T) { +func TestHistogramInWALAndMmapChunk_AppenderV2(t *testing.T) { head, _ := newTestHead(t, 3000, compression.None, false) t.Cleanup(func() { require.NoError(t, head.Close()) @@ -4258,9 +1874,9 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { numHistograms := 300 exp := map[string][]chunks.Sample{} ts := int64(0) - var app storage.Appender + var app storage.AppenderV2 for _, gauge := range []bool{true, false} { - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) var hists []*histogram.Histogram if gauge { hists = tsdbutil.GenerateTestGaugeHistograms(numHistograms) @@ -4270,19 +1886,19 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { for _, h := range hists { h.NegativeSpans = h.PositiveSpans h.NegativeBuckets = h.PositiveBuckets - _, err := app.AppendHistogram(0, s1, ts, h, nil) + _, err := app.Append(0, s1, 0, ts, 0, h, nil, storage.AOptions{}) require.NoError(t, err) exp[k1] = append(exp[k1], sample{t: ts, h: h.Copy()}) ts++ if ts%5 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) } for _, gauge := range []bool{true, false} { - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) var hists []*histogram.FloatHistogram if gauge { hists = tsdbutil.GenerateTestGaugeFloatHistograms(numHistograms) @@ -4292,13 +1908,13 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { for _, h := range hists { h.NegativeSpans = h.PositiveSpans h.NegativeBuckets = h.PositiveBuckets - _, err := app.AppendHistogram(0, s1, ts, nil, h) + _, err := app.Append(0, s1, 0, ts, 0, nil, h, storage.AOptions{}) require.NoError(t, err) exp[k1] = append(exp[k1], sample{t: ts, fh: h.Copy()}) ts++ if ts%5 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) @@ -4322,7 +1938,7 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { k2 := s2.String() ts = 0 for _, gauge := range []bool{true, false} { - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) var hists []*histogram.Histogram if gauge { hists = tsdbutil.GenerateTestGaugeHistograms(100) @@ -4333,7 +1949,7 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { ts++ h.NegativeSpans = h.PositiveSpans h.NegativeBuckets = h.PositiveBuckets - _, err := app.AppendHistogram(0, s2, ts, h, nil) + _, err := app.Append(0, s2, 0, ts, 0, h, nil, storage.AOptions{}) require.NoError(t, err) eh := h.Copy() if !gauge && ts > 30 && (ts-10)%20 == 1 { @@ -4343,22 +1959,22 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { exp[k2] = append(exp[k2], sample{t: ts, h: eh}) if ts%20 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) // Add some float. for range 10 { ts++ - _, err := app.Append(0, s2, ts, float64(ts)) + _, err := app.Append(0, s2, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) exp[k2] = append(exp[k2], sample{t: ts, f: float64(ts)}) } require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) } for _, gauge := range []bool{true, false} { - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) var hists []*histogram.FloatHistogram if gauge { hists = tsdbutil.GenerateTestGaugeFloatHistograms(100) @@ -4369,7 +1985,7 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { ts++ h.NegativeSpans = h.PositiveSpans h.NegativeBuckets = h.PositiveBuckets - _, err := app.AppendHistogram(0, s2, ts, nil, h) + _, err := app.Append(0, s2, 0, ts, 0, nil, h, storage.AOptions{}) require.NoError(t, err) eh := h.Copy() if !gauge && ts > 30 && (ts-10)%20 == 1 { @@ -4379,16 +1995,16 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { exp[k2] = append(exp[k2], sample{t: ts, fh: eh}) if ts%20 == 0 { require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) // Add some float. for range 10 { ts++ - _, err := app.Append(0, s2, ts, float64(ts)) + _, err := app.Append(0, s2, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) exp[k2] = append(exp[k2], sample{t: ts, f: float64(ts)}) } require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) @@ -4425,7 +2041,7 @@ func TestHistogramInWALAndMmapChunk(t *testing.T) { testQuery() } -func TestChunkSnapshot(t *testing.T) { +func TestChunkSnapshot_AppenderV2(t *testing.T) { head, _ := newTestHead(t, 120*4, compression.None, false) defer func() { head.opts.EnableMemorySnapshotOnShutdown = false @@ -4446,7 +2062,7 @@ func TestChunkSnapshot(t *testing.T) { histograms := tsdbutil.GenerateTestGaugeHistograms(481) floatHistogram := tsdbutil.GenerateTestGaugeFloatHistograms(481) - addExemplar := func(app storage.Appender, ref storage.SeriesRef, lbls labels.Labels, ts int64) { + newExemplar := func(lbls labels.Labels, ts int64) exemplar.Exemplar { e := ex{ seriesLabels: lbls, e: exemplar.Exemplar{ @@ -4456,8 +2072,7 @@ func TestChunkSnapshot(t *testing.T) { }, } expExemplars = append(expExemplars, e) - _, err := app.AppendExemplar(ref, e.seriesLabels, e.e) - require.NoError(t, err) + return e.e } checkSamples := func() { @@ -4534,7 +2149,7 @@ func TestChunkSnapshot(t *testing.T) { { // Initial data that goes into snapshot. // Add some initial samples with >=1 m-map chunk. - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) for i := 1; i <= numSeries; i++ { lbls := labels.FromStrings("foo", fmt.Sprintf("bar%d", i)) lblStr := lbls.String() @@ -4545,26 +2160,30 @@ func TestChunkSnapshot(t *testing.T) { // 240 samples should m-map at least 1 chunk. for ts := int64(1); ts <= 240; ts++ { + // Add an exemplar, but only to float sample. + aOpts := storage.AOptions{} + if ts%10 == 0 { + aOpts.Exemplars = []exemplar.Exemplar{newExemplar(lbls, ts)} + } val := rand.Float64() expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil}) - ref, err := app.Append(0, lbls, ts, val) + _, err := app.Append(0, lbls, 0, ts, val, nil, nil, aOpts) require.NoError(t, err) hist := histograms[int(ts)] expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil}) - _, err = app.AppendHistogram(0, lblsHist, ts, hist, nil) + _, err = app.Append(0, lblsHist, 0, ts, 0, hist, nil, storage.AOptions{}) require.NoError(t, err) floatHist := floatHistogram[int(ts)] expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist}) - _, err = app.AppendHistogram(0, lblsFloatHist, ts, nil, floatHist) + _, err = app.Append(0, lblsFloatHist, 0, ts, 0, nil, floatHist, storage.AOptions{}) require.NoError(t, err) - // Add an exemplar and to create multiple WAL records. + // Create multiple WAL records (commit). if ts%10 == 0 { - addExemplar(app, ref, lbls, ts) require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } } @@ -4608,7 +2227,7 @@ func TestChunkSnapshot(t *testing.T) { { // Additional data to only include in WAL and m-mapped chunks and not snapshot. This mimics having an old snapshot on disk. // Add more samples. - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) for i := 1; i <= numSeries; i++ { lbls := labels.FromStrings("foo", fmt.Sprintf("bar%d", i)) lblStr := lbls.String() @@ -4619,26 +2238,30 @@ func TestChunkSnapshot(t *testing.T) { // 240 samples should m-map at least 1 chunk. for ts := int64(241); ts <= 480; ts++ { + // Add an exemplar, but only to float sample. + aOpts := storage.AOptions{} + if ts%10 == 0 { + aOpts.Exemplars = []exemplar.Exemplar{newExemplar(lbls, ts)} + } val := rand.Float64() expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil}) - ref, err := app.Append(0, lbls, ts, val) + _, err := app.Append(0, lbls, 0, ts, val, nil, nil, aOpts) require.NoError(t, err) hist := histograms[int(ts)] expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil}) - _, err = app.AppendHistogram(0, lblsHist, ts, hist, nil) + _, err = app.Append(0, lblsHist, 0, ts, 0, hist, nil, storage.AOptions{}) require.NoError(t, err) floatHist := floatHistogram[int(ts)] expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist}) - _, err = app.AppendHistogram(0, lblsFloatHist, ts, nil, floatHist) + _, err = app.Append(0, lblsFloatHist, 0, ts, 0, nil, floatHist, storage.AOptions{}) require.NoError(t, err) - // Add an exemplar and to create multiple WAL records. + // Create multiple WAL records (commit). if ts%10 == 0 { - addExemplar(app, ref, lbls, ts) require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) } } } @@ -4712,7 +2335,7 @@ func TestChunkSnapshot(t *testing.T) { } } -func TestSnapshotError(t *testing.T) { +func TestSnapshotError_AppenderV2(t *testing.T) { head, _ := newTestHead(t, 120*4, compression.None, false) defer func() { head.opts.EnableMemorySnapshotOnShutdown = false @@ -4720,9 +2343,9 @@ func TestSnapshotError(t *testing.T) { }() // Add a sample. - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) lbls := labels.FromStrings("foo", "bar") - _, err := app.Append(0, lbls, 99, 99) + _, err := app.Append(0, lbls, 0, 99, 99, nil, nil, storage.AOptions{}) require.NoError(t, err) // Add histograms @@ -4731,10 +2354,10 @@ func TestSnapshotError(t *testing.T) { lblsHist := labels.FromStrings("hist", "bar") lblsFloatHist := labels.FromStrings("floathist", "bar") - _, err = app.AppendHistogram(0, lblsHist, 99, hist, nil) + _, err = app.Append(0, lblsHist, 0, 99, 0, hist, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.AppendHistogram(0, lblsFloatHist, 99, nil, floatHist) + _, err = app.Append(0, lblsFloatHist, 0, 99, 0, nil, floatHist, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -4822,7 +2445,7 @@ func TestSnapshotError(t *testing.T) { require.Equal(t, 2.0, prom_testutil.ToFloat64(head.metrics.seriesCreated)) } -func TestHistogramMetrics(t *testing.T) { +func TestHeadAppenderV2_Append_HistogramSamplesAppendedMetric(t *testing.T) { numHistograms := 10 head, _ := newTestHead(t, 1000, compression.None, false) t.Cleanup(func() { @@ -4836,15 +2459,15 @@ func TestHistogramMetrics(t *testing.T) { expHSeries++ l := labels.FromStrings("a", fmt.Sprintf("b%d", x)) for i, h := range tsdbutil.GenerateTestHistograms(numHistograms) { - app := head.Appender(context.Background()) - _, err := app.AppendHistogram(0, l, int64(i), h, nil) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, l, 0, int64(i), 0, h, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) expHSamples++ } for i, fh := range tsdbutil.GenerateTestFloatHistograms(numHistograms) { - app := head.Appender(context.Background()) - _, err := app.AppendHistogram(0, l, int64(numHistograms+i), nil, fh) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, l, 0, int64(numHistograms+i), 0, nil, fh, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) expHSamples++ @@ -4863,16 +2486,16 @@ func TestHistogramMetrics(t *testing.T) { require.Equal(t, float64(0), prom_testutil.ToFloat64(head.metrics.samplesAppended.WithLabelValues(sampleMetricTypeHistogram))) // Counter reset. } -func TestHistogramStaleSample(t *testing.T) { +func TestHeadAppenderV2_Append_StaleHistogram(t *testing.T) { t.Run("integer histogram", func(t *testing.T) { - testHistogramStaleSampleHelper(t, false) + testHeadAppenderV2AppendStaleHistogram(t, false) }) t.Run("float histogram", func(t *testing.T) { - testHistogramStaleSampleHelper(t, true) + testHeadAppenderV2AppendStaleHistogram(t, true) }) } -func testHistogramStaleSampleHelper(t *testing.T, floatHistogram bool) { +func testHeadAppenderV2AppendStaleHistogram(t *testing.T, floatHistogram bool) { t.Helper() l := labels.FromStrings("a", "b") numHistograms := 20 @@ -4958,20 +2581,20 @@ func testHistogramStaleSampleHelper(t *testing.T, floatHistogram bool) { } // Adding stale in the same appender. - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) for _, h := range tsdbutil.GenerateTestHistograms(numHistograms) { var err error if floatHistogram { - _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), nil, h.ToFloat(nil)) + _, err = app.Append(0, l, 0, 100*int64(len(expHistograms)), 0, nil, h.ToFloat(nil), storage.AOptions{}) expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), fh: h.ToFloat(nil)}) } else { - _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), h, nil) + _, err = app.Append(0, l, 0, 100*int64(len(expHistograms)), 0, h, nil, storage.AOptions{}) expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), h: h}) } require.NoError(t, err) } // +1 so that delta-of-delta is not 0. - _, err := app.Append(0, l, 100*int64(len(expHistograms))+1, math.Float64frombits(value.StaleNaN)) + _, err := app.Append(0, l, 0, 100*int64(len(expHistograms))+1, math.Float64frombits(value.StaleNaN), nil, nil, storage.AOptions{}) require.NoError(t, err) if floatHistogram { expHistograms = append(expHistograms, timedHistogram{t: 100*int64(len(expHistograms)) + 1, fh: &histogram.FloatHistogram{Sum: math.Float64frombits(value.StaleNaN)}}) @@ -4989,23 +2612,23 @@ func testHistogramStaleSampleHelper(t *testing.T, floatHistogram bool) { testQuery(1) // Adding stale in different appender and continuing series after a stale sample. - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) for _, h := range tsdbutil.GenerateTestHistograms(2 * numHistograms)[numHistograms:] { var err error if floatHistogram { - _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), nil, h.ToFloat(nil)) + _, err = app.Append(0, l, 0, 100*int64(len(expHistograms)), 0, nil, h.ToFloat(nil), storage.AOptions{}) expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), fh: h.ToFloat(nil)}) } else { - _, err = app.AppendHistogram(0, l, 100*int64(len(expHistograms)), h, nil) + _, err = app.Append(0, l, 0, 100*int64(len(expHistograms)), 0, h, nil, storage.AOptions{}) expHistograms = append(expHistograms, timedHistogram{t: 100 * int64(len(expHistograms)), h: h}) } require.NoError(t, err) } require.NoError(t, app.Commit()) - app = head.Appender(context.Background()) + app = head.AppenderV2(context.Background()) // +1 so that delta-of-delta is not 0. - _, err = app.Append(0, l, 100*int64(len(expHistograms))+1, math.Float64frombits(value.StaleNaN)) + _, err = app.Append(0, l, 0, 100*int64(len(expHistograms))+1, math.Float64frombits(value.StaleNaN), nil, nil, storage.AOptions{}) require.NoError(t, err) if floatHistogram { expHistograms = append(expHistograms, timedHistogram{t: 100*int64(len(expHistograms)) + 1, fh: &histogram.FloatHistogram{Sum: math.Float64frombits(value.StaleNaN)}}) @@ -5024,7 +2647,7 @@ func testHistogramStaleSampleHelper(t *testing.T, floatHistogram bool) { testQuery(2) } -func TestHistogramCounterResetHeader(t *testing.T) { +func TestHeadAppenderV2_Append_CounterResetHeader(t *testing.T) { for _, floatHisto := range []bool{true} { // FIXME t.Run(fmt.Sprintf("floatHistogram=%t", floatHisto), func(t *testing.T) { l := labels.FromStrings("a", "b") @@ -5037,12 +2660,12 @@ func TestHistogramCounterResetHeader(t *testing.T) { ts := int64(0) appendHistogram := func(h *histogram.Histogram) { ts++ - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) var err error if floatHisto { - _, err = app.AppendHistogram(0, l, ts, nil, h.ToFloat(nil)) + _, err = app.Append(0, l, 0, ts, 0, nil, h.ToFloat(nil), storage.AOptions{}) } else { - _, err = app.AppendHistogram(0, l, ts, h.Copy(), nil) + _, err = app.Append(0, l, 0, ts, 0, h.Copy(), nil, storage.AOptions{}) } require.NoError(t, err) require.NoError(t, app.Commit()) @@ -5145,7 +2768,7 @@ func TestHistogramCounterResetHeader(t *testing.T) { } } -func TestOOOHistogramCounterResetHeaders(t *testing.T) { +func TestHeadAppenderV2_Append_OOOHistogramCounterResetHeaders(t *testing.T) { for _, floatHisto := range []bool{true, false} { t.Run(fmt.Sprintf("floatHistogram=%t", floatHisto), func(t *testing.T) { l := labels.FromStrings("a", "b") @@ -5158,12 +2781,12 @@ func TestOOOHistogramCounterResetHeaders(t *testing.T) { require.NoError(t, head.Init(0)) appendHistogram := func(ts int64, h *histogram.Histogram) { - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) var err error if floatHisto { - _, err = app.AppendHistogram(0, l, ts, nil, h.ToFloat(nil)) + _, err = app.Append(0, l, 0, ts, 0, nil, h.ToFloat(nil), storage.AOptions{}) } else { - _, err = app.AppendHistogram(0, l, ts, h.Copy(), nil) + _, err = app.Append(0, l, 0, ts, 0, h.Copy(), nil, storage.AOptions{}) } require.NoError(t, err) require.NoError(t, app.Commit()) @@ -5305,7 +2928,7 @@ func TestOOOHistogramCounterResetHeaders(t *testing.T) { } } -func TestAppendingDifferentEncodingToSameSeries(t *testing.T) { +func TestHeadAppenderV2_Append_DifferentEncodingSameSeries(t *testing.T) { dir := t.TempDir() opts := DefaultOptions() db, err := Open(dir, nil, nil, opts, nil) @@ -5398,13 +3021,13 @@ func TestAppendingDifferentEncodingToSameSeries(t *testing.T) { } for _, a := range appends { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for _, s := range a.samples { var err error if s.H() != nil || s.FH() != nil { - _, err = app.AppendHistogram(0, lbls, s.T(), s.H(), s.FH()) + _, err = app.Append(0, lbls, 0, s.T(), 0, s.H(), s.FH(), storage.AOptions{}) } else { - _, err = app.Append(0, lbls, s.T(), s.F()) + _, err = app.Append(0, lbls, 0, s.T(), s.F(), nil, nil, storage.AOptions{}) } require.Equal(t, a.err, err) } @@ -5434,75 +3057,7 @@ func TestAppendingDifferentEncodingToSameSeries(t *testing.T) { require.Equal(t, map[string][]chunks.Sample{lbls.String(): expResult}, series) } -// Tests https://github.com/prometheus/prometheus/issues/9725. -func TestChunkSnapshotReplayBug(t *testing.T) { - dir := t.TempDir() - wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) - require.NoError(t, err) - - // Write few series records and samples such that the series references are not in order in the WAL - // for status_code="200". - var buf []byte - for i := 1; i <= 1000; i++ { - var ref chunks.HeadSeriesRef - if i <= 500 { - ref = chunks.HeadSeriesRef(i * 100) - } else { - ref = chunks.HeadSeriesRef((i - 500) * 50) - } - seriesRec := record.RefSeries{ - Ref: ref, - Labels: labels.FromStrings( - "__name__", "request_duration", - "status_code", "200", - "foo", fmt.Sprintf("baz%d", rand.Int()), - ), - } - // Add a sample so that the series is not garbage collected. - samplesRec := record.RefSample{Ref: ref, T: 1000, V: 1000} - var enc record.Encoder - - rec := enc.Series([]record.RefSeries{seriesRec}, buf) - buf = rec[:0] - require.NoError(t, wal.Log(rec)) - rec = enc.Samples([]record.RefSample{samplesRec}, buf) - buf = rec[:0] - require.NoError(t, wal.Log(rec)) - } - - // Write a corrupt snapshot to fail the replay on startup. - snapshotName := chunkSnapshotDir(0, 100) - cpdir := filepath.Join(dir, snapshotName) - require.NoError(t, os.MkdirAll(cpdir, 0o777)) - - err = os.WriteFile(filepath.Join(cpdir, "00000000"), []byte{1, 5, 3, 5, 6, 7, 4, 2, 2}, 0o777) - require.NoError(t, err) - - opts := DefaultHeadOptions() - opts.ChunkDirRoot = dir - opts.EnableMemorySnapshotOnShutdown = true - head, err := NewHead(nil, nil, wal, nil, opts, nil) - require.NoError(t, err) - require.NoError(t, head.Init(math.MinInt64)) - defer func() { - require.NoError(t, head.Close()) - }() - - // Snapshot replay should error out. - require.Equal(t, 1.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) - - // Querying `request_duration{status_code!="200"}` should return no series since all of - // them have status_code="200". - q, err := NewBlockQuerier(head, math.MinInt64, math.MaxInt64) - require.NoError(t, err) - series := query(t, q, - labels.MustNewMatcher(labels.MatchEqual, "__name__", "request_duration"), - labels.MustNewMatcher(labels.MatchNotEqual, "status_code", "200"), - ) - require.Empty(t, series, "there should be no series found") -} - -func TestChunkSnapshotTakenAfterIncompleteSnapshot(t *testing.T) { +func TestChunkSnapshotTakenAfterIncompleteSnapshot_AppenderV2(t *testing.T) { dir := t.TempDir() wlTemp, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) require.NoError(t, err) @@ -5522,8 +3077,8 @@ func TestChunkSnapshotTakenAfterIncompleteSnapshot(t *testing.T) { require.Equal(t, 0.0, prom_testutil.ToFloat64(head.metrics.snapshotReplayErrorTotal)) // Add some samples for the snapshot. - app := head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 10, 10) + app := head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 10, 10, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -5539,15 +3094,15 @@ func TestChunkSnapshotTakenAfterIncompleteSnapshot(t *testing.T) { } // TestWBLReplay checks the replay at a low level. -func TestWBLReplay(t *testing.T) { +func TestWBLReplay_AppenderV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testWBLReplay(t, scenario) + testWBLReplayAppenderV2(t, scenario) }) } } -func testWBLReplay(t *testing.T, scenario sampleTypeScenario) { +func testWBLReplayAppenderV2(t *testing.T, scenario sampleTypeScenario) { dir := t.TempDir() wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) require.NoError(t, err) @@ -5566,8 +3121,8 @@ func testWBLReplay(t *testing.T, scenario sampleTypeScenario) { var expOOOSamples []chunks.Sample l := labels.FromStrings("foo", "bar") appendSample := func(mins int64, _ float64, isOOO bool) { - app := h.Appender(context.Background()) - _, s, err := scenario.appendFunc(app, l, mins*time.Minute.Milliseconds(), mins) + app := h.AppenderV2(context.Background()) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), l, mins*time.Minute.Milliseconds(), mins) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -5631,15 +3186,15 @@ func testWBLReplay(t *testing.T, scenario sampleTypeScenario) { } // TestOOOMmapReplay checks the replay at a low level. -func TestOOOMmapReplay(t *testing.T) { +func TestOOOMmapReplay_AppenderV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOMmapReplay(t, scenario) + testOOOMmapReplayAppenderV2(t, scenario) }) } } -func testOOOMmapReplay(t *testing.T, scenario sampleTypeScenario) { +func testOOOMmapReplayAppenderV2(t *testing.T, scenario sampleTypeScenario) { dir := t.TempDir() wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) require.NoError(t, err) @@ -5658,8 +3213,8 @@ func testOOOMmapReplay(t *testing.T, scenario sampleTypeScenario) { l := labels.FromStrings("foo", "bar") appendSample := func(mins int64) { - app := h.Appender(context.Background()) - _, _, err := scenario.appendFunc(app, l, mins*time.Minute.Milliseconds(), mins) + app := h.AppenderV2(context.Background()) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), l, mins*time.Minute.Milliseconds(), mins) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -5721,7 +3276,7 @@ func testOOOMmapReplay(t *testing.T, scenario sampleTypeScenario) { require.NoError(t, h.Close()) } -func TestHeadInit_DiscardChunksWithUnsupportedEncoding(t *testing.T) { +func TestHead_Init_DiscardChunksWithUnsupportedEncoding(t *testing.T) { h, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, h.Close()) @@ -5730,12 +3285,12 @@ func TestHeadInit_DiscardChunksWithUnsupportedEncoding(t *testing.T) { require.NoError(t, h.Init(0)) ctx := context.Background() - app := h.Appender(ctx) + app := h.AppenderV2(ctx) seriesLabels := labels.FromStrings("a", "1") var seriesRef storage.SeriesRef var err error for i := range 400 { - seriesRef, err = app.Append(0, seriesLabels, int64(i), float64(i)) + seriesRef, err = app.Append(0, seriesLabels, 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) } @@ -5746,9 +3301,9 @@ func TestHeadInit_DiscardChunksWithUnsupportedEncoding(t *testing.T) { // Make this chunk not overlap with the previous and the next h.chunkDiskMapper.WriteChunk(chunks.HeadSeriesRef(seriesRef), 500, 600, uc, false, func(err error) { require.NoError(t, err) }) - app = h.Appender(ctx) + app = h.AppenderV2(ctx) for i := 700; i < 1200; i++ { - _, err := app.Append(0, seriesLabels, int64(i), float64(i)) + _, err := app.Append(0, seriesLabels, 0, int64(i), float64(i), nil, nil, storage.AOptions{}) require.NoError(t, err) } @@ -5780,26 +3335,8 @@ func TestHeadInit_DiscardChunksWithUnsupportedEncoding(t *testing.T) { require.Equal(t, expChunks, series.mmappedChunks) } -const ( - UnsupportedMask = 0b10000000 - EncUnsupportedXOR = chunkenc.EncXOR | UnsupportedMask -) - -// unsupportedChunk holds a XORChunk and overrides the Encoding() method. -type unsupportedChunk struct { - *chunkenc.XORChunk -} - -func newUnsupportedChunk() *unsupportedChunk { - return &unsupportedChunk{chunkenc.NewXORChunk()} -} - -func (*unsupportedChunk) Encoding() chunkenc.Encoding { - return EncUnsupportedXOR -} - // Tests https://github.com/prometheus/prometheus/issues/10277. -func TestMmapPanicAfterMmapReplayCorruption(t *testing.T) { +func TestMmapPanicAfterMmapReplayCorruption_AppenderV2(t *testing.T) { dir := t.TempDir() wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.None) require.NoError(t, err) @@ -5819,13 +3356,13 @@ func TestMmapPanicAfterMmapReplayCorruption(t *testing.T) { lbls := labels.FromStrings("__name__", "testing", "foo", "bar") addChunks := func() { interval := DefaultBlockDuration / (4 * 120) - app := h.Appender(context.Background()) + app := h.AppenderV2(context.Background()) for i := range 250 { - ref, err = app.Append(ref, lbls, lastTs, float64(lastTs)) + ref, err = app.Append(ref, lbls, 0, lastTs, float64(lastTs), nil, nil, storage.AOptions{}) lastTs += interval if i%10 == 0 { require.NoError(t, app.Commit()) - app = h.Appender(context.Background()) + app = h.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) @@ -5854,7 +3391,7 @@ func TestMmapPanicAfterMmapReplayCorruption(t *testing.T) { } // Tests https://github.com/prometheus/prometheus/issues/10277. -func TestReplayAfterMmapReplayError(t *testing.T) { +func TestReplayAfterMmapReplayError_AppenderV2(t *testing.T) { dir := t.TempDir() var h *Head var err error @@ -5881,16 +3418,16 @@ func TestReplayAfterMmapReplayError(t *testing.T) { lbls := labels.FromStrings("__name__", "testing", "foo", "bar") var expSamples []chunks.Sample addSamples := func(numSamples int) { - app := h.Appender(context.Background()) + app := h.AppenderV2(context.Background()) var ref storage.SeriesRef for i := range numSamples { - ref, err = app.Append(ref, lbls, lastTs, float64(lastTs)) + ref, err = app.Append(ref, lbls, 0, lastTs, float64(lastTs), nil, nil, storage.AOptions{}) expSamples = append(expSamples, sample{t: lastTs, f: float64(lastTs)}) require.NoError(t, err) lastTs += itvl if i%10 == 0 { require.NoError(t, app.Commit()) - app = h.Appender(context.Background()) + app = h.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) @@ -5933,15 +3470,15 @@ func TestReplayAfterMmapReplayError(t *testing.T) { require.NoError(t, h.Close()) } -func TestOOOAppendWithNoSeries(t *testing.T) { +func TestHeadAppenderV2_Append_OOOWithNoSeries(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOAppendWithNoSeries(t, scenario.appendFunc) + testHeadAppenderV2AppendOOOWithNoSeries(t, scenario.appendFunc) }) } } -func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error)) { +func testHeadAppenderV2AppendOOOWithNoSeries(t *testing.T, appendFunc func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error)) { dir := t.TempDir() wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) require.NoError(t, err) @@ -5961,8 +3498,8 @@ func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.Ap require.NoError(t, h.Init(0)) appendSample := func(lbls labels.Labels, ts int64) { - app := h.Appender(context.Background()) - _, _, err := appendFunc(app, lbls, ts*time.Minute.Milliseconds(), ts) + app := h.AppenderV2(context.Background()) + _, _, err := appendFunc(storage.AppenderV2AsLimitedV1(app), lbls, ts*time.Minute.Milliseconds(), ts) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -6009,8 +3546,8 @@ func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.Ap // Now 179m is too old. s4 := newLabels(4) - app := h.Appender(context.Background()) - _, _, err = appendFunc(app, s4, 179*time.Minute.Milliseconds(), 179) + app := h.AppenderV2(context.Background()) + _, _, err = appendFunc(storage.AppenderV2AsLimitedV1(app), s4, 179*time.Minute.Milliseconds(), 179) require.Equal(t, storage.ErrTooOldSample, err) require.NoError(t, app.Rollback()) verifyOOOSamples(s3, 1) @@ -6022,17 +3559,17 @@ func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.Ap verifyInOrderSamples(s5, 1) } -func TestHeadMinOOOTimeUpdate(t *testing.T) { +func TestHead_MinOOOTime_Update_AppenderV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { if scenario.sampleType == sampleMetricTypeFloat { - testHeadMinOOOTimeUpdate(t, scenario) + testHeadMinOOOTimeUpdateAppenderV2(t, scenario) } }) } } -func testHeadMinOOOTimeUpdate(t *testing.T, scenario sampleTypeScenario) { +func testHeadMinOOOTimeUpdateAppenderV2(t *testing.T, scenario sampleTypeScenario) { dir := t.TempDir() wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) require.NoError(t, err) @@ -6051,8 +3588,8 @@ func testHeadMinOOOTimeUpdate(t *testing.T, scenario sampleTypeScenario) { require.NoError(t, h.Init(0)) appendSample := func(ts int64) { - app := h.Appender(context.Background()) - _, _, err = scenario.appendFunc(app, labels.FromStrings("a", "b"), ts*time.Minute.Milliseconds(), ts) + app := h.AppenderV2(context.Background()) + _, _, err = scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), labels.FromStrings("a", "b"), ts*time.Minute.Milliseconds(), ts) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -6077,7 +3614,7 @@ func testHeadMinOOOTimeUpdate(t *testing.T, scenario sampleTypeScenario) { require.Equal(t, 295*time.Minute.Milliseconds(), h.MinOOOTime()) } -func TestGaugeHistogramWALAndChunkHeader(t *testing.T) { +func TestGaugeHistogramWALAndChunkHeader_AppenderV2(t *testing.T) { l := labels.FromStrings("a", "b") head, _ := newTestHead(t, 1000, compression.None, false) t.Cleanup(func() { @@ -6088,8 +3625,8 @@ func TestGaugeHistogramWALAndChunkHeader(t *testing.T) { ts := int64(0) appendHistogram := func(h *histogram.Histogram) { ts++ - app := head.Appender(context.Background()) - _, err := app.AppendHistogram(0, l, ts, h.Copy(), nil) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, l, 0, ts, 0, h.Copy(), nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -6153,7 +3690,7 @@ func TestGaugeHistogramWALAndChunkHeader(t *testing.T) { checkHeaders() } -func TestGaugeFloatHistogramWALAndChunkHeader(t *testing.T) { +func TestGaugeFloatHistogramWALAndChunkHeader_AppenderV2(t *testing.T) { l := labels.FromStrings("a", "b") head, _ := newTestHead(t, 1000, compression.None, false) t.Cleanup(func() { @@ -6164,8 +3701,8 @@ func TestGaugeFloatHistogramWALAndChunkHeader(t *testing.T) { ts := int64(0) appendHistogram := func(h *histogram.FloatHistogram) { ts++ - app := head.Appender(context.Background()) - _, err := app.AppendHistogram(0, l, ts, nil, h.Copy()) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, l, 0, ts, 0, nil, h.Copy(), storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -6229,12 +3766,12 @@ func TestGaugeFloatHistogramWALAndChunkHeader(t *testing.T) { checkHeaders() } -func TestSnapshotAheadOfWALError(t *testing.T) { +func TestSnapshotAheadOfWALError_AppenderV2(t *testing.T) { head, _ := newTestHead(t, 120*4, compression.None, false) head.opts.EnableMemorySnapshotOnShutdown = true // Add a sample to fill WAL. - app := head.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("foo", "bar"), 10, 10) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 10, 10, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -6257,8 +3794,8 @@ func TestSnapshotAheadOfWALError(t *testing.T) { head, err = NewHead(nil, nil, w, nil, head.opts, nil) require.NoError(t, err) // Add a sample to fill WAL. - app = head.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 10, 10) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 10, 10, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) lastSegment, _, _ := w.LastSegmentAndOffset() @@ -6284,31 +3821,7 @@ func TestSnapshotAheadOfWALError(t *testing.T) { require.NoError(t, head.Close()) } -func BenchmarkCuttingHeadHistogramChunks(b *testing.B) { - const ( - numSamples = 50000 - numBuckets = 100 - ) - samples := histogram.GenerateBigTestHistograms(numSamples, numBuckets) - - h, _ := newTestHead(b, DefaultBlockDuration, compression.None, false) - defer func() { - require.NoError(b, h.Close()) - }() - - a := h.Appender(context.Background()) - ts := time.Now().UnixMilli() - lbls := labels.FromStrings("foo", "bar") - - b.ResetTimer() - - for _, s := range samples { - _, err := a.AppendHistogram(0, lbls, ts, s, nil) - require.NoError(b, err) - } -} - -func TestCuttingNewHeadChunks(t *testing.T) { +func TestCuttingNewHeadChunks_AppenderV2(t *testing.T) { ctx := context.Background() testCases := map[string]struct { numTotalSamples int @@ -6413,7 +3926,7 @@ func TestCuttingNewHeadChunks(t *testing.T) { require.NoError(t, h.Close()) }() - a := h.Appender(context.Background()) + a := h.AppenderV2(context.Background()) ts := int64(10000) lbls := labels.FromStrings("foo", "bar") @@ -6421,10 +3934,10 @@ func TestCuttingNewHeadChunks(t *testing.T) { for i := 0; i < tc.numTotalSamples; i++ { if tc.floatValFunc != nil { - _, err := a.Append(0, lbls, ts, tc.floatValFunc(i)) + _, err := a.Append(0, lbls, 0, ts, tc.floatValFunc(i), nil, nil, storage.AOptions{}) require.NoError(t, err) } else if tc.histValFunc != nil { - _, err := a.AppendHistogram(0, lbls, ts, tc.histValFunc(i), nil) + _, err := a.Append(0, lbls, 0, ts, 0, tc.histValFunc(i), nil, storage.AOptions{}) require.NoError(t, err) } @@ -6472,7 +3985,7 @@ func TestCuttingNewHeadChunks(t *testing.T) { // is appended to the head, right when the head chunk is at the size limit. // The test adds all samples as duplicate, thus expecting that the result has // exactly half of the samples. -func TestHeadDetectsDuplicateSampleAtSizeLimit(t *testing.T) { +func TestHeadDetectsDuplicateSampleAtSizeLimit_AppenderV2(t *testing.T) { numSamples := 1000 baseTS := int64(1695209650) @@ -6481,15 +3994,15 @@ func TestHeadDetectsDuplicateSampleAtSizeLimit(t *testing.T) { require.NoError(t, h.Close()) }() - a := h.Appender(context.Background()) + a := h.AppenderV2(context.Background()) var err error vals := []float64{math.MaxFloat64, 0x00} // Use the worst case scenario for the XOR encoding. Otherwise we hit the sample limit before the size limit. for i := range numSamples { ts := baseTS + int64(i/2)*10000 - a.Append(0, labels.FromStrings("foo", "bar"), ts, vals[(i/2)%len(vals)]) + a.Append(0, labels.FromStrings("foo", "bar"), 0, ts, vals[(i/2)%len(vals)], nil, nil, storage.AOptions{}) err = a.Commit() require.NoError(t, err) - a = h.Appender(context.Background()) + a = h.AppenderV2(context.Background()) } indexReader, err := h.Index() @@ -6515,29 +4028,29 @@ func TestHeadDetectsDuplicateSampleAtSizeLimit(t *testing.T) { require.Equal(t, numSamples/2, storedSampleCount) } -func TestWALSampleAndExemplarOrder(t *testing.T) { +func TestWALSampleAndExemplarOrder_AppenderV2(t *testing.T) { lbls := labels.FromStrings("foo", "bar") testcases := map[string]struct { - appendF func(app storage.Appender, ts int64) (storage.SeriesRef, error) + appendF func(app storage.AppenderV2, ts int64) (storage.SeriesRef, error) expectedType reflect.Type }{ "float sample": { - appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { - return app.Append(0, lbls, ts, 1.0) + appendF: func(app storage.AppenderV2, ts int64) (storage.SeriesRef, error) { + return app.Append(0, lbls, 0, ts, 1.0, nil, nil, storage.AOptions{Exemplars: []exemplar.Exemplar{{Value: 1.0, Ts: 5}}}) }, - expectedType: reflect.TypeOf([]record.RefSample{}), + expectedType: reflect.TypeFor[[]record.RefSample](), }, "histogram sample": { - appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { - return app.AppendHistogram(0, lbls, ts, tsdbutil.GenerateTestHistogram(1), nil) + appendF: func(app storage.AppenderV2, ts int64) (storage.SeriesRef, error) { + return app.Append(0, lbls, 0, ts, 0, tsdbutil.GenerateTestHistogram(1), nil, storage.AOptions{Exemplars: []exemplar.Exemplar{{Value: 1.0, Ts: 5}}}) }, - expectedType: reflect.TypeOf([]record.RefHistogramSample{}), + expectedType: reflect.TypeFor[[]record.RefHistogramSample](), }, "float histogram sample": { - appendF: func(app storage.Appender, ts int64) (storage.SeriesRef, error) { - return app.AppendHistogram(0, lbls, ts, nil, tsdbutil.GenerateTestFloatHistogram(1)) + appendF: func(app storage.AppenderV2, ts int64) (storage.SeriesRef, error) { + return app.Append(0, lbls, 0, ts, 0, nil, tsdbutil.GenerateTestFloatHistogram(1), storage.AOptions{Exemplars: []exemplar.Exemplar{{Value: 1.0, Ts: 5}}}) }, - expectedType: reflect.TypeOf([]record.RefFloatHistogramSample{}), + expectedType: reflect.TypeFor[[]record.RefFloatHistogramSample](), }, } @@ -6548,12 +4061,11 @@ func TestWALSampleAndExemplarOrder(t *testing.T) { require.NoError(t, h.Close()) }() - app := h.Appender(context.Background()) - ref, err := tc.appendF(app, 10) + app := h.AppenderV2(context.Background()) + _, err := tc.appendF(app, 10) require.NoError(t, err) - app.AppendExemplar(ref, lbls, exemplar.Exemplar{Value: 1.0, Ts: 5}) - app.Commit() + require.NoError(t, app.Commit()) recs := readTestWAL(t, w.Dir()) require.Len(t, recs, 3) @@ -6567,126 +4079,7 @@ func TestWALSampleAndExemplarOrder(t *testing.T) { } } -// TestHeadCompactionWhileAppendAndCommitExemplar simulates a use case where -// a series is removed from the head while an exemplar is being appended to it. -// This can happen in theory by compacting the head at the right time due to -// a series being idle. -// The test cheats a little bit by not appending a sample with the exemplar. -// If you also add a sample and run Truncate in a concurrent goroutine and run -// the test around a million(!) times, you can get -// `unknown HeadSeriesRef when trying to add exemplar: 1` error on push. -// It is likely that running the test for much longer and with more time variations -// would trigger the -// `signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xbb03d1` -// panic, that we have seen in the wild once. -func TestHeadCompactionWhileAppendAndCommitExemplar(t *testing.T) { - h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) - app := h.Appender(context.Background()) - lbls := labels.FromStrings("foo", "bar") - ref, err := app.Append(0, lbls, 1, 1) - require.NoError(t, err) - app.Commit() - // Not adding a sample here to trigger the fault. - app = h.Appender(context.Background()) - _, err = app.AppendExemplar(ref, lbls, exemplar.Exemplar{Value: 1, Ts: 20}) - require.NoError(t, err) - h.Truncate(10) - app.Commit() - h.Close() -} - -func labelsWithHashCollision() (labels.Labels, labels.Labels) { - // These two series have the same XXHash; thanks to https://github.com/pstibrany/labels_hash_collisions - ls1 := labels.FromStrings("__name__", "metric", "lbl", "HFnEaGl") - ls2 := labels.FromStrings("__name__", "metric", "lbl", "RqcXatm") - - if ls1.Hash() != ls2.Hash() { - // These ones are the same when using -tags slicelabels - ls1 = labels.FromStrings("__name__", "metric", "lbl1", "value", "lbl2", "l6CQ5y") - ls2 = labels.FromStrings("__name__", "metric", "lbl1", "value", "lbl2", "v7uDlF") - } - - if ls1.Hash() != ls2.Hash() { - panic("This code needs to be updated: find new labels with colliding hash values.") - } - - return ls1, ls2 -} - -// stripeSeriesWithCollidingSeries returns a stripeSeries with two memSeries having the same, colliding, hash. -func stripeSeriesWithCollidingSeries(t *testing.T) (*stripeSeries, *memSeries, *memSeries) { - t.Helper() - - lbls1, lbls2 := labelsWithHashCollision() - ms1 := memSeries{ - lset: lbls1, - } - ms2 := memSeries{ - lset: lbls2, - } - hash := lbls1.Hash() - s := newStripeSeries(1, noopSeriesLifecycleCallback{}) - - got, created := s.setUnlessAlreadySet(hash, lbls1, &ms1) - require.True(t, created) - require.Same(t, &ms1, got) - - // Add a conflicting series - got, created = s.setUnlessAlreadySet(hash, lbls2, &ms2) - require.True(t, created) - require.Same(t, &ms2, got) - - return s, &ms1, &ms2 -} - -func TestStripeSeries_getOrSet(t *testing.T) { - s, ms1, ms2 := stripeSeriesWithCollidingSeries(t) - hash := ms1.lset.Hash() - - // Verify that we can get both of the series despite the hash collision - got := s.getByHash(hash, ms1.lset) - require.Same(t, ms1, got) - got = s.getByHash(hash, ms2.lset) - require.Same(t, ms2, got) -} - -func TestStripeSeries_gc(t *testing.T) { - s, ms1, ms2 := stripeSeriesWithCollidingSeries(t) - hash := ms1.lset.Hash() - - s.gc(0, 0, nil) - - // Verify that we can get neither ms1 nor ms2 after gc-ing corresponding series - got := s.getByHash(hash, ms1.lset) - require.Nil(t, got) - got = s.getByHash(hash, ms2.lset) - require.Nil(t, got) -} - -func TestPostingsCardinalityStats(t *testing.T) { - head := &Head{postings: index.NewMemPostings()} - head.postings.Add(1, labels.FromStrings(labels.MetricName, "t", "n", "v1")) - head.postings.Add(2, labels.FromStrings(labels.MetricName, "t", "n", "v2")) - - statsForMetricName := head.PostingsCardinalityStats(labels.MetricName, 10) - head.postings.Add(3, labels.FromStrings(labels.MetricName, "t", "n", "v3")) - // Using cache. - require.Equal(t, statsForMetricName, head.PostingsCardinalityStats(labels.MetricName, 10)) - - statsForSomeLabel := head.PostingsCardinalityStats("n", 10) - // Cache should be evicted because of the change of label name. - require.NotEqual(t, statsForMetricName, statsForSomeLabel) - head.postings.Add(4, labels.FromStrings(labels.MetricName, "t", "n", "v4")) - // Using cache. - require.Equal(t, statsForSomeLabel, head.PostingsCardinalityStats("n", 10)) - // Cache should be evicted because of the change of limit parameter. - statsForSomeLabel1 := head.PostingsCardinalityStats("n", 1) - require.NotEqual(t, statsForSomeLabel1, statsForSomeLabel) - // Using cache. - require.Equal(t, statsForSomeLabel1, head.PostingsCardinalityStats("n", 1)) -} - -func TestHeadAppender_AppendFloatWithSameTimestampAsPreviousHistogram(t *testing.T) { +func TestHeadAppenderV2_Append_FloatWithSameTimestampAsPreviousHistogram(t *testing.T) { head, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) t.Cleanup(func() { head.Close() }) @@ -6694,28 +4087,29 @@ func TestHeadAppender_AppendFloatWithSameTimestampAsPreviousHistogram(t *testing { // Append a float 10.0 @ 1_000 - app := head.Appender(context.Background()) - _, err := app.Append(0, ls, 1_000, 10.0) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, ls, 0, 1_000, 10.0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } { // Append a float histogram @ 2_000 - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) h := tsdbutil.GenerateTestHistogram(1) - _, err := app.AppendHistogram(0, ls, 2_000, h, nil) + _, err := app.Append(0, ls, 0, 2_000, 0, h, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } - app := head.Appender(context.Background()) - _, err := app.Append(0, ls, 2_000, 10.0) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, ls, 0, 2_000, 10.0, nil, nil, storage.AOptions{}) require.Error(t, err) require.ErrorIs(t, err, storage.NewDuplicateHistogramToFloatErr(2_000, 10.0)) } -func TestHeadAppender_AppendST(t *testing.T) { +func TestHeadAppenderV2_Append_EnableSTAsZeroSample(t *testing.T) { + // Make sure counter resets hints are non-zero, so we can detect ST histogram samples. testHistogram := tsdbutil.GenerateTestHistogram(1) testHistogram.CounterResetHint = histogram.NotCounterReset testFloatHistogram := tsdbutil.GenerateTestFloatHistogram(1) @@ -6909,30 +4303,121 @@ func TestHeadAppender_AppendST(t *testing.T) { } }(), }, + { + name: "ST lower than minValidTime/float", + appendableSamples: []appendableSamples{ + {ts: 100, fSample: 10, st: -1}, + }, + // ST results ErrOutOfBounds, but ST append is best effort, so + // ST should be ignored, but sample appended. + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 100, f: 10}, + } + }(), + }, + { + name: "ST lower than minValidTime/histogram", + appendableSamples: []appendableSamples{ + {ts: 100, h: testHistogram, st: -1}, + }, + // ST results ErrOutOfBounds, but ST append is best effort, so + // ST should be ignored, but sample appended. + expectedSamples: func() []chunks.Sample { + // NOTE: Without ST, on query, first histogram sample will get + // CounterReset adjusted to 0. + firstSample := testHistogram.Copy() + firstSample.CounterResetHint = histogram.UnknownCounterReset + return []chunks.Sample{ + sample{t: 100, h: firstSample}, + } + }(), + }, + { + name: "ST lower than minValidTime/floathistogram", + appendableSamples: []appendableSamples{ + {ts: 100, fh: testFloatHistogram, st: -1}, + }, + // ST results ErrOutOfBounds, but ST append is best effort, so + // ST should be ignored, but sample appended. + expectedSamples: func() []chunks.Sample { + // NOTE: Without ST, on query, first histogram sample will get + // CounterReset adjusted to 0. + firstSample := testFloatHistogram.Copy() + firstSample.CounterResetHint = histogram.UnknownCounterReset + return []chunks.Sample{ + sample{t: 100, fh: firstSample}, + } + }(), + }, + { + name: "ST duplicates an existing sample/float", + appendableSamples: []appendableSamples{ + {ts: 100, fSample: 10}, + {ts: 200, fSample: 10, st: 100}, + }, + // ST results ErrOutOfBounds, but ST append is best effort, so + // ST should be ignored, but sample appended. + expectedSamples: func() []chunks.Sample { + return []chunks.Sample{ + sample{t: 100, f: 10}, + sample{t: 200, f: 10}, + } + }(), + }, + { + name: "ST duplicates an existing sample/histogram", + appendableSamples: []appendableSamples{ + {ts: 100, h: testHistogram}, + {ts: 200, h: testHistogram, st: 100}, + }, + // ST results ErrDuplicateSampleForTimestamp, but ST append is best effort, so + // ST should be ignored, but sample appended. + expectedSamples: func() []chunks.Sample { + // NOTE: Without ST, on query, first histogram sample will get + // CounterReset adjusted to 0. + firstSample := testHistogram.Copy() + firstSample.CounterResetHint = histogram.UnknownCounterReset + return []chunks.Sample{ + sample{t: 100, h: firstSample}, + sample{t: 200, h: testHistogram}, + } + }(), + }, + { + name: "ST duplicates an existing sample/floathistogram", + appendableSamples: []appendableSamples{ + {ts: 100, fh: testFloatHistogram}, + {ts: 200, fh: testFloatHistogram, st: 100}, + }, + // ST results ErrDuplicateSampleForTimestamp, but ST append is best effort, so + // ST should ignored, but sample appended. + expectedSamples: func() []chunks.Sample { + // NOTE: Without ST, on query, first histogram sample will get + // CounterReset adjusted to 0. + firstSample := testFloatHistogram.Copy() + firstSample.CounterResetHint = histogram.UnknownCounterReset + return []chunks.Sample{ + sample{t: 100, fh: firstSample}, + sample{t: 200, fh: testFloatHistogram}, + } + }(), + }, } { t.Run(tc.name, func(t *testing.T) { - h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) + opts := newTestHeadDefaultOptions(DefaultBlockDuration, false) + opts.EnableSTAsZeroSample = true + h, _ := newTestHeadWithOptions(t, compression.None, opts) defer func() { require.NoError(t, h.Close()) }() - a := h.Appender(context.Background()) - lbls := labels.FromStrings("foo", "bar") - for _, sample := range tc.appendableSamples { - // Append float if it's a float test case - if sample.fSample != 0 { - _, err := a.AppendSTZeroSample(0, lbls, sample.ts, sample.st) - require.NoError(t, err) - _, err = a.Append(0, lbls, sample.ts, sample.fSample) - require.NoError(t, err) - } - // Append histograms if it's a histogram test case - if sample.h != nil || sample.fh != nil { - ref, err := a.AppendHistogramSTZeroSample(0, lbls, sample.ts, sample.st, sample.h, sample.fh) - require.NoError(t, err) - _, err = a.AppendHistogram(ref, lbls, sample.ts, sample.h, sample.fh) - require.NoError(t, err) - } + a := h.AppenderV2(context.Background()) + lbls := labels.FromStrings("foo", "bar") + + for _, s := range tc.appendableSamples { + _, err := a.Append(0, lbls, s.st, s.ts, s.fSample, s.h, s.fh, storage.AOptions{}) + require.NoError(t, err) } require.NoError(t, a.Commit()) @@ -6944,122 +4429,29 @@ func TestHeadAppender_AppendST(t *testing.T) { } } -func TestHeadAppender_AppendHistogramSTZeroSample(t *testing.T) { - type appendableSamples struct { - ts int64 - h *histogram.Histogram - fh *histogram.FloatHistogram - st int64 // 0 if no created timestamp. - } - for _, tc := range []struct { - name string - appendableSamples []appendableSamples - expectedError error - }{ - { - name: "integer histogram ST lower than minValidTime initiates ErrOutOfBounds", - appendableSamples: []appendableSamples{ - {ts: 100, h: tsdbutil.GenerateTestHistogram(1), st: -1}, - }, - expectedError: storage.ErrOutOfBounds, - }, - { - name: "float histograms ST lower than minValidTime initiates ErrOutOfBounds", - appendableSamples: []appendableSamples{ - {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1), st: -1}, - }, - expectedError: storage.ErrOutOfBounds, - }, - { - name: "integer histogram ST duplicates an existing sample", - appendableSamples: []appendableSamples{ - {ts: 100, h: tsdbutil.GenerateTestHistogram(1)}, - {ts: 200, h: tsdbutil.GenerateTestHistogram(1), st: 100}, - }, - expectedError: storage.ErrDuplicateSampleForTimestamp, - }, - { - name: "float histogram ST duplicates an existing sample", - appendableSamples: []appendableSamples{ - {ts: 100, fh: tsdbutil.GenerateTestFloatHistogram(1)}, - {ts: 200, fh: tsdbutil.GenerateTestFloatHistogram(1), st: 100}, - }, - expectedError: storage.ErrDuplicateSampleForTimestamp, - }, - } { - t.Run(tc.name, func(t *testing.T) { - h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) - - defer func() { - require.NoError(t, h.Close()) - }() - - lbls := labels.FromStrings("foo", "bar") - - var ref storage.SeriesRef - for _, sample := range tc.appendableSamples { - a := h.Appender(context.Background()) - var err error - if sample.st != 0 { - ref, err = a.AppendHistogramSTZeroSample(ref, lbls, sample.ts, sample.st, sample.h, sample.fh) - require.ErrorIs(t, err, tc.expectedError) - } - - ref, err = a.AppendHistogram(ref, lbls, sample.ts, sample.h, sample.fh) - require.NoError(t, err) - require.NoError(t, a.Commit()) - } - }) - } -} - -func TestHeadCompactableDoesNotCompactEmptyHead(t *testing.T) { - // Use a chunk range of 1 here so that if we attempted to determine if the head - // was compactable using default values for min and max times, `Head.compactable()` - // would return true which is incorrect. This test verifies that we short-circuit - // the check when the head has not yet had any samples added. - head, _ := newTestHead(t, 1, compression.None, false) - defer func() { - require.NoError(t, head.Close()) - }() - - require.False(t, head.compactable()) -} - -type countSeriesLifecycleCallback struct { - created atomic.Int64 - deleted atomic.Int64 -} - -func (*countSeriesLifecycleCallback) PreCreation(labels.Labels) error { return nil } -func (c *countSeriesLifecycleCallback) PostCreation(labels.Labels) { c.created.Inc() } -func (c *countSeriesLifecycleCallback) PostDeletion(s map[chunks.HeadSeriesRef]labels.Labels) { - c.deleted.Add(int64(len(s))) -} - // Regression test for data race https://github.com/prometheus/prometheus/issues/15139. -func TestHeadAppendHistogramAndCommitConcurrency(t *testing.T) { +func TestHeadAppenderV2_Append_HistogramAndCommitConcurrency(t *testing.T) { h := tsdbutil.GenerateTestHistogram(1) fh := tsdbutil.GenerateTestFloatHistogram(1) - testCases := map[string]func(storage.Appender, int) error{ - "integer histogram": func(app storage.Appender, i int) error { - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar", "serial", strconv.Itoa(i)), 1, h, nil) + testCases := map[string]func(storage.AppenderV2, int) error{ + "integer histogram": func(app storage.AppenderV2, i int) error { + _, err := app.Append(0, labels.FromStrings("foo", "bar", "serial", strconv.Itoa(i)), 0, 1, 0, h, nil, storage.AOptions{}) return err }, - "float histogram": func(app storage.Appender, i int) error { - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar", "serial", strconv.Itoa(i)), 1, nil, fh) + "float histogram": func(app storage.AppenderV2, i int) error { + _, err := app.Append(0, labels.FromStrings("foo", "bar", "serial", strconv.Itoa(i)), 0, 1, 0, nil, fh, storage.AOptions{}) return err }, } for name, tc := range testCases { t.Run(name, func(t *testing.T) { - testHeadAppendHistogramAndCommitConcurrency(t, tc) + testHeadAppenderV2AppendHistogramAndCommitConcurrency(t, tc) }) } } -func testHeadAppendHistogramAndCommitConcurrency(t *testing.T, appendFn func(storage.Appender, int) error) { +func testHeadAppenderV2AppendHistogramAndCommitConcurrency(t *testing.T, appendFn func(storage.AppenderV2, int) error) { head, _ := newTestHead(t, 1000, compression.None, false) defer func() { require.NoError(t, head.Close()) @@ -7076,7 +4468,7 @@ func testHeadAppendHistogramAndCommitConcurrency(t *testing.T, appendFn func(sto go func() { defer wg.Done() for i := range 10000 { - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) require.NoError(t, appendFn(app, i)) require.NoError(t, app.Commit()) } @@ -7085,7 +4477,7 @@ func testHeadAppendHistogramAndCommitConcurrency(t *testing.T, appendFn func(sto go func() { defer wg.Done() for i := range 10000 { - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) require.NoError(t, appendFn(app, i)) require.NoError(t, app.Commit()) } @@ -7094,7 +4486,7 @@ func testHeadAppendHistogramAndCommitConcurrency(t *testing.T, appendFn func(sto wg.Wait() } -func TestHead_NumStaleSeries(t *testing.T) { +func TestHeadAppenderV2_NumStaleSeries(t *testing.T) { head, _ := newTestHead(t, 1000, compression.None, false) t.Cleanup(func() { require.NoError(t, head.Close()) @@ -7105,20 +4497,20 @@ func TestHead_NumStaleSeries(t *testing.T) { require.Equal(t, uint64(0), head.NumStaleSeries()) appendSample := func(lbls labels.Labels, ts int64, val float64) { - app := head.Appender(context.Background()) - _, err := app.Append(0, lbls, ts, val) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, lbls, 0, ts, val, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } appendHistogram := func(lbls labels.Labels, ts int64, val *histogram.Histogram) { - app := head.Appender(context.Background()) - _, err := app.AppendHistogram(0, lbls, ts, val, nil) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, lbls, 0, ts, 0, val, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } appendFloatHistogram := func(lbls labels.Labels, ts int64, val *histogram.FloatHistogram) { - app := head.Appender(context.Background()) - _, err := app.AppendHistogram(0, lbls, ts, nil, val) + app := head.AppenderV2(context.Background()) + _, err := app.Append(0, lbls, 0, ts, 0, nil, val, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -7244,22 +4636,22 @@ func TestHead_NumStaleSeries(t *testing.T) { // TestHistogramStalenessConversionMetrics verifies that staleness marker conversion correctly // increments the right appender metrics for both histogram and float histogram scenarios. -func TestHistogramStalenessConversionMetrics(t *testing.T) { +func TestHeadAppenderV2_Append_HistogramStalenessConversionMetrics(t *testing.T) { testCases := []struct { name string - setupHistogram func(app storage.Appender, lbls labels.Labels) error + setupHistogram func(app storage.AppenderV2, lbls labels.Labels) error }{ { name: "float_staleness_to_histogram", - setupHistogram: func(app storage.Appender, lbls labels.Labels) error { - _, err := app.AppendHistogram(0, lbls, 1000, tsdbutil.GenerateTestHistograms(1)[0], nil) + setupHistogram: func(app storage.AppenderV2, lbls labels.Labels) error { + _, err := app.Append(0, lbls, 0, 1000, 0, tsdbutil.GenerateTestHistograms(1)[0], nil, storage.AOptions{}) return err }, }, { name: "float_staleness_to_float_histogram", - setupHistogram: func(app storage.Appender, lbls labels.Labels) error { - _, err := app.AppendHistogram(0, lbls, 1000, nil, tsdbutil.GenerateTestFloatHistograms(1)[0]) + setupHistogram: func(app storage.AppenderV2, lbls labels.Labels) error { + _, err := app.Append(0, lbls, 0, 1000, 0, nil, tsdbutil.GenerateTestFloatHistograms(1)[0], storage.AOptions{}) return err }, }, @@ -7283,14 +4675,14 @@ func TestHistogramStalenessConversionMetrics(t *testing.T) { } // Step 1: Establish a series with histogram data - app := head.Appender(context.Background()) + app := head.AppenderV2(context.Background()) err := tc.setupHistogram(app, lbls) require.NoError(t, err) require.NoError(t, app.Commit()) // Step 2: Add a float staleness marker - app = head.Appender(context.Background()) - _, err = app.Append(0, lbls, 2000, math.Float64frombits(value.StaleNaN)) + app = head.AppenderV2(context.Background()) + _, err = app.Append(0, lbls, 0, 2000, math.Float64frombits(value.StaleNaN), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) diff --git a/tsdb/head_bench_test.go b/tsdb/head_bench_test.go index c98fb6613d..a63b0ced50 100644 --- a/tsdb/head_bench_test.go +++ b/tsdb/head_bench_test.go @@ -14,7 +14,6 @@ package tsdb import ( - "context" "errors" "fmt" "math/rand" @@ -32,6 +31,228 @@ import ( "github.com/prometheus/prometheus/util/compression" ) +type benchAppendFunc func(b *testing.B, h *Head, ts int64, series []storage.Series, samplesPerAppend int64) storage.AppenderTransaction + +func appendV1Float(b *testing.B, h *Head, ts int64, series []storage.Series, samplesPerAppend int64) storage.AppenderTransaction { + var err error + app := h.Appender(b.Context()) + for _, s := range series { + var ref storage.SeriesRef + for sampleIndex := range samplesPerAppend { + ref, err = app.Append(ref, s.Labels(), ts+sampleIndex, float64(ts+sampleIndex)) + require.NoError(b, err) + } + } + return app +} + +func appendV2Float(b *testing.B, h *Head, ts int64, series []storage.Series, samplesPerAppend int64) storage.AppenderTransaction { + var err error + app := h.AppenderV2(b.Context()) + for _, s := range series { + var ref storage.SeriesRef + for sampleIndex := range samplesPerAppend { + ref, err = app.Append(ref, s.Labels(), 0, ts+sampleIndex, float64(ts+sampleIndex), nil, nil, storage.AOptions{}) + require.NoError(b, err) + } + } + return app +} + +func appendV1FloatOrHistogramWithExemplars(b *testing.B, h *Head, ts int64, series []storage.Series, samplesPerAppend int64) storage.AppenderTransaction { + var err error + app := h.Appender(b.Context()) + for i, s := range series { + var ref storage.SeriesRef + for sampleIndex := range samplesPerAppend { + // if i is even, append a sample, else append a histogram. + if i%2 == 0 { + ref, err = app.Append(ref, s.Labels(), ts+sampleIndex, float64(ts+sampleIndex)) + require.NoError(b, err) + // Every sample also has an exemplar attached. + _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }) + require.NoError(b, err) + continue + } + + h := &histogram.Histogram{ + Count: 7 + uint64(ts*5), + ZeroCount: 2 + uint64(ts), + ZeroThreshold: 0.001, + Sum: 18.4 * rand.Float64(), + Schema: 1, + PositiveSpans: []histogram.Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + PositiveBuckets: []int64{ts + 1, 1, -1, 0}, + } + ref, err = app.AppendHistogram(ref, s.Labels(), ts, h, nil) + require.NoError(b, err) + // Every histogram sample also has 3 exemplars attached. + _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }) + require.NoError(b, err) + _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }) + require.NoError(b, err) + _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }) + require.NoError(b, err) + } + } + return app +} + +func appendV2FloatOrHistogramWithExemplars(b *testing.B, h *Head, ts int64, series []storage.Series, samplesPerAppend int64) storage.AppenderTransaction { + var ( + err error + ex = make([]exemplar.Exemplar, 3) + ) + + app := h.AppenderV2(b.Context()) + for i, s := range series { + var ref storage.SeriesRef + for sampleIndex := range samplesPerAppend { + aOpts := storage.AOptions{Exemplars: ex[:0]} + + // if i is even, append a sample, else append a histogram. + if i%2 == 0 { + // Every sample also has an exemplar attached. + aOpts.Exemplars = append(aOpts.Exemplars, exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }) + ref, err = app.Append(ref, s.Labels(), 0, ts, float64(ts), nil, nil, aOpts) + require.NoError(b, err) + continue + } + h := &histogram.Histogram{ + Count: 7 + uint64(ts*5), + ZeroCount: 2 + uint64(ts), + ZeroThreshold: 0.001, + Sum: 18.4 * rand.Float64(), + Schema: 1, + PositiveSpans: []histogram.Span{ + {Offset: 0, Length: 2}, + {Offset: 1, Length: 2}, + }, + PositiveBuckets: []int64{ts + 1, 1, -1, 0}, + } + + // Every histogram sample also has 3 exemplars attached. + aOpts.Exemplars = append(aOpts.Exemplars, + exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }, + exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }, + exemplar.Exemplar{ + Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), + Value: rand.Float64(), + Ts: ts + sampleIndex, + }, + ) + ref, err = app.Append(ref, s.Labels(), 0, ts, 0, h, nil, aOpts) + require.NoError(b, err) + } + } + return app +} + +type appendCase struct { + name string + appendFunc benchAppendFunc +} + +func appendCases() []appendCase { + return []appendCase{ + { + name: "appender=v1/case=floats", + appendFunc: appendV1Float, + }, + { + name: "appender=v2/case=floats", + appendFunc: appendV2Float, + }, + { + name: "appender=v1/case=floatsHistogramsExemplars", + appendFunc: appendV1FloatOrHistogramWithExemplars, + }, + { + name: "appender=v2/case=floatsHistogramsExemplars", + appendFunc: appendV2FloatOrHistogramWithExemplars, + }, + } +} + +/* + export bench=append && go test \ + -run '^$' -bench '^BenchmarkHeadAppender_AppendCommit$' \ + -benchtime 5s -count 6 -cpu 2 -timeout 999m \ + | tee ${bench}.txt +*/ +func BenchmarkHeadAppender_AppendCommit(b *testing.B) { + // NOTE(bwplotka): Previously we also had 1k and 10k series case. There is nothing + // special happening in 100 vs 1k vs 10k, so let's save considerable amount of benchmark time + // for quicker feedback. In return, we add more sample type cases. + // Similarly, we removed the 2 sample in append case. + // + // TODO(bwplotka): This still takes ~6500s (~2h) for -benchtime 5s -count 6 to complete. + // We might want to reduce the time bit more. 5s is really important as the slowest + // case (appender=v1/case=floatsHistogramsExemplars/series=100/samples_per_append=100-2) + // in 5s yields only 255 iters 23184892 ns/op. Perhaps -benchtime=300x would be better? + seriesCounts := []int{10, 100} + series := genSeries(100, 10, 0, 0) // Only using the generated labels. + for _, appendCase := range appendCases() { + for _, seriesCount := range seriesCounts { + for _, samplesPerAppend := range []int64{1, 5, 100} { + b.Run(fmt.Sprintf("%s/series=%d/samples_per_append=%d", appendCase.name, seriesCount, samplesPerAppend), func(b *testing.B) { + opts := newTestHeadDefaultOptions(10000, false) + opts.EnableExemplarStorage = true // We benchmark with exemplars, benchmark with them. + h, _ := newTestHeadWithOptions(b, compression.None, opts) + b.Cleanup(func() { require.NoError(b, h.Close()) }) + + ts := int64(1000) + + // Init series, that's not what we're benchmarking here. + app := appendCase.appendFunc(b, h, ts, series[:seriesCount], samplesPerAppend) + require.NoError(b, app.Commit()) + ts += 1000 // should increment more than highest samplesPerAppend + + b.ReportAllocs() + b.ResetTimer() + + for b.Loop() { + app := appendCase.appendFunc(b, h, ts, series[:seriesCount], samplesPerAppend) + require.NoError(b, app.Commit()) + ts += 1000 // should increment more than highest samplesPerAppend + } + }) + } + } + } +} + func BenchmarkHeadStripeSeriesCreate(b *testing.B) { chunkDir := b.TempDir() // Put a series, select it. GC it and then access it. @@ -86,86 +307,6 @@ func BenchmarkHeadStripeSeriesCreate_PreCreationFailure(b *testing.B) { } } -func BenchmarkHead_WalCommit(b *testing.B) { - seriesCounts := []int{100, 1000, 10000} - series := genSeries(10000, 10, 0, 0) // Only using the generated labels. - - appendSamples := func(b *testing.B, app storage.Appender, seriesCount int, ts int64) { - var err error - for i, s := range series[:seriesCount] { - var ref storage.SeriesRef - // if i is even, append a sample, else append a histogram. - if i%2 == 0 { - ref, err = app.Append(ref, s.Labels(), ts, float64(ts)) - } else { - h := &histogram.Histogram{ - Count: 7 + uint64(ts*5), - ZeroCount: 2 + uint64(ts), - ZeroThreshold: 0.001, - Sum: 18.4 * rand.Float64(), - Schema: 1, - PositiveSpans: []histogram.Span{ - {Offset: 0, Length: 2}, - {Offset: 1, Length: 2}, - }, - PositiveBuckets: []int64{ts + 1, 1, -1, 0}, - } - ref, err = app.AppendHistogram(ref, s.Labels(), ts, h, nil) - } - require.NoError(b, err) - - _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ - Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), - Value: rand.Float64(), - Ts: ts, - }) - require.NoError(b, err) - } - } - - for _, seriesCount := range seriesCounts { - b.Run(fmt.Sprintf("%d series", seriesCount), func(b *testing.B) { - for _, commits := range []int64{1, 2} { // To test commits that create new series and when the series already exists. - b.Run(fmt.Sprintf("%d commits", commits), func(b *testing.B) { - b.ReportAllocs() - b.ResetTimer() - - for b.Loop() { - b.StopTimer() - h, w := newTestHead(b, 10000, compression.None, false) - b.Cleanup(func() { - if h != nil { - h.Close() - } - if w != nil { - w.Close() - } - }) - app := h.Appender(context.Background()) - - appendSamples(b, app, seriesCount, 0) - - b.StartTimer() - require.NoError(b, app.Commit()) - if commits == 2 { - b.StopTimer() - app = h.Appender(context.Background()) - appendSamples(b, app, seriesCount, 1) - b.StartTimer() - require.NoError(b, app.Commit()) - } - b.StopTimer() - h.Close() - h = nil - w.Close() - w = nil - } - }) - } - }) - } -} - type failingSeriesLifecycleCallback struct{} func (failingSeriesLifecycleCallback) PreCreation(labels.Labels) error { return errors.New("failed") } diff --git a/tsdb/head_bench_v2_test.go b/tsdb/head_bench_v2_test.go deleted file mode 100644 index c98fb6613d..0000000000 --- a/tsdb/head_bench_v2_test.go +++ /dev/null @@ -1,173 +0,0 @@ -// Copyright 2018 The Prometheus Authors -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package tsdb - -import ( - "context" - "errors" - "fmt" - "math/rand" - "strconv" - "testing" - - "github.com/stretchr/testify/require" - "go.uber.org/atomic" - - "github.com/prometheus/prometheus/model/exemplar" - "github.com/prometheus/prometheus/model/histogram" - "github.com/prometheus/prometheus/model/labels" - "github.com/prometheus/prometheus/storage" - "github.com/prometheus/prometheus/tsdb/chunks" - "github.com/prometheus/prometheus/util/compression" -) - -func BenchmarkHeadStripeSeriesCreate(b *testing.B) { - chunkDir := b.TempDir() - // Put a series, select it. GC it and then access it. - opts := DefaultHeadOptions() - opts.ChunkRange = 1000 - opts.ChunkDirRoot = chunkDir - h, err := NewHead(nil, nil, nil, nil, opts, nil) - require.NoError(b, err) - defer h.Close() - - for i := 0; b.Loop(); i++ { - h.getOrCreate(uint64(i), labels.FromStrings("a", strconv.Itoa(i)), false) - } -} - -func BenchmarkHeadStripeSeriesCreateParallel(b *testing.B) { - chunkDir := b.TempDir() - // Put a series, select it. GC it and then access it. - opts := DefaultHeadOptions() - opts.ChunkRange = 1000 - opts.ChunkDirRoot = chunkDir - h, err := NewHead(nil, nil, nil, nil, opts, nil) - require.NoError(b, err) - defer h.Close() - - var count atomic.Int64 - - b.RunParallel(func(pb *testing.PB) { - for pb.Next() { - i := count.Inc() - h.getOrCreate(uint64(i), labels.FromStrings("a", strconv.Itoa(int(i))), false) - } - }) -} - -func BenchmarkHeadStripeSeriesCreate_PreCreationFailure(b *testing.B) { - chunkDir := b.TempDir() - // Put a series, select it. GC it and then access it. - opts := DefaultHeadOptions() - opts.ChunkRange = 1000 - opts.ChunkDirRoot = chunkDir - - // Mock the PreCreation() callback to fail on each series. - opts.SeriesCallback = failingSeriesLifecycleCallback{} - - h, err := NewHead(nil, nil, nil, nil, opts, nil) - require.NoError(b, err) - defer h.Close() - - for i := 0; b.Loop(); i++ { - h.getOrCreate(uint64(i), labels.FromStrings("a", strconv.Itoa(i)), false) - } -} - -func BenchmarkHead_WalCommit(b *testing.B) { - seriesCounts := []int{100, 1000, 10000} - series := genSeries(10000, 10, 0, 0) // Only using the generated labels. - - appendSamples := func(b *testing.B, app storage.Appender, seriesCount int, ts int64) { - var err error - for i, s := range series[:seriesCount] { - var ref storage.SeriesRef - // if i is even, append a sample, else append a histogram. - if i%2 == 0 { - ref, err = app.Append(ref, s.Labels(), ts, float64(ts)) - } else { - h := &histogram.Histogram{ - Count: 7 + uint64(ts*5), - ZeroCount: 2 + uint64(ts), - ZeroThreshold: 0.001, - Sum: 18.4 * rand.Float64(), - Schema: 1, - PositiveSpans: []histogram.Span{ - {Offset: 0, Length: 2}, - {Offset: 1, Length: 2}, - }, - PositiveBuckets: []int64{ts + 1, 1, -1, 0}, - } - ref, err = app.AppendHistogram(ref, s.Labels(), ts, h, nil) - } - require.NoError(b, err) - - _, err = app.AppendExemplar(ref, s.Labels(), exemplar.Exemplar{ - Labels: labels.FromStrings("trace_id", strconv.Itoa(rand.Int())), - Value: rand.Float64(), - Ts: ts, - }) - require.NoError(b, err) - } - } - - for _, seriesCount := range seriesCounts { - b.Run(fmt.Sprintf("%d series", seriesCount), func(b *testing.B) { - for _, commits := range []int64{1, 2} { // To test commits that create new series and when the series already exists. - b.Run(fmt.Sprintf("%d commits", commits), func(b *testing.B) { - b.ReportAllocs() - b.ResetTimer() - - for b.Loop() { - b.StopTimer() - h, w := newTestHead(b, 10000, compression.None, false) - b.Cleanup(func() { - if h != nil { - h.Close() - } - if w != nil { - w.Close() - } - }) - app := h.Appender(context.Background()) - - appendSamples(b, app, seriesCount, 0) - - b.StartTimer() - require.NoError(b, app.Commit()) - if commits == 2 { - b.StopTimer() - app = h.Appender(context.Background()) - appendSamples(b, app, seriesCount, 1) - b.StartTimer() - require.NoError(b, app.Commit()) - } - b.StopTimer() - h.Close() - h = nil - w.Close() - w = nil - } - }) - } - }) - } -} - -type failingSeriesLifecycleCallback struct{} - -func (failingSeriesLifecycleCallback) PreCreation(labels.Labels) error { return errors.New("failed") } -func (failingSeriesLifecycleCallback) PostCreation(labels.Labels) {} -func (failingSeriesLifecycleCallback) PostDeletion(map[chunks.HeadSeriesRef]labels.Labels) {} diff --git a/tsdb/head_test.go b/tsdb/head_test.go index 552db13d07..84605d31fa 100644 --- a/tsdb/head_test.go +++ b/tsdb/head_test.go @@ -107,49 +107,6 @@ func BenchmarkCreateSeries(b *testing.B) { } } -func BenchmarkHeadAppender_Append_Commit_ExistingSeries(b *testing.B) { - seriesCounts := []int{100, 1000, 10000} - series := genSeries(10000, 10, 0, 0) - - for _, seriesCount := range seriesCounts { - b.Run(fmt.Sprintf("%d series", seriesCount), func(b *testing.B) { - for _, samplesPerAppend := range []int64{1, 2, 5, 100} { - b.Run(fmt.Sprintf("%d samples per append", samplesPerAppend), func(b *testing.B) { - h, _ := newTestHead(b, 10000, compression.None, false) - b.Cleanup(func() { require.NoError(b, h.Close()) }) - - ts := int64(1000) - appendSamples := func() error { - var err error - app := h.Appender(context.Background()) - for _, s := range series[:seriesCount] { - var ref storage.SeriesRef - for sampleIndex := range samplesPerAppend { - ref, err = app.Append(ref, s.Labels(), ts+sampleIndex, float64(ts+sampleIndex)) - if err != nil { - return err - } - } - } - ts += 1000 // should increment more than highest samplesPerAppend - return app.Commit() - } - - // Init series, that's not what we're benchmarking here. - require.NoError(b, appendSamples()) - - b.ReportAllocs() - b.ResetTimer() - - for b.Loop() { - require.NoError(b, appendSamples()) - } - }) - } - }) - } -} - func populateTestWL(t testing.TB, w *wlog.WL, recs []any, buf []byte) []byte { var enc record.Encoder for _, r := range recs { @@ -5941,7 +5898,7 @@ func TestOOOAppendWithNoSeries(t *testing.T) { } } -func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error)) { +func testOOOAppendWithNoSeries(t *testing.T, appendFunc func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error)) { dir := t.TempDir() wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, compression.Snappy) require.NoError(t, err) @@ -6284,6 +6241,7 @@ func TestSnapshotAheadOfWALError(t *testing.T) { require.NoError(t, head.Close()) } +// TODO(bwplotka): Bad benchmark (no b.Loop/b.N), fix or remove. func BenchmarkCuttingHeadHistogramChunks(b *testing.B) { const ( numSamples = 50000 @@ -6579,6 +6537,8 @@ func TestWALSampleAndExemplarOrder(t *testing.T) { // would trigger the // `signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xbb03d1` // panic, that we have seen in the wild once. +// +// TODO(bwplotka): This no longer can happen in AppenderV2, remove once AppenderV1 is removed, see #17632. func TestHeadCompactionWhileAppendAndCommitExemplar(t *testing.T) { h, _ := newTestHead(t, DefaultBlockDuration, compression.None, false) app := h.Appender(context.Background()) diff --git a/tsdb/testutil.go b/tsdb/testutil.go index 4d413322c8..d41591750b 100644 --- a/tsdb/testutil.go +++ b/tsdb/testutil.go @@ -44,14 +44,14 @@ type testValue struct { type sampleTypeScenario struct { sampleType string - appendFunc func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) + appendFunc func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) sampleFunc func(ts, value int64) sample } var sampleTypeScenarios = map[string]sampleTypeScenario{ float: { sampleType: sampleMetricTypeFloat, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, f: float64(value)} ref, err := appender.Append(0, lbls, ts, s.f) return ref, s, err @@ -62,7 +62,7 @@ var sampleTypeScenarios = map[string]sampleTypeScenario{ }, intHistogram: { sampleType: sampleMetricTypeHistogram, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, h: tsdbutil.GenerateTestHistogram(value)} ref, err := appender.AppendHistogram(0, lbls, ts, s.h, nil) return ref, s, err @@ -73,7 +73,7 @@ var sampleTypeScenarios = map[string]sampleTypeScenario{ }, floatHistogram: { sampleType: sampleMetricTypeHistogram, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, fh: tsdbutil.GenerateTestFloatHistogram(value)} ref, err := appender.AppendHistogram(0, lbls, ts, nil, s.fh) return ref, s, err @@ -84,7 +84,7 @@ var sampleTypeScenarios = map[string]sampleTypeScenario{ }, customBucketsIntHistogram: { sampleType: sampleMetricTypeHistogram, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, h: tsdbutil.GenerateTestCustomBucketsHistogram(value)} ref, err := appender.AppendHistogram(0, lbls, ts, s.h, nil) return ref, s, err @@ -95,7 +95,7 @@ var sampleTypeScenarios = map[string]sampleTypeScenario{ }, customBucketsFloatHistogram: { sampleType: sampleMetricTypeHistogram, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, fh: tsdbutil.GenerateTestCustomBucketsFloatHistogram(value)} ref, err := appender.AppendHistogram(0, lbls, ts, nil, s.fh) return ref, s, err @@ -106,7 +106,7 @@ var sampleTypeScenarios = map[string]sampleTypeScenario{ }, gaugeIntHistogram: { sampleType: sampleMetricTypeHistogram, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, h: tsdbutil.GenerateTestGaugeHistogram(value)} ref, err := appender.AppendHistogram(0, lbls, ts, s.h, nil) return ref, s, err @@ -117,7 +117,7 @@ var sampleTypeScenarios = map[string]sampleTypeScenario{ }, gaugeFloatHistogram: { sampleType: sampleMetricTypeHistogram, - appendFunc: func(appender storage.Appender, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { + appendFunc: func(appender storage.LimitedAppenderV1, lbls labels.Labels, ts, value int64) (storage.SeriesRef, sample, error) { s := sample{t: ts, fh: tsdbutil.GenerateTestGaugeFloatHistogram(value)} ref, err := appender.AppendHistogram(0, lbls, ts, nil, s.fh) return ref, s, err From e7e45090e4561775350e8ee5e23a398916165046 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Tue, 2 Dec 2025 14:35:53 +0000 Subject: [PATCH 119/439] refactor(appenderV2): port TSDB non-head tests Signed-off-by: bwplotka --- tsdb/blockwriter.go | 6 + tsdb/blockwriter_test.go | 35 + tsdb/db.go | 16 + tsdb/db_append_v2_test.go | 2683 +++++++------------------------------ 4 files changed, 566 insertions(+), 2174 deletions(-) diff --git a/tsdb/blockwriter.go b/tsdb/blockwriter.go index 14137f12cc..e038812224 100644 --- a/tsdb/blockwriter.go +++ b/tsdb/blockwriter.go @@ -86,6 +86,12 @@ func (w *BlockWriter) Appender(ctx context.Context) storage.Appender { return w.head.Appender(ctx) } +// AppenderV2 returns a new appender on the database. +// AppenderV2 can't be called concurrently. However, the returned AppenderV2 can safely be used concurrently. +func (w *BlockWriter) AppenderV2(ctx context.Context) storage.AppenderV2 { + return w.head.AppenderV2(ctx) +} + // Flush implements the Writer interface. This is where actual block writing // happens. After flush completes, no writes can be done. func (w *BlockWriter) Flush(ctx context.Context) (ulid.ULID, error) { diff --git a/tsdb/blockwriter_test.go b/tsdb/blockwriter_test.go index e7c3146247..becae6aa04 100644 --- a/tsdb/blockwriter_test.go +++ b/tsdb/blockwriter_test.go @@ -23,6 +23,7 @@ import ( "github.com/stretchr/testify/require" "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/storage" "github.com/prometheus/prometheus/tsdb/chunks" ) @@ -59,3 +60,37 @@ func TestBlockWriter(t *testing.T) { require.NoError(t, w.Close()) } + +func TestBlockWriter_AppenderV2(t *testing.T) { + ctx := context.Background() + outputDir := t.TempDir() + w, err := NewBlockWriter(promslog.NewNopLogger(), outputDir, DefaultBlockDuration) + require.NoError(t, err) + + // Add some series. + app := w.AppenderV2(ctx) + ts1, v1 := int64(44), float64(7) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, ts1, v1, nil, nil, storage.AOptions{}) + require.NoError(t, err) + ts2, v2 := int64(55), float64(12) + _, err = app.Append(0, labels.FromStrings("c", "d"), 0, ts2, v2, nil, nil, storage.AOptions{}) + require.NoError(t, err) + require.NoError(t, app.Commit()) + id, err := w.Flush(ctx) + require.NoError(t, err) + + // Confirm the block has the correct data. + blockpath := filepath.Join(outputDir, id.String()) + b, err := OpenBlock(nil, blockpath, nil, nil) + require.NoError(t, err) + defer func() { require.NoError(t, b.Close()) }() + q, err := NewBlockQuerier(b, math.MinInt64, math.MaxInt64) + require.NoError(t, err) + series := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "", ".*")) + sample1 := []chunks.Sample{sample{t: ts1, f: v1}} + sample2 := []chunks.Sample{sample{t: ts2, f: v2}} + expectedSeries := map[string][]chunks.Sample{"{a=\"b\"}": sample1, "{c=\"d\"}": sample2} + require.Equal(t, expectedSeries, series) + + require.NoError(t, w.Close()) +} diff --git a/tsdb/db.go b/tsdb/db.go index c4f29c225f..73300d74f1 100644 --- a/tsdb/db.go +++ b/tsdb/db.go @@ -220,6 +220,20 @@ type Options struct { // UseUncachedIO allows bypassing the page cache when appropriate. UseUncachedIO bool + // EnableSTAsZeroSample represents 'created-timestamp-zero-ingestion' feature flag. + // If true, ST, if non-zero and earlier than sample timestamp, will be stored + // as a zero sample before the actual sample. + // + // The zero sample is best-effort, only debug log on failure is emitted. + // NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 + // is implemented. + EnableSTAsZeroSample bool + + // EnableMetadataWALRecords represents 'metadata-wal-records' feature flag. + // NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 + // is implemented. + EnableMetadataWALRecords bool + // BlockCompactionExcludeFunc is a function which returns true for blocks that should NOT be compacted. // It's passed down to the TSDB compactor. BlockCompactionExcludeFunc BlockExcludeFilterFunc @@ -973,6 +987,8 @@ func open(dir string, l *slog.Logger, r prometheus.Registerer, opts *Options, rn headOpts.OutOfOrderTimeWindow.Store(opts.OutOfOrderTimeWindow) headOpts.OutOfOrderCapMax.Store(opts.OutOfOrderCapMax) headOpts.EnableSharding = opts.EnableSharding + headOpts.EnableSTAsZeroSample = opts.EnableSTAsZeroSample + headOpts.EnableMetadataWALRecords = opts.EnableMetadataWALRecords if opts.WALReplayConcurrency > 0 { headOpts.WALReplayConcurrency = opts.WALReplayConcurrency } diff --git a/tsdb/db_append_v2_test.go b/tsdb/db_append_v2_test.go index 4e084ef0d8..344b1d6943 100644 --- a/tsdb/db_append_v2_test.go +++ b/tsdb/db_append_v2_test.go @@ -1,4 +1,4 @@ -// Copyright 2017 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -15,46 +15,32 @@ package tsdb import ( "bufio" - "bytes" "context" - "encoding/binary" - "errors" - "flag" "fmt" - "hash/crc32" "log/slog" "math" "math/rand" - "net/http" - "net/http/httptest" "os" "path" "path/filepath" "runtime" "sort" "strconv" - "sync" "testing" "time" - "github.com/gogo/protobuf/proto" - "github.com/golang/snappy" "github.com/oklog/ulid/v2" "github.com/prometheus/client_golang/prometheus" prom_testutil "github.com/prometheus/client_golang/prometheus/testutil" - "github.com/prometheus/common/model" "github.com/prometheus/common/promslog" "github.com/stretchr/testify/require" "go.uber.org/atomic" - "go.uber.org/goleak" "github.com/prometheus/prometheus/config" "github.com/prometheus/prometheus/model/histogram" "github.com/prometheus/prometheus/model/labels" "github.com/prometheus/prometheus/model/metadata" - "github.com/prometheus/prometheus/prompb" "github.com/prometheus/prometheus/storage" - "github.com/prometheus/prometheus/storage/remote" "github.com/prometheus/prometheus/tsdb/chunkenc" "github.com/prometheus/prometheus/tsdb/chunks" "github.com/prometheus/prometheus/tsdb/fileutil" @@ -68,192 +54,20 @@ import ( "github.com/prometheus/prometheus/util/testutil" ) -func TestMain(m *testing.M) { - var isolationEnabled bool - flag.BoolVar(&isolationEnabled, "test.tsdb-isolation", true, "enable isolation") - flag.Parse() - defaultIsolationDisabled = !isolationEnabled +// TODO(bwplotka): Ensure non-ported tests are not deleted from db_test.go when removing AppenderV1 flow (#17632): +// * TestQuerier_ShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks +// * TestChunkQuerier_ShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks +// * TestEmptyLabelsetCausesError +// * TestQueryHistogramFromBlocksWithCompaction - goleak.VerifyTestMain(m, - goleak.IgnoreTopFunction("github.com/prometheus/prometheus/tsdb.(*SegmentWAL).cut.func1"), - goleak.IgnoreTopFunction("github.com/prometheus/prometheus/tsdb.(*SegmentWAL).cut.func2"), - goleak.IgnoreTopFunction("go.opencensus.io/stats/view.(*worker).start")) -} - -type testDBOptions struct { - dir string - opts *Options - rngs []int64 -} -type testDBOpt func(o *testDBOptions) - -func withDir(dir string) testDBOpt { - return func(o *testDBOptions) { - o.dir = dir - } -} - -func withOpts(opts *Options) testDBOpt { - return func(o *testDBOptions) { - o.opts = opts - } -} - -func withRngs(rngs ...int64) testDBOpt { - return func(o *testDBOptions) { - o.rngs = rngs - } -} - -func newTestDB(t testing.TB, opts ...testDBOpt) (db *DB) { - var o testDBOptions - for _, opt := range opts { - opt(&o) - } - if o.opts == nil { - o.opts = DefaultOptions() - } - if o.dir == "" { - o.dir = t.TempDir() - } - - var err error - if len(o.rngs) == 0 { - db, err = Open(o.dir, nil, nil, o.opts, nil) - } else { - o.opts, o.rngs = validateOpts(o.opts, o.rngs) - db, err = open(o.dir, nil, nil, o.opts, o.rngs, nil) - } - require.NoError(t, err) - t.Cleanup(func() { - // Always close. DB is safe for close-after-close. - require.NoError(t, db.Close()) - }) - return db -} - -func TestDBClose_AfterClose(t *testing.T) { - db := newTestDB(t) - require.NoError(t, db.Close()) - require.NoError(t, db.Close()) - - // Double check if we are closing correct DB after reuse. - db = newTestDB(t) - require.NoError(t, db.Close()) - require.NoError(t, db.Close()) -} - -// query runs a matcher query against the querier and fully expands its data. -func query(t testing.TB, q storage.Querier, matchers ...*labels.Matcher) map[string][]chunks.Sample { - ss := q.Select(context.Background(), false, nil, matchers...) - defer func() { - require.NoError(t, q.Close()) - }() - - var it chunkenc.Iterator - result := map[string][]chunks.Sample{} - for ss.Next() { - series := ss.At() - - it = series.Iterator(it) - samples, err := storage.ExpandSamples(it, newSample) - require.NoError(t, err) - require.NoError(t, it.Err()) - - if len(samples) == 0 { - continue - } - - name := series.Labels().String() - result[name] = samples - } - require.NoError(t, ss.Err()) - require.Empty(t, ss.Warnings()) - - return result -} - -// queryAndExpandChunks runs a matcher query against the querier and fully expands its data into samples. -func queryAndExpandChunks(t testing.TB, q storage.ChunkQuerier, matchers ...*labels.Matcher) map[string][][]chunks.Sample { - s := queryChunks(t, q, matchers...) - - res := make(map[string][][]chunks.Sample) - for k, v := range s { - var samples [][]chunks.Sample - for _, chk := range v { - sam, err := storage.ExpandSamples(chk.Chunk.Iterator(nil), nil) - require.NoError(t, err) - samples = append(samples, sam) - } - res[k] = samples - } - - return res -} - -// queryChunks runs a matcher query against the querier and expands its data. -func queryChunks(t testing.TB, q storage.ChunkQuerier, matchers ...*labels.Matcher) map[string][]chunks.Meta { - ss := q.Select(context.Background(), false, nil, matchers...) - defer func() { - require.NoError(t, q.Close()) - }() - - var it chunks.Iterator - result := map[string][]chunks.Meta{} - for ss.Next() { - series := ss.At() - - chks := []chunks.Meta{} - it = series.Iterator(it) - for it.Next() { - chks = append(chks, it.At()) - } - require.NoError(t, it.Err()) - - if len(chks) == 0 { - continue - } - - name := series.Labels().String() - result[name] = chks - } - require.NoError(t, ss.Err()) - require.Empty(t, ss.Warnings()) - return result -} - -// Ensure that blocks are held in memory in their time order -// and not in ULID order as they are read from the directory. -func TestDB_reloadOrder(t *testing.T) { - db := newTestDB(t) - - metas := []BlockMeta{ - {MinTime: 90, MaxTime: 100}, - {MinTime: 70, MaxTime: 80}, - {MinTime: 100, MaxTime: 110}, - } - for _, m := range metas { - createBlock(t, db.Dir(), genSeries(1, 1, m.MinTime, m.MaxTime)) - } - - require.NoError(t, db.reloadBlocks()) - blocks := db.Blocks() - require.Len(t, blocks, 3) - require.Equal(t, metas[1].MinTime, blocks[0].Meta().MinTime) - require.Equal(t, metas[1].MaxTime, blocks[0].Meta().MaxTime) - require.Equal(t, metas[0].MinTime, blocks[1].Meta().MinTime) - require.Equal(t, metas[0].MaxTime, blocks[1].Meta().MaxTime) - require.Equal(t, metas[2].MinTime, blocks[2].Meta().MinTime) - require.Equal(t, metas[2].MaxTime, blocks[2].Meta().MaxTime) -} - -func TestDataAvailableOnlyAfterCommit(t *testing.T) { +// TODO(krajorama): Add histograms test cases. +func TestDataAvailableOnlyAfterCommit_AppendV2(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) - _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) querier, err := db.Querier(0, 1) @@ -275,7 +89,7 @@ func TestDataAvailableOnlyAfterCommit(t *testing.T) { // TestNoPanicAfterWALCorruption ensures that querying the db after a WAL corruption doesn't cause a panic. // https://github.com/prometheus/prometheus/issues/7548 -func TestNoPanicAfterWALCorruption(t *testing.T) { +func TestNoPanicAfterWALCorruption_AppendV2(t *testing.T) { db := newTestDB(t, withOpts(&Options{WALSegmentSize: 32 * 1024})) // Append until the first mmapped head chunk. @@ -286,8 +100,8 @@ func TestNoPanicAfterWALCorruption(t *testing.T) { { // Appending 121 samples because on the 121st a new chunk will be created. for range 121 { - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("foo", "bar"), maxt, 0) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, maxt, 0, nil, nil, storage.AOptions{}) expSamples = append(expSamples, sample{t: maxt, f: 0}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -326,22 +140,24 @@ func TestNoPanicAfterWALCorruption(t *testing.T) { } } -func TestDataNotAvailableAfterRollback(t *testing.T) { +func TestDataNotAvailableAfterRollback_AppendV2(t *testing.T) { db := newTestDB(t) - app := db.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("type", "float"), 0, 0) + app := db.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("type", "float"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.AppendHistogram( - 0, labels.FromStrings("type", "histogram"), 0, + _, err = app.Append( + 0, labels.FromStrings("type", "histogram"), 0, 0, 0, &histogram.Histogram{Count: 42, Sum: math.NaN()}, nil, + storage.AOptions{}, ) require.NoError(t, err) - _, err = app.AppendHistogram( - 0, labels.FromStrings("type", "floathistogram"), 0, + _, err = app.Append( + 0, labels.FromStrings("type", "floathistogram"), 0, 0, 0, nil, &histogram.FloatHistogram{Count: 42, Sum: math.NaN()}, + storage.AOptions{}, ) require.NoError(t, err) @@ -413,41 +229,41 @@ func TestDataNotAvailableAfterRollback(t *testing.T) { require.Equal(t, 0, walFloatHistogramCount, "float histograms should not have been written to WAL") } -func TestDBAppenderAddRef(t *testing.T) { +func TestDBAppenderV2_AddRef(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app1 := db.Appender(ctx) + app1 := db.AppenderV2(ctx) - ref1, err := app1.Append(0, labels.FromStrings("a", "b"), 123, 0) + ref1, err := app1.Append(0, labels.FromStrings("a", "b"), 0, 123, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) // Reference should already work before commit. - ref2, err := app1.Append(ref1, labels.EmptyLabels(), 124, 1) + ref2, err := app1.Append(ref1, labels.EmptyLabels(), 0, 124, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, ref1, ref2) err = app1.Commit() require.NoError(t, err) - app2 := db.Appender(ctx) + app2 := db.AppenderV2(ctx) // first ref should already work in next transaction. - ref3, err := app2.Append(ref1, labels.EmptyLabels(), 125, 0) + ref3, err := app2.Append(ref1, labels.EmptyLabels(), 0, 125, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, ref1, ref3) - ref4, err := app2.Append(ref1, labels.FromStrings("a", "b"), 133, 1) + ref4, err := app2.Append(ref1, labels.FromStrings("a", "b"), 0, 133, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, ref1, ref4) // Reference must be valid to add another sample. - ref5, err := app2.Append(ref2, labels.EmptyLabels(), 143, 2) + ref5, err := app2.Append(ref2, labels.EmptyLabels(), 0, 143, 2, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, ref1, ref5) // Missing labels & invalid refs should fail. - _, err = app2.Append(9999999, labels.EmptyLabels(), 1, 1) + _, err = app2.Append(9999999, labels.EmptyLabels(), 0, 1, 1, nil, nil, storage.AOptions{}) require.ErrorIs(t, err, ErrInvalidSample) require.NoError(t, app2.Commit()) @@ -468,17 +284,17 @@ func TestDBAppenderAddRef(t *testing.T) { }, res) } -func TestAppendEmptyLabelsIgnored(t *testing.T) { +func TestDBAppenderV2_EmptyLabelsIgnored(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app1 := db.Appender(ctx) + app1 := db.AppenderV2(ctx) - ref1, err := app1.Append(0, labels.FromStrings("a", "b"), 123, 0) + ref1, err := app1.Append(0, labels.FromStrings("a", "b"), 0, 123, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) // Add with empty label. - ref2, err := app1.Append(0, labels.FromStrings("a", "b", "c", ""), 124, 0) + ref2, err := app1.Append(0, labels.FromStrings("a", "b", "c", ""), 0, 124, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) // Should be the same series. @@ -488,7 +304,17 @@ func TestAppendEmptyLabelsIgnored(t *testing.T) { require.NoError(t, err) } -func TestDeleteSimple(t *testing.T) { +func TestDBAppenderV2_EmptyLabelsetCausesError(t *testing.T) { + db := newTestDB(t) + + ctx := context.Background() + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.Labels{}, 0, 0, 0, nil, nil, storage.AOptions{}) + require.Error(t, err) + require.Equal(t, "empty labelset: invalid sample", err.Error()) +} + +func TestDeleteSimple_AppendV2(t *testing.T) { const numSamples int64 = 10 cases := []struct { @@ -522,12 +348,12 @@ func TestDeleteSimple(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) smpls := make([]float64, numSamples) for i := range numSamples { smpls[i] = rand.Float64() - app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + app.Append(0, labels.FromStrings("a", "b"), 0, i, smpls[i], nil, nil, storage.AOptions{}) } require.NoError(t, app.Commit()) @@ -576,19 +402,19 @@ func TestDeleteSimple(t *testing.T) { } } -func TestAmendHistogramDatapointCausesError(t *testing.T) { +func TestAmendHistogramDatapointCausesError_AppendV2(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 0) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) - app = db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 1) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0, 1, nil, nil, storage.AOptions{}) require.ErrorIs(t, err, storage.ErrDuplicateSampleForTimestamp) require.NoError(t, app.Rollback()) @@ -606,81 +432,71 @@ func TestAmendHistogramDatapointCausesError(t *testing.T) { } fh := h.ToFloat(nil) - app = db.Appender(ctx) - _, err = app.AppendHistogram(0, labels.FromStrings("a", "c"), 0, h.Copy(), nil) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "c"), 0, 0, 0, h.Copy(), nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) - app = db.Appender(ctx) - _, err = app.AppendHistogram(0, labels.FromStrings("a", "c"), 0, h.Copy(), nil) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "c"), 0, 0, 0, h.Copy(), nil, storage.AOptions{}) require.NoError(t, err) h.Schema = 2 - _, err = app.AppendHistogram(0, labels.FromStrings("a", "c"), 0, h.Copy(), nil) + _, err = app.Append(0, labels.FromStrings("a", "c"), 0, 0, 0, h.Copy(), nil, storage.AOptions{}) require.Equal(t, storage.ErrDuplicateSampleForTimestamp, err) require.NoError(t, app.Rollback()) // Float histogram. - app = db.Appender(ctx) - _, err = app.AppendHistogram(0, labels.FromStrings("a", "d"), 0, nil, fh.Copy()) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "d"), 0, 0, 0, nil, fh.Copy(), storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) - app = db.Appender(ctx) - _, err = app.AppendHistogram(0, labels.FromStrings("a", "d"), 0, nil, fh.Copy()) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "d"), 0, 0, 0, nil, fh.Copy(), storage.AOptions{}) require.NoError(t, err) fh.Schema = 2 - _, err = app.AppendHistogram(0, labels.FromStrings("a", "d"), 0, nil, fh.Copy()) + _, err = app.Append(0, labels.FromStrings("a", "d"), 0, 0, 0, nil, fh.Copy(), storage.AOptions{}) require.Equal(t, storage.ErrDuplicateSampleForTimestamp, err) require.NoError(t, app.Rollback()) } -func TestDuplicateNaNDatapointNoAmendError(t *testing.T) { +func TestDuplicateNaNDatapointNoAmendError_AppendV2(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("a", "b"), 0, math.NaN()) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 0, math.NaN(), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) - app = db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("a", "b"), 0, math.NaN()) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0, math.NaN(), nil, nil, storage.AOptions{}) require.NoError(t, err) } -func TestNonDuplicateNaNDatapointsCausesAmendError(t *testing.T) { +func TestNonDuplicateNaNDatapointsCausesAmendError_AppendV2(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("a", "b"), 0, math.Float64frombits(0x7ff0000000000001)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 0, math.Float64frombits(0x7ff0000000000001), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) - app = db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("a", "b"), 0, math.Float64frombits(0x7ff0000000000002)) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0, math.Float64frombits(0x7ff0000000000002), nil, nil, storage.AOptions{}) require.ErrorIs(t, err, storage.ErrDuplicateSampleForTimestamp) } -func TestEmptyLabelsetCausesError(t *testing.T) { - db := newTestDB(t) - - ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.Labels{}, 0, 0) - require.Error(t, err) - require.Equal(t, "empty labelset: invalid sample", err.Error()) -} - -func TestSkippingInvalidValuesInSameTxn(t *testing.T) { +func TestSkippingInvalidValuesInSameTxn_AppendV2(t *testing.T) { db := newTestDB(t) // Append AmendedValue. ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 1) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 0, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 2) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 0, 2, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -695,10 +511,10 @@ func TestSkippingInvalidValuesInSameTxn(t *testing.T) { }, ssMap) // Append Out of Order Value. - app = db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("a", "b"), 10, 3) + app = db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 10, 3, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("a", "b"), 7, 5) + _, err = app.Append(0, labels.FromStrings("a", "b"), 0, 7, 5, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -712,15 +528,15 @@ func TestSkippingInvalidValuesInSameTxn(t *testing.T) { }, ssMap) } -func TestDB_Snapshot(t *testing.T) { +func TestDB_Snapshot_AppendV2(t *testing.T) { db := newTestDB(t) // append data ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) mint := int64(1414141414000) for i := range 1000 { - _, err := app.Append(0, labels.FromStrings("foo", "bar"), mint+int64(i), 1.0) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, mint+int64(i), 1.0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -757,14 +573,14 @@ func TestDB_Snapshot(t *testing.T) { // TestDB_Snapshot_ChunksOutsideOfCompactedRange ensures that a snapshot removes chunks samples // that are outside the set block time range. // See https://github.com/prometheus/prometheus/issues/5105 -func TestDB_Snapshot_ChunksOutsideOfCompactedRange(t *testing.T) { +func TestDB_Snapshot_ChunksOutsideOfCompactedRange_AppendV2(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) mint := int64(1414141414000) for i := range 1000 { - _, err := app.Append(0, labels.FromStrings("foo", "bar"), mint+int64(i), 1.0) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, mint+int64(i), 1.0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -803,18 +619,18 @@ func TestDB_Snapshot_ChunksOutsideOfCompactedRange(t *testing.T) { require.Equal(t, 1000.0-10, sum) } -func TestDB_SnapshotWithDelete(t *testing.T) { +func TestDB_SnapshotWithDelete_AppendV2(t *testing.T) { const numSamples int64 = 10 db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) smpls := make([]float64, numSamples) for i := range numSamples { smpls[i] = rand.Float64() - app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + app.Append(0, labels.FromStrings("a", "b"), 0, i, smpls[i], nil, nil, storage.AOptions{}) } require.NoError(t, app.Commit()) @@ -888,7 +704,7 @@ func TestDB_SnapshotWithDelete(t *testing.T) { } } -func TestDB_e2e(t *testing.T) { +func TestDB_e2e_AppendV2(t *testing.T) { const ( numDatapoints = 1000 numRanges = 1000 @@ -946,7 +762,7 @@ func TestDB_e2e(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) for _, l := range lbls { lset := labels.New(l...) @@ -958,7 +774,7 @@ func TestDB_e2e(t *testing.T) { series = append(series, sample{ts, v, nil, nil}) - _, err := app.Append(0, lset, ts, v) + _, err := app.Append(0, lset, 0, ts, v, nil, nil, storage.AOptions{}) require.NoError(t, err) ts += rand.Int63n(timeInterval) + 1 @@ -1044,14 +860,14 @@ func TestDB_e2e(t *testing.T) { } } -func TestWALFlushedOnDBClose(t *testing.T) { +func TestWALFlushedOnDBClose_AppendV2(t *testing.T) { db := newTestDB(t) lbls := labels.FromStrings("labelname", "labelvalue") ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, lbls, 0, 1) + app := db.AppenderV2(ctx) + _, err := app.Append(0, lbls, 0, 0, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -1068,7 +884,7 @@ func TestWALFlushedOnDBClose(t *testing.T) { require.Equal(t, []string{"labelvalue"}, values) } -func TestWALSegmentSizeOptions(t *testing.T) { +func TestWALSegmentSizeOptions_AppendV2(t *testing.T) { tests := map[int]func(dbdir string, segmentSize int){ // Default Wal Size. 0: func(dbDir string, _ int) { @@ -1126,11 +942,11 @@ func TestWALSegmentSizeOptions(t *testing.T) { db := newTestDB(t, withOpts(opts)) for i := range int64(155) { - app := db.Appender(context.Background()) - ref, err := app.Append(0, labels.FromStrings("wal"+strconv.Itoa(int(i)), "size"), i, rand.Float64()) + app := db.AppenderV2(context.Background()) + ref, err := app.Append(0, labels.FromStrings("wal"+strconv.Itoa(int(i)), "size"), 0, i, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) for j := int64(1); j <= 78; j++ { - _, err := app.Append(ref, labels.EmptyLabels(), i+j, rand.Float64()) + _, err := app.Append(ref, labels.EmptyLabels(), 0, i+j, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -1144,7 +960,7 @@ func TestWALSegmentSizeOptions(t *testing.T) { // https://github.com/prometheus/prometheus/issues/9846 // https://github.com/prometheus/prometheus/issues/9859 -func TestWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T) { +func TestWALReplayRaceOnSamplesLoggedBeforeSeries_AppendV2(t *testing.T) { const ( numRuns = 1 numSamplesBeforeSeriesCreation = 1000 @@ -1155,13 +971,13 @@ func TestWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T) { for _, numSamplesAfterSeriesCreation := range []int{1, 1000} { for run := 1; run <= numRuns; run++ { t.Run(fmt.Sprintf("samples after series creation = %d, run = %d", numSamplesAfterSeriesCreation, run), func(t *testing.T) { - testWALReplayRaceOnSamplesLoggedBeforeSeries(t, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation) + testWALReplayRaceOnSamplesLoggedBeforeSeriesAppendV2(t, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation) }) } } } -func testWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation int) { +func testWALReplayRaceOnSamplesLoggedBeforeSeriesAppendV2(t *testing.T, numSamplesBeforeSeriesCreation, numSamplesAfterSeriesCreation int) { const numSeries = 1000 db := newTestDB(t) @@ -1184,11 +1000,11 @@ func testWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T, numSamplesBefore require.NoError(t, err) // Add samples via appender so that they're logged after the series in the WAL. - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) lbls := labels.FromStrings("series_id", strconv.Itoa(seriesRef)) for ts := numSamplesBeforeSeriesCreation; ts < numSamplesBeforeSeriesCreation+numSamplesAfterSeriesCreation; ts++ { - _, err := app.Append(0, lbls, int64(ts), float64(ts)) + _, err := app.Append(0, lbls, 0, int64(ts), float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -1225,19 +1041,19 @@ func testWALReplayRaceOnSamplesLoggedBeforeSeries(t *testing.T, numSamplesBefore require.Equal(t, numSeries, actualSeries) } -func TestTombstoneClean(t *testing.T) { +func TestTombstoneClean_AppendV2(t *testing.T) { t.Parallel() const numSamples int64 = 10 db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) smpls := make([]float64, numSamples) for i := range numSamples { smpls[i] = rand.Float64() - app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + app.Append(0, labels.FromStrings("a", "b"), 0, i, smpls[i], nil, nil, storage.AOptions{}) } require.NoError(t, app.Commit()) @@ -1318,19 +1134,19 @@ func TestTombstoneClean(t *testing.T) { // TestTombstoneCleanResultEmptyBlock tests that a TombstoneClean that results in empty blocks (no timeseries) // will also delete the resultant block. -func TestTombstoneCleanResultEmptyBlock(t *testing.T) { +func TestTombstoneCleanResultEmptyBlock_AppendV2(t *testing.T) { t.Parallel() numSamples := int64(10) db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) smpls := make([]float64, numSamples) for i := range numSamples { smpls[i] = rand.Float64() - app.Append(0, labels.FromStrings("a", "b"), i, smpls[i]) + app.Append(0, labels.FromStrings("a", "b"), 0, i, smpls[i], nil, nil, storage.AOptions{}) } require.NoError(t, app.Commit()) @@ -1358,180 +1174,7 @@ func TestTombstoneCleanResultEmptyBlock(t *testing.T) { require.Empty(t, actualBlockDirs) } -// TestTombstoneCleanFail tests that a failing TombstoneClean doesn't leave any blocks behind. -// When TombstoneClean errors the original block that should be rebuilt doesn't get deleted so -// if TombstoneClean leaves any blocks behind these will overlap. -func TestTombstoneCleanFail(t *testing.T) { - t.Parallel() - db := newTestDB(t) - - var oldBlockDirs []string - - // Create some blocks pending for compaction. - // totalBlocks should be >=2 so we have enough blocks to trigger compaction failure. - totalBlocks := 2 - for i := range totalBlocks { - blockDir := createBlock(t, db.Dir(), genSeries(1, 1, int64(i), int64(i)+1)) - block, err := OpenBlock(nil, blockDir, nil, nil) - require.NoError(t, err) - // Add some fake tombstones to trigger the compaction. - tomb := tombstones.NewMemTombstones() - tomb.AddInterval(0, tombstones.Interval{Mint: int64(i), Maxt: int64(i) + 1}) - block.tombstones = tomb - - db.blocks = append(db.blocks, block) - oldBlockDirs = append(oldBlockDirs, blockDir) - } - - // Initialize the mockCompactorFailing with a room for a single compaction iteration. - // mockCompactorFailing will fail on the second iteration so we can check if the cleanup works as expected. - db.compactor = &mockCompactorFailing{ - t: t, - blocks: db.blocks, - max: totalBlocks + 1, - } - - // The compactor should trigger a failure here. - require.Error(t, db.CleanTombstones()) - - // Now check that the CleanTombstones replaced the old block even after a failure. - actualBlockDirs, err := blockDirs(db.Dir()) - require.NoError(t, err) - // Only one block should have been replaced by a new block. - require.Len(t, actualBlockDirs, len(oldBlockDirs)) - require.Len(t, intersection(oldBlockDirs, actualBlockDirs), len(actualBlockDirs)-1) -} - -func intersection(oldBlocks, actualBlocks []string) (intersection []string) { - hash := make(map[string]bool) - for _, e := range oldBlocks { - hash[e] = true - } - for _, e := range actualBlocks { - // If block present in the hashmap then append intersection list. - if hash[e] { - intersection = append(intersection, e) - } - } - return intersection -} - -// mockCompactorFailing creates a new empty block on every write and fails when reached the max allowed total. -// For CompactOOO, it always fails. -type mockCompactorFailing struct { - t *testing.T - blocks []*Block - max int -} - -func (*mockCompactorFailing) Plan(string) ([]string, error) { - return nil, nil -} - -func (c *mockCompactorFailing) Write(dest string, _ BlockReader, _, _ int64, _ *BlockMeta) ([]ulid.ULID, error) { - if len(c.blocks) >= c.max { - return []ulid.ULID{}, errors.New("the compactor already did the maximum allowed blocks so it is time to fail") - } - - block, err := OpenBlock(nil, createBlock(c.t, dest, genSeries(1, 1, 0, 1)), nil, nil) - require.NoError(c.t, err) - require.NoError(c.t, block.Close()) // Close block as we won't be using anywhere. - c.blocks = append(c.blocks, block) - - // Now check that all expected blocks are actually persisted on disk. - // This way we make sure that we have some blocks that are supposed to be removed. - var expectedBlocks []string - for _, b := range c.blocks { - expectedBlocks = append(expectedBlocks, filepath.Join(dest, b.Meta().ULID.String())) - } - actualBlockDirs, err := blockDirs(dest) - require.NoError(c.t, err) - - require.Equal(c.t, expectedBlocks, actualBlockDirs) - - return []ulid.ULID{block.Meta().ULID}, nil -} - -func (*mockCompactorFailing) Compact(string, []string, []*Block) ([]ulid.ULID, error) { - return []ulid.ULID{}, nil -} - -func (*mockCompactorFailing) CompactOOO(string, *OOOCompactionHead) (result []ulid.ULID, err error) { - return nil, errors.New("mock compaction failing CompactOOO") -} - -func TestTimeRetention(t *testing.T) { - t.Parallel() - testCases := []struct { - name string - blocks []*BlockMeta - expBlocks []*BlockMeta - retentionDuration int64 - }{ - { - name: "Block max time delta greater than retention duration", - blocks: []*BlockMeta{ - {MinTime: 500, MaxTime: 900}, // Oldest block, beyond retention - {MinTime: 1000, MaxTime: 1500}, - {MinTime: 1500, MaxTime: 2000}, // Newest block - }, - expBlocks: []*BlockMeta{ - {MinTime: 1000, MaxTime: 1500}, - {MinTime: 1500, MaxTime: 2000}, - }, - retentionDuration: 1000, - }, - { - name: "Block max time delta equal to retention duration", - blocks: []*BlockMeta{ - {MinTime: 500, MaxTime: 900}, // Oldest block - {MinTime: 1000, MaxTime: 1500}, // Coinciding exactly with the retention duration. - {MinTime: 1500, MaxTime: 2000}, // Newest block - }, - expBlocks: []*BlockMeta{ - {MinTime: 1500, MaxTime: 2000}, - }, - retentionDuration: 500, - }, - } - for _, tc := range testCases { - t.Run(tc.name, func(t *testing.T) { - db := newTestDB(t, withRngs(1000)) - - for _, m := range tc.blocks { - createBlock(t, db.Dir(), genSeries(10, 10, m.MinTime, m.MaxTime)) - } - - require.NoError(t, db.reloadBlocks()) // Reload the db to register the new blocks. - require.Len(t, db.Blocks(), len(tc.blocks)) // Ensure all blocks are registered. - - db.opts.RetentionDuration = tc.retentionDuration - // Reloading should truncate the blocks which are >= the retention duration vs the first block. - require.NoError(t, db.reloadBlocks()) - - actBlocks := db.Blocks() - - require.Equal(t, 1, int(prom_testutil.ToFloat64(db.metrics.timeRetentionCount)), "metric retention count mismatch") - require.Len(t, actBlocks, len(tc.expBlocks)) - for i, eb := range tc.expBlocks { - require.Equal(t, eb.MinTime, actBlocks[i].meta.MinTime) - require.Equal(t, eb.MaxTime, actBlocks[i].meta.MaxTime) - } - }) - } -} - -func TestRetentionDurationMetric(t *testing.T) { - db := newTestDB(t, withOpts(&Options{ - RetentionDuration: 1000, - }), withRngs(100)) - - expRetentionDuration := 1.0 - actRetentionDuration := prom_testutil.ToFloat64(db.metrics.retentionDuration) - require.Equal(t, expRetentionDuration, actRetentionDuration, "metric retention duration mismatch") -} - -func TestSizeRetention(t *testing.T) { +func TestSizeRetention_AppendV2(t *testing.T) { t.Parallel() opts := DefaultOptions() opts.OutOfOrderTimeWindow = 100 @@ -1554,7 +1197,7 @@ func TestSizeRetention(t *testing.T) { } // Add some data to the WAL. - headApp := db.Head().Appender(context.Background()) + headApp := db.Head().AppenderV2(context.Background()) var aSeries labels.Labels var it chunkenc.Iterator for _, m := range headBlocks { @@ -1564,7 +1207,7 @@ func TestSizeRetention(t *testing.T) { it = s.Iterator(it) for it.Next() == chunkenc.ValFloat { tim, v := it.At() - _, err := headApp.Append(0, s.Labels(), tim, v) + _, err := headApp.Append(0, s.Labels(), 0, tim, v, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, it.Err()) @@ -1620,9 +1263,9 @@ func TestSizeRetention(t *testing.T) { require.Equal(t, expSize, actSize, "registered size doesn't match actual disk size") // Add some out of order samples to check the size of WBL. - headApp = db.Head().Appender(context.Background()) + headApp = db.Head().AppenderV2(context.Background()) for ts := int64(750); ts < 800; ts++ { - _, err := headApp.Append(0, aSeries, ts, float64(ts)) + _, err := headApp.Append(0, aSeries, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, headApp.Commit()) @@ -1668,93 +1311,7 @@ func TestSizeRetention(t *testing.T) { require.Equal(t, expBlocks[len(expBlocks)-1].MaxTime, actBlocks[len(actBlocks)-1].meta.MaxTime, "maxT mismatch of the last block") } -func TestSizeRetentionMetric(t *testing.T) { - cases := []struct { - maxBytes int64 - expMaxBytes int64 - }{ - {maxBytes: 1000, expMaxBytes: 1000}, - {maxBytes: 0, expMaxBytes: 0}, - {maxBytes: -1000, expMaxBytes: 0}, - } - - for _, c := range cases { - db := newTestDB(t, withOpts(&Options{ - MaxBytes: c.maxBytes, - }), withRngs(100)) - - actMaxBytes := int64(prom_testutil.ToFloat64(db.metrics.maxBytes)) - require.Equal(t, c.expMaxBytes, actMaxBytes, "metric retention limit bytes mismatch") - } -} - -// TestRuntimeRetentionConfigChange tests that retention configuration can be -// changed at runtime via ApplyConfig and that the retention logic properly -// deletes blocks when retention is shortened. This test also ensures race-free -// concurrent access to retention settings. -func TestRuntimeRetentionConfigChange(t *testing.T) { - const ( - initialRetentionDuration = int64(10 * time.Hour / time.Millisecond) // 10 hours - shorterRetentionDuration = int64(1 * time.Hour / time.Millisecond) // 1 hour - ) - - db := newTestDB(t, withOpts(&Options{ - RetentionDuration: initialRetentionDuration, - }), withRngs(100)) - - nineHoursMs := int64(9 * time.Hour / time.Millisecond) - nineAndHalfHoursMs := int64((9*time.Hour + 30*time.Minute) / time.Millisecond) - blocks := []*BlockMeta{ - {MinTime: 0, MaxTime: 100}, // 10 hours old (beyond new retention) - {MinTime: 100, MaxTime: 200}, // 9.9 hours old (beyond new retention) - {MinTime: nineHoursMs, MaxTime: nineAndHalfHoursMs}, // 1 hour old (within new retention) - {MinTime: nineAndHalfHoursMs, MaxTime: initialRetentionDuration}, // 0.5 hours old (within new retention) - } - - for _, m := range blocks { - createBlock(t, db.Dir(), genSeriesFromSampleGenerator(10, 10, m.MinTime, m.MaxTime, int64(time.Minute/time.Millisecond), func(ts int64) chunks.Sample { - return sample{t: ts, f: rand.Float64()} - })) - } - - // Reload blocks and verify all are loaded. - require.NoError(t, db.reloadBlocks()) - require.Len(t, db.Blocks(), len(blocks), "expected all blocks to be loaded initially") - - cfg := &config.Config{ - StorageConfig: config.StorageConfig{ - TSDBConfig: &config.TSDBConfig{ - Retention: &config.TSDBRetentionConfig{ - Time: model.Duration(shorterRetentionDuration), - }, - }, - }, - } - - require.NoError(t, db.ApplyConfig(cfg), "ApplyConfig should succeed") - - actualRetention := db.getRetentionDuration() - require.Equal(t, shorterRetentionDuration, actualRetention, "retention duration should be updated") - - expectedRetentionSeconds := (time.Duration(shorterRetentionDuration) * time.Millisecond).Seconds() - actualRetentionSeconds := prom_testutil.ToFloat64(db.metrics.retentionDuration) - require.Equal(t, expectedRetentionSeconds, actualRetentionSeconds, "retention duration metric should be updated") - - require.NoError(t, db.reloadBlocks()) - - // Verify that blocks beyond the new retention were deleted. - // We expect only the last 2 blocks to remain (those within 1 hour). - actBlocks := db.Blocks() - require.Len(t, actBlocks, 2, "expected old blocks to be deleted after retention change") - - // Verify the remaining blocks are the newest ones. - require.Equal(t, nineHoursMs, actBlocks[0].meta.MinTime, "first remaining block should be within retention") - require.Equal(t, nineAndHalfHoursMs, actBlocks[1].meta.MinTime, "last remaining block should be the newest") - - require.Positive(t, int(prom_testutil.ToFloat64(db.metrics.timeRetentionCount)), "time retention count should be incremented") -} - -func TestNotMatcherSelectsLabelsUnsetSeries(t *testing.T) { +func TestNotMatcherSelectsLabelsUnsetSeries_AppendV2(t *testing.T) { db := newTestDB(t) labelpairs := []labels.Labels{ @@ -1763,9 +1320,9 @@ func TestNotMatcherSelectsLabelsUnsetSeries(t *testing.T) { } ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) for _, lbls := range labelpairs { - _, err := app.Append(0, lbls, 0, 1) + _, err := app.Append(0, lbls, 0, 0, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -1825,133 +1382,21 @@ func TestNotMatcherSelectsLabelsUnsetSeries(t *testing.T) { } } -// expandSeriesSet returns the raw labels in the order they are retrieved from -// the series set and the samples keyed by Labels().String(). -func expandSeriesSet(ss storage.SeriesSet) ([]labels.Labels, map[string][]sample, annotations.Annotations, error) { - resultLabels := []labels.Labels{} - resultSamples := map[string][]sample{} - var it chunkenc.Iterator - for ss.Next() { - series := ss.At() - samples := []sample{} - it = series.Iterator(it) - for it.Next() == chunkenc.ValFloat { - t, v := it.At() - samples = append(samples, sample{t: t, f: v}) - } - resultLabels = append(resultLabels, series.Labels()) - resultSamples[series.Labels().String()] = samples - } - return resultLabels, resultSamples, ss.Warnings(), ss.Err() -} - -func TestOverlappingBlocksDetectsAllOverlaps(t *testing.T) { - // Create 10 blocks that does not overlap (0-10, 10-20, ..., 100-110) but in reverse order to ensure our algorithm - // will handle that. - metas := make([]BlockMeta, 11) - for i := 10; i >= 0; i-- { - metas[i] = BlockMeta{MinTime: int64(i * 10), MaxTime: int64((i + 1) * 10)} - } - - require.Empty(t, OverlappingBlocks(metas), "we found unexpected overlaps") - - // Add overlapping blocks. We've to establish order again since we aren't interested - // in trivial overlaps caused by unorderedness. - add := func(ms ...BlockMeta) []BlockMeta { - repl := append(append([]BlockMeta{}, metas...), ms...) - sort.Slice(repl, func(i, j int) bool { - return repl[i].MinTime < repl[j].MinTime - }) - return repl - } - - // o1 overlaps with 10-20. - o1 := BlockMeta{MinTime: 15, MaxTime: 17} - require.Equal(t, Overlaps{ - {Min: 15, Max: 17}: {metas[1], o1}, - }, OverlappingBlocks(add(o1))) - - // o2 overlaps with 20-30 and 30-40. - o2 := BlockMeta{MinTime: 21, MaxTime: 31} - require.Equal(t, Overlaps{ - {Min: 21, Max: 30}: {metas[2], o2}, - {Min: 30, Max: 31}: {o2, metas[3]}, - }, OverlappingBlocks(add(o2))) - - // o3a and o3b overlaps with 30-40 and each other. - o3a := BlockMeta{MinTime: 33, MaxTime: 39} - o3b := BlockMeta{MinTime: 34, MaxTime: 36} - require.Equal(t, Overlaps{ - {Min: 34, Max: 36}: {metas[3], o3a, o3b}, - }, OverlappingBlocks(add(o3a, o3b))) - - // o4 is 1:1 overlap with 50-60. - o4 := BlockMeta{MinTime: 50, MaxTime: 60} - require.Equal(t, Overlaps{ - {Min: 50, Max: 60}: {metas[5], o4}, - }, OverlappingBlocks(add(o4))) - - // o5 overlaps with 60-70, 70-80 and 80-90. - o5 := BlockMeta{MinTime: 61, MaxTime: 85} - require.Equal(t, Overlaps{ - {Min: 61, Max: 70}: {metas[6], o5}, - {Min: 70, Max: 80}: {o5, metas[7]}, - {Min: 80, Max: 85}: {o5, metas[8]}, - }, OverlappingBlocks(add(o5))) - - // o6a overlaps with 90-100, 100-110 and o6b, o6b overlaps with 90-100 and o6a. - o6a := BlockMeta{MinTime: 92, MaxTime: 105} - o6b := BlockMeta{MinTime: 94, MaxTime: 99} - require.Equal(t, Overlaps{ - {Min: 94, Max: 99}: {metas[9], o6a, o6b}, - {Min: 100, Max: 105}: {o6a, metas[10]}, - }, OverlappingBlocks(add(o6a, o6b))) - - // All together. - require.Equal(t, Overlaps{ - {Min: 15, Max: 17}: {metas[1], o1}, - {Min: 21, Max: 30}: {metas[2], o2}, {Min: 30, Max: 31}: {o2, metas[3]}, - {Min: 34, Max: 36}: {metas[3], o3a, o3b}, - {Min: 50, Max: 60}: {metas[5], o4}, - {Min: 61, Max: 70}: {metas[6], o5}, {Min: 70, Max: 80}: {o5, metas[7]}, {Min: 80, Max: 85}: {o5, metas[8]}, - {Min: 94, Max: 99}: {metas[9], o6a, o6b}, {Min: 100, Max: 105}: {o6a, metas[10]}, - }, OverlappingBlocks(add(o1, o2, o3a, o3b, o4, o5, o6a, o6b))) - - // Additional case. - var nc1 []BlockMeta - nc1 = append(nc1, BlockMeta{MinTime: 1, MaxTime: 5}) - nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) - nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) - nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) - nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 3}) - nc1 = append(nc1, BlockMeta{MinTime: 2, MaxTime: 6}) - nc1 = append(nc1, BlockMeta{MinTime: 3, MaxTime: 5}) - nc1 = append(nc1, BlockMeta{MinTime: 5, MaxTime: 7}) - nc1 = append(nc1, BlockMeta{MinTime: 7, MaxTime: 10}) - nc1 = append(nc1, BlockMeta{MinTime: 8, MaxTime: 9}) - require.Equal(t, Overlaps{ - {Min: 2, Max: 3}: {nc1[0], nc1[1], nc1[2], nc1[3], nc1[4], nc1[5]}, // 1-5, 2-3, 2-3, 2-3, 2-3, 2,6 - {Min: 3, Max: 5}: {nc1[0], nc1[5], nc1[6]}, // 1-5, 2-6, 3-5 - {Min: 5, Max: 6}: {nc1[5], nc1[7]}, // 2-6, 5-7 - {Min: 8, Max: 9}: {nc1[8], nc1[9]}, // 7-10, 8-9 - }, OverlappingBlocks(nc1)) -} - // Regression test for https://github.com/prometheus/tsdb/issues/347 -func TestChunkAtBlockBoundary(t *testing.T) { +func TestChunkAtBlockBoundary_AppendV2(t *testing.T) { t.Parallel() db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) blockRange := db.compactor.(*LeveledCompactor).ranges[0] label := labels.FromStrings("foo", "bar") for i := range int64(3) { - _, err := app.Append(0, label, i*blockRange, 0) + _, err := app.Append(0, label, 0, i*blockRange, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, label, i*blockRange+1000, 0) + _, err = app.Append(0, label, 0, i*blockRange+1000, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } @@ -1992,20 +1437,20 @@ func TestChunkAtBlockBoundary(t *testing.T) { } } -func TestQuerierWithBoundaryChunks(t *testing.T) { +func TestQuerierWithBoundaryChunks_AppendV2(t *testing.T) { t.Parallel() db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) blockRange := db.compactor.(*LeveledCompactor).ranges[0] label := labels.FromStrings("foo", "bar") for i := range int64(5) { - _, err := app.Append(0, label, i*blockRange, 0) + _, err := app.Append(0, label, 0, i*blockRange, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, labels.FromStrings("blockID", strconv.FormatInt(i, 10)), i*blockRange, 0) + _, err = app.Append(0, labels.FromStrings("blockID", strconv.FormatInt(i, 10)), 0, i*blockRange, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } @@ -2034,7 +1479,7 @@ func TestQuerierWithBoundaryChunks(t *testing.T) { // - no blocks with WAL: set to the smallest sample from the WAL // - with blocks no WAL: set to the last block maxT // - with blocks with WAL: same as above -func TestInitializeHeadTimestamp(t *testing.T) { +func TestInitializeHeadTimestamp_AppendV2(t *testing.T) { t.Parallel() t.Run("clean", func(t *testing.T) { db := newTestDB(t) @@ -2046,8 +1491,8 @@ func TestInitializeHeadTimestamp(t *testing.T) { // First added sample initializes the writable range. ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("a", "b"), 1000, 1) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 1000, 1, nil, nil, storage.AOptions{}) require.NoError(t, err) require.Equal(t, int64(1000), db.head.MinTime()) @@ -2125,7 +1570,7 @@ func TestInitializeHeadTimestamp(t *testing.T) { }) } -func TestNoEmptyBlocks(t *testing.T) { +func TestNoEmptyBlocks_AppendV2(t *testing.T) { t.Parallel() db := newTestDB(t, withRngs(100)) ctx := context.Background() @@ -2146,12 +1591,12 @@ func TestNoEmptyBlocks(t *testing.T) { }) t.Run("Test no blocks after deleting all samples from head.", func(t *testing.T) { - app := db.Appender(ctx) - _, err := app.Append(0, defaultLabel, 1, 0) + app := db.AppenderV2(ctx) + _, err := app.Append(0, defaultLabel, 0, 1, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, defaultLabel, 2, 0) + _, err = app.Append(0, defaultLabel, 0, 2, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, defaultLabel, 3+rangeToTriggerCompaction, 0) + _, err = app.Append(0, defaultLabel, 0, 3+rangeToTriggerCompaction, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.NoError(t, db.Delete(ctx, math.MinInt64, math.MaxInt64, defaultMatcher)) @@ -2163,17 +1608,17 @@ func TestNoEmptyBlocks(t *testing.T) { require.Len(t, actBlocks, len(db.Blocks())) require.Empty(t, actBlocks) - app = db.Appender(ctx) - _, err = app.Append(0, defaultLabel, 1, 0) + app = db.AppenderV2(ctx) + _, err = app.Append(0, defaultLabel, 0, 1, 0, nil, nil, storage.AOptions{}) require.Equal(t, storage.ErrOutOfBounds, err, "the head should be truncated so no samples in the past should be allowed") // Adding new blocks. currentTime := db.Head().MaxTime() - _, err = app.Append(0, defaultLabel, currentTime, 0) + _, err = app.Append(0, defaultLabel, 0, currentTime, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, defaultLabel, currentTime+1, 0) + _, err = app.Append(0, defaultLabel, 0, currentTime+1, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, defaultLabel, currentTime+rangeToTriggerCompaction, 0) + _, err = app.Append(0, defaultLabel, 0, currentTime+rangeToTriggerCompaction, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -2188,13 +1633,13 @@ func TestNoEmptyBlocks(t *testing.T) { t.Run(`When no new block is created from head, and there are some blocks on disk compaction should not run into infinite loop (was seen during development).`, func(t *testing.T) { oldBlocks := db.Blocks() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) currentTime := db.Head().MaxTime() - _, err := app.Append(0, defaultLabel, currentTime, 0) + _, err := app.Append(0, defaultLabel, 0, currentTime, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, defaultLabel, currentTime+1, 0) + _, err = app.Append(0, defaultLabel, 0, currentTime+1, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, defaultLabel, currentTime+rangeToTriggerCompaction, 0) + _, err = app.Append(0, defaultLabel, 0, currentTime+rangeToTriggerCompaction, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.NoError(t, db.head.Delete(ctx, math.MinInt64, math.MaxInt64, defaultMatcher)) @@ -2227,7 +1672,7 @@ func TestNoEmptyBlocks(t *testing.T) { }) } -func TestDB_LabelNames(t *testing.T) { +func TestDB_LabelNames_AppendV2(t *testing.T) { ctx := context.Background() tests := []struct { // Add 'sampleLabels1' -> Test Head -> Compact -> Test Disk -> @@ -2272,11 +1717,11 @@ func TestDB_LabelNames(t *testing.T) { // Appends samples into the database. appendSamples := func(db *DB, mint, maxt int64, sampleLabels [][2]string) { t.Helper() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) for i := mint; i <= maxt; i++ { for _, tuple := range sampleLabels { label := labels.FromStrings(tuple[0], tuple[1]) - _, err := app.Append(0, label, i*blockRange, 0) + _, err := app.Append(0, label, 0, i*blockRange, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } } @@ -2329,7 +1774,7 @@ func TestDB_LabelNames(t *testing.T) { } } -func TestCorrectNumTombstones(t *testing.T) { +func TestCorrectNumTombstones_AppendV2(t *testing.T) { t.Parallel() db := newTestDB(t) @@ -2339,10 +1784,10 @@ func TestCorrectNumTombstones(t *testing.T) { defaultMatcher := labels.MustNewMatcher(labels.MatchEqual, name, value) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) for i := range int64(3) { for j := range int64(15) { - _, err := app.Append(0, defaultLabel, i*blockRange+j, 0) + _, err := app.Append(0, defaultLabel, 0, i*blockRange+j, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } } @@ -2377,7 +1822,7 @@ func TestCorrectNumTombstones(t *testing.T) { // // This ensures that a snapshot that includes the head and creates a block with a custom time range // will not overlap with the first block created by the next compaction. -func TestBlockRanges(t *testing.T) { +func TestBlockRanges_AppendV2(t *testing.T) { t.Parallel() logger := promslog.New(&promslog.Config{}) ctx := context.Background() @@ -2393,16 +1838,16 @@ func TestBlockRanges(t *testing.T) { rangeToTriggerCompaction := db.compactor.(*LeveledCompactor).ranges[0]/2*3 + 1 - app := db.Appender(ctx) + app := db.AppenderV2(ctx) lbl := labels.FromStrings("a", "b") - _, err = app.Append(0, lbl, firstBlockMaxT-1, rand.Float64()) + _, err = app.Append(0, lbl, 0, firstBlockMaxT-1, rand.Float64(), nil, nil, storage.AOptions{}) require.Error(t, err, "appending a sample with a timestamp covered by a previous block shouldn't be possible") - _, err = app.Append(0, lbl, firstBlockMaxT+1, rand.Float64()) + _, err = app.Append(0, lbl, 0, firstBlockMaxT+1, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, lbl, firstBlockMaxT+2, rand.Float64()) + _, err = app.Append(0, lbl, 0, firstBlockMaxT+2, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) secondBlockMaxt := firstBlockMaxT + rangeToTriggerCompaction - _, err = app.Append(0, lbl, secondBlockMaxt, rand.Float64()) // Add samples to trigger a new compaction + _, err = app.Append(0, lbl, 0, secondBlockMaxt, rand.Float64(), nil, nil, storage.AOptions{}) // Add samples to trigger a new compaction require.NoError(t, err) require.NoError(t, app.Commit()) @@ -2419,15 +1864,15 @@ func TestBlockRanges(t *testing.T) { // Test that wal records are skipped when an existing block covers the same time ranges // and compaction doesn't create an overlapping block. - app = db.Appender(ctx) + app = db.AppenderV2(ctx) db.DisableCompactions() - _, err = app.Append(0, lbl, secondBlockMaxt+1, rand.Float64()) + _, err = app.Append(0, lbl, 0, secondBlockMaxt+1, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, lbl, secondBlockMaxt+2, rand.Float64()) + _, err = app.Append(0, lbl, 0, secondBlockMaxt+2, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, lbl, secondBlockMaxt+3, rand.Float64()) + _, err = app.Append(0, lbl, 0, secondBlockMaxt+3, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, lbl, secondBlockMaxt+4, rand.Float64()) + _, err = app.Append(0, lbl, 0, secondBlockMaxt+4, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) require.NoError(t, db.Close()) @@ -2442,8 +1887,8 @@ func TestBlockRanges(t *testing.T) { require.Len(t, db.Blocks(), 3, "db doesn't include expected number of blocks") require.Equal(t, db.Blocks()[2].Meta().MaxTime, thirdBlockMaxt, "unexpected maxt of the last block") - app = db.Appender(ctx) - _, err = app.Append(0, lbl, thirdBlockMaxt+rangeToTriggerCompaction, rand.Float64()) // Trigger a compaction + app = db.AppenderV2(ctx) + _, err = app.Append(0, lbl, 0, thirdBlockMaxt+rangeToTriggerCompaction, rand.Float64(), nil, nil, storage.AOptions{}) // Trigger a compaction require.NoError(t, err) require.NoError(t, app.Commit()) for range 100 { @@ -2461,10 +1906,11 @@ func TestBlockRanges(t *testing.T) { // TestDBReadOnly ensures that opening a DB in readonly mode doesn't modify any files on the disk. // It also checks that the API calls return equivalent results as a normal db.Open() mode. -func TestDBReadOnly(t *testing.T) { +func TestDBReadOnly_AppendV2(t *testing.T) { t.Parallel() var ( dbDir = t.TempDir() + logger = promslog.New(&promslog.Config{}) expBlocks []*Block expBlock *Block expSeries map[string][]chunks.Sample @@ -2488,7 +1934,7 @@ func TestDBReadOnly(t *testing.T) { } // Add head to test DBReadOnly WAL reading capabilities. - w, err := wlog.New(nil, nil, filepath.Join(dbDir, "wal"), compression.Snappy) + w, err := wlog.New(logger, nil, filepath.Join(dbDir, "wal"), compression.Snappy) require.NoError(t, err) h := createHead(t, w, genSeries(1, 1, 16, 18), dbDir) require.NoError(t, h.Close()) @@ -2501,8 +1947,8 @@ func TestDBReadOnly(t *testing.T) { dbSizeBeforeAppend, err := fileutil.DirSize(dbWritable.Dir()) require.NoError(t, err) - app := dbWritable.Appender(context.Background()) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), dbWritable.Head().MaxTime()+1, 0) + app := dbWritable.AppenderV2(context.Background()) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, dbWritable.Head().MaxTime()+1, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -2524,7 +1970,7 @@ func TestDBReadOnly(t *testing.T) { } // Open a read only db and ensure that the API returns the same result as the normal DB. - dbReadOnly, err := OpenDBReadOnly(dbDir, "", nil) + dbReadOnly, err := OpenDBReadOnly(dbDir, "", logger) require.NoError(t, err) defer func() { require.NoError(t, dbReadOnly.Close()) }() @@ -2575,32 +2021,14 @@ func TestDBReadOnly(t *testing.T) { }) } -// TestDBReadOnlyClosing ensures that after closing the db -// all api methods return an ErrClosed. -func TestDBReadOnlyClosing(t *testing.T) { - t.Parallel() - sandboxDir := t.TempDir() - db, err := OpenDBReadOnly(t.TempDir(), sandboxDir, promslog.New(&promslog.Config{})) - require.NoError(t, err) - // The sandboxDir was there. - require.DirExists(t, db.sandboxDir) - require.NoError(t, db.Close()) - // The sandboxDir was deleted when closing. - require.NoDirExists(t, db.sandboxDir) - require.Equal(t, db.Close(), ErrClosed) - _, err = db.Blocks() - require.Equal(t, err, ErrClosed) - _, err = db.Querier(0, 1) - require.Equal(t, err, ErrClosed) -} - -func TestDBReadOnly_FlushWAL(t *testing.T) { +func TestDBReadOnly_FlushWAL_AppendV2(t *testing.T) { t.Parallel() var ( - dbDir = t.TempDir() - err error - maxt int - ctx = context.Background() + dbDir = t.TempDir() + logger = promslog.New(&promslog.Config{}) + err error + maxt int + ctx = context.Background() ) // Bootstrap the db. @@ -2608,10 +2036,10 @@ func TestDBReadOnly_FlushWAL(t *testing.T) { // Append data to the WAL. db := newTestDB(t, withDir(dbDir)) db.DisableCompactions() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) maxt = 1000 for i := 0; i < maxt; i++ { - _, err := app.Append(0, labels.FromStrings(defaultLabelName, "flush"), int64(i), 1.0) + _, err := app.Append(0, labels.FromStrings(defaultLabelName, "flush"), 0, int64(i), 1.0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -2619,7 +2047,7 @@ func TestDBReadOnly_FlushWAL(t *testing.T) { } // Flush WAL. - db, err := OpenDBReadOnly(dbDir, "", nil) + db, err := OpenDBReadOnly(dbDir, "", logger) require.NoError(t, err) flush := t.TempDir() @@ -2627,7 +2055,7 @@ func TestDBReadOnly_FlushWAL(t *testing.T) { require.NoError(t, db.Close()) // Reopen the DB from the flushed WAL block. - db, err = OpenDBReadOnly(flush, "", nil) + db, err = OpenDBReadOnly(flush, "", logger) require.NoError(t, err) defer func() { require.NoError(t, db.Close()) }() blocks, err := db.Blocks() @@ -2656,7 +2084,7 @@ func TestDBReadOnly_FlushWAL(t *testing.T) { require.Equal(t, 1000.0, sum) } -func TestDBReadOnly_Querier_NoAlteration(t *testing.T) { +func TestDBReadOnly_Querier_NoAlteration_AppendV2(t *testing.T) { countChunks := func(dir string) int { files, err := os.ReadDir(mmappedChunksDir(dir)) require.NoError(t, err) @@ -2692,8 +2120,8 @@ func TestDBReadOnly_Querier_NoAlteration(t *testing.T) { // Append until the first mmapped head chunk. for i := range 121 { - app := db.Appender(context.Background()) - _, err := app.Append(0, labels.FromStrings("foo", "bar"), int64(i), 0) + app := db.AppenderV2(context.Background()) + _, err := app.Append(0, labels.FromStrings("foo", "bar"), 0, int64(i), 0, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) } @@ -2726,7 +2154,7 @@ func TestDBReadOnly_Querier_NoAlteration(t *testing.T) { }) } -func TestDBCannotSeePartialCommits(t *testing.T) { +func TestDBCannotSeePartialCommits_AppendV2(t *testing.T) { if defaultIsolationDisabled { t.Skip("skipping test since tsdb isolation is disabled") } @@ -2741,10 +2169,10 @@ func TestDBCannotSeePartialCommits(t *testing.T) { go func() { iter := 0 for { - app := db.Appender(ctx) + app := db.AppenderV2(ctx) for j := range 100 { - _, err := app.Append(0, labels.FromStrings("foo", "bar", "a", strconv.Itoa(j)), int64(iter), float64(iter)) + _, err := app.Append(0, labels.FromStrings("foo", "bar", "a", strconv.Itoa(j)), 0, int64(iter), float64(iter), nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -2792,7 +2220,7 @@ func TestDBCannotSeePartialCommits(t *testing.T) { require.Equal(t, 0, inconsistencies, "Some queries saw inconsistent results.") } -func TestDBQueryDoesntSeeAppendsAfterCreation(t *testing.T) { +func TestDBQueryDoesntSeeAppendsAfterCreation_AppendV2(t *testing.T) { if defaultIsolationDisabled { t.Skip("skipping test since tsdb isolation is disabled") } @@ -2803,8 +2231,8 @@ func TestDBQueryDoesntSeeAppendsAfterCreation(t *testing.T) { defer querierBeforeAdd.Close() ctx := context.Background() - app := db.Appender(ctx) - _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 0) + app := db.AppenderV2(ctx) + _, err = app.Append(0, labels.FromStrings("foo", "bar"), 0, 0, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) querierAfterAddButBeforeCommit, err := db.Querier(0, 1000000) @@ -2854,257 +2282,6 @@ func TestDBQueryDoesntSeeAppendsAfterCreation(t *testing.T) { require.Equal(t, map[string][]sample{`{foo="bar"}`: {{t: 0, f: 0}}}, seriesSet) } -func assureChunkFromSamples(t *testing.T, samples []chunks.Sample) chunks.Meta { - chks, err := chunks.ChunkFromSamples(samples) - require.NoError(t, err) - return chks -} - -// TestChunkWriter_ReadAfterWrite ensures that chunk segment are cut at the set segment size and -// that the resulted segments includes the expected chunks data. -func TestChunkWriter_ReadAfterWrite(t *testing.T) { - chk1 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 1, nil, nil}}) - chk2 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 2, nil, nil}}) - chk3 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 3, nil, nil}}) - chk4 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 4, nil, nil}}) - chk5 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 5, nil, nil}}) - chunkSize := len(chk1.Chunk.Bytes()) + chunks.MaxChunkLengthFieldSize + chunks.ChunkEncodingSize + crc32.Size - - tests := []struct { - chks [][]chunks.Meta - segmentSize, - expSegmentsCount int - expSegmentSizes []int - }{ - // 0:Last chunk ends at the segment boundary so - // all chunks should fit in a single segment. - { - chks: [][]chunks.Meta{ - { - chk1, - chk2, - chk3, - }, - }, - segmentSize: 3 * chunkSize, - expSegmentSizes: []int{3 * chunkSize}, - expSegmentsCount: 1, - }, - // 1:Two chunks can fit in a single segment so the last one should result in a new segment. - { - chks: [][]chunks.Meta{ - { - chk1, - chk2, - chk3, - chk4, - chk5, - }, - }, - segmentSize: 2 * chunkSize, - expSegmentSizes: []int{2 * chunkSize, 2 * chunkSize, chunkSize}, - expSegmentsCount: 3, - }, - // 2:When the segment size is smaller than the size of 2 chunks - // the last segment should still create a new segment. - { - chks: [][]chunks.Meta{ - { - chk1, - chk2, - chk3, - }, - }, - segmentSize: 2*chunkSize - 1, - expSegmentSizes: []int{chunkSize, chunkSize, chunkSize}, - expSegmentsCount: 3, - }, - // 3:When the segment is smaller than a single chunk - // it should still be written by ignoring the max segment size. - { - chks: [][]chunks.Meta{ - { - chk1, - }, - }, - segmentSize: chunkSize - 1, - expSegmentSizes: []int{chunkSize}, - expSegmentsCount: 1, - }, - // 4:All chunks are bigger than the max segment size, but - // these should still be written even when this will result in bigger segment than the set size. - // Each segment will hold a single chunk. - { - chks: [][]chunks.Meta{ - { - chk1, - chk2, - chk3, - }, - }, - segmentSize: 1, - expSegmentSizes: []int{chunkSize, chunkSize, chunkSize}, - expSegmentsCount: 3, - }, - // 5:Adding multiple batches of chunks. - { - chks: [][]chunks.Meta{ - { - chk1, - chk2, - chk3, - }, - { - chk4, - chk5, - }, - }, - segmentSize: 3 * chunkSize, - expSegmentSizes: []int{3 * chunkSize, 2 * chunkSize}, - expSegmentsCount: 2, - }, - // 6:Adding multiple batches of chunks. - { - chks: [][]chunks.Meta{ - { - chk1, - }, - { - chk2, - chk3, - }, - { - chk4, - }, - }, - segmentSize: 2 * chunkSize, - expSegmentSizes: []int{2 * chunkSize, 2 * chunkSize}, - expSegmentsCount: 2, - }, - } - - for i, test := range tests { - t.Run(strconv.Itoa(i), func(t *testing.T) { - tempDir := t.TempDir() - - chunkw, err := chunks.NewWriter(tempDir, chunks.WithSegmentSize(chunks.SegmentHeaderSize+int64(test.segmentSize))) - require.NoError(t, err) - - for _, chks := range test.chks { - require.NoError(t, chunkw.WriteChunks(chks...)) - } - require.NoError(t, chunkw.Close()) - - files, err := os.ReadDir(tempDir) - require.NoError(t, err) - require.Len(t, files, test.expSegmentsCount, "expected segments count mismatch") - - // Verify that all data is written to the segments. - sizeExp := 0 - sizeAct := 0 - - for _, chks := range test.chks { - for _, chk := range chks { - l := make([]byte, binary.MaxVarintLen32) - sizeExp += binary.PutUvarint(l, uint64(len(chk.Chunk.Bytes()))) // The length field. - sizeExp += chunks.ChunkEncodingSize - sizeExp += len(chk.Chunk.Bytes()) // The data itself. - sizeExp += crc32.Size // The 4 bytes of crc32 - } - } - sizeExp += test.expSegmentsCount * chunks.SegmentHeaderSize // The segment header bytes. - - for i, f := range files { - fi, err := f.Info() - require.NoError(t, err) - size := int(fi.Size()) - // Verify that the segment is the same or smaller than the expected size. - require.GreaterOrEqual(t, chunks.SegmentHeaderSize+test.expSegmentSizes[i], size, "Segment:%v should NOT be bigger than:%v actual:%v", i, chunks.SegmentHeaderSize+test.expSegmentSizes[i], size) - - sizeAct += size - } - require.Equal(t, sizeExp, sizeAct) - - // Check the content of the chunks. - r, err := chunks.NewDirReader(tempDir, nil) - require.NoError(t, err) - defer func() { require.NoError(t, r.Close()) }() - - for _, chks := range test.chks { - for _, chkExp := range chks { - chkAct, iterable, err := r.ChunkOrIterable(chkExp) - require.NoError(t, err) - require.Nil(t, iterable) - require.Equal(t, chkExp.Chunk.Bytes(), chkAct.Bytes()) - } - } - }) - } -} - -func TestRangeForTimestamp(t *testing.T) { - type args struct { - t int64 - width int64 - } - tests := []struct { - args args - expected int64 - }{ - {args{0, 5}, 5}, - {args{1, 5}, 5}, - {args{5, 5}, 10}, - {args{6, 5}, 10}, - {args{13, 5}, 15}, - {args{95, 5}, 100}, - } - for _, tt := range tests { - got := rangeForTimestamp(tt.args.t, tt.args.width) - require.Equal(t, tt.expected, got) - } -} - -// TestChunkReader_ConcurrentReads checks that the chunk result can be read concurrently. -// Regression test for https://github.com/prometheus/prometheus/pull/6514. -func TestChunkReader_ConcurrentReads(t *testing.T) { - t.Parallel() - chks := []chunks.Meta{ - assureChunkFromSamples(t, []chunks.Sample{sample{1, 1, nil, nil}}), - assureChunkFromSamples(t, []chunks.Sample{sample{1, 2, nil, nil}}), - assureChunkFromSamples(t, []chunks.Sample{sample{1, 3, nil, nil}}), - assureChunkFromSamples(t, []chunks.Sample{sample{1, 4, nil, nil}}), - assureChunkFromSamples(t, []chunks.Sample{sample{1, 5, nil, nil}}), - } - - tempDir := t.TempDir() - - chunkw, err := chunks.NewWriter(tempDir) - require.NoError(t, err) - - require.NoError(t, chunkw.WriteChunks(chks...)) - require.NoError(t, chunkw.Close()) - - r, err := chunks.NewDirReader(tempDir, nil) - require.NoError(t, err) - - var wg sync.WaitGroup - for _, chk := range chks { - for range 100 { - wg.Add(1) - go func(chunk chunks.Meta) { - defer wg.Done() - - chkAct, iterable, err := r.ChunkOrIterable(chunk) - require.NoError(t, err) - require.Nil(t, iterable) - require.Equal(t, chunk.Chunk.Bytes(), chkAct.Bytes()) - }(chk) - } - wg.Wait() - } - require.NoError(t, r.Close()) -} - // TestCompactHead ensures that the head compaction // creates a block that is ready for loading and // does not cause data loss. @@ -3113,7 +2290,7 @@ func TestChunkReader_ConcurrentReads(t *testing.T) { // * appends values; // * compacts the head; and // * queries the db to ensure the samples are present from the compacted head. -func TestCompactHead(t *testing.T) { +func TestCompactHead_AppendV2(t *testing.T) { t.Parallel() // Open a DB and append data to the WAL. @@ -3126,12 +2303,12 @@ func TestCompactHead(t *testing.T) { } db := newTestDB(t, withOpts(opts)) ctx := context.Background() - app := db.Appender(ctx) + app := db.AppenderV2(ctx) var expSamples []sample maxt := 100 for i := range maxt { val := rand.Float64() - _, err := app.Append(0, labels.FromStrings("a", "b"), int64(i), val) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, int64(i), val, nil, nil, storage.AOptions{}) require.NoError(t, err) expSamples = append(expSamples, sample{int64(i), val, nil, nil}) } @@ -3169,13 +2346,13 @@ func TestCompactHead(t *testing.T) { } // TestCompactHeadWithDeletion tests https://github.com/prometheus/prometheus/issues/11585. -func TestCompactHeadWithDeletion(t *testing.T) { +func TestCompactHeadWithDeletion_AppendV2(t *testing.T) { db := newTestDB(t) ctx := context.Background() - app := db.Appender(ctx) - _, err := app.Append(0, labels.FromStrings("a", "b"), 10, rand.Float64()) + app := db.AppenderV2(ctx) + _, err := app.Append(0, labels.FromStrings("a", "b"), 0, 10, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -3184,146 +2361,10 @@ func TestCompactHeadWithDeletion(t *testing.T) { // This recreates the bug. require.NoError(t, db.CompactHead(NewRangeHead(db.Head(), 0, 100))) -} - -func deleteNonBlocks(dbDir string) error { - dirs, err := os.ReadDir(dbDir) - if err != nil { - return err - } - for _, dir := range dirs { - if ok := isBlockDir(dir); !ok { - if err := os.RemoveAll(filepath.Join(dbDir, dir.Name())); err != nil { - return err - } - } - } - dirs, err = os.ReadDir(dbDir) - if err != nil { - return err - } - for _, dir := range dirs { - if ok := isBlockDir(dir); !ok { - return fmt.Errorf("root folder:%v still hase non block directory:%v", dbDir, dir.Name()) - } - } - return nil -} - -func TestOpen_VariousBlockStates(t *testing.T) { - tmpDir := t.TempDir() - - var ( - expectedLoadedDirs = map[string]struct{}{} - expectedRemovedDirs = map[string]struct{}{} - expectedIgnoredDirs = map[string]struct{}{} - ) - - { - // Ok blocks; should be loaded. - expectedLoadedDirs[createBlock(t, tmpDir, genSeries(10, 2, 0, 10))] = struct{}{} - expectedLoadedDirs[createBlock(t, tmpDir, genSeries(10, 2, 10, 20))] = struct{}{} - } - { - // Block to repair; should be repaired & loaded. - dbDir := filepath.Join("testdata", "repair_index_version", "01BZJ9WJQPWHGNC2W4J9TA62KC") - outDir := filepath.Join(tmpDir, "01BZJ9WJQPWHGNC2W4J9TA62KC") - expectedLoadedDirs[outDir] = struct{}{} - - // Touch chunks dir in block. - require.NoError(t, os.MkdirAll(filepath.Join(dbDir, "chunks"), 0o777)) - defer func() { - require.NoError(t, os.RemoveAll(filepath.Join(dbDir, "chunks"))) - }() - require.NoError(t, os.Mkdir(outDir, os.ModePerm)) - require.NoError(t, fileutil.CopyDirs(dbDir, outDir)) - } - { - // Missing meta.json; should be ignored and only logged. - // TODO(bwplotka): Probably add metric. - dir := createBlock(t, tmpDir, genSeries(10, 2, 20, 30)) - expectedIgnoredDirs[dir] = struct{}{} - require.NoError(t, os.Remove(filepath.Join(dir, metaFilename))) - } - { - // Tmp blocks during creation; those should be removed on start. - dir := createBlock(t, tmpDir, genSeries(10, 2, 30, 40)) - require.NoError(t, fileutil.Replace(dir, dir+tmpForCreationBlockDirSuffix)) - expectedRemovedDirs[dir+tmpForCreationBlockDirSuffix] = struct{}{} - - // Tmp blocks during deletion; those should be removed on start. - dir = createBlock(t, tmpDir, genSeries(10, 2, 40, 50)) - require.NoError(t, fileutil.Replace(dir, dir+tmpForDeletionBlockDirSuffix)) - expectedRemovedDirs[dir+tmpForDeletionBlockDirSuffix] = struct{}{} - - // Pre-2.21 tmp blocks; those should be removed on start. - dir = createBlock(t, tmpDir, genSeries(10, 2, 50, 60)) - require.NoError(t, fileutil.Replace(dir, dir+tmpLegacy)) - expectedRemovedDirs[dir+tmpLegacy] = struct{}{} - } - { - // One ok block; but two should be replaced. - dir := createBlock(t, tmpDir, genSeries(10, 2, 50, 60)) - expectedLoadedDirs[dir] = struct{}{} - - m, _, err := readMetaFile(dir) - require.NoError(t, err) - - compacted := createBlock(t, tmpDir, genSeries(10, 2, 50, 55)) - expectedRemovedDirs[compacted] = struct{}{} - - m.Compaction.Parents = append(m.Compaction.Parents, - BlockDesc{ULID: ulid.MustParse(filepath.Base(compacted))}, - BlockDesc{ULID: ulid.MustNew(1, nil)}, - BlockDesc{ULID: ulid.MustNew(123, nil)}, - ) - - // Regression test: Already removed parent can be still in list, which was causing Open errors. - m.Compaction.Parents = append(m.Compaction.Parents, BlockDesc{ULID: ulid.MustParse(filepath.Base(compacted))}) - m.Compaction.Parents = append(m.Compaction.Parents, BlockDesc{ULID: ulid.MustParse(filepath.Base(compacted))}) - _, err = writeMetaFile(promslog.New(&promslog.Config{}), dir, m) - require.NoError(t, err) - } - tmpCheckpointDir := path.Join(tmpDir, "wal/checkpoint.00000001.tmp") - err := os.MkdirAll(tmpCheckpointDir, 0o777) - require.NoError(t, err) - tmpChunkSnapshotDir := path.Join(tmpDir, chunkSnapshotPrefix+"0000.00000001.tmp") - err = os.MkdirAll(tmpChunkSnapshotDir, 0o777) - require.NoError(t, err) - - opts := DefaultOptions() - opts.RetentionDuration = 0 - db := newTestDB(t, withDir(tmpDir), withOpts(opts)) - loadedBlocks := db.Blocks() - - var loaded int - for _, l := range loadedBlocks { - _, ok := expectedLoadedDirs[filepath.Join(tmpDir, l.meta.ULID.String())] - require.True(t, ok, "unexpected block", l.meta.ULID, "was loaded") - loaded++ - } - require.Len(t, expectedLoadedDirs, loaded) require.NoError(t, db.Close()) - - files, err := os.ReadDir(tmpDir) - require.NoError(t, err) - - var ignored int - for _, f := range files { - _, ok := expectedRemovedDirs[filepath.Join(tmpDir, f.Name())] - require.False(t, ok, "expected", filepath.Join(tmpDir, f.Name()), "to be removed, but still exists") - if _, ok := expectedIgnoredDirs[filepath.Join(tmpDir, f.Name())]; ok { - ignored++ - } - } - require.Len(t, expectedIgnoredDirs, ignored) - _, err = os.Stat(tmpCheckpointDir) - require.True(t, os.IsNotExist(err)) - _, err = os.Stat(tmpChunkSnapshotDir) - require.True(t, os.IsNotExist(err)) } -func TestOneCheckpointPerCompactCall(t *testing.T) { +func TestOneCheckpointPerCompactCall_AppendV2(t *testing.T) { t.Parallel() blockRange := int64(1000) opts := &Options{ @@ -3342,11 +2383,11 @@ func TestOneCheckpointPerCompactCall(t *testing.T) { lbls := labels.FromStrings("foo_d", "choco_bar") // Append samples spanning 59 block ranges. - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for i := range int64(60) { - _, err := app.Append(0, lbls, blockRange*i, rand.Float64()) + _, err := app.Append(0, lbls, 0, blockRange*i, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.Append(0, lbls, (blockRange*i)+blockRange/2, rand.Float64()) + _, err = app.Append(0, lbls, 0, (blockRange*i)+blockRange/2, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) // Rotate the WAL file so that there is >3 files for checkpoint to happen. _, err = db.head.wal.NextSegment() @@ -3402,8 +2443,8 @@ func TestOneCheckpointPerCompactCall(t *testing.T) { require.Equal(t, 0, int(db.head.NumSeries())) // Adding sample way into the future. - app = db.Appender(context.Background()) - _, err = app.Append(0, lbls, blockRange*120, rand.Float64()) + app = db.AppenderV2(context.Background()) + _, err = app.Append(0, lbls, 0, blockRange*120, rand.Float64(), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -3437,288 +2478,7 @@ func TestOneCheckpointPerCompactCall(t *testing.T) { require.Equal(t, 54, cno) } -func TestNoPanicOnTSDBOpenError(t *testing.T) { - tmpdir := t.TempDir() - - // Taking the lock will cause a TSDB startup error. - l, err := tsdbutil.NewDirLocker(tmpdir, "tsdb", promslog.NewNopLogger(), nil) - require.NoError(t, err) - require.NoError(t, l.Lock()) - - _, err = Open(tmpdir, nil, nil, DefaultOptions(), nil) - require.Error(t, err) - - require.NoError(t, l.Release()) -} - -func TestLockfile(t *testing.T) { - tsdbutil.TestDirLockerUsage(t, func(t *testing.T, data string, createLock bool) (*tsdbutil.DirLocker, testutil.Closer) { - opts := DefaultOptions() - opts.NoLockfile = !createLock - - // Create the DB. This should create lockfile and its metrics. - db, err := Open(data, nil, nil, opts, nil) - require.NoError(t, err) - - return db.locker, testutil.NewCallbackCloser(func() { - require.NoError(t, db.Close()) - }) - }) -} - -func TestQuerier_ShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { - t.Skip("TODO: investigate why process crash in CI") - - const numRuns = 5 - - for i := 1; i <= numRuns; i++ { - t.Run(strconv.Itoa(i), func(t *testing.T) { - testQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t) - }) - } -} - -func testQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { - const ( - numSeries = 1000 - numStressIterations = 10000 - minStressAllocationBytes = 128 * 1024 - maxStressAllocationBytes = 512 * 1024 - ) - - db := newTestDB(t) - - // Disable compactions so we can control it. - db.DisableCompactions() - - // Generate the metrics we're going to append. - metrics := make([]labels.Labels, 0, numSeries) - for i := range numSeries { - metrics = append(metrics, labels.FromStrings(labels.MetricName, fmt.Sprintf("test_%d", i))) - } - - // Push 1 sample every 15s for 2x the block duration period. - ctx := context.Background() - interval := int64(15 * time.Second / time.Millisecond) - ts := int64(0) - - for ; ts < 2*DefaultBlockDuration; ts += interval { - app := db.Appender(ctx) - - for _, metric := range metrics { - _, err := app.Append(0, metric, ts, float64(ts)) - require.NoError(t, err) - } - - require.NoError(t, app.Commit()) - } - - // Compact the TSDB head for the first time. We expect the head chunks file has been cut. - require.NoError(t, db.Compact(ctx)) - require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) - - // Push more samples for another 1x block duration period. - for ; ts < 3*DefaultBlockDuration; ts += interval { - app := db.Appender(ctx) - - for _, metric := range metrics { - _, err := app.Append(0, metric, ts, float64(ts)) - require.NoError(t, err) - } - - require.NoError(t, app.Commit()) - } - - // At this point we expect 2 mmap-ed head chunks. - - // Get a querier and make sure it's closed only once the test is over. - querier, err := db.Querier(0, math.MaxInt64) - require.NoError(t, err) - defer func() { - require.NoError(t, querier.Close()) - }() - - // Query back all series. - hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} - seriesSet := querier.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchRegexp, labels.MetricName, ".+")) - - // Fetch samples iterators from all series. - var iterators []chunkenc.Iterator - actualSeries := 0 - for seriesSet.Next() { - actualSeries++ - - // Get the iterator and call Next() so that we're sure the chunk is loaded. - it := seriesSet.At().Iterator(nil) - it.Next() - it.At() - - iterators = append(iterators, it) - } - require.NoError(t, seriesSet.Err()) - require.Equal(t, numSeries, actualSeries) - - // Compact the TSDB head again. - require.NoError(t, db.Compact(ctx)) - require.Equal(t, float64(2), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) - - // At this point we expect 1 head chunk has been deleted. - - // Stress the memory and call GC. This is required to increase the chances - // the chunk memory area is released to the kernel. - var buf []byte - for i := range numStressIterations { - //nolint:staticcheck - buf = append(buf, make([]byte, minStressAllocationBytes+rand.Int31n(maxStressAllocationBytes-minStressAllocationBytes))...) - if i%1000 == 0 { - buf = nil - } - } - - // Iterate samples. Here we're summing it just to make sure no golang compiler - // optimization triggers in case we discard the result of it.At(). - var sum float64 - var firstErr error - for _, it := range iterators { - for it.Next() == chunkenc.ValFloat { - _, v := it.At() - sum += v - } - - if err := it.Err(); err != nil { - firstErr = err - } - } - - // After having iterated all samples we also want to be sure no error occurred or - // the "cannot populate chunk XXX: not found" error occurred. This error can occur - // when the iterator tries to fetch an head chunk which has been offloaded because - // of the head compaction in the meanwhile. - if firstErr != nil { - require.ErrorContains(t, firstErr, "cannot populate chunk") - } -} - -func TestChunkQuerier_ShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { - t.Skip("TODO: investigate why process crash in CI") - - const numRuns = 5 - - for i := 1; i <= numRuns; i++ { - t.Run(strconv.Itoa(i), func(t *testing.T) { - testChunkQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t) - }) - } -} - -func testChunkQuerierShouldNotPanicIfHeadChunkIsTruncatedWhileReadingQueriedChunks(t *testing.T) { - const ( - numSeries = 1000 - numStressIterations = 10000 - minStressAllocationBytes = 128 * 1024 - maxStressAllocationBytes = 512 * 1024 - ) - - db := newTestDB(t) - - // Disable compactions so we can control it. - db.DisableCompactions() - - // Generate the metrics we're going to append. - metrics := make([]labels.Labels, 0, numSeries) - for i := range numSeries { - metrics = append(metrics, labels.FromStrings(labels.MetricName, fmt.Sprintf("test_%d", i))) - } - - // Push 1 sample every 15s for 2x the block duration period. - ctx := context.Background() - interval := int64(15 * time.Second / time.Millisecond) - ts := int64(0) - - for ; ts < 2*DefaultBlockDuration; ts += interval { - app := db.Appender(ctx) - - for _, metric := range metrics { - _, err := app.Append(0, metric, ts, float64(ts)) - require.NoError(t, err) - } - - require.NoError(t, app.Commit()) - } - - // Compact the TSDB head for the first time. We expect the head chunks file has been cut. - require.NoError(t, db.Compact(ctx)) - require.Equal(t, float64(1), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) - - // Push more samples for another 1x block duration period. - for ; ts < 3*DefaultBlockDuration; ts += interval { - app := db.Appender(ctx) - - for _, metric := range metrics { - _, err := app.Append(0, metric, ts, float64(ts)) - require.NoError(t, err) - } - - require.NoError(t, app.Commit()) - } - - // At this point we expect 2 mmap-ed head chunks. - - // Get a querier and make sure it's closed only once the test is over. - querier, err := db.ChunkQuerier(0, math.MaxInt64) - require.NoError(t, err) - defer func() { - require.NoError(t, querier.Close()) - }() - - // Query back all series. - hints := &storage.SelectHints{Start: 0, End: math.MaxInt64, Step: interval} - seriesSet := querier.Select(ctx, true, hints, labels.MustNewMatcher(labels.MatchRegexp, labels.MetricName, ".+")) - - // Iterate all series and get their chunks. - var it chunks.Iterator - var chunks []chunkenc.Chunk - actualSeries := 0 - for seriesSet.Next() { - actualSeries++ - it = seriesSet.At().Iterator(it) - for it.Next() { - chunks = append(chunks, it.At().Chunk) - } - } - require.NoError(t, seriesSet.Err()) - require.Equal(t, numSeries, actualSeries) - - // Compact the TSDB head again. - require.NoError(t, db.Compact(ctx)) - require.Equal(t, float64(2), prom_testutil.ToFloat64(db.Head().metrics.headTruncateTotal)) - - // At this point we expect 1 head chunk has been deleted. - - // Stress the memory and call GC. This is required to increase the chances - // the chunk memory area is released to the kernel. - var buf []byte - for i := range numStressIterations { - //nolint:staticcheck - buf = append(buf, make([]byte, minStressAllocationBytes+rand.Int31n(maxStressAllocationBytes-minStressAllocationBytes))...) - if i%1000 == 0 { - buf = nil - } - } - - // Iterate chunks and read their bytes slice. Here we're computing the CRC32 - // just to iterate through the bytes slice. We don't really care the reason why - // we read this data, we just need to read it to make sure the memory address - // of the []byte is still valid. - chkCRC32 := crc32.New(crc32.MakeTable(crc32.Castagnoli)) - for _, chunk := range chunks { - chkCRC32.Reset() - _, err := chkCRC32.Write(chunk.Bytes()) - require.NoError(t, err) - } -} - -func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier(t *testing.T) { +func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier_AppendV2(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration db := newTestDB(t, withOpts(opts)) @@ -3738,16 +2498,16 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier(t *test // Push samples after the OOO sample we'll write below. for ; ts < 10*interval; ts += interval { - app := db.Appender(ctx) - _, err := app.Append(0, metric, ts, float64(ts)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, metric, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) samplesWritten++ } // Push a single OOO sample. - app := db.Appender(ctx) - _, err := app.Append(0, metric, oooTS, float64(ts)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, metric, 0, oooTS, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) samplesWritten++ @@ -3809,7 +2569,7 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingQuerier(t *test require.NoError(t, querierCreatedAfterCompaction.Close()) } -func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting(t *testing.T) { +func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting_AppendV2(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration db := newTestDB(t, withOpts(opts)) @@ -3829,16 +2589,16 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting(t *testing.T) { // Push samples after the OOO sample we'll write below. for ; ts < 10*interval; ts += interval { - app := db.Appender(ctx) - _, err := app.Append(0, metric, ts, float64(ts)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, metric, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) samplesWritten++ } // Push a single OOO sample. - app := db.Appender(ctx) - _, err := app.Append(0, metric, oooTS, float64(ts)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, metric, 0, oooTS, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) samplesWritten++ @@ -3888,7 +2648,7 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterSelecting(t *testing.T) { require.Eventually(t, compactionComplete.Load, time.Second, 10*time.Millisecond, "compaction should complete after querier was closed") } -func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators(t *testing.T) { +func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators_AppendV2(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 3 * DefaultBlockDuration db := newTestDB(t, withOpts(opts)) @@ -3908,16 +2668,16 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators(t *te // Push samples after the OOO sample we'll write below. for ; ts < 10*interval; ts += interval { - app := db.Appender(ctx) - _, err := app.Append(0, metric, ts, float64(ts)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, metric, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) samplesWritten++ } // Push a single OOO sample. - app := db.Appender(ctx) - _, err := app.Append(0, metric, oooTS, float64(ts)) + app := db.AppenderV2(ctx) + _, err := app.Append(0, metric, 0, oooTS, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) samplesWritten++ @@ -3967,7 +2727,7 @@ func TestQuerierShouldNotFailIfOOOCompactionOccursAfterRetrievingIterators(t *te require.Eventually(t, compactionComplete.Load, time.Second, 10*time.Millisecond, "compaction should complete after querier was closed") } -func TestOOOWALWrite(t *testing.T) { +func TestOOOWALWrite_AppendV2(t *testing.T) { minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } s := labels.NewSymbolTable() @@ -3979,13 +2739,13 @@ func TestOOOWALWrite(t *testing.T) { s2 := scratchBuilder2.Labels() scenarios := map[string]struct { - appendSample func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) + appendSample func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error) expectedOOORecords []any expectedInORecords []any }{ "float": { - appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { - seriesRef, err := app.Append(0, l, minutes(mins), float64(mins)) + appendSample: func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.Append(0, l, 0, minutes(mins), float64(mins), nil, nil, storage.AOptions{}) require.NoError(t, err) return seriesRef, nil }, @@ -4075,8 +2835,8 @@ func TestOOOWALWrite(t *testing.T) { }, }, "integer histogram": { - appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { - seriesRef, err := app.AppendHistogram(0, l, minutes(mins), tsdbutil.GenerateTestHistogram(mins), nil) + appendSample: func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.Append(0, l, 0, minutes(mins), 0, tsdbutil.GenerateTestHistogram(mins), nil, storage.AOptions{}) require.NoError(t, err) return seriesRef, nil }, @@ -4166,8 +2926,8 @@ func TestOOOWALWrite(t *testing.T) { }, }, "float histogram": { - appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { - seriesRef, err := app.AppendHistogram(0, l, minutes(mins), nil, tsdbutil.GenerateTestFloatHistogram(mins)) + appendSample: func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.Append(0, l, 0, minutes(mins), 0, nil, tsdbutil.GenerateTestFloatHistogram(mins), storage.AOptions{}) require.NoError(t, err) return seriesRef, nil }, @@ -4257,8 +3017,8 @@ func TestOOOWALWrite(t *testing.T) { }, }, "custom buckets histogram": { - appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { - seriesRef, err := app.AppendHistogram(0, l, minutes(mins), tsdbutil.GenerateTestCustomBucketsHistogram(mins), nil) + appendSample: func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.Append(0, l, 0, minutes(mins), 0, tsdbutil.GenerateTestCustomBucketsHistogram(mins), nil, storage.AOptions{}) require.NoError(t, err) return seriesRef, nil }, @@ -4348,8 +3108,8 @@ func TestOOOWALWrite(t *testing.T) { }, }, "custom buckets float histogram": { - appendSample: func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error) { - seriesRef, err := app.AppendHistogram(0, l, minutes(mins), nil, tsdbutil.GenerateTestCustomBucketsFloatHistogram(mins)) + appendSample: func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error) { + seriesRef, err := app.Append(0, l, 0, minutes(mins), 0, nil, tsdbutil.GenerateTestCustomBucketsFloatHistogram(mins), storage.AOptions{}) require.NoError(t, err) return seriesRef, nil }, @@ -4441,13 +3201,13 @@ func TestOOOWALWrite(t *testing.T) { } for name, scenario := range scenarios { t.Run(name, func(t *testing.T) { - testOOOWALWrite(t, scenario.appendSample, scenario.expectedOOORecords, scenario.expectedInORecords) + testOOOWALWriteAppendV2(t, scenario.appendSample, scenario.expectedOOORecords, scenario.expectedInORecords) }) } } -func testOOOWALWrite(t *testing.T, - appendSample func(app storage.Appender, l labels.Labels, mins int64) (storage.SeriesRef, error), +func testOOOWALWriteAppendV2(t *testing.T, + appendSample func(app storage.AppenderV2, l labels.Labels, mins int64) (storage.SeriesRef, error), expectedOOORecords []any, expectedInORecords []any, ) { @@ -4459,23 +3219,23 @@ func testOOOWALWrite(t *testing.T, s1, s2 := labels.FromStrings("l", "v1"), labels.FromStrings("l", "v2") // Ingest sample at 1h. - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) appendSample(app, s1, 60) appendSample(app, s2, 60) require.NoError(t, app.Commit()) // OOO for s1. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) appendSample(app, s1, 40) require.NoError(t, app.Commit()) // OOO for s2. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) appendSample(app, s2, 42) require.NoError(t, app.Commit()) // OOO for both s1 and s2 in the same commit. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) appendSample(app, s2, 45) appendSample(app, s1, 35) appendSample(app, s1, 36) // m-maps. @@ -4483,13 +3243,13 @@ func testOOOWALWrite(t *testing.T, require.NoError(t, app.Commit()) // OOO for s1 but not for s2 in the same commit. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) appendSample(app, s1, 50) // m-maps. appendSample(app, s2, 65) require.NoError(t, app.Commit()) // Single commit has 2 times m-mapping and more samples after m-map. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) appendSample(app, s2, 50) // m-maps. appendSample(app, s2, 51) appendSample(app, s2, 52) // m-maps. @@ -4547,7 +3307,7 @@ func testOOOWALWrite(t *testing.T, } // Tests https://github.com/prometheus/prometheus/issues/10291#issuecomment-1044373110. -func TestDBPanicOnMmappingHeadChunk(t *testing.T) { +func TestDBPanicOnMmappingHeadChunk_AppendV2(t *testing.T) { var err error ctx := context.Background() @@ -4559,16 +3319,16 @@ func TestDBPanicOnMmappingHeadChunk(t *testing.T) { lastTs := int64(0) addSamples := func(numSamples int) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) var ref storage.SeriesRef lbls := labels.FromStrings("__name__", "testing", "foo", "bar") for i := range numSamples { - ref, err = app.Append(ref, lbls, lastTs, float64(lastTs)) + ref, err = app.Append(ref, lbls, 0, lastTs, float64(lastTs), nil, nil, storage.AOptions{}) require.NoError(t, err) lastTs += itvl if i%10 == 0 { require.NoError(t, app.Commit()) - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) } } require.NoError(t, app.Commit()) @@ -4603,37 +3363,34 @@ func TestDBPanicOnMmappingHeadChunk(t *testing.T) { require.NoError(t, db.Close()) } -func TestMetadataInWAL(t *testing.T) { - updateMetadata := func(t *testing.T, app storage.Appender, s labels.Labels, m metadata.Metadata) { - _, err := app.UpdateMetadata(0, s, m) - require.NoError(t, err) - } - - db := newTestDB(t) +// TODO(bwplotka): Add cases ensuring stale sample appends will skipp metadata persisting. +func TestMetadataInWAL_AppenderV2(t *testing.T) { + opts := DefaultOptions() + opts.EnableMetadataWALRecords = true + db := newTestDB(t, withOpts(opts)) ctx := context.Background() - // Add some series so we can append metadata to them. - app := db.Appender(ctx) + // Add some series so we can attach metadata to them. s1 := labels.FromStrings("a", "b") s2 := labels.FromStrings("c", "d") s3 := labels.FromStrings("e", "f") s4 := labels.FromStrings("g", "h") - for _, s := range []labels.Labels{s1, s2, s3, s4} { - _, err := app.Append(0, s, 0, 0) - require.NoError(t, err) - } - require.NoError(t, app.Commit()) - // Add a first round of metadata to the first three series. - // Re-take the Appender, as the previous Commit will have it closed. m1 := metadata.Metadata{Type: "gauge", Unit: "unit_1", Help: "help_1"} m2 := metadata.Metadata{Type: "gauge", Unit: "unit_2", Help: "help_2"} m3 := metadata.Metadata{Type: "gauge", Unit: "unit_3", Help: "help_3"} - app = db.Appender(ctx) - updateMetadata(t, app, s1, m1) - updateMetadata(t, app, s2, m2) - updateMetadata(t, app, s3, m3) + + app := db.AppenderV2(ctx) + ts := int64(0) + _, err := app.Append(0, s1, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m1}) + require.NoError(t, err) + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m2}) + require.NoError(t, err) + _, err = app.Append(0, s3, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m3}) + require.NoError(t, err) + _, err = app.Append(0, s4, 0, ts, 0, nil, nil, storage.AOptions{}) + require.NoError(t, err) require.NoError(t, app.Commit()) // Add a replicated metadata entry to the first series, @@ -4641,10 +3398,14 @@ func TestMetadataInWAL(t *testing.T) { // and a changed metadata entry to the second series. m4 := metadata.Metadata{Type: "counter", Unit: "unit_4", Help: "help_4"} m5 := metadata.Metadata{Type: "counter", Unit: "unit_5", Help: "help_5"} - app = db.Appender(ctx) - updateMetadata(t, app, s1, m1) - updateMetadata(t, app, s4, m4) - updateMetadata(t, app, s2, m5) + app = db.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s1, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m1}) + require.NoError(t, err) + _, err = app.Append(0, s4, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m4}) + require.NoError(t, err) + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m5}) + require.NoError(t, err) require.NoError(t, app.Commit()) // Read the WAL to see if the disk storage format is correct. @@ -4668,70 +3429,75 @@ func TestMetadataInWAL(t *testing.T) { require.Equal(t, expectedMetadata[3:], gotMetadataBlocks[1]) } -func TestMetadataCheckpointingOnlyKeepsLatestEntry(t *testing.T) { - updateMetadata := func(t *testing.T, app storage.Appender, s labels.Labels, m metadata.Metadata) { - _, err := app.UpdateMetadata(0, s, m) - require.NoError(t, err) - } - +func TestMetadataCheckpointingOnlyKeepsLatestEntry_AppendV2(t *testing.T) { ctx := context.Background() numSamples := 10000 hb, w := newTestHead(t, int64(numSamples)*10, compression.None, false) + hb.opts.EnableMetadataWALRecords = true // Add some series so we can append metadata to them. - app := hb.Appender(ctx) s1 := labels.FromStrings("a", "b") s2 := labels.FromStrings("c", "d") s3 := labels.FromStrings("e", "f") s4 := labels.FromStrings("g", "h") - for _, s := range []labels.Labels{s1, s2, s3, s4} { - _, err := app.Append(0, s, 0, 0) - require.NoError(t, err) - } - require.NoError(t, app.Commit()) - - // Add a first round of metadata to the first three series. - // Re-take the Appender, as the previous Commit will have it closed. m1 := metadata.Metadata{Type: "gauge", Unit: "unit_1", Help: "help_1"} m2 := metadata.Metadata{Type: "gauge", Unit: "unit_2", Help: "help_2"} m3 := metadata.Metadata{Type: "gauge", Unit: "unit_3", Help: "help_3"} m4 := metadata.Metadata{Type: "gauge", Unit: "unit_4", Help: "help_4"} - app = hb.Appender(ctx) - updateMetadata(t, app, s1, m1) - updateMetadata(t, app, s2, m2) - updateMetadata(t, app, s3, m3) - updateMetadata(t, app, s4, m4) + + app := hb.AppenderV2(ctx) + ts := int64(0) + _, err := app.Append(0, s1, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m1}) + require.NoError(t, err) + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m2}) + require.NoError(t, err) + _, err = app.Append(0, s3, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m3}) + require.NoError(t, err) + _, err = app.Append(0, s4, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m4}) + require.NoError(t, err) require.NoError(t, app.Commit()) // Update metadata for first series. m5 := metadata.Metadata{Type: "counter", Unit: "unit_5", Help: "help_5"} - app = hb.Appender(ctx) - updateMetadata(t, app, s1, m5) + app = hb.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s1, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m5}) + require.NoError(t, err) require.NoError(t, app.Commit()) // Switch back-and-forth metadata for second series. // Since it ended on a new metadata record, we expect a single new entry. m6 := metadata.Metadata{Type: "counter", Unit: "unit_6", Help: "help_6"} - app = hb.Appender(ctx) - updateMetadata(t, app, s2, m6) + app = hb.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m6}) + require.NoError(t, err) require.NoError(t, app.Commit()) - app = hb.Appender(ctx) - updateMetadata(t, app, s2, m2) + app = hb.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m2}) + require.NoError(t, err) require.NoError(t, app.Commit()) - app = hb.Appender(ctx) - updateMetadata(t, app, s2, m6) + app = hb.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m6}) + require.NoError(t, err) require.NoError(t, app.Commit()) - app = hb.Appender(ctx) - updateMetadata(t, app, s2, m2) + app = hb.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m2}) + require.NoError(t, err) require.NoError(t, app.Commit()) - app = hb.Appender(ctx) - updateMetadata(t, app, s2, m6) + app = hb.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m6}) + require.NoError(t, err) require.NoError(t, app.Commit()) // Let's create a checkpoint. @@ -4772,37 +3538,34 @@ func TestMetadataCheckpointingOnlyKeepsLatestEntry(t *testing.T) { require.NoError(t, hb.Close()) } -func TestMetadataAssertInMemoryData(t *testing.T) { - updateMetadata := func(t *testing.T, app storage.Appender, s labels.Labels, m metadata.Metadata) { - _, err := app.UpdateMetadata(0, s, m) - require.NoError(t, err) - } - - db := newTestDB(t) +func TestMetadataAssertInMemoryData_AppendV2(t *testing.T) { + opts := DefaultOptions() + opts.EnableMetadataWALRecords = true + db := newTestDB(t, withOpts(opts)) ctx := context.Background() // Add some series so we can append metadata to them. - app := db.Appender(ctx) s1 := labels.FromStrings("a", "b") s2 := labels.FromStrings("c", "d") s3 := labels.FromStrings("e", "f") s4 := labels.FromStrings("g", "h") - for _, s := range []labels.Labels{s1, s2, s3, s4} { - _, err := app.Append(0, s, 0, 0) - require.NoError(t, err) - } - require.NoError(t, app.Commit()) - // Add a first round of metadata to the first three series. // The in-memory data held in the db Head should hold the metadata. m1 := metadata.Metadata{Type: "gauge", Unit: "unit_1", Help: "help_1"} m2 := metadata.Metadata{Type: "gauge", Unit: "unit_2", Help: "help_2"} m3 := metadata.Metadata{Type: "gauge", Unit: "unit_3", Help: "help_3"} - app = db.Appender(ctx) - updateMetadata(t, app, s1, m1) - updateMetadata(t, app, s2, m2) - updateMetadata(t, app, s3, m3) + + app := db.AppenderV2(ctx) + ts := int64(0) + _, err := app.Append(0, s1, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m1}) + require.NoError(t, err) + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m2}) + require.NoError(t, err) + _, err = app.Append(0, s3, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m3}) + require.NoError(t, err) + _, err = app.Append(0, s4, 0, ts, 0, nil, nil, storage.AOptions{}) + require.NoError(t, err) require.NoError(t, app.Commit()) series1 := db.head.series.getByHash(s1.Hash(), s1) @@ -4820,10 +3583,14 @@ func TestMetadataAssertInMemoryData(t *testing.T) { // The in-memory data held in the db Head should be correctly updated. m4 := metadata.Metadata{Type: "counter", Unit: "unit_4", Help: "help_4"} m5 := metadata.Metadata{Type: "counter", Unit: "unit_5", Help: "help_5"} - app = db.Appender(ctx) - updateMetadata(t, app, s1, m1) - updateMetadata(t, app, s4, m4) - updateMetadata(t, app, s2, m5) + app = db.AppenderV2(ctx) + ts++ + _, err = app.Append(0, s1, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m1}) + require.NoError(t, err) + _, err = app.Append(0, s4, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m4}) + require.NoError(t, err) + _, err = app.Append(0, s2, 0, ts, 0, nil, nil, storage.AOptions{Metadata: m5}) + require.NoError(t, err) require.NoError(t, app.Commit()) series1 = db.head.series.getByHash(s1.Hash(), s1) @@ -4839,19 +3606,24 @@ func TestMetadataAssertInMemoryData(t *testing.T) { // Reopen the DB, replaying the WAL. The Head must have been replayed // correctly in memory. - db = newTestDB(t, withDir(db.Dir())) - _, err := db.head.wal.Size() + reopenDB, err := Open(db.Dir(), nil, nil, nil, nil) + require.NoError(t, err) + t.Cleanup(func() { + require.NoError(t, reopenDB.Close()) + }) + + _, err = reopenDB.head.wal.Size() require.NoError(t, err) - require.Equal(t, *db.head.series.getByHash(s1.Hash(), s1).meta, m1) - require.Equal(t, *db.head.series.getByHash(s2.Hash(), s2).meta, m5) - require.Equal(t, *db.head.series.getByHash(s3.Hash(), s3).meta, m3) - require.Equal(t, *db.head.series.getByHash(s4.Hash(), s4).meta, m4) + require.Equal(t, *reopenDB.head.series.getByHash(s1.Hash(), s1).meta, m1) + require.Equal(t, *reopenDB.head.series.getByHash(s2.Hash(), s2).meta, m5) + require.Equal(t, *reopenDB.head.series.getByHash(s3.Hash(), s3).meta, m3) + require.Equal(t, *reopenDB.head.series.getByHash(s4.Hash(), s4).meta, m4) } // TestMultipleEncodingsCommitOrder mainly serves to demonstrate when happens when committing a batch of samples for the // same series when there are multiple encodings. With issue #15177 fixed, this now all works as expected. -func TestMultipleEncodingsCommitOrder(t *testing.T) { +func TestMultipleEncodingsCommitOrder_AppendV2(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() @@ -4860,20 +3632,20 @@ func TestMultipleEncodingsCommitOrder(t *testing.T) { db.DisableCompactions() series1 := labels.FromStrings("foo", "bar1") - addSample := func(app storage.Appender, ts int64, valType chunkenc.ValueType) chunks.Sample { + addSample := func(app storage.AppenderV2, ts int64, valType chunkenc.ValueType) chunks.Sample { if valType == chunkenc.ValFloat { - _, err := app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) return sample{t: ts, f: float64(ts)} } if valType == chunkenc.ValHistogram { h := tsdbutil.GenerateTestHistogram(ts) - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) require.NoError(t, err) return sample{t: ts, h: h} } fh := tsdbutil.GenerateTestFloatHistogram(ts) - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, nil, fh, storage.AOptions{}) require.NoError(t, err) return sample{t: ts, fh: fh} } @@ -4913,7 +3685,7 @@ func TestMultipleEncodingsCommitOrder(t *testing.T) { var expSamples []chunks.Sample // Append samples with different encoding types and then commit them at once. - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for i := 100; i < 105; i++ { s := addSample(app, int64(i), chunkenc.ValFloat) @@ -4947,7 +3719,7 @@ func TestMultipleEncodingsCommitOrder(t *testing.T) { verifySamples(100, 150, expSamples, 5) // Append and commit some in-order histograms by themselves. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) for i := 150; i < 160; i++ { s := addSample(app, int64(i), chunkenc.ValHistogram) expSamples = append(expSamples, s) @@ -4959,7 +3731,7 @@ func TestMultipleEncodingsCommitOrder(t *testing.T) { // Append and commit samples for all encoding types. This time all samples will be treated as OOO because samples // with newer timestamps have already been committed. - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) for i := 50; i < 55; i++ { s := addSample(app, int64(i), chunkenc.ValFloat) expSamples = append(expSamples, s) @@ -4990,18 +3762,18 @@ func TestMultipleEncodingsCommitOrder(t *testing.T) { // TODO(codesome): test more samples incoming once compaction has started. To verify new samples after the start // // are not included in this compaction. -func TestOOOCompaction(t *testing.T) { +func TestOOOCompaction_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOCompaction(t, scenario, false) + testOOOCompactionAppenderV2(t, scenario, false) }) t.Run(name+"+extra", func(t *testing.T) { - testOOOCompaction(t, scenario, true) + testOOOCompactionAppenderV2(t, scenario, true) }) } } -func testOOOCompaction(t *testing.T, scenario sampleTypeScenario, addExtraSamples bool) { +func testOOOCompactionAppenderV2(t *testing.T, scenario sampleTypeScenario, addExtraSamples bool) { ctx := context.Background() opts := DefaultOptions() @@ -5014,12 +3786,12 @@ func testOOOCompaction(t *testing.T, scenario sampleTypeScenario, addExtraSample series2 := labels.FromStrings("foo", "bar2") addSample := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, _, err := scenario.appendFunc(app, series1, ts, ts) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) - _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + _, _, err = scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series2, ts, 2*ts) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -5188,16 +3960,16 @@ func testOOOCompaction(t *testing.T, scenario sampleTypeScenario, addExtraSample // TestOOOCompactionWithNormalCompaction tests if OOO compaction is performed // when the normal head's compaction is done. -func TestOOOCompactionWithNormalCompaction(t *testing.T) { +func TestOOOCompactionWithNormalCompaction_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOCompactionWithNormalCompaction(t, scenario) + testOOOCompactionWithNormalCompactionAppendV2(t, scenario) }) } } -func testOOOCompactionWithNormalCompaction(t *testing.T, scenario sampleTypeScenario) { +func testOOOCompactionWithNormalCompactionAppendV2(t *testing.T, scenario sampleTypeScenario) { t.Parallel() ctx := context.Background() @@ -5212,12 +3984,12 @@ func testOOOCompactionWithNormalCompaction(t *testing.T, scenario sampleTypeScen series2 := labels.FromStrings("foo", "bar2") addSamples := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, _, err := scenario.appendFunc(app, series1, ts, ts) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) - _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + _, _, err = scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series2, ts, 2*ts) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -5293,16 +4065,16 @@ func testOOOCompactionWithNormalCompaction(t *testing.T, scenario sampleTypeScen // TestOOOCompactionWithDisabledWriteLog tests the scenario where the TSDB is // configured to not have wal and wbl but its able to compact both the in-order // and out-of-order head. -func TestOOOCompactionWithDisabledWriteLog(t *testing.T) { +func TestOOOCompactionWithDisabledWriteLog_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOCompactionWithDisabledWriteLog(t, scenario) + testOOOCompactionWithDisabledWriteLogAppend2(t, scenario) }) } } -func testOOOCompactionWithDisabledWriteLog(t *testing.T, scenario sampleTypeScenario) { +func testOOOCompactionWithDisabledWriteLogAppend2(t *testing.T, scenario sampleTypeScenario) { t.Parallel() ctx := context.Background() @@ -5318,12 +4090,12 @@ func testOOOCompactionWithDisabledWriteLog(t *testing.T, scenario sampleTypeScen series2 := labels.FromStrings("foo", "bar2") addSamples := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, _, err := scenario.appendFunc(app, series1, ts, ts) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) - _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + _, _, err = scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series2, ts, 2*ts) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -5396,19 +4168,19 @@ func testOOOCompactionWithDisabledWriteLog(t *testing.T, scenario sampleTypeScen verifySamples(db.Blocks()[1], 250, 350) } -// TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL tests the scenario where the WBL goes +// TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL_AppendV2 tests the scenario where the WBL goes // missing after a restart while snapshot was enabled, but the query still returns the right // data from the mmap chunks. -func TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T) { +func TestOOOQueryAfterRestartWithSnapshotAndRemovedWBL_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t, scenario) + testOOOQueryAfterRestartWithSnapshotAndRemovedWBLAppendV2(t, scenario) }) } } -func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sampleTypeScenario) { +func testOOOQueryAfterRestartWithSnapshotAndRemovedWBLAppendV2(t *testing.T, scenario sampleTypeScenario) { ctx := context.Background() opts := DefaultOptions() @@ -5423,12 +4195,12 @@ func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sa series2 := labels.FromStrings("foo", "bar2") addSamples := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, _, err := scenario.appendFunc(app, series1, ts, ts) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) - _, _, err = scenario.appendFunc(app, series2, ts, 2*ts) + _, _, err = scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series2, ts, 2*ts) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -5509,38 +4281,38 @@ func testOOOQueryAfterRestartWithSnapshotAndRemovedWBL(t *testing.T, scenario sa verifySamples(90, 109) } -func TestQuerierOOOQuery(t *testing.T) { +func TestQuerierOOOQuery_AppendV2(t *testing.T) { scenarios := map[string]struct { - appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) + appendFunc func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error) sampleFunc func(ts int64) chunks.Sample }{ "float": { - appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { - return app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) + appendFunc: func(app storage.AppenderV2, ts int64, _ bool) (storage.SeriesRef, error) { + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, float64(ts), nil, nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, f: float64(ts)} }, }, "integer histogram": { - appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error) { h := tsdbutil.GenerateTestHistogram(ts) if counterReset { h.CounterResetHint = histogram.CounterReset } - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} }, }, "float histogram": { - appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error) { fh := tsdbutil.GenerateTestFloatHistogram(ts) if counterReset { fh.CounterResetHint = histogram.CounterReset } - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, nil, fh, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, fh: tsdbutil.GenerateTestFloatHistogram(ts)} @@ -5548,10 +4320,10 @@ func TestQuerierOOOQuery(t *testing.T) { }, "integer histogram counter resets": { // Adding counter reset to all histograms means each histogram will have its own chunk. - appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, _ bool) (storage.SeriesRef, error) { h := tsdbutil.GenerateTestHistogram(ts) h.CounterResetHint = histogram.CounterReset // For this scenario, ignore the counterReset argument. - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} @@ -5561,13 +4333,13 @@ func TestQuerierOOOQuery(t *testing.T) { for name, scenario := range scenarios { t.Run(name, func(t *testing.T) { - testQuerierOOOQuery(t, scenario.appendFunc, scenario.sampleFunc) + testQuerierOOOQueryAppendV2(t, scenario.appendFunc, scenario.sampleFunc) }) } } -func testQuerierOOOQuery(t *testing.T, - appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error), +func testQuerierOOOQueryAppendV2(t *testing.T, + appendFunc func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error), sampleFunc func(ts int64) chunks.Sample, ) { opts := DefaultOptions() @@ -5580,7 +4352,7 @@ func testQuerierOOOQuery(t *testing.T, minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } addSample := func(db *DB, fromMins, toMins, queryMinT, queryMaxT int64, expSamples []chunks.Sample, filter filterFunc, counterReset bool) ([]chunks.Sample, int) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) totalAppended := 0 for m := fromMins; m <= toMins; m += time.Minute.Milliseconds() { if !filter(m / time.Minute.Milliseconds()) { @@ -5803,7 +4575,7 @@ func testQuerierOOOQuery(t *testing.T, } } -func TestChunkQuerierOOOQuery(t *testing.T) { +func TestChunkQuerierOOOQuery_AppendV2(t *testing.T) { nBucketHistogram := func(n int64) *histogram.Histogram { h := &histogram.Histogram{ Count: uint64(n), @@ -5821,37 +4593,37 @@ func TestChunkQuerierOOOQuery(t *testing.T) { } scenarios := map[string]struct { - appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) + appendFunc func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error) sampleFunc func(ts int64) chunks.Sample checkInUseBucket bool }{ "float": { - appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { - return app.Append(0, labels.FromStrings("foo", "bar1"), ts, float64(ts)) + appendFunc: func(app storage.AppenderV2, ts int64, _ bool) (storage.SeriesRef, error) { + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, float64(ts), nil, nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, f: float64(ts)} }, }, "integer histogram": { - appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error) { h := tsdbutil.GenerateTestHistogram(ts) if counterReset { h.CounterResetHint = histogram.CounterReset } - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} }, }, "float histogram": { - appendFunc: func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error) { fh := tsdbutil.GenerateTestFloatHistogram(ts) if counterReset { fh.CounterResetHint = histogram.CounterReset } - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, nil, fh, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, fh: tsdbutil.GenerateTestFloatHistogram(ts)} @@ -5859,10 +4631,10 @@ func TestChunkQuerierOOOQuery(t *testing.T) { }, "integer histogram counter resets": { // Adding counter reset to all histograms means each histogram will have its own chunk. - appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, _ bool) (storage.SeriesRef, error) { h := tsdbutil.GenerateTestHistogram(ts) h.CounterResetHint = histogram.CounterReset // For this scenario, ignore the counterReset argument. - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { return sample{t: ts, h: tsdbutil.GenerateTestHistogram(ts)} @@ -5870,9 +4642,9 @@ func TestChunkQuerierOOOQuery(t *testing.T) { }, "integer histogram with recode": { // Histograms have increasing number of buckets so their chunks are recoded. - appendFunc: func(app storage.Appender, ts int64, _ bool) (storage.SeriesRef, error) { + appendFunc: func(app storage.AppenderV2, ts int64, _ bool) (storage.SeriesRef, error) { n := ts / time.Minute.Milliseconds() - return app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nBucketHistogram(n), nil) + return app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, nBucketHistogram(n), nil, storage.AOptions{}) }, sampleFunc: func(ts int64) chunks.Sample { n := ts / time.Minute.Milliseconds() @@ -5885,13 +4657,13 @@ func TestChunkQuerierOOOQuery(t *testing.T) { } for name, scenario := range scenarios { t.Run(name, func(t *testing.T) { - testChunkQuerierOOOQuery(t, scenario.appendFunc, scenario.sampleFunc, scenario.checkInUseBucket) + testChunkQuerierOOOQueryAppendV2(t, scenario.appendFunc, scenario.sampleFunc, scenario.checkInUseBucket) }) } } -func testChunkQuerierOOOQuery(t *testing.T, - appendFunc func(app storage.Appender, ts int64, counterReset bool) (storage.SeriesRef, error), +func testChunkQuerierOOOQueryAppendV2(t *testing.T, + appendFunc func(app storage.AppenderV2, ts int64, counterReset bool) (storage.SeriesRef, error), sampleFunc func(ts int64) chunks.Sample, checkInUseBuckets bool, ) { @@ -5906,7 +4678,7 @@ func testChunkQuerierOOOQuery(t *testing.T, minutes := func(m int64) int64 { return m * time.Minute.Milliseconds() } addSample := func(db *DB, fromMins, toMins, queryMinT, queryMaxT int64, expSamples []chunks.Sample, filter filterFunc, counterReset bool) ([]chunks.Sample, int) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) totalAppended := 0 for m := fromMins; m <= toMins; m += time.Minute.Milliseconds() { if !filter(m / time.Minute.Milliseconds()) { @@ -6161,17 +4933,17 @@ func testChunkQuerierOOOQuery(t *testing.T, // could potentially come in that would change the status of the header. In this case, the UnknownCounterReset // headers would be re-checked at query time and updated as needed. However, this test is checking the counter // reset headers at the time of storage. -func TestOOONativeHistogramsWithCounterResets(t *testing.T) { +func TestOOONativeHistogramsWithCounterResets_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { if name == intHistogram || name == floatHistogram { - testOOONativeHistogramsWithCounterResets(t, scenario) + testOOONativeHistogramsWithCounterResetsAppendV2(t, scenario) } }) } } -func testOOONativeHistogramsWithCounterResets(t *testing.T, scenario sampleTypeScenario) { +func testOOONativeHistogramsWithCounterResetsAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 24 * time.Hour.Milliseconds() @@ -6278,7 +5050,7 @@ func testOOONativeHistogramsWithCounterResets(t *testing.T, scenario sampleTypeS db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) expSamples := make(map[string][]chunks.Sample) @@ -6290,7 +5062,7 @@ func testOOONativeHistogramsWithCounterResets(t *testing.T, scenario sampleTypeS if resetCount { j = 0 } - _, s, err := scenario.appendFunc(app, lbls, minutes(i), j) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), lbls, minutes(i), j) require.NoError(t, err) if s.Type() == chunkenc.ValHistogram { s.H().CounterResetHint = batch.expCounterResetHints[smplIdx] @@ -6324,16 +5096,16 @@ func testOOONativeHistogramsWithCounterResets(t *testing.T, scenario sampleTypeS } } -func TestOOOInterleavedImplicitCounterResets(t *testing.T) { +func TestOOOInterleavedImplicitCounterResets_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOInterleavedImplicitCounterResets(t, name, scenario) + testOOOInterleavedImplicitCounterResetsV2(t, name, scenario) }) } } -func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario sampleTypeScenario) { - var appendFunc func(app storage.Appender, ts, v int64) error +func testOOOInterleavedImplicitCounterResetsV2(t *testing.T, name string, scenario sampleTypeScenario) { + var appendFunc func(app storage.AppenderV2, ts, v int64) error if scenario.sampleType != sampleMetricTypeHistogram { return @@ -6341,29 +5113,29 @@ func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario switch name { case intHistogram: - appendFunc = func(app storage.Appender, ts, v int64) error { + appendFunc = func(app storage.AppenderV2, ts, v int64) error { h := &histogram.Histogram{ Count: uint64(v), Sum: float64(v), PositiveSpans: []histogram.Span{{Offset: 0, Length: 1}}, PositiveBuckets: []int64{v}, } - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) return err } case floatHistogram: - appendFunc = func(app storage.Appender, ts, v int64) error { + appendFunc = func(app storage.AppenderV2, ts, v int64) error { fh := &histogram.FloatHistogram{ Count: float64(v), Sum: float64(v), PositiveSpans: []histogram.Span{{Offset: 0, Length: 1}}, PositiveBuckets: []float64{float64(v)}, } - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, nil, fh, storage.AOptions{}) return err } case customBucketsIntHistogram: - appendFunc = func(app storage.Appender, ts, v int64) error { + appendFunc = func(app storage.AppenderV2, ts, v int64) error { h := &histogram.Histogram{ Schema: -53, Count: uint64(v), @@ -6372,11 +5144,11 @@ func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario PositiveBuckets: []int64{v}, CustomValues: []float64{float64(1), float64(2), float64(3)}, } - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, h, nil) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, h, nil, storage.AOptions{}) return err } case customBucketsFloatHistogram: - appendFunc = func(app storage.Appender, ts, v int64) error { + appendFunc = func(app storage.AppenderV2, ts, v int64) error { fh := &histogram.FloatHistogram{ Schema: -53, Count: float64(v), @@ -6385,7 +5157,7 @@ func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario PositiveBuckets: []float64{float64(v)}, CustomValues: []float64{float64(1), float64(2), float64(3)}, } - _, err := app.AppendHistogram(0, labels.FromStrings("foo", "bar1"), ts, nil, fh) + _, err := app.Append(0, labels.FromStrings("foo", "bar1"), 0, ts, 0, nil, fh, storage.AOptions{}) return err } case gaugeIntHistogram, gaugeFloatHistogram: @@ -6512,7 +5284,7 @@ func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario db := newTestDB(t, withOpts(opts)) db.DisableCompactions() - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for _, s := range tc.samples { require.NoError(t, appendFunc(app, s.ts, s.v)) } @@ -6594,15 +5366,15 @@ func testOOOInterleavedImplicitCounterResets(t *testing.T, name string, scenario } } -func TestOOOAppendAndQuery(t *testing.T) { +func TestOOOAppendAndQuery_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOAppendAndQuery(t, scenario) + testOOOAppendAndQueryAppendV2(t, scenario) }) } } -func testOOOAppendAndQuery(t *testing.T, scenario sampleTypeScenario) { +func testOOOAppendAndQueryAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 4 * time.Hour.Milliseconds() @@ -6617,12 +5389,12 @@ func testOOOAppendAndQuery(t *testing.T, scenario sampleTypeScenario) { appendedSamples := make(map[string][]chunks.Sample) totalSamples := 0 addSample := func(lbls labels.Labels, fromMins, toMins int64, faceError bool) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) key := lbls.String() from, to := minutes(fromMins), minutes(toMins) for m := from; m <= to; m += time.Minute.Milliseconds() { val := rand.Intn(1000) - _, s, err := scenario.appendFunc(app, lbls, m, int64(val)) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), lbls, m, int64(val)) if faceError { require.Error(t, err) } else { @@ -6724,15 +5496,15 @@ func testOOOAppendAndQuery(t *testing.T, scenario sampleTypeScenario) { testQuery(math.MinInt64, math.MaxInt64) } -func TestOOODisabled(t *testing.T) { +func TestOOODisabled_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOODisabled(t, scenario) + testOOODisabledAppendV2(t, scenario) }) } } -func testOOODisabled(t *testing.T, scenario sampleTypeScenario) { +func testOOODisabledAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 0 db := newTestDB(t, withOpts(opts)) @@ -6745,11 +5517,11 @@ func testOOODisabled(t *testing.T, scenario sampleTypeScenario) { failedSamples := 0 addSample := func(db *DB, lbls labels.Labels, fromMins, toMins int64, faceError bool) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) key := lbls.String() from, to := minutes(fromMins), minutes(toMins) for m := from; m <= to; m += time.Minute.Milliseconds() { - _, _, err := scenario.appendFunc(app, lbls, m, m) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), lbls, m, m) if faceError { require.Error(t, err) failedSamples++ @@ -6794,15 +5566,15 @@ func testOOODisabled(t *testing.T, scenario sampleTypeScenario) { require.Nil(t, ms.ooo) } -func TestWBLAndMmapReplay(t *testing.T) { +func TestWBLAndMmapReplay_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testWBLAndMmapReplay(t, scenario) + testWBLAndMmapReplayAppendV2(t, scenario) }) } } -func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { +func testWBLAndMmapReplayAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 4 * time.Hour.Milliseconds() @@ -6816,12 +5588,12 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { expSamples := make(map[string][]chunks.Sample) totalSamples := 0 addSample := func(lbls labels.Labels, fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) key := lbls.String() from, to := minutes(fromMins), minutes(toMins) for m := from; m <= to; m += time.Minute.Milliseconds() { val := rand.Intn(1000) - _, s, err := scenario.appendFunc(app, lbls, m, int64(val)) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), lbls, m, int64(val)) require.NoError(t, err) expSamples[key] = append(expSamples[key], s) totalSamples++ @@ -6975,7 +5747,7 @@ func testWBLAndMmapReplay(t *testing.T, scenario sampleTypeScenario) { }) } -func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { +func TestOOOHistogramCompactionWithCounterResets_AppendV2(t *testing.T) { for _, floatHistogram := range []bool{false, true} { ctx := context.Background() @@ -6992,12 +5764,12 @@ func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { var series1ExpSamplesPreCompact, series2ExpSamplesPreCompact, series1ExpSamplesPostCompact, series2ExpSamplesPostCompact []chunks.Sample addSample := func(ts int64, l labels.Labels, val int, hint histogram.CounterResetHint) sample { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) tsMs := ts * time.Minute.Milliseconds() if floatHistogram { h := tsdbutil.GenerateTestFloatHistogram(int64(val)) h.CounterResetHint = hint - _, err := app.AppendHistogram(0, l, tsMs, nil, h) + _, err := app.Append(0, l, 0, tsMs, 0, nil, h, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, fh: h.Copy()} @@ -7005,7 +5777,7 @@ func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { h := tsdbutil.GenerateTestHistogram(int64(val)) h.CounterResetHint = hint - _, err := app.AppendHistogram(0, l, tsMs, h, nil) + _, err := app.Append(0, l, 0, tsMs, 0, h, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, h: h.Copy()} @@ -7330,7 +6102,7 @@ func TestOOOHistogramCompactionWithCounterResets(t *testing.T) { } } -func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing.T) { +func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets_AppendV2(t *testing.T) { for _, floatHistogram := range []bool{false, true} { ctx := context.Background() @@ -7344,18 +6116,18 @@ func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing series1 := labels.FromStrings("foo", "bar1") addSample := func(ts int64, l labels.Labels, val int) sample { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) tsMs := ts if floatHistogram { h := tsdbutil.GenerateTestFloatHistogram(int64(val)) - _, err := app.AppendHistogram(0, l, tsMs, nil, h) + _, err := app.Append(0, l, 0, tsMs, 0, nil, h, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, fh: h.Copy()} } h := tsdbutil.GenerateTestHistogram(int64(val)) - _, err := app.AppendHistogram(0, l, tsMs, h, nil) + _, err := app.Append(0, l, 0, tsMs, 0, h, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) return sample{t: tsMs, h: h.Copy()} @@ -7403,7 +6175,8 @@ func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing // Compact the in-order head and expect another block. // Since this is a forced compaction, this block is not aligned with 2h. - require.NoError(t, db.CompactHead(NewRangeHead(db.head, 0, 3))) + err := db.CompactHead(NewRangeHead(db.head, 0, 3)) + require.NoError(t, err) require.Len(t, db.Blocks(), 2) // Blocks created out of normal and OOO head now. But not merged. @@ -7419,28 +6192,16 @@ func TestInterleavedInOrderAndOOOHistogramCompactionWithCounterResets(t *testing } } -func copyWithCounterReset(s sample, hint histogram.CounterResetHint) sample { - if s.h != nil { - h := s.h.Copy() - h.CounterResetHint = hint - return sample{t: s.t, h: h} - } - - h := s.fh.Copy() - h.CounterResetHint = hint - return sample{t: s.t, fh: h} -} - -func TestOOOCompactionFailure(t *testing.T) { +func TestOOOCompactionFailure_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOCompactionFailure(t, scenario) + testOOOCompactionFailureAppendV2(t, scenario) }) } } -func testOOOCompactionFailure(t *testing.T, scenario sampleTypeScenario) { +func testOOOCompactionFailureAppendV2(t *testing.T, scenario sampleTypeScenario) { ctx := context.Background() opts := DefaultOptions() @@ -7448,14 +6209,17 @@ func testOOOCompactionFailure(t *testing.T, scenario sampleTypeScenario) { opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() db := newTestDB(t, withOpts(opts)) db.DisableCompactions() // We want to manually call it. + t.Cleanup(func() { + require.NoError(t, db.Close()) + }) series1 := labels.FromStrings("foo", "bar1") addSample := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, _, err := scenario.appendFunc(app, series1, ts, ts) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -7572,7 +6336,7 @@ func testOOOCompactionFailure(t *testing.T, scenario sampleTypeScenario) { verifyMmapFiles("000001") } -func TestWBLCorruption(t *testing.T) { +func TestWBLCorruption_AppendV2(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderCapMax = 30 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() @@ -7582,10 +6346,10 @@ func TestWBLCorruption(t *testing.T) { series1 := labels.FromStrings("foo", "bar1") var allSamples, expAfterRestart []chunks.Sample addSamples := func(fromMins, toMins int64, afterRestart bool) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, err := app.Append(0, series1, ts, float64(ts)) + _, err := app.Append(0, series1, 0, ts, float64(ts), nil, nil, storage.AOptions{}) require.NoError(t, err) allSamples = append(allSamples, sample{t: ts, f: float64(ts)}) if afterRestart { @@ -7711,15 +6475,15 @@ func TestWBLCorruption(t *testing.T) { verifySamples(expAfterRestart) } -func TestOOOMmapCorruption(t *testing.T) { +func TestOOOMmapCorruption_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOOOMmapCorruption(t, scenario) + testOOOMmapCorruptionAppendV2(t, scenario) }) } } -func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { +func testOOOMmapCorruptionAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderCapMax = 10 opts.OutOfOrderTimeWindow = 300 * time.Minute.Milliseconds() @@ -7729,10 +6493,10 @@ func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { series1 := labels.FromStrings("foo", "bar1") var allSamples, expInMmapChunks []chunks.Sample addSamples := func(fromMins, toMins int64, inMmapAfterCorruption bool) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, s, err := scenario.appendFunc(app, series1, ts, ts) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) allSamples = append(allSamples, s) if inMmapAfterCorruption { @@ -7835,16 +6599,16 @@ func testOOOMmapCorruption(t *testing.T, scenario sampleTypeScenario) { verifySamples(allSamples) } -func TestOutOfOrderRuntimeConfig(t *testing.T) { +func TestOutOfOrderRuntimeConfig_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testOutOfOrderRuntimeConfig(t, scenario) + testOutOfOrderRuntimeConfigAppendV2(t, scenario) }) } } -func testOutOfOrderRuntimeConfig(t *testing.T, scenario sampleTypeScenario) { +func testOutOfOrderRuntimeConfigAppendV2(t *testing.T, scenario sampleTypeScenario) { ctx := context.Background() getDB := func(oooTimeWindow int64) *DB { @@ -7867,10 +6631,10 @@ func testOutOfOrderRuntimeConfig(t *testing.T, scenario sampleTypeScenario) { series1 := labels.FromStrings("foo", "bar1") addSamples := func(t *testing.T, db *DB, fromMins, toMins int64, success bool, allSamples []chunks.Sample) []chunks.Sample { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, s, err := scenario.appendFunc(app, series1, ts, ts) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) if success { require.NoError(t, err) allSamples = append(allSamples, s) @@ -8067,22 +6831,22 @@ func testOutOfOrderRuntimeConfig(t *testing.T, scenario sampleTypeScenario) { }) } -func TestNoGapAfterRestartWithOOO(t *testing.T) { +func TestNoGapAfterRestartWithOOO_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testNoGapAfterRestartWithOOO(t, scenario) + testNoGapAfterRestartWithOOOAppendV2(t, scenario) }) } } -func testNoGapAfterRestartWithOOO(t *testing.T, scenario sampleTypeScenario) { +func testNoGapAfterRestartWithOOOAppendV2(t *testing.T, scenario sampleTypeScenario) { series1 := labels.FromStrings("foo", "bar1") addSamples := func(t *testing.T, db *DB, fromMins, toMins int64, success bool) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, _, err := scenario.appendFunc(app, series1, ts, ts) + _, _, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) if success { require.NoError(t, err) } else { @@ -8176,15 +6940,15 @@ func testNoGapAfterRestartWithOOO(t *testing.T, scenario sampleTypeScenario) { } } -func TestWblReplayAfterOOODisableAndRestart(t *testing.T) { +func TestWblReplayAfterOOODisableAndRestart_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testWblReplayAfterOOODisableAndRestart(t, scenario) + testWblReplayAfterOOODisableAndRestartAppendV2(t, scenario) }) } } -func testWblReplayAfterOOODisableAndRestart(t *testing.T, scenario sampleTypeScenario) { +func testWblReplayAfterOOODisableAndRestartAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() @@ -8193,10 +6957,10 @@ func testWblReplayAfterOOODisableAndRestart(t *testing.T, scenario sampleTypeSce series1 := labels.FromStrings("foo", "bar1") var allSamples []chunks.Sample addSamples := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, s, err := scenario.appendFunc(app, series1, ts, ts) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) allSamples = append(allSamples, s) } @@ -8236,15 +7000,15 @@ func testWblReplayAfterOOODisableAndRestart(t *testing.T, scenario sampleTypeSce verifySamples(allSamples) } -func TestPanicOnApplyConfig(t *testing.T) { +func TestPanicOnApplyConfig_AppendV2(t *testing.T) { for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testPanicOnApplyConfig(t, scenario) + testPanicOnApplyConfigAppendV2(t, scenario) }) } } -func testPanicOnApplyConfig(t *testing.T, scenario sampleTypeScenario) { +func testPanicOnApplyConfigAppendV2(t *testing.T, scenario sampleTypeScenario) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = 60 * time.Minute.Milliseconds() @@ -8253,10 +7017,10 @@ func testPanicOnApplyConfig(t *testing.T, scenario sampleTypeScenario) { series1 := labels.FromStrings("foo", "bar1") var allSamples []chunks.Sample addSamples := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, s, err := scenario.appendFunc(app, series1, ts, ts) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) allSamples = append(allSamples, s) } @@ -8285,16 +7049,16 @@ func testPanicOnApplyConfig(t *testing.T, scenario sampleTypeScenario) { require.NoError(t, err) } -func TestDiskFillingUpAfterDisablingOOO(t *testing.T) { +func TestDiskFillingUpAfterDisablingOOO_AppendV2(t *testing.T) { t.Parallel() for name, scenario := range sampleTypeScenarios { t.Run(name, func(t *testing.T) { - testDiskFillingUpAfterDisablingOOO(t, scenario) + testDiskFillingUpAfterDisablingOOOAppenderV2(t, scenario) }) } } -func testDiskFillingUpAfterDisablingOOO(t *testing.T, scenario sampleTypeScenario) { +func testDiskFillingUpAfterDisablingOOOAppenderV2(t *testing.T, scenario sampleTypeScenario) { t.Parallel() ctx := context.Background() @@ -8307,10 +7071,10 @@ func testDiskFillingUpAfterDisablingOOO(t *testing.T, scenario sampleTypeScenari series1 := labels.FromStrings("foo", "bar1") var allSamples []chunks.Sample addSamples := func(fromMins, toMins int64) { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for m := fromMins; m <= toMins; m++ { ts := m * time.Minute.Milliseconds() - _, s, err := scenario.appendFunc(app, series1, ts, ts) + _, s, err := scenario.appendFunc(storage.AppenderV2AsLimitedV1(app), series1, ts, ts) require.NoError(t, err) allSamples = append(allSamples, s) } @@ -8376,19 +7140,22 @@ func testDiskFillingUpAfterDisablingOOO(t *testing.T, scenario sampleTypeScenari require.Equal(t, int64(0), finfo.Size()) } -func TestHistogramAppendAndQuery(t *testing.T) { +func TestHistogramAppendAndQuery_AppendV2(t *testing.T) { t.Run("integer histograms", func(t *testing.T) { - testHistogramAppendAndQueryHelper(t, false) + testHistogramAppendAndQueryHelperAppendV2(t, false) }) t.Run("float histograms", func(t *testing.T) { - testHistogramAppendAndQueryHelper(t, true) + testHistogramAppendAndQueryHelperAppendV2(t, true) }) } -func testHistogramAppendAndQueryHelper(t *testing.T, floatHistogram bool) { +func testHistogramAppendAndQueryHelperAppendV2(t *testing.T, floatHistogram bool) { t.Helper() db := newTestDB(t) minute := func(m int) int64 { return int64(m) * time.Minute.Milliseconds() } + t.Cleanup(func() { + require.NoError(t, db.Close()) + }) ctx := context.Background() appendHistogram := func(t *testing.T, @@ -8397,14 +7164,14 @@ func testHistogramAppendAndQueryHelper(t *testing.T, floatHistogram bool) { ) { t.Helper() var err error - app := db.Appender(ctx) + app := db.AppenderV2(ctx) if floatHistogram { - _, err = app.AppendHistogram(0, lbls, minute(tsMinute), nil, h.ToFloat(nil)) + _, err = app.Append(0, lbls, 0, minute(tsMinute), 0, nil, h.ToFloat(nil), storage.AOptions{}) efh := h.ToFloat(nil) efh.CounterResetHint = expCRH *exp = append(*exp, sample{t: minute(tsMinute), fh: efh}) } else { - _, err = app.AppendHistogram(0, lbls, minute(tsMinute), h.Copy(), nil) + _, err = app.Append(0, lbls, 0, minute(tsMinute), 0, h.Copy(), nil, storage.AOptions{}) eh := h.Copy() eh.CounterResetHint = expCRH *exp = append(*exp, sample{t: minute(tsMinute), h: eh}) @@ -8414,8 +7181,8 @@ func testHistogramAppendAndQueryHelper(t *testing.T, floatHistogram bool) { } appendFloat := func(t *testing.T, lbls labels.Labels, tsMinute int, val float64, exp *[]chunks.Sample) { t.Helper() - app := db.Appender(ctx) - _, err := app.Append(0, lbls, minute(tsMinute), val) + app := db.AppenderV2(ctx) + _, err := app.Append(0, lbls, 0, minute(tsMinute), val, nil, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) *exp = append(*exp, sample{t: minute(tsMinute), f: val}) @@ -8647,139 +7414,7 @@ func testHistogramAppendAndQueryHelper(t *testing.T, floatHistogram bool) { }) } -func TestQueryHistogramFromBlocksWithCompaction(t *testing.T) { - t.Parallel() - minute := func(m int) int64 { return int64(m) * time.Minute.Milliseconds() } - - testBlockQuerying := func(t *testing.T, blockSeries ...[]storage.Series) { - t.Helper() - - opts := DefaultOptions() - db := newTestDB(t, withOpts(opts)) - - var it chunkenc.Iterator - exp := make(map[string][]chunks.Sample) - for _, series := range blockSeries { - createBlock(t, db.Dir(), series) - - for _, s := range series { - lbls := s.Labels().String() - slice := exp[lbls] - it = s.Iterator(it) - smpls, err := storage.ExpandSamples(it, nil) - require.NoError(t, err) - slice = append(slice, smpls...) - sort.Slice(slice, func(i, j int) bool { - return slice[i].T() < slice[j].T() - }) - exp[lbls] = slice - } - } - - require.Empty(t, db.Blocks()) - require.NoError(t, db.reload()) - require.Len(t, db.Blocks(), len(blockSeries)) - - q, err := db.Querier(math.MinInt64, math.MaxInt64) - require.NoError(t, err) - res := query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*")) - compareSeries(t, exp, res) - - // Compact all the blocks together and query again. - blocks := db.Blocks() - blockDirs := make([]string, 0, len(blocks)) - for _, b := range blocks { - blockDirs = append(blockDirs, b.Dir()) - } - ids, err := db.compactor.Compact(db.Dir(), blockDirs, blocks) - require.NoError(t, err) - require.Len(t, ids, 1) - require.NoError(t, db.reload()) - require.Len(t, db.Blocks(), 1) - - q, err = db.Querier(math.MinInt64, math.MaxInt64) - require.NoError(t, err) - res = query(t, q, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*")) - - // After compaction, we do not require "unknown" counter resets - // due to origin from different overlapping chunks anymore. - for _, ss := range exp { - for i, s := range ss[1:] { - if s.Type() == chunkenc.ValHistogram && ss[i].Type() == chunkenc.ValHistogram && s.H().CounterResetHint == histogram.UnknownCounterReset { - s.H().CounterResetHint = histogram.NotCounterReset - } - if s.Type() == chunkenc.ValFloatHistogram && ss[i].Type() == chunkenc.ValFloatHistogram && s.FH().CounterResetHint == histogram.UnknownCounterReset { - s.FH().CounterResetHint = histogram.NotCounterReset - } - } - } - compareSeries(t, exp, res) - } - - for _, floatHistogram := range []bool{false, true} { - t.Run(fmt.Sprintf("floatHistogram=%t", floatHistogram), func(t *testing.T) { - t.Run("serial blocks with only histograms", func(t *testing.T) { - testBlockQuerying(t, - genHistogramSeries(10, 5, minute(0), minute(119), minute(1), floatHistogram), - genHistogramSeries(10, 5, minute(120), minute(239), minute(1), floatHistogram), - genHistogramSeries(10, 5, minute(240), minute(359), minute(1), floatHistogram), - ) - }) - - t.Run("serial blocks with either histograms or floats in a block and not both", func(t *testing.T) { - testBlockQuerying(t, - genHistogramSeries(10, 5, minute(0), minute(119), minute(1), floatHistogram), - genSeriesFromSampleGenerator(10, 5, minute(120), minute(239), minute(1), func(ts int64) chunks.Sample { - return sample{t: ts, f: rand.Float64()} - }), - genHistogramSeries(10, 5, minute(240), minute(359), minute(1), floatHistogram), - ) - }) - - t.Run("serial blocks with mix of histograms and float64", func(t *testing.T) { - testBlockQuerying(t, - genHistogramAndFloatSeries(10, 5, minute(0), minute(60), minute(1), floatHistogram), - genHistogramSeries(10, 5, minute(61), minute(120), minute(1), floatHistogram), - genHistogramAndFloatSeries(10, 5, minute(121), minute(180), minute(1), floatHistogram), - genSeriesFromSampleGenerator(10, 5, minute(181), minute(240), minute(1), func(ts int64) chunks.Sample { - return sample{t: ts, f: rand.Float64()} - }), - ) - }) - - t.Run("overlapping blocks with only histograms", func(t *testing.T) { - testBlockQuerying(t, - genHistogramSeries(10, 5, minute(0), minute(120), minute(3), floatHistogram), - genHistogramSeries(10, 5, minute(1), minute(120), minute(3), floatHistogram), - genHistogramSeries(10, 5, minute(2), minute(120), minute(3), floatHistogram), - ) - }) - - t.Run("overlapping blocks with only histograms and only float in a series", func(t *testing.T) { - testBlockQuerying(t, - genHistogramSeries(10, 5, minute(0), minute(120), minute(3), floatHistogram), - genSeriesFromSampleGenerator(10, 5, minute(1), minute(120), minute(3), func(ts int64) chunks.Sample { - return sample{t: ts, f: rand.Float64()} - }), - genHistogramSeries(10, 5, minute(2), minute(120), minute(3), floatHistogram), - ) - }) - - t.Run("overlapping blocks with mix of histograms and float64", func(t *testing.T) { - testBlockQuerying(t, - genHistogramAndFloatSeries(10, 5, minute(0), minute(60), minute(3), floatHistogram), - genHistogramSeries(10, 5, minute(46), minute(100), minute(3), floatHistogram), - genHistogramAndFloatSeries(10, 5, minute(89), minute(140), minute(3), floatHistogram), - genSeriesFromSampleGenerator(10, 5, minute(126), minute(200), minute(3), func(ts int64) chunks.Sample { - return sample{t: ts, f: rand.Float64()} - }), - ) - }) - }) - } -} - -func TestOOONativeHistogramsSettings(t *testing.T) { +func TestOOONativeHistogramsSettings_AppendV2(t *testing.T) { h := &histogram.Histogram{ Count: 9, ZeroCount: 4, @@ -8800,11 +7435,11 @@ func TestOOONativeHistogramsSettings(t *testing.T) { opts.OutOfOrderTimeWindow = 0 db := newTestDB(t, withOpts(opts), withRngs(100)) - app := db.Appender(context.Background()) - _, err := app.AppendHistogram(0, l, 100, h, nil) + app := db.AppenderV2(context.Background()) + _, err := app.Append(0, l, 0, 100, 0, h, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.AppendHistogram(0, l, 50, h, nil) + _, err = app.Append(0, l, 0, 50, 0, h, nil, storage.AOptions{}) require.NoError(t, err) // The OOO sample is not detected until it is committed, so no error is returned require.NoError(t, app.Commit()) @@ -8822,14 +7457,14 @@ func TestOOONativeHistogramsSettings(t *testing.T) { db := newTestDB(t, withOpts(opts), withRngs(100)) // Add in-order samples - app := db.Appender(context.Background()) - _, err := app.AppendHistogram(0, l, 200, h, nil) + app := db.AppenderV2(context.Background()) + _, err := app.Append(0, l, 0, 200, 0, h, nil, storage.AOptions{}) require.NoError(t, err) // Add OOO samples - _, err = app.AppendHistogram(0, l, 100, h, nil) + _, err = app.Append(0, l, 0, 100, 0, h, nil, storage.AOptions{}) require.NoError(t, err) - _, err = app.AppendHistogram(0, l, 150, h, nil) + _, err = app.Append(0, l, 0, 150, 0, h, nil, storage.AOptions{}) require.NoError(t, err) require.NoError(t, app.Commit()) @@ -8843,71 +7478,11 @@ func TestOOONativeHistogramsSettings(t *testing.T) { }) } -// compareSeries essentially replaces `require.Equal(t, expected, actual)` in -// situations where the actual series might contain more counter reset hints -// "unknown" than the expected series. This can easily happen for long series -// that trigger new chunks. This function therefore tolerates counter reset -// hints "CounterReset" and "NotCounterReset" in an expected series where the -// actual series contains a counter reset hint "UnknownCounterReset". -// "GaugeType" hints are still strictly checked, and any "UnknownCounterReset" -// in an expected series has to be matched precisely by the actual series. -func compareSeries(t require.TestingT, expected, actual map[string][]chunks.Sample) { - if len(expected) != len(actual) { - // The reason for the difference is not the counter reset hints - // (alone), so let's use the pretty diffing by the require - // package. - require.Equal(t, expected, actual, "number of series differs") - } - for key, expSamples := range expected { - actSamples, ok := actual[key] - if !ok { - require.Equal(t, expected, actual, "expected series %q not found", key) - } - if len(expSamples) != len(actSamples) { - require.Equal(t, expSamples, actSamples, "number of samples for series %q differs", key) - } - - for i, eS := range expSamples { - aS := actSamples[i] - - // Must use the interface as Equal does not work when actual types differ - // not only does the type differ, but chunk.Sample.FH() interface may auto convert from chunk.Sample.H()! - require.Equal(t, eS.T(), aS.T(), "timestamp of sample %d in series %q differs", i, key) - - require.Equal(t, eS.Type(), aS.Type(), "type of sample %d in series %q differs", i, key) - - switch eS.Type() { - case chunkenc.ValFloat: - require.Equal(t, eS.F(), aS.F(), "sample %d in series %q differs", i, key) - case chunkenc.ValHistogram: - eH, aH := eS.H(), aS.H() - if aH.CounterResetHint == histogram.UnknownCounterReset { - eH = eH.Copy() - // It is always safe to set the counter reset hint to UnknownCounterReset - eH.CounterResetHint = histogram.UnknownCounterReset - eS = sample{t: eS.T(), h: eH} - } - require.Equal(t, eH, aH, "histogram sample %d in series %q differs", i, key) - - case chunkenc.ValFloatHistogram: - eFH, aFH := eS.FH(), aS.FH() - if aFH.CounterResetHint == histogram.UnknownCounterReset { - eFH = eFH.Copy() - // It is always safe to set the counter reset hint to UnknownCounterReset - eFH.CounterResetHint = histogram.UnknownCounterReset - eS = sample{t: eS.T(), fh: eFH} - } - require.Equal(t, eFH, aFH, "float histogram sample %d in series %q differs", i, key) - } - } - } -} - // TestChunkQuerierReadWriteRace looks for any possible race between appending // samples and reading chunks because the head chunk that is being appended to // can be read in parallel and we should be able to make a copy of the chunk without // worrying about the parallel write. -func TestChunkQuerierReadWriteRace(t *testing.T) { +func TestChunkQuerierReadWriteRace_AppendV2(t *testing.T) { t.Parallel() db := newTestDB(t) @@ -8917,10 +7492,10 @@ func TestChunkQuerierReadWriteRace(t *testing.T) { <-time.After(5 * time.Millisecond) // Initial pause while readers start. ts := 0 for range 500 { - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for range 10 { ts++ - _, err := app.Append(0, lbls, int64(ts), float64(ts*100)) + _, err := app.Append(0, lbls, 0, int64(ts), float64(ts*100), nil, nil, storage.AOptions{}) if err != nil { return err } @@ -8974,26 +7549,8 @@ Outer: require.NoError(t, writerErr) } -type mockCompactorFn struct { - planFn func() ([]string, error) - compactFn func() ([]ulid.ULID, error) - writeFn func() ([]ulid.ULID, error) -} - -func (c *mockCompactorFn) Plan(string) ([]string, error) { - return c.planFn() -} - -func (c *mockCompactorFn) Compact(string, []string, []*Block) ([]ulid.ULID, error) { - return c.compactFn() -} - -func (c *mockCompactorFn) Write(string, BlockReader, int64, int64, *BlockMeta) ([]ulid.ULID, error) { - return c.writeFn() -} - // Regression test for https://github.com/prometheus/prometheus/pull/13754 -func TestAbortBlockCompactions(t *testing.T) { +func TestAbortBlockCompactions_AppendV2(t *testing.T) { // Create a test DB db := newTestDB(t) // It should NOT be compactable at the beginning of the test @@ -9031,7 +7588,7 @@ func TestAbortBlockCompactions(t *testing.T) { require.Equal(t, 4, compactions, "expected 4 compactions to be completed") } -func TestNewCompactorFunc(t *testing.T) { +func TestNewCompactorFunc_AppendV2(t *testing.T) { opts := DefaultOptions() block1 := ulid.MustNew(1, nil) block2 := ulid.MustNew(2, nil) @@ -9062,225 +7619,3 @@ func TestNewCompactorFunc(t *testing.T) { require.Len(t, ulids, 1) require.Equal(t, block2, ulids[0]) } - -func TestBlockQuerierAndBlockChunkQuerier(t *testing.T) { - opts := DefaultOptions() - opts.BlockQuerierFunc = func(b BlockReader, mint, maxt int64) (storage.Querier, error) { - // Only block with hints can be queried. - if len(b.Meta().Compaction.Hints) > 0 { - return NewBlockQuerier(b, mint, maxt) - } - return storage.NoopQuerier(), nil - } - opts.BlockChunkQuerierFunc = func(b BlockReader, mint, maxt int64) (storage.ChunkQuerier, error) { - // Only level 4 compaction block can be queried. - if b.Meta().Compaction.Level == 4 { - return NewBlockChunkQuerier(b, mint, maxt) - } - return storage.NoopChunkedQuerier(), nil - } - - db := newTestDB(t, withOpts(opts)) - - metas := []BlockMeta{ - {Compaction: BlockMetaCompaction{Hints: []string{"test-hint"}}}, - {Compaction: BlockMetaCompaction{Level: 4}}, - } - for i := range metas { - // Include blockID into series to identify which block got touched. - serieses := []storage.Series{storage.NewListSeries(labels.FromMap(map[string]string{"block": fmt.Sprintf("block-%d", i), labels.MetricName: "test_metric"}), []chunks.Sample{sample{t: 0, f: 1}})} - blockDir := createBlock(t, db.Dir(), serieses) - b, err := OpenBlock(db.logger, blockDir, db.chunkPool, nil) - require.NoError(t, err) - - // Overwrite meta.json with compaction section for testing purpose. - b.meta.Compaction = metas[i].Compaction - _, err = writeMetaFile(db.logger, blockDir, &b.meta) - require.NoError(t, err) - require.NoError(t, b.Close()) - } - require.NoError(t, db.reloadBlocks()) - require.Len(t, db.Blocks(), 2) - - querier, err := db.Querier(0, 500) - require.NoError(t, err) - defer querier.Close() - matcher := labels.MustNewMatcher(labels.MatchEqual, labels.MetricName, "test_metric") - seriesSet := querier.Select(context.Background(), false, nil, matcher) - count := 0 - var lbls labels.Labels - for seriesSet.Next() { - count++ - lbls = seriesSet.At().Labels() - } - require.NoError(t, seriesSet.Err()) - require.Equal(t, 1, count) - // Make sure only block-0 is queried. - require.Equal(t, "block-0", lbls.Get("block")) - - chunkQuerier, err := db.ChunkQuerier(0, 500) - require.NoError(t, err) - defer chunkQuerier.Close() - css := chunkQuerier.Select(context.Background(), false, nil, matcher) - count = 0 - // Reset lbls variable. - lbls = labels.EmptyLabels() - for css.Next() { - count++ - lbls = css.At().Labels() - } - require.NoError(t, css.Err()) - require.Equal(t, 1, count) - // Make sure only block-1 is queried. - require.Equal(t, "block-1", lbls.Get("block")) -} - -func TestGenerateCompactionDelay(t *testing.T) { - assertDelay := func(delay time.Duration, expectedMaxPercentDelay int) { - t.Helper() - require.GreaterOrEqual(t, delay, time.Duration(0)) - // Expect to generate a delay up to MaxPercentDelay of the head chunk range - require.LessOrEqual(t, delay, (time.Duration(60000*expectedMaxPercentDelay/100) * time.Millisecond)) - } - - opts := DefaultOptions() - cases := []struct { - compactionDelayPercent int - }{ - { - compactionDelayPercent: 1, - }, - { - compactionDelayPercent: 10, - }, - { - compactionDelayPercent: 60, - }, - { - compactionDelayPercent: 100, - }, - } - - opts.EnableDelayedCompaction = true - - for _, c := range cases { - opts.CompactionDelayMaxPercent = c.compactionDelayPercent - db := newTestDB(t, withOpts(opts), withRngs(60000)) - - // The offset is generated and changed while opening. - assertDelay(db.opts.CompactionDelay, c.compactionDelayPercent) - - for range 1000 { - assertDelay(db.generateCompactionDelay(), c.compactionDelayPercent) - } - } -} - -type blockedResponseRecorder struct { - r *httptest.ResponseRecorder - - // writeBlocked is used to block writing until the test wants it to resume. - writeBlocked chan struct{} - // writeStarted is closed by blockedResponseRecorder to signal that writing has started. - writeStarted chan struct{} -} - -func (br *blockedResponseRecorder) Write(buf []byte) (int, error) { - select { - case <-br.writeStarted: - default: - close(br.writeStarted) - } - - <-br.writeBlocked - return br.r.Write(buf) -} - -func (br *blockedResponseRecorder) Header() http.Header { return br.r.Header() } - -func (br *blockedResponseRecorder) WriteHeader(code int) { br.r.WriteHeader(code) } - -func (br *blockedResponseRecorder) Flush() { br.r.Flush() } - -// TestBlockClosingBlockedDuringRemoteRead ensures that a TSDB Block is not closed while it is being queried -// through remote read. This is a regression test for https://github.com/prometheus/prometheus/issues/14422. -// TODO: Ideally, this should reside in storage/remote/read_handler_test.go once the necessary TSDB utils are accessible there. -func TestBlockClosingBlockedDuringRemoteRead(t *testing.T) { - dir := t.TempDir() - - createBlock(t, dir, genSeries(2, 1, 0, 10)) - - // Not using newTestDB as db.Close is expected to return error. - db, err := Open(dir, nil, nil, nil, nil) - require.NoError(t, err) - defer db.Close() - - readAPI := remote.NewReadHandler( - nil, nil, db, - func() config.Config { - return config.Config{} - }, 0, 1, 0, - ) - - matcher, err := labels.NewMatcher(labels.MatchRegexp, "__name__", ".*") - require.NoError(t, err) - - query, err := remote.ToQuery(0, 10, []*labels.Matcher{matcher}, nil) - require.NoError(t, err) - - req := &prompb.ReadRequest{ - Queries: []*prompb.Query{query}, - AcceptedResponseTypes: []prompb.ReadRequest_ResponseType{prompb.ReadRequest_STREAMED_XOR_CHUNKS}, - } - data, err := proto.Marshal(req) - require.NoError(t, err) - - request, err := http.NewRequest(http.MethodPost, "", bytes.NewBuffer(snappy.Encode(nil, data))) - require.NoError(t, err) - - blockedRecorder := &blockedResponseRecorder{ - r: httptest.NewRecorder(), - writeBlocked: make(chan struct{}), - writeStarted: make(chan struct{}), - } - - readDone := make(chan struct{}) - go func() { - readAPI.ServeHTTP(blockedRecorder, request) - require.Equal(t, http.StatusOK, blockedRecorder.r.Code) - close(readDone) - }() - - // Wait for the read API to start streaming data. - <-blockedRecorder.writeStarted - - // Try to close the queried block. - blockClosed := make(chan struct{}) - go func() { - for _, block := range db.Blocks() { - block.Close() - } - close(blockClosed) - }() - - // Closing the queried block should block. - // Wait a little bit to make sure of that. - select { - case <-time.After(100 * time.Millisecond): - case <-readDone: - require.Fail(t, "read API should still be streaming data.") - case <-blockClosed: - require.Fail(t, "Block shouldn't get closed while being queried.") - } - - // Resume the read API data streaming. - close(blockedRecorder.writeBlocked) - <-readDone - - // The block should be no longer needed and closing it should end. - select { - case <-time.After(10 * time.Millisecond): - require.Fail(t, "Closing the block timed out.") - case <-blockClosed: - } -} From a1fcac9078af55156058371139c364bd8f3d7a7a Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Tue, 9 Dec 2025 10:48:44 +0000 Subject: [PATCH 120/439] maintainers: Add krajoma as TSDB maintainer This is to help with interface switch we are doing https://github.com/prometheus/prometheus/issues/17632 See https://cloud-native.slack.com/archives/C01AUBA4PFE/p1765277201832839?thread_ts=1765199584.855979&cid=C01AUBA4PFE Signed-off-by: Bartlomiej Plotka --- MAINTAINERS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index d36f82ca61..8d107b9774 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -18,7 +18,7 @@ George Krajcsovits ( / @krajorama) * `storage` * `remote`: Callum Styan ( / @cstyan), Bartłomiej Płotka ( / @bwplotka), Tom Wilkie (tom.wilkie@gmail.com / @tomwilkie), Nicolás Pazos ( / @npazosmendez), Alex Greenbank ( / @alexgreenbank) * `otlptranslator`: Arthur Silva Sens ( / @ArthurSens), Arve Knudsen ( / @aknuds1), Jesús Vázquez ( / @jesusvazquez) -* `tsdb`: Ganesh Vernekar ( / @codesome), Bartłomiej Płotka ( / @bwplotka), Jesús Vázquez ( / @jesusvazquez) +* `tsdb`: Ganesh Vernekar ( / @codesome), Bartłomiej Płotka ( / @bwplotka), Jesús Vázquez ( / @jesusvazquez), George Krajcsovits ( / @krajorama) * `web` * `ui`: Julius Volz ( / @juliusv) * `module`: Augustin Husson ( @nexucis) From 5b2661956506daa84148611db52eb32741739704 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Thu, 4 Dec 2025 10:57:50 +0100 Subject: [PATCH 121/439] web/api: Add maximum limit validation to TSDB status endpoint Add a maximum limit of 10,000 to the TSDB status endpoint to prevent resource exhaustion from excessively large limit values, as we preallocate []Stat for up to the limit: `make([]Stat, 0, length)`. Note that the endpoint acquires a cardinality mutex during stats calculation, so this can not be run in parallel. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- docs/querying/api.md | 2 +- web/api/v1/api.go | 4 ++++ web/api/v1/api_test.go | 12 ++++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/querying/api.md b/docs/querying/api.md index b377c6174e..4804443343 100644 --- a/docs/querying/api.md +++ b/docs/querying/api.md @@ -1346,7 +1346,7 @@ GET /api/v1/status/tsdb ``` URL query parameters: -- `limit=`: Limit the number of returned items to a given number for each set of statistics. By default, 10 items are returned. +- `limit=`: Limit the number of returned items to a given number for each set of statistics. By default, 10 items are returned. The maximum allowed limit is 10000. The `data` section of the query result consists of: diff --git a/web/api/v1/api.go b/web/api/v1/api.go index 86c0461087..fd3652f4e4 100644 --- a/web/api/v1/api.go +++ b/web/api/v1/api.go @@ -1837,12 +1837,16 @@ func (api *API) serveTSDBBlocks(*http.Request) apiFuncResult { } func (api *API) serveTSDBStatus(r *http.Request) apiFuncResult { + const maxTSDBLimit = 10000 limit := 10 if s := r.FormValue("limit"); s != "" { var err error if limit, err = strconv.Atoi(s); err != nil || limit < 1 { return apiFuncResult{nil, &apiError{errorBadData, errors.New("limit must be a positive number")}, nil, nil} } + if limit > maxTSDBLimit { + return apiFuncResult{nil, &apiError{errorBadData, fmt.Errorf("limit must not exceed %d", maxTSDBLimit)}, nil, nil} + } } s, err := api.db.Stats(labels.MetricName, limit) if err != nil { diff --git a/web/api/v1/api_test.go b/web/api/v1/api_test.go index 8e0adc0802..83e8618630 100644 --- a/web/api/v1/api_test.go +++ b/web/api/v1/api_test.go @@ -4465,6 +4465,18 @@ func TestTSDBStatus(t *testing.T) { values: map[string][]string{"limit": {"0"}}, errType: errorBadData, }, + { + db: tsdb, + endpoint: tsdbStatusAPI, + values: map[string][]string{"limit": {"10000"}}, + errType: errorNone, + }, + { + db: tsdb, + endpoint: tsdbStatusAPI, + values: map[string][]string{"limit": {"10001"}}, + errType: errorBadData, + }, } { t.Run(strconv.Itoa(i), func(t *testing.T) { api := &API{db: tc.db, gatherer: prometheus.DefaultGatherer} From a5671a002fd28da02f80c9dacff3bec947637630 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Wed, 29 Oct 2025 15:46:20 +0100 Subject: [PATCH 122/439] API: Add a /api/v1/features endpoint Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- Makefile | 5 + cmd/prometheus/features_test.go | 125 +++++++++++++ cmd/prometheus/main.go | 24 ++- cmd/prometheus/testdata/features.json | 249 ++++++++++++++++++++++++++ discovery/manager.go | 20 +++ discovery/registry.go | 10 ++ docs/querying/api.md | 77 ++++++++ model/labels/labels_dedupelabels.go | 3 + model/labels/labels_slicelabels.go | 3 + model/labels/labels_stringlabels.go | 3 + promql/engine.go | 16 ++ promql/parser/features.go | 57 ++++++ rules/manager.go | 11 ++ scrape/manager.go | 11 ++ template/template.go | 27 +++ tsdb/db.go | 13 ++ util/features/features.go | 127 +++++++++++++ web/api/v1/api.go | 29 +++ web/api/v1/errors_test.go | 1 + web/web.go | 25 ++- 20 files changed, 830 insertions(+), 6 deletions(-) create mode 100644 cmd/prometheus/features_test.go create mode 100644 cmd/prometheus/testdata/features.json create mode 100644 promql/parser/features.go create mode 100644 util/features/features.go diff --git a/Makefile b/Makefile index 43020998ef..30295c56e5 100644 --- a/Makefile +++ b/Makefile @@ -184,6 +184,11 @@ check-go-mod-version: @echo ">> checking go.mod version matching" @./scripts/check-go-mod-version.sh +.PHONY: update-features-testdata +update-features-testdata: + @echo ">> updating features testdata" + @$(GO) test ./cmd/prometheus -run TestFeaturesAPI -update-features + .PHONY: update-all-go-deps update-all-go-deps: @$(MAKE) update-go-deps diff --git a/cmd/prometheus/features_test.go b/cmd/prometheus/features_test.go new file mode 100644 index 0000000000..5907c87247 --- /dev/null +++ b/cmd/prometheus/features_test.go @@ -0,0 +1,125 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package main + +import ( + "bytes" + "encoding/json" + "flag" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "testing" + "time" + + "github.com/stretchr/testify/require" + + "github.com/prometheus/prometheus/util/testutil" +) + +var updateFeatures = flag.Bool("update-features", false, "update features.json golden file") + +func TestFeaturesAPI(t *testing.T) { + if testing.Short() { + t.Skip("skipping test in short mode.") + } + t.Parallel() + + tmpDir := t.TempDir() + configFile := filepath.Join(tmpDir, "prometheus.yml") + require.NoError(t, os.WriteFile(configFile, []byte{}, 0o644)) + + port := testutil.RandomUnprivilegedPort(t) + prom := prometheusCommandWithLogging( + t, + configFile, + port, + fmt.Sprintf("--storage.tsdb.path=%s", tmpDir), + ) + require.NoError(t, prom.Start()) + + baseURL := fmt.Sprintf("http://127.0.0.1:%d", port) + + // Wait for Prometheus to be ready. + require.Eventually(t, func() bool { + resp, err := http.Get(baseURL + "/-/ready") + if err != nil { + return false + } + defer resp.Body.Close() + return resp.StatusCode == http.StatusOK + }, 10*time.Second, 100*time.Millisecond, "Prometheus didn't become ready in time") + + // Fetch features from the API. + resp, err := http.Get(baseURL + "/api/v1/features") + require.NoError(t, err) + defer resp.Body.Close() + require.Equal(t, http.StatusOK, resp.StatusCode) + + body, err := io.ReadAll(resp.Body) + require.NoError(t, err) + + // Parse API response. + var apiResponse struct { + Status string `json:"status"` + Data map[string]map[string]bool `json:"data"` + } + require.NoError(t, json.Unmarshal(body, &apiResponse)) + require.Equal(t, "success", apiResponse.Status) + + goldenPath := filepath.Join("testdata", "features.json") + + // If update flag is set, write the current features to the golden file. + if *updateFeatures { + var buf bytes.Buffer + encoder := json.NewEncoder(&buf) + encoder.SetEscapeHTML(false) + encoder.SetIndent("", " ") + require.NoError(t, encoder.Encode(apiResponse.Data)) + // Ensure testdata directory exists. + require.NoError(t, os.MkdirAll(filepath.Dir(goldenPath), 0o755)) + require.NoError(t, os.WriteFile(goldenPath, buf.Bytes(), 0o644)) + t.Logf("Updated golden file: %s", goldenPath) + return + } + + // Load golden file. + goldenData, err := os.ReadFile(goldenPath) + require.NoError(t, err, "Failed to read golden file %s. Run 'make update-features-testdata' to generate it.", goldenPath) + + var expectedFeatures map[string]map[string]bool + require.NoError(t, json.Unmarshal(goldenData, &expectedFeatures)) + + // The labels implementation depends on build tags (stringlabels, slicelabels, or dedupelabels). + // We need to update the expected features to match the current build. + if prometheusFeatures, ok := expectedFeatures["prometheus"]; ok { + // Remove all label implementation features from expected. + delete(prometheusFeatures, "stringlabels") + delete(prometheusFeatures, "slicelabels") + delete(prometheusFeatures, "dedupelabels") + // Add the current implementation. + if actualPrometheus, ok := apiResponse.Data["prometheus"]; ok { + for _, impl := range []string{"stringlabels", "slicelabels", "dedupelabels"} { + if actualPrometheus[impl] { + prometheusFeatures[impl] = true + } + } + } + } + + // Compare the features data with the golden file. + require.Equal(t, expectedFeatures, apiResponse.Data, "Features mismatch. Run 'make update-features-testdata' to update the golden file.") +} diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go index f7757968b7..53379dc940 100644 --- a/cmd/prometheus/main.go +++ b/cmd/prometheus/main.go @@ -73,11 +73,13 @@ import ( "github.com/prometheus/prometheus/scrape" "github.com/prometheus/prometheus/storage" "github.com/prometheus/prometheus/storage/remote" + "github.com/prometheus/prometheus/template" "github.com/prometheus/prometheus/tracing" "github.com/prometheus/prometheus/tsdb" "github.com/prometheus/prometheus/tsdb/agent" "github.com/prometheus/prometheus/util/compression" "github.com/prometheus/prometheus/util/documentcli" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/logging" "github.com/prometheus/prometheus/util/notifications" prom_runtime "github.com/prometheus/prometheus/util/runtime" @@ -236,6 +238,7 @@ func (c *flagConfig) setFeatureListOptions(logger *slog.Logger) error { case "metadata-wal-records": c.scrape.AppendMetadata = true c.web.AppendMetadata = true + features.Enable(features.TSDB, "metadata_wal_records") logger.Info("Experimental metadata records in WAL enabled") case "promql-per-step-stats": c.enablePerStepStats = true @@ -342,10 +345,14 @@ func main() { Registerer: prometheus.DefaultRegisterer, }, web: web.Options{ - Registerer: prometheus.DefaultRegisterer, - Gatherer: prometheus.DefaultGatherer, + Registerer: prometheus.DefaultRegisterer, + Gatherer: prometheus.DefaultGatherer, + FeatureRegistry: features.DefaultRegistry, }, promslogConfig: promslog.Config{}, + scrape: scrape.Options{ + FeatureRegistry: features.DefaultRegistry, + }, } a := kingpin.New(filepath.Base(os.Args[0]), "The Prometheus monitoring server").UsageWriter(os.Stdout) @@ -797,6 +804,12 @@ func main() { "vm_limits", prom_runtime.VMLimits(), ) + features.Set(features.Prometheus, "agent_mode", agentMode) + features.Set(features.Prometheus, "server_mode", !agentMode) + features.Set(features.Prometheus, "auto_reload_config", cfg.enableAutoReload) + features.Enable(features.Prometheus, labels.ImplementationName) + template.RegisterFeatures(features.DefaultRegistry) + var ( localStorage = &readyStorage{stats: tsdb.NewDBStats()} scraper = &readyScrapeManager{} @@ -833,13 +846,13 @@ func main() { os.Exit(1) } - discoveryManagerScrape = discovery.NewManager(ctxScrape, logger.With("component", "discovery manager scrape"), prometheus.DefaultRegisterer, sdMetrics, discovery.Name("scrape")) + discoveryManagerScrape = discovery.NewManager(ctxScrape, logger.With("component", "discovery manager scrape"), prometheus.DefaultRegisterer, sdMetrics, discovery.Name("scrape"), discovery.FeatureRegistry(features.DefaultRegistry)) if discoveryManagerScrape == nil { logger.Error("failed to create a discovery manager scrape") os.Exit(1) } - discoveryManagerNotify = discovery.NewManager(ctxNotify, logger.With("component", "discovery manager notify"), prometheus.DefaultRegisterer, sdMetrics, discovery.Name("notify")) + discoveryManagerNotify = discovery.NewManager(ctxNotify, logger.With("component", "discovery manager notify"), prometheus.DefaultRegisterer, sdMetrics, discovery.Name("notify"), discovery.FeatureRegistry(features.DefaultRegistry)) if discoveryManagerNotify == nil { logger.Error("failed to create a discovery manager notify") os.Exit(1) @@ -880,6 +893,7 @@ func main() { EnablePerStepStats: cfg.enablePerStepStats, EnableDelayedNameRemoval: cfg.promqlEnableDelayedNameRemoval, EnableTypeAndUnitLabels: cfg.scrape.EnableTypeAndUnitLabels, + FeatureRegistry: features.DefaultRegistry, } queryEngine = promql.NewEngine(opts) @@ -902,6 +916,7 @@ func main() { DefaultRuleQueryOffset: func() time.Duration { return time.Duration(cfgFile.GlobalConfig.RuleQueryOffset) }, + FeatureRegistry: features.DefaultRegistry, }) } @@ -1919,6 +1934,7 @@ func (opts tsdbOptions) ToTSDBOptions() tsdb.Options { EnableOverlappingCompaction: opts.EnableOverlappingCompaction, UseUncachedIO: opts.UseUncachedIO, BlockCompactionExcludeFunc: opts.BlockCompactionExcludeFunc, + FeatureRegistry: features.DefaultRegistry, } } diff --git a/cmd/prometheus/testdata/features.json b/cmd/prometheus/testdata/features.json new file mode 100644 index 0000000000..fbffd941fd --- /dev/null +++ b/cmd/prometheus/testdata/features.json @@ -0,0 +1,249 @@ +{ + "api": { + "admin": false, + "exclude_alerts": true, + "label_values_match": true, + "lifecycle": false, + "otlp_write_receiver": false, + "query_stats": true, + "query_warnings": true, + "remote_write_receiver": false, + "time_range_labels": true, + "time_range_series": true + }, + "otlp_receiver": { + "delta_conversion": false, + "native_delta_ingestion": false + }, + "prometheus": { + "agent_mode": false, + "auto_reload_config": false, + "server_mode": true, + "stringlabels": true + }, + "promql": { + "anchored": false, + "at_modifier": true, + "bool": true, + "by": true, + "delayed_name_removal": false, + "duration_expr": false, + "group_left": true, + "group_right": true, + "ignoring": true, + "negative_offset": true, + "offset": true, + "on": true, + "per_query_lookback_delta": true, + "per_step_stats": false, + "smoothed": false, + "subqueries": true, + "type_and_unit_labels": false, + "without": true + }, + "promql_functions": { + "abs": true, + "absent": true, + "absent_over_time": true, + "acos": true, + "acosh": true, + "asin": true, + "asinh": true, + "atan": true, + "atanh": true, + "avg_over_time": true, + "ceil": true, + "changes": true, + "clamp": true, + "clamp_max": true, + "clamp_min": true, + "cos": true, + "cosh": true, + "count_over_time": true, + "day_of_month": true, + "day_of_week": true, + "day_of_year": true, + "days_in_month": true, + "deg": true, + "delta": true, + "deriv": true, + "double_exponential_smoothing": false, + "exp": true, + "first_over_time": false, + "floor": true, + "histogram_avg": true, + "histogram_count": true, + "histogram_fraction": true, + "histogram_quantile": true, + "histogram_stddev": true, + "histogram_stdvar": true, + "histogram_sum": true, + "hour": true, + "idelta": true, + "increase": true, + "info": false, + "irate": true, + "label_join": true, + "label_replace": true, + "last_over_time": true, + "ln": true, + "log10": true, + "log2": true, + "mad_over_time": false, + "max_over_time": true, + "min_over_time": true, + "minute": true, + "month": true, + "pi": true, + "predict_linear": true, + "present_over_time": true, + "quantile_over_time": true, + "rad": true, + "rate": true, + "resets": true, + "round": true, + "scalar": true, + "sgn": true, + "sin": true, + "sinh": true, + "sort": true, + "sort_by_label": false, + "sort_by_label_desc": false, + "sort_desc": true, + "sqrt": true, + "stddev_over_time": true, + "stdvar_over_time": true, + "sum_over_time": true, + "tan": true, + "tanh": true, + "time": true, + "timestamp": true, + "ts_of_first_over_time": false, + "ts_of_last_over_time": false, + "ts_of_max_over_time": false, + "ts_of_min_over_time": false, + "vector": true, + "year": true + }, + "promql_operators": { + "!=": true, + "!~": true, + "%": true, + "*": true, + "+": true, + "-": true, + "/": true, + "<": true, + "<=": true, + "==": true, + "=~": true, + ">": true, + ">=": true, + "@": true, + "^": true, + "and": true, + "atan2": true, + "avg": true, + "bottomk": true, + "count": true, + "count_values": true, + "group": true, + "limit_ratio": false, + "limitk": false, + "max": true, + "min": true, + "or": true, + "quantile": true, + "stddev": true, + "stdvar": true, + "sum": true, + "topk": true, + "unless": true + }, + "rules": { + "concurrent_rule_eval": false, + "keep_firing_for": true, + "query_offset": true + }, + "scrape": { + "extra_scrape_metrics": false, + "start_timestamp_zero_ingestion": false, + "type_and_unit_labels": false + }, + "service_discovery_providers": { + "aws": true, + "azure": true, + "consul": true, + "digitalocean": true, + "dns": true, + "docker": true, + "dockerswarm": true, + "ec2": true, + "ecs": true, + "eureka": true, + "file": true, + "gce": true, + "hetzner": true, + "http": true, + "ionos": true, + "kubernetes": true, + "kuma": true, + "lightsail": true, + "linode": true, + "marathon": true, + "nerve": true, + "nomad": true, + "openstack": true, + "ovhcloud": true, + "puppetdb": true, + "scaleway": true, + "serverset": true, + "stackit": true, + "static": true, + "triton": true, + "uyuni": true, + "vultr": true + }, + "templating_functions": { + "args": true, + "externalURL": true, + "first": true, + "graphLink": true, + "humanize": true, + "humanize1024": true, + "humanizeDuration": true, + "humanizePercentage": true, + "humanizeTimestamp": true, + "label": true, + "match": true, + "now": true, + "parseDuration": true, + "pathPrefix": true, + "query": true, + "reReplaceAll": true, + "safeHtml": true, + "sortByLabel": true, + "stripDomain": true, + "stripPort": true, + "strvalue": true, + "tableLink": true, + "title": true, + "toDuration": true, + "toLower": true, + "toTime": true, + "toUpper": true, + "urlQueryEscape": true, + "value": true + }, + "tsdb": { + "delayed_compaction": false, + "exemplar_storage": false, + "isolation": true, + "native_histograms": true, + "use_uncached_io": false + }, + "ui": { + "ui_v2": false, + "ui_v3": true + } +} diff --git a/discovery/manager.go b/discovery/manager.go index 878bc5f6d4..431050aa0b 100644 --- a/discovery/manager.go +++ b/discovery/manager.go @@ -27,6 +27,7 @@ import ( "github.com/prometheus/common/promslog" "github.com/prometheus/prometheus/discovery/targetgroup" + "github.com/prometheus/prometheus/util/features" ) type poolKey struct { @@ -111,6 +112,13 @@ func NewManager(ctx context.Context, logger *slog.Logger, registerer prometheus. } mgr.metrics = metrics + // Register all available service discovery providers with the feature registry. + if mgr.featureRegistry != nil { + for _, sdName := range RegisteredConfigNames() { + mgr.featureRegistry.Enable(features.ServiceDiscoveryProviders, sdName) + } + } + return mgr } @@ -141,6 +149,15 @@ func HTTPClientOptions(opts ...config.HTTPClientOption) func(*Manager) { } } +// FeatureRegistry sets the feature registry for the manager. +func FeatureRegistry(fr features.Collector) func(*Manager) { + return func(m *Manager) { + m.mtx.Lock() + defer m.mtx.Unlock() + m.featureRegistry = fr + } +} + // Manager maintains a set of discovery providers and sends each update to a map channel. // Targets are grouped by the target set name. type Manager struct { @@ -175,6 +192,9 @@ type Manager struct { metrics *Metrics sdMetrics map[string]DiscovererMetrics + + // featureRegistry is used to track which service discovery providers are configured. + featureRegistry features.Collector } // Providers returns the currently configured SD providers. diff --git a/discovery/registry.go b/discovery/registry.go index 33938cef3e..b3b82cdeec 100644 --- a/discovery/registry.go +++ b/discovery/registry.go @@ -280,3 +280,13 @@ func RegisterSDMetrics(registerer prometheus.Registerer, rmm RefreshMetricsManag } return metrics, nil } + +// RegisteredConfigNames returns the names of all registered service discovery providers. +func RegisteredConfigNames() []string { + names := make([]string, 0, len(configNames)) + for name := range configNames { + names = append(names, name) + } + sort.Strings(names) + return names +} diff --git a/docs/querying/api.md b/docs/querying/api.md index b377c6174e..19d4a339e4 100644 --- a/docs/querying/api.md +++ b/docs/querying/api.md @@ -1700,3 +1700,80 @@ GET /api/v1/notifications/live ``` *New in v3.0* + +### Features + +The following endpoint returns a list of enabled features in the Prometheus server: + +``` +GET /api/v1/features +``` + +This endpoint provides information about which features are currently enabled or disabled in the Prometheus instance. Features are organized into categories such as `api`, `promql`, `promql_functions`, etc. + +The `data` section contains a map where each key is a feature category, and each value is a map of feature names to their enabled status (boolean). + +```bash +curl http://localhost:9090/api/v1/features +``` + +```json +{ + "status": "success", + "data": { + "api": { + "admin": false, + "exclude_alerts": true + }, + "otlp_receiver": { + "delta_conversion": false, + "native_delta_ingestion": false + }, + "prometheus": { + "agent_mode": false, + "auto_reload_config": false + }, + "promql": { + "anchored": false, + "at_modifier": true + }, + "promql_functions": { + "abs": true, + "absent": true + }, + "promql_operators": { + "!=": true, + "!~": true + }, + "rules": { + "concurrent_rule_eval": false, + "keep_firing_for": true + }, + "scrape": { + "start_timestamp_zero_ingestion": false, + "extra_metrics": false + }, + "service_discovery": { + "azure": true, + "consul": true + }, + "templating": { + "args": true, + "externalURL": true + }, + "tsdb": { + "delayed_compaction": false, + "exemplar_storage": false + } + } +} +``` + +**Notes:** + +- All feature names use `snake_case` naming convention +- Features set to `false` may be omitted from the response +- Clients should treat absent features as equivalent to `false` +- Clients must ignore unknown feature names and categories for forward compatibility + +*New in v3.8* diff --git a/model/labels/labels_dedupelabels.go b/model/labels/labels_dedupelabels.go index 1e736c832e..4518482c96 100644 --- a/model/labels/labels_dedupelabels.go +++ b/model/labels/labels_dedupelabels.go @@ -24,6 +24,9 @@ import ( "github.com/cespare/xxhash/v2" ) +// ImplementationName is the name of the labels implementation. +const ImplementationName = "dedupelabels" + // Labels is implemented by a SymbolTable and string holding name/value // pairs encoded as indexes into the table in varint encoding. // Names are in alphabetical order. diff --git a/model/labels/labels_slicelabels.go b/model/labels/labels_slicelabels.go index 21ad145c1c..df3524abf6 100644 --- a/model/labels/labels_slicelabels.go +++ b/model/labels/labels_slicelabels.go @@ -25,6 +25,9 @@ import ( "github.com/cespare/xxhash/v2" ) +// ImplementationName is the name of the labels implementation. +const ImplementationName = "slicelabels" + // Labels is a sorted set of labels. Order has to be guaranteed upon // instantiation. type Labels []Label diff --git a/model/labels/labels_stringlabels.go b/model/labels/labels_stringlabels.go index f087223802..1460e7db93 100644 --- a/model/labels/labels_stringlabels.go +++ b/model/labels/labels_stringlabels.go @@ -23,6 +23,9 @@ import ( "github.com/cespare/xxhash/v2" ) +// ImplementationName is the name of the labels implementation. +const ImplementationName = "stringlabels" + // Labels is implemented by a single flat string holding name/value pairs. // Each name and value is preceded by its length, encoded as a single byte // for size 0-254, or the following 3 bytes little-endian, if the first byte is 255. diff --git a/promql/engine.go b/promql/engine.go index d3b67e3d81..8f922abaab 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -49,6 +49,7 @@ import ( "github.com/prometheus/prometheus/storage" "github.com/prometheus/prometheus/tsdb/chunkenc" "github.com/prometheus/prometheus/util/annotations" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/logging" "github.com/prometheus/prometheus/util/stats" "github.com/prometheus/prometheus/util/zeropool" @@ -330,6 +331,9 @@ type EngineOpts struct { EnableDelayedNameRemoval bool // EnableTypeAndUnitLabels will allow PromQL Engine to make decisions based on the type and unit labels. EnableTypeAndUnitLabels bool + + // FeatureRegistry is the registry for tracking enabled/disabled features. + FeatureRegistry features.Collector } // Engine handles the lifetime of queries from beginning to end. @@ -446,6 +450,18 @@ func NewEngine(opts EngineOpts) *Engine { ) } + if r := opts.FeatureRegistry; r != nil { + r.Set(features.PromQL, "at_modifier", opts.EnableAtModifier) + r.Set(features.PromQL, "negative_offset", opts.EnableNegativeOffset) + r.Set(features.PromQL, "per_step_stats", opts.EnablePerStepStats) + r.Set(features.PromQL, "delayed_name_removal", opts.EnableDelayedNameRemoval) + r.Set(features.PromQL, "type_and_unit_labels", opts.EnableTypeAndUnitLabels) + r.Enable(features.PromQL, "per_query_lookback_delta") + r.Enable(features.PromQL, "subqueries") + + parser.RegisterFeatures(r) + } + return &Engine{ timeout: opts.Timeout, logger: opts.Logger, diff --git a/promql/parser/features.go b/promql/parser/features.go new file mode 100644 index 0000000000..ec64678237 --- /dev/null +++ b/promql/parser/features.go @@ -0,0 +1,57 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package parser + +import "github.com/prometheus/prometheus/util/features" + +// RegisterFeatures registers all PromQL features with the feature registry. +// This includes operators (arithmetic and comparison/set), aggregators (standard +// and experimental), and functions. +func RegisterFeatures(r features.Collector) { + // Register core PromQL language keywords. + for keyword, itemType := range key { + if itemType.IsKeyword() { + // Handle experimental keywords separately. + switch keyword { + case "anchored", "smoothed": + r.Set(features.PromQL, keyword, EnableExtendedRangeSelectors) + default: + r.Enable(features.PromQL, keyword) + } + } + } + + // Register operators. + for o := ItemType(operatorsStart + 1); o < operatorsEnd; o++ { + if o.IsOperator() { + r.Set(features.PromQLOperators, o.String(), true) + } + } + + // Register aggregators. + for a := ItemType(aggregatorsStart + 1); a < aggregatorsEnd; a++ { + if a.IsAggregator() { + experimental := a.IsExperimentalAggregator() && !EnableExperimentalFunctions + r.Set(features.PromQLOperators, a.String(), !experimental) + } + } + + // Register functions. + for f, fc := range Functions { + r.Set(features.PromQLFunctions, f, !fc.Experimental || EnableExperimentalFunctions) + } + + // Register experimental parser features. + r.Set(features.PromQL, "duration_expr", ExperimentalDurationExpr) +} diff --git a/rules/manager.go b/rules/manager.go index 7d07217336..d610c154be 100644 --- a/rules/manager.go +++ b/rules/manager.go @@ -37,6 +37,7 @@ import ( "github.com/prometheus/prometheus/promql" "github.com/prometheus/prometheus/promql/parser" "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/strutil" ) @@ -134,6 +135,9 @@ type ManagerOptions struct { RestoreNewRuleGroups bool Metrics *Metrics + + // FeatureRegistry is used to register rule manager features. + FeatureRegistry features.Collector } // NewManager returns an implementation of Manager, ready to be started @@ -174,6 +178,13 @@ func NewManager(o *ManagerOptions) *Manager { o.Logger = promslog.NewNopLogger() } + // Register rule manager features if a registry is provided. + if o.FeatureRegistry != nil { + o.FeatureRegistry.Set(features.Rules, "concurrent_rule_eval", o.ConcurrentEvalsEnabled) + o.FeatureRegistry.Enable(features.Rules, "query_offset") + o.FeatureRegistry.Enable(features.Rules, "keep_firing_for") + } + m := &Manager{ groups: map[string]*Group{}, opts: o, diff --git a/scrape/manager.go b/scrape/manager.go index c63d7d0eae..9bb6988df9 100644 --- a/scrape/manager.go +++ b/scrape/manager.go @@ -33,6 +33,7 @@ import ( "github.com/prometheus/prometheus/discovery/targetgroup" "github.com/prometheus/prometheus/model/labels" "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/logging" "github.com/prometheus/prometheus/util/osutil" "github.com/prometheus/prometheus/util/pool" @@ -67,6 +68,13 @@ func NewManager(o *Options, logger *slog.Logger, newScrapeFailureLogger func(str m.metrics.setTargetMetadataCacheGatherer(m) + // Register scrape features. + if r := o.FeatureRegistry; r != nil { + r.Set(features.Scrape, "extra_scrape_metrics", o.ExtraMetrics) + r.Set(features.Scrape, "start_timestamp_zero_ingestion", o.EnableStartTimestampZeroIngestion) + r.Set(features.Scrape, "type_and_unit_labels", o.EnableTypeAndUnitLabels) + } + return m, nil } @@ -93,6 +101,9 @@ type Options struct { // Optional HTTP client options to use when scraping. HTTPClientOptions []config_util.HTTPClientOption + // FeatureRegistry is the registry for tracking enabled/disabled features. + FeatureRegistry features.Collector + // private option for testability. skipOffsetting bool } diff --git a/template/template.go b/template/template.go index ea7e93b18c..572e8450d3 100644 --- a/template/template.go +++ b/template/template.go @@ -36,6 +36,7 @@ import ( "golang.org/x/text/language" "github.com/prometheus/prometheus/promql" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/strutil" ) @@ -413,3 +414,29 @@ func floatToTime(v float64) (*time.Time, error) { t := model.TimeFromUnixNano(int64(timestamp)).Time().UTC() return &t, nil } + +// templateFunctions returns a representative funcMap with all available template functions. +// This is used to discover which functions are available for feature registration. +func templateFunctions() text_template.FuncMap { + // Create a dummy expander to get the function map. + expander := NewTemplateExpander( + context.Background(), + "", + "", + nil, + 0, + nil, + &url.URL{}, + nil, + ) + return expander.funcMap +} + +// RegisterFeatures registers all template functions with the feature registry. +func RegisterFeatures(r features.Collector) { + // Get all function names from the template function map. + funcMap := templateFunctions() + for name := range funcMap { + r.Enable(features.TemplatingFunctions, name) + } +} diff --git a/tsdb/db.go b/tsdb/db.go index dac5689b09..c946a9e329 100644 --- a/tsdb/db.go +++ b/tsdb/db.go @@ -47,6 +47,7 @@ import ( "github.com/prometheus/prometheus/tsdb/tsdbutil" "github.com/prometheus/prometheus/tsdb/wlog" "github.com/prometheus/prometheus/util/compression" + "github.com/prometheus/prometheus/util/features" ) const ( @@ -223,6 +224,9 @@ type Options struct { // BlockCompactionExcludeFunc is a function which returns true for blocks that should NOT be compacted. // It's passed down to the TSDB compactor. BlockCompactionExcludeFunc BlockExcludeFilterFunc + + // FeatureRegistry is used to register TSDB features. + FeatureRegistry features.Collector } type NewCompactorFunc func(ctx context.Context, r prometheus.Registerer, l *slog.Logger, ranges []int64, pool chunkenc.Pool, opts *Options) (Compactor, error) @@ -783,6 +787,15 @@ func Open(dir string, l *slog.Logger, r prometheus.Registerer, opts *Options, st var rngs []int64 opts, rngs = validateOpts(opts, nil) + // Register TSDB features if a registry is provided. + if opts.FeatureRegistry != nil { + opts.FeatureRegistry.Set(features.TSDB, "exemplar_storage", opts.EnableExemplarStorage) + opts.FeatureRegistry.Set(features.TSDB, "delayed_compaction", opts.EnableDelayedCompaction) + opts.FeatureRegistry.Set(features.TSDB, "isolation", !opts.IsolationDisabled) + opts.FeatureRegistry.Set(features.TSDB, "use_uncached_io", opts.UseUncachedIO) + opts.FeatureRegistry.Enable(features.TSDB, "native_histograms") + } + return open(dir, l, r, opts, rngs, stats) } diff --git a/util/features/features.go b/util/features/features.go new file mode 100644 index 0000000000..d52384dbd8 --- /dev/null +++ b/util/features/features.go @@ -0,0 +1,127 @@ +// Copyright The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package features + +import ( + "maps" + "sync" +) + +// Category constants define the standard feature flag categories used in Prometheus. +const ( + API = "api" + OTLPReceiver = "otlp_receiver" + Prometheus = "prometheus" + PromQL = "promql" + PromQLFunctions = "promql_functions" + PromQLOperators = "promql_operators" + Rules = "rules" + Scrape = "scrape" + ServiceDiscoveryProviders = "service_discovery_providers" + TemplatingFunctions = "templating_functions" + TSDB = "tsdb" + UI = "ui" +) + +// Collector defines the interface for collecting and managing feature flags. +// It provides methods to enable, disable, and retrieve feature states. +type Collector interface { + // Enable marks a feature as enabled in the registry. + // The category and name should use snake_case naming convention. + Enable(category, name string) + + // Disable marks a feature as disabled in the registry. + // The category and name should use snake_case naming convention. + Disable(category, name string) + + // Set sets a feature to the specified enabled state. + // The category and name should use snake_case naming convention. + Set(category, name string, enabled bool) + + // Get returns a copy of all registered features organized by category. + // Returns a map where the keys are category names and values are maps + // of feature names to their enabled status. + Get() map[string]map[string]bool +} + +// registry is the private implementation of the Collector interface. +// It stores feature information organized by category. +type registry struct { + mu sync.RWMutex + features map[string]map[string]bool +} + +// DefaultRegistry is the package-level registry used by Prometheus. +var DefaultRegistry = NewRegistry() + +// NewRegistry creates a new feature registry. +func NewRegistry() Collector { + return ®istry{ + features: make(map[string]map[string]bool), + } +} + +// Enable marks a feature as enabled in the registry. +func (r *registry) Enable(category, name string) { + r.Set(category, name, true) +} + +// Disable marks a feature as disabled in the registry. +func (r *registry) Disable(category, name string) { + r.Set(category, name, false) +} + +// Set sets a feature to the specified enabled state. +func (r *registry) Set(category, name string, enabled bool) { + r.mu.Lock() + defer r.mu.Unlock() + + if r.features[category] == nil { + r.features[category] = make(map[string]bool) + } + r.features[category][name] = enabled +} + +// Get returns a copy of all registered features organized by category. +func (r *registry) Get() map[string]map[string]bool { + r.mu.RLock() + defer r.mu.RUnlock() + + result := make(map[string]map[string]bool, len(r.features)) + for category, features := range r.features { + result[category] = make(map[string]bool, len(features)) + maps.Copy(result[category], features) + } + return result +} + +// Enable marks a feature as enabled in the default registry. +func Enable(category, name string) { + DefaultRegistry.Enable(category, name) +} + +// Disable marks a feature as disabled in the default registry. +func Disable(category, name string) { + DefaultRegistry.Disable(category, name) +} + +// Set sets a feature to the specified enabled state in the default registry. +func Set(category, name string, enabled bool) { + DefaultRegistry.Set(category, name, enabled) +} + +// Get returns all features from the default registry. +func Get() map[string]map[string]bool { + return DefaultRegistry.Get() +} diff --git a/web/api/v1/api.go b/web/api/v1/api.go index 86c0461087..a4e9a6b62a 100644 --- a/web/api/v1/api.go +++ b/web/api/v1/api.go @@ -56,6 +56,7 @@ import ( "github.com/prometheus/prometheus/tsdb" "github.com/prometheus/prometheus/tsdb/index" "github.com/prometheus/prometheus/util/annotations" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/httputil" "github.com/prometheus/prometheus/util/notifications" "github.com/prometheus/prometheus/util/stats" @@ -255,6 +256,8 @@ type API struct { otlpWriteHandler http.Handler codecs []Codec + + featureRegistry features.Collector } // NewAPI returns an initialized API type. @@ -295,6 +298,7 @@ func NewAPI( enableTypeAndUnitLabels bool, appendMetadata bool, overrideErrorCode OverrideErrorCode, + featureRegistry features.Collector, ) *API { a := &API{ QueryEngine: qe, @@ -324,6 +328,7 @@ func NewAPI( notificationsGetter: notificationsGetter, notificationsSub: notificationsSub, overrideErrorCode: overrideErrorCode, + featureRegistry: featureRegistry, remoteReadHandler: remote.NewReadHandler(logger, registerer, q, configFunc, remoteReadSampleLimit, remoteReadConcurrencyLimit, remoteReadMaxBytesInFrame), } @@ -445,6 +450,7 @@ func (api *API) Register(r *route.Router) { r.Get("/status/flags", wrap(api.serveFlags)) r.Get("/status/tsdb", wrapAgent(api.serveTSDBStatus)) r.Get("/status/tsdb/blocks", wrapAgent(api.serveTSDBBlocks)) + r.Get("/features", wrap(api.features)) r.Get("/status/walreplay", api.serveWALReplayStatus) r.Get("/notifications", api.notifications) r.Get("/notifications/live", api.notificationsSSE) @@ -1789,6 +1795,29 @@ func (api *API) serveFlags(*http.Request) apiFuncResult { return apiFuncResult{api.flagsMap, nil, nil, nil} } +// featuresData wraps feature flags data to provide custom JSON marshaling without HTML escaping. +// featuresData does not contain user-provided input, and it is more convenient to have unescaped +// representation of PromQL operators like >=. +type featuresData struct { + data map[string]map[string]bool +} + +func (f featuresData) MarshalJSON() ([]byte, error) { + json := jsoniter.Config{ + EscapeHTML: false, + SortMapKeys: true, + ValidateJsonRawMessage: true, + }.Froze() + return json.Marshal(f.data) +} + +func (api *API) features(*http.Request) apiFuncResult { + if api.featureRegistry == nil { + return apiFuncResult{nil, &apiError{errorInternal, errors.New("feature registry not configured")}, nil, nil} + } + return apiFuncResult{featuresData{data: api.featureRegistry.Get()}, nil, nil, nil} +} + // TSDBStat holds the information about individual cardinality. type TSDBStat struct { Name string `json:"name"` diff --git a/web/api/v1/errors_test.go b/web/api/v1/errors_test.go index c44444404b..5bd943ba98 100644 --- a/web/api/v1/errors_test.go +++ b/web/api/v1/errors_test.go @@ -168,6 +168,7 @@ func createPrometheusAPI(t *testing.T, q storage.SampleAndChunkQueryable, overri false, false, overrideErrorCode, + nil, ) promRouter := route.New().WithPrefix("/api/v1") diff --git a/web/web.go b/web/web.go index d7b647e3db..2d216502c1 100644 --- a/web/web.go +++ b/web/web.go @@ -57,6 +57,7 @@ import ( "github.com/prometheus/prometheus/scrape" "github.com/prometheus/prometheus/storage" "github.com/prometheus/prometheus/template" + "github.com/prometheus/prometheus/util/features" "github.com/prometheus/prometheus/util/httputil" "github.com/prometheus/prometheus/util/netconnlimit" "github.com/prometheus/prometheus/util/notifications" @@ -300,8 +301,9 @@ type Options struct { AcceptRemoteWriteProtoMsgs remoteapi.MessageTypes - Gatherer prometheus.Gatherer - Registerer prometheus.Registerer + Gatherer prometheus.Gatherer + Registerer prometheus.Registerer + FeatureRegistry features.Collector } // New initializes a new web Handler. @@ -399,8 +401,27 @@ func New(logger *slog.Logger, o *Options) *Handler { o.EnableTypeAndUnitLabels, o.AppendMetadata, nil, + o.FeatureRegistry, ) + if r := o.FeatureRegistry; r != nil { + // Set dynamic API features (based on configuration). + r.Set(features.API, "lifecycle", o.EnableLifecycle) + r.Set(features.API, "admin", o.EnableAdminAPI) + r.Set(features.API, "remote_write_receiver", o.EnableRemoteWriteReceiver) + r.Set(features.API, "otlp_write_receiver", o.EnableOTLPWriteReceiver) + r.Set(features.OTLPReceiver, "delta_conversion", o.ConvertOTLPDelta) + r.Set(features.OTLPReceiver, "native_delta_ingestion", o.NativeOTLPDeltaIngestion) + r.Enable(features.API, "label_values_match") // match[] parameter for label values endpoint. + r.Enable(features.API, "query_warnings") // warnings in query responses. + r.Enable(features.API, "query_stats") // stats parameter for query endpoints. + r.Enable(features.API, "time_range_series") // start/end parameters for /series endpoint. + r.Enable(features.API, "time_range_labels") // start/end parameters for /labels endpoints. + r.Enable(features.API, "exclude_alerts") // exclude_alerts parameter for /rules endpoint. + r.Set(features.UI, "ui_v3", !o.UseOldUI) + r.Set(features.UI, "ui_v2", o.UseOldUI) + } + if o.RoutePrefix != "/" { // If the prefix is missing for the root path, prepend it. router.Get("/", func(w http.ResponseWriter, r *http.Request) { From 4cad87cae831a3e92e18bb8df99154f25c3c14fd Mon Sep 17 00:00:00 2001 From: Julius Volz Date: Wed, 10 Dec 2025 11:20:43 +0100 Subject: [PATCH 123/439] PromQL: Fix insufficient cardinality checking for filter ops Generally, binary operations between two vectors fail if there is a many-to-one or one-to-many matching situation between series within a match group and no `group_left()` or `group_right()` modifier is present. For filter ops this is also generally the case, but there can be situations where multiple series on one side can match a single series on the other side, but only 0 or 1 of those multiple series remains after the filter operator has been applied. In this case, the PromQL engine does not produce a matching error, since it only tracks series matching for those series that survive the filtering. IMO this is incorrect behavior (which can also erratically make a query sometimes fail and sometimes succeed, depending on current sample values), and we should always produce an error if there is a match error prior to applying the filter op. This PR ensures that we do the cardinality / match tracking independently of the result of the filter operation. Signed-off-by: Julius Volz --- promql/engine.go | 10 ++++++---- promql/promqltest/testdata/operators.test | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 8f922abaab..6ba6008b19 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -2936,17 +2936,15 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * if info != nil { lastErr = info } - switch { - case returnBool: + if returnBool { histogramValue = nil if keep { floatValue = 1.0 } else { floatValue = 0.0 } - case !keep: - continue } + metric := resultMetric(ls.Metric, rs.Metric, op, matching, enh) if !ev.enableDelayedNameRemoval && returnBool { metric = metric.DropReserved(schema.IsMetadataLabel) @@ -2972,6 +2970,10 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching * insertedSigs[insertSig] = struct{}{} } + if !keep && !returnBool { + continue + } + enh.Out = append(enh.Out, Sample{ Metric: metric, F: floatValue, diff --git a/promql/promqltest/testdata/operators.test b/promql/promqltest/testdata/operators.test index 0e779f192c..e570be9630 100644 --- a/promql/promqltest/testdata/operators.test +++ b/promql/promqltest/testdata/operators.test @@ -316,6 +316,27 @@ eval instant at 5m http_requests_histogram == http_requests_histogram eval instant at 5m http_requests_histogram != http_requests_histogram expect no_info +clear + +# Check that we track many-to-one vector matching errors even when all but 0 or 1 +# series on the "many" side are filtered away. +load 5m + many_side{label="foo",job="test"} 0 + many_side{label="bar",job="test"} 1 + one_side{job="test"} 1 + +# Check 0 series surviving the filtering producing an error. +eval instant at 0m many_side > on(job) one_side + expect fail + +# Check 1 series surviving the filtering producing an error. +eval instant at 0m many_side >= on(job) one_side + expect fail + +# Check 2 series surviving the filtering producing an error. +eval instant at 0m many_side <= on(job) one_side + expect fail + # group_left/group_right. clear From d0b122a7116b1cd85590fa44e29bbf34ed567aaf Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Wed, 10 Dec 2025 16:16:08 +0100 Subject: [PATCH 124/439] PromQL: duration expression: add range() Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- docs/feature_flags.md | 7 +- promql/durations.go | 5 +- promql/durations_test.go | 33 +- promql/engine.go | 2 +- promql/parser/ast.go | 6 +- promql/parser/generated_parser.y | 39 +- promql/parser/generated_parser.y.go | 1007 +++++++++-------- promql/parser/lex.go | 6 +- promql/parser/parse_test.go | 100 +- promql/parser/printer.go | 2 + promql/parser/printer_test.go | 15 + promql/promqltest/test.go | 4 + .../testdata/duration_expression.test | 25 +- 13 files changed, 748 insertions(+), 503 deletions(-) diff --git a/docs/feature_flags.md b/docs/feature_flags.md index 0051859d66..74daa11c13 100644 --- a/docs/feature_flags.md +++ b/docs/feature_flags.md @@ -197,7 +197,12 @@ the offset calculation. `step()` can be used in duration expressions. For a **range query**, it resolves to the step width of the range query. -For an **instant query**, it resolves to `0s`. +For an **instant query**, it resolves to `0s`. + +`range()` can be used in duration expressions. +For a **range query**, it resolves to the full range of the query (end time - start time). +For an **instant query**, it resolves to `0s`. +This is particularly useful in combination with `@end()` to look back over the entire query range, e.g., `max_over_time(metric[range()] @ end())`. `min(, )` and `max(, )` can be used to find the minimum or maximum of two duration expressions. diff --git a/promql/durations.go b/promql/durations.go index c882adfbb6..216dd02725 100644 --- a/promql/durations.go +++ b/promql/durations.go @@ -28,7 +28,8 @@ import ( // in OriginalOffsetExpr representing (1h / 2). This visitor evaluates // such duration expression, setting OriginalOffset to 30m. type durationVisitor struct { - step time.Duration + step time.Duration + queryRange time.Duration } // Visit finds any duration expressions in AST Nodes and modifies the Node to @@ -121,6 +122,8 @@ func (v *durationVisitor) evaluateDurationExpr(expr parser.Expr) (float64, error switch n.Op { case parser.STEP: return float64(v.step.Seconds()), nil + case parser.RANGE: + return float64(v.queryRange.Seconds()), nil case parser.MIN: return math.Min(lhs, rhs), nil case parser.MAX: diff --git a/promql/durations_test.go b/promql/durations_test.go index 18592a0d0a..7a5e8f00a4 100644 --- a/promql/durations_test.go +++ b/promql/durations_test.go @@ -213,6 +213,37 @@ func TestCalculateDuration(t *testing.T) { }, expected: 3 * time.Second, }, + { + name: "range", + expr: &parser.DurationExpr{ + Op: parser.RANGE, + }, + expected: 5 * time.Minute, + }, + { + name: "range division", + expr: &parser.DurationExpr{ + LHS: &parser.DurationExpr{ + Op: parser.RANGE, + }, + RHS: &parser.NumberLiteral{Val: 2}, + Op: parser.DIV, + }, + expected: 150 * time.Second, + }, + { + name: "max of step and range", + expr: &parser.DurationExpr{ + LHS: &parser.DurationExpr{ + Op: parser.STEP, + }, + RHS: &parser.DurationExpr{ + Op: parser.RANGE, + }, + Op: parser.MAX, + }, + expected: 5 * time.Minute, + }, { name: "division by zero", expr: &parser.DurationExpr{ @@ -243,7 +274,7 @@ func TestCalculateDuration(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - v := &durationVisitor{step: 1 * time.Second} + v := &durationVisitor{step: 1 * time.Second, queryRange: 5 * time.Minute} result, err := v.calculateDuration(tt.expr, tt.allowedNegative) if tt.errorMessage != "" { require.Error(t, err) diff --git a/promql/engine.go b/promql/engine.go index 6ba6008b19..52c52b9617 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -3991,7 +3991,7 @@ func unwrapStepInvariantExpr(e parser.Expr) parser.Expr { func PreprocessExpr(expr parser.Expr, start, end time.Time, step time.Duration) (parser.Expr, error) { detectHistogramStatsDecoding(expr) - if err := parser.Walk(&durationVisitor{step: step}, expr, nil); err != nil { + if err := parser.Walk(&durationVisitor{step: step, queryRange: end.Sub(start)}, expr, nil); err != nil { return nil, err } diff --git a/promql/parser/ast.go b/promql/parser/ast.go index 67ecb190fe..8a1a094b79 100644 --- a/promql/parser/ast.go +++ b/promql/parser/ast.go @@ -116,8 +116,8 @@ type DurationExpr struct { LHS, RHS Expr // The operands on the respective sides of the operator. Wrapped bool // Set when the duration is wrapped in parentheses. - StartPos posrange.Pos // For unary operations and step(), the start position of the operator. - EndPos posrange.Pos // For step(), the end position of the operator. + StartPos posrange.Pos // For unary operations, step(), and range(), the start position of the operator. + EndPos posrange.Pos // For step() and range(), the end position of the operator. } // Call represents a function call. @@ -474,7 +474,7 @@ func (e *BinaryExpr) PositionRange() posrange.PositionRange { } func (e *DurationExpr) PositionRange() posrange.PositionRange { - if e.Op == STEP { + if e.Op == STEP || e.Op == RANGE { return posrange.PositionRange{ Start: e.StartPos, End: e.EndPos, diff --git a/promql/parser/generated_parser.y b/promql/parser/generated_parser.y index d9bbb10b28..47776f53d0 100644 --- a/promql/parser/generated_parser.y +++ b/promql/parser/generated_parser.y @@ -153,6 +153,7 @@ WITHOUT START END STEP +RANGE %token preprocessorEnd // Counter reset hints. @@ -465,7 +466,7 @@ offset_expr: expr OFFSET offset_duration_expr $$ = $1 } | expr OFFSET error - { yylex.(*parser).unexpected("offset", "number, duration, or step()"); $$ = $1 } + { yylex.(*parser).unexpected("offset", "number, duration, step(), or range()"); $$ = $1 } ; /* @@ -575,11 +576,11 @@ subquery_expr : expr LEFT_BRACKET positive_duration_expr COLON positive_durati | expr LEFT_BRACKET positive_duration_expr COLON positive_duration_expr error { yylex.(*parser).unexpected("subquery selector", "\"]\""); $$ = $1 } | expr LEFT_BRACKET positive_duration_expr COLON error - { yylex.(*parser).unexpected("subquery selector", "number, duration, or step() or \"]\""); $$ = $1 } + { yylex.(*parser).unexpected("subquery selector", "number, duration, step(), range(), or \"]\""); $$ = $1 } | expr LEFT_BRACKET positive_duration_expr error { yylex.(*parser).unexpected("subquery or range", "\":\" or \"]\""); $$ = $1 } | expr LEFT_BRACKET error - { yylex.(*parser).unexpected("subquery or range selector", "number, duration, or step()"); $$ = $1 } + { yylex.(*parser).unexpected("subquery or range selector", "number, duration, step(), or range()"); $$ = $1 } ; /* @@ -696,7 +697,7 @@ metric : metric_identifier label_set ; -metric_identifier: AVG | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | IDENTIFIER | LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | QUANTILE | STDDEV | STDVAR | SUM | TOPK | WITHOUT | START | END | LIMITK | LIMIT_RATIO | STEP | ANCHORED | SMOOTHED; +metric_identifier: AVG | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | IDENTIFIER | LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | QUANTILE | STDDEV | STDVAR | SUM | TOPK | WITHOUT | START | END | LIMITK | LIMIT_RATIO | STEP | RANGE | ANCHORED | SMOOTHED; label_set : LEFT_BRACE label_set_list RIGHT_BRACE { $$ = labels.New($2...) } @@ -953,7 +954,7 @@ counter_reset_hint : UNKNOWN_COUNTER_RESET | COUNTER_RESET | NOT_COUNTER_RESET | aggregate_op : AVG | BOTTOMK | COUNT | COUNT_VALUES | GROUP | MAX | MIN | QUANTILE | STDDEV | STDVAR | SUM | TOPK | LIMITK | LIMIT_RATIO; // Inside of grouping options label names can be recognized as keywords by the lexer. This is a list of keywords that could also be a label name. -maybe_label : AVG | BOOL | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | GROUP_LEFT | GROUP_RIGHT | IDENTIFIER | IGNORING | LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | ON | QUANTILE | STDDEV | STDVAR | SUM | TOPK | START | END | ATAN2 | LIMITK | LIMIT_RATIO | STEP | ANCHORED | SMOOTHED; +maybe_label : AVG | BOOL | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | GROUP_LEFT | GROUP_RIGHT | IDENTIFIER | IGNORING | LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | ON | QUANTILE | STDDEV | STDVAR | SUM | TOPK | START | END | ATAN2 | LIMITK | LIMIT_RATIO | STEP | RANGE | ANCHORED | SMOOTHED; unary_op : ADD | SUB; @@ -1088,6 +1089,14 @@ offset_duration_expr : number_duration_literal EndPos: $3.PositionRange().End, } } + | RANGE LEFT_PAREN RIGHT_PAREN + { + $$ = &DurationExpr{ + Op: RANGE, + StartPos: $1.PositionRange().Start, + EndPos: $3.PositionRange().End, + } + } | unary_op STEP LEFT_PAREN RIGHT_PAREN { $$ = &DurationExpr{ @@ -1100,6 +1109,18 @@ offset_duration_expr : number_duration_literal StartPos: $1.Pos, } } + | unary_op RANGE LEFT_PAREN RIGHT_PAREN + { + $$ = &DurationExpr{ + Op: $1.Typ, + RHS: &DurationExpr{ + Op: RANGE, + StartPos: $2.PositionRange().Start, + EndPos: $4.PositionRange().End, + }, + StartPos: $1.Pos, + } + } | min_max LEFT_PAREN duration_expr COMMA duration_expr RIGHT_PAREN { $$ = &DurationExpr{ @@ -1234,6 +1255,14 @@ duration_expr : number_duration_literal EndPos: $3.PositionRange().End, } } + | RANGE LEFT_PAREN RIGHT_PAREN + { + $$ = &DurationExpr{ + Op: RANGE, + StartPos: $1.PositionRange().Start, + EndPos: $3.PositionRange().End, + } + } | min_max LEFT_PAREN duration_expr COMMA duration_expr RIGHT_PAREN { $$ = &DurationExpr{ diff --git a/promql/parser/generated_parser.y.go b/promql/parser/generated_parser.y.go index eb4b32129a..f5feec0b55 100644 --- a/promql/parser/generated_parser.y.go +++ b/promql/parser/generated_parser.y.go @@ -124,19 +124,20 @@ const preprocessorStart = 57431 const START = 57432 const END = 57433 const STEP = 57434 -const preprocessorEnd = 57435 -const counterResetHintsStart = 57436 -const UNKNOWN_COUNTER_RESET = 57437 -const COUNTER_RESET = 57438 -const NOT_COUNTER_RESET = 57439 -const GAUGE_TYPE = 57440 -const counterResetHintsEnd = 57441 -const startSymbolsStart = 57442 -const START_METRIC = 57443 -const START_SERIES_DESCRIPTION = 57444 -const START_EXPRESSION = 57445 -const START_METRIC_SELECTOR = 57446 -const startSymbolsEnd = 57447 +const RANGE = 57435 +const preprocessorEnd = 57436 +const counterResetHintsStart = 57437 +const UNKNOWN_COUNTER_RESET = 57438 +const COUNTER_RESET = 57439 +const NOT_COUNTER_RESET = 57440 +const GAUGE_TYPE = 57441 +const counterResetHintsEnd = 57442 +const startSymbolsStart = 57443 +const START_METRIC = 57444 +const START_SERIES_DESCRIPTION = 57445 +const START_EXPRESSION = 57446 +const START_METRIC_SELECTOR = 57447 +const startSymbolsEnd = 57448 var yyToknames = [...]string{ "$end", @@ -231,6 +232,7 @@ var yyToknames = [...]string{ "START", "END", "STEP", + "RANGE", "preprocessorEnd", "counterResetHintsStart", "UNKNOWN_COUNTER_RESET", @@ -256,344 +258,344 @@ var yyExca = [...]int16{ -1, 1, 1, -1, -2, 0, - -1, 40, - 1, 149, - 10, 149, - 24, 149, + -1, 41, + 1, 150, + 10, 150, + 24, 150, -2, 0, - -1, 70, - 2, 192, - 15, 192, - 79, 192, - 87, 192, - -2, 107, - -1, 71, + -1, 72, 2, 193, 15, 193, 79, 193, 87, 193, - -2, 108, - -1, 72, + -2, 107, + -1, 73, 2, 194, 15, 194, 79, 194, 87, 194, - -2, 110, - -1, 73, + -2, 108, + -1, 74, 2, 195, 15, 195, 79, 195, 87, 195, - -2, 111, - -1, 74, + -2, 110, + -1, 75, 2, 196, 15, 196, 79, 196, 87, 196, - -2, 112, - -1, 75, + -2, 111, + -1, 76, 2, 197, 15, 197, 79, 197, 87, 197, - -2, 117, - -1, 76, + -2, 112, + -1, 77, 2, 198, 15, 198, 79, 198, 87, 198, - -2, 119, - -1, 77, + -2, 117, + -1, 78, 2, 199, 15, 199, 79, 199, 87, 199, - -2, 121, - -1, 78, + -2, 119, + -1, 79, 2, 200, 15, 200, 79, 200, 87, 200, - -2, 122, - -1, 79, + -2, 121, + -1, 80, 2, 201, 15, 201, 79, 201, 87, 201, - -2, 123, - -1, 80, + -2, 122, + -1, 81, 2, 202, 15, 202, 79, 202, 87, 202, - -2, 124, - -1, 81, + -2, 123, + -1, 82, 2, 203, 15, 203, 79, 203, 87, 203, - -2, 125, - -1, 82, + -2, 124, + -1, 83, 2, 204, 15, 204, 79, 204, 87, 204, - -2, 129, - -1, 83, + -2, 125, + -1, 84, 2, 205, 15, 205, 79, 205, 87, 205, + -2, 129, + -1, 85, + 2, 206, + 15, 206, + 79, 206, + 87, 206, -2, 130, - -1, 135, - 41, 270, - 42, 270, - 52, 270, - 53, 270, - 57, 270, + -1, 137, + 41, 274, + 42, 274, + 52, 274, + 53, 274, + 57, 274, -2, 22, - -1, 245, - 9, 257, - 12, 257, - 13, 257, - 18, 257, - 19, 257, - 25, 257, - 41, 257, - 47, 257, - 48, 257, - 51, 257, - 57, 257, - 62, 257, - 63, 257, - 64, 257, - 65, 257, - 66, 257, - 67, 257, - 68, 257, - 69, 257, - 70, 257, - 71, 257, - 72, 257, - 73, 257, - 74, 257, - 75, 257, - 79, 257, - 83, 257, - 84, 257, - 85, 257, - 87, 257, - 90, 257, - 91, 257, - 92, 257, + -1, 251, + 9, 259, + 12, 259, + 13, 259, + 18, 259, + 19, 259, + 25, 259, + 41, 259, + 47, 259, + 48, 259, + 51, 259, + 57, 259, + 62, 259, + 63, 259, + 64, 259, + 65, 259, + 66, 259, + 67, 259, + 68, 259, + 69, 259, + 70, 259, + 71, 259, + 72, 259, + 73, 259, + 74, 259, + 75, 259, + 79, 259, + 83, 259, + 84, 259, + 85, 259, + 87, 259, + 90, 259, + 91, 259, + 92, 259, + 93, 259, -2, 0, - -1, 246, - 9, 257, - 12, 257, - 13, 257, - 18, 257, - 19, 257, - 25, 257, - 41, 257, - 47, 257, - 48, 257, - 51, 257, - 57, 257, - 62, 257, - 63, 257, - 64, 257, - 65, 257, - 66, 257, - 67, 257, - 68, 257, - 69, 257, - 70, 257, - 71, 257, - 72, 257, - 73, 257, - 74, 257, - 75, 257, - 79, 257, - 83, 257, - 84, 257, - 85, 257, - 87, 257, - 90, 257, - 91, 257, - 92, 257, + -1, 252, + 9, 259, + 12, 259, + 13, 259, + 18, 259, + 19, 259, + 25, 259, + 41, 259, + 47, 259, + 48, 259, + 51, 259, + 57, 259, + 62, 259, + 63, 259, + 64, 259, + 65, 259, + 66, 259, + 67, 259, + 68, 259, + 69, 259, + 70, 259, + 71, 259, + 72, 259, + 73, 259, + 74, 259, + 75, 259, + 79, 259, + 83, 259, + 84, 259, + 85, 259, + 87, 259, + 90, 259, + 91, 259, + 92, 259, + 93, 259, -2, 0, } const yyPrivate = 57344 -const yyLast = 1071 +const yyLast = 1050 var yyAct = [...]int16{ - 57, 182, 401, 399, 185, 406, 278, 237, 193, 332, - 93, 47, 346, 141, 68, 221, 91, 413, 414, 415, - 416, 127, 128, 64, 156, 186, 66, 126, 347, 326, - 129, 243, 122, 125, 130, 244, 245, 246, 119, 122, - 118, 124, 123, 121, 327, 151, 124, 118, 214, 123, - 121, 396, 373, 124, 120, 364, 395, 366, 323, 385, - 328, 354, 352, 133, 216, 135, 6, 98, 100, 101, - 364, 102, 103, 104, 105, 106, 107, 108, 109, 110, - 111, 324, 112, 113, 117, 99, 42, 131, 315, 112, - 144, 117, 136, 400, 241, 350, 191, 143, 128, 349, - 142, 137, 270, 314, 322, 320, 129, 268, 317, 114, - 116, 115, 192, 95, 233, 178, 114, 116, 115, 195, - 199, 200, 201, 202, 203, 204, 174, 321, 319, 177, - 196, 196, 196, 196, 196, 196, 196, 232, 175, 217, - 267, 130, 197, 197, 197, 197, 197, 197, 197, 132, - 196, 134, 138, 205, 390, 407, 239, 207, 210, 227, - 206, 223, 197, 229, 428, 2, 3, 4, 5, 360, - 190, 194, 429, 389, 359, 7, 266, 240, 61, 86, - 189, 231, 269, 427, 181, 150, 426, 262, 60, 358, - 264, 119, 122, 196, 425, 209, 271, 272, 266, 197, - 152, 225, 123, 121, 230, 197, 124, 120, 208, 196, - 84, 224, 226, 119, 122, 38, 384, 213, 222, 383, - 223, 197, 10, 382, 123, 121, 85, 235, 124, 120, - 143, 190, 88, 318, 238, 381, 180, 179, 241, 242, - 380, 189, 379, 378, 247, 248, 249, 250, 251, 252, - 253, 254, 255, 256, 257, 258, 259, 260, 261, 348, - 225, 198, 325, 191, 94, 377, 351, 376, 97, 353, - 224, 226, 344, 345, 92, 195, 375, 196, 374, 192, - 196, 39, 228, 355, 61, 55, 196, 95, 1, 197, - 181, 87, 197, 149, 60, 148, 172, 69, 197, 54, - 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, - 167, 168, 169, 170, 171, 417, 84, 362, 65, 53, - 190, 9, 9, 144, 52, 51, 363, 365, 196, 367, - 189, 155, 85, 142, 275, 368, 369, 184, 274, 50, - 197, 140, 180, 179, 190, 49, 95, 48, 372, 119, - 122, 386, 191, 273, 189, 8, 46, 153, 211, 40, - 123, 121, 196, 371, 124, 120, 392, 198, 192, 394, - 370, 388, 94, 45, 197, 154, 191, 402, 403, 404, - 398, 44, 92, 405, 43, 409, 408, 411, 410, 418, - 90, 281, 192, 56, 236, 95, 422, 316, 419, 420, - 196, 291, 361, 421, 393, 119, 122, 297, 329, 423, - 96, 391, 197, 234, 280, 276, 123, 121, 424, 89, - 124, 120, 412, 119, 122, 187, 188, 183, 431, 196, - 279, 119, 122, 58, 123, 121, 293, 294, 124, 120, - 295, 197, 123, 121, 139, 0, 124, 120, 308, 0, - 0, 282, 284, 286, 287, 288, 296, 298, 301, 302, - 303, 304, 305, 309, 310, 0, 281, 283, 285, 289, - 290, 292, 299, 313, 312, 300, 291, 0, 220, 306, - 307, 311, 297, 219, 0, 0, 277, 387, 0, 280, - 147, 0, 190, 61, 0, 146, 218, 0, 0, 265, - 0, 0, 189, 60, 430, 0, 119, 122, 145, 0, - 0, 293, 294, 0, 0, 295, 0, 123, 121, 0, - 0, 124, 120, 308, 191, 84, 282, 284, 286, 287, - 288, 296, 298, 301, 302, 303, 304, 305, 309, 310, - 192, 85, 283, 285, 289, 290, 292, 299, 313, 312, - 300, 180, 179, 0, 306, 307, 311, 61, 0, 118, - 59, 86, 0, 62, 0, 0, 22, 60, 0, 0, - 212, 0, 0, 63, 0, 0, 263, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 98, 100, 0, 84, - 0, 0, 0, 0, 0, 18, 19, 109, 110, 20, - 0, 112, 113, 117, 99, 85, 0, 0, 0, 0, - 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, - 80, 81, 82, 83, 0, 0, 0, 13, 114, 116, - 115, 24, 37, 36, 215, 30, 0, 0, 31, 32, - 67, 61, 41, 0, 59, 86, 0, 62, 0, 0, - 22, 60, 0, 119, 122, 0, 0, 63, 0, 0, - 0, 0, 0, 0, 123, 121, 0, 0, 124, 120, - 0, 357, 0, 84, 0, 0, 0, 0, 61, 18, - 19, 0, 0, 20, 181, 0, 0, 0, 60, 85, - 356, 0, 0, 0, 70, 71, 72, 73, 74, 75, - 76, 77, 78, 79, 80, 81, 82, 83, 0, 0, - 84, 13, 0, 0, 0, 24, 37, 36, 0, 30, - 0, 0, 31, 32, 67, 61, 85, 0, 59, 86, - 0, 62, 331, 0, 22, 60, 180, 179, 0, 330, - 0, 63, 0, 334, 335, 333, 340, 342, 339, 341, - 336, 337, 338, 343, 0, 0, 0, 84, 0, 0, - 0, 198, 0, 18, 19, 0, 0, 20, 0, 0, - 0, 0, 0, 85, 0, 0, 0, 0, 70, 71, - 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, - 82, 83, 17, 86, 0, 13, 0, 0, 22, 24, - 37, 36, 397, 30, 0, 0, 31, 32, 67, 0, - 0, 0, 0, 334, 335, 333, 340, 342, 339, 341, - 336, 337, 338, 343, 0, 0, 0, 18, 19, 0, - 0, 20, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 11, 12, 14, 15, 16, 21, 23, 25, - 26, 27, 28, 29, 33, 34, 17, 38, 0, 13, - 0, 0, 22, 24, 37, 36, 0, 30, 0, 0, - 31, 32, 35, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 18, 19, 0, 0, 20, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 11, 12, 14, 15, - 16, 21, 23, 25, 26, 27, 28, 29, 33, 34, - 118, 0, 0, 13, 0, 0, 0, 24, 37, 36, - 0, 30, 0, 0, 31, 32, 35, 0, 0, 118, - 0, 0, 0, 0, 0, 0, 0, 98, 100, 101, - 0, 102, 103, 104, 105, 106, 107, 108, 109, 110, - 111, 0, 112, 113, 117, 99, 98, 100, 101, 0, - 102, 103, 104, 0, 106, 107, 108, 109, 110, 111, - 173, 112, 113, 117, 99, 118, 0, 61, 0, 114, - 116, 115, 0, 181, 118, 0, 0, 60, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 114, 116, - 115, 0, 98, 100, 101, 0, 102, 103, 0, 84, - 106, 107, 100, 109, 110, 111, 0, 112, 113, 117, - 99, 0, 109, 110, 0, 85, 112, 0, 117, 99, - 0, 0, 0, 0, 0, 180, 179, 0, 0, 0, - 0, 0, 0, 0, 114, 116, 115, 0, 0, 0, - 0, 0, 0, 114, 116, 115, 0, 0, 0, 0, - 176, + 58, 186, 413, 411, 341, 418, 286, 243, 197, 95, + 189, 48, 355, 144, 70, 227, 93, 251, 252, 356, + 159, 190, 65, 120, 17, 88, 127, 130, 128, 129, + 22, 425, 426, 427, 428, 131, 249, 121, 124, 335, + 250, 67, 132, 126, 408, 407, 377, 332, 125, 123, + 331, 102, 126, 122, 336, 154, 324, 6, 397, 18, + 19, 111, 112, 20, 135, 114, 137, 119, 101, 375, + 337, 323, 375, 330, 11, 12, 14, 15, 16, 21, + 23, 25, 26, 27, 28, 29, 33, 34, 43, 133, + 329, 13, 116, 118, 117, 24, 38, 37, 146, 30, + 402, 124, 31, 32, 35, 36, 130, 412, 138, 396, + 194, 125, 123, 328, 131, 126, 365, 182, 239, 401, + 193, 199, 204, 205, 206, 207, 208, 209, 177, 363, + 362, 181, 200, 200, 200, 200, 200, 200, 200, 178, + 120, 238, 223, 201, 201, 201, 201, 201, 201, 201, + 212, 215, 134, 200, 136, 211, 210, 2, 3, 4, + 5, 222, 233, 221, 201, 245, 235, 384, 333, 371, + 228, 247, 229, 360, 370, 359, 246, 358, 188, 273, + 140, 368, 114, 195, 119, 194, 277, 139, 62, 369, + 268, 237, 229, 271, 185, 193, 441, 200, 61, 196, + 367, 201, 273, 383, 155, 278, 279, 280, 201, 116, + 118, 117, 231, 200, 236, 121, 124, 195, 382, 440, + 86, 218, 230, 232, 201, 381, 125, 123, 276, 275, + 126, 122, 231, 196, 274, 146, 87, 132, 439, 327, + 429, 438, 230, 232, 248, 141, 184, 183, 419, 253, + 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, + 264, 265, 266, 267, 334, 357, 191, 192, 214, 353, + 354, 202, 203, 361, 121, 124, 88, 364, 283, 7, + 39, 213, 282, 199, 200, 125, 123, 395, 200, 126, + 122, 366, 10, 194, 200, 201, 394, 281, 393, 201, + 392, 391, 90, 193, 390, 201, 160, 161, 162, 163, + 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, + 174, 389, 194, 388, 120, 195, 373, 387, 386, 385, + 153, 99, 193, 62, 442, 374, 376, 200, 378, 185, + 56, 196, 40, 61, 379, 380, 89, 152, 201, 151, + 1, 100, 102, 103, 195, 104, 105, 175, 71, 108, + 109, 398, 111, 112, 113, 86, 114, 115, 119, 101, + 196, 66, 200, 55, 9, 9, 54, 404, 8, 53, + 406, 87, 41, 201, 52, 158, 410, 51, 414, 415, + 416, 184, 183, 116, 118, 117, 421, 420, 423, 422, + 417, 430, 50, 49, 289, 47, 156, 216, 147, 46, + 431, 432, 200, 372, 299, 433, 202, 203, 145, 96, + 305, 435, 157, 201, 403, 437, 326, 288, 147, 94, + 436, 97, 45, 44, 57, 242, 434, 234, 145, 338, + 443, 200, 97, 98, 121, 124, 143, 240, 284, 301, + 302, 97, 201, 303, 91, 125, 123, 424, 187, 126, + 122, 316, 287, 59, 290, 292, 294, 295, 296, 304, + 306, 309, 310, 311, 312, 313, 317, 318, 142, 0, + 291, 293, 297, 298, 300, 307, 322, 321, 308, 289, + 96, 0, 314, 315, 319, 320, 226, 150, 405, 299, + 94, 225, 149, 0, 0, 305, 0, 0, 92, 285, + 0, 0, 288, 97, 224, 148, 62, 121, 124, 0, + 0, 0, 272, 0, 0, 0, 61, 0, 125, 123, + 0, 0, 126, 122, 301, 302, 0, 0, 303, 0, + 0, 0, 0, 0, 0, 0, 316, 0, 86, 290, + 292, 294, 295, 296, 304, 306, 309, 310, 311, 312, + 313, 317, 318, 0, 87, 291, 293, 297, 298, 300, + 307, 322, 321, 308, 184, 183, 0, 314, 315, 319, + 320, 62, 0, 120, 60, 88, 0, 63, 0, 0, + 22, 61, 0, 0, 217, 0, 0, 64, 0, 269, + 270, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 100, 102, 0, 86, 0, 0, 0, 0, 0, 18, + 19, 111, 112, 20, 0, 114, 115, 119, 101, 87, + 0, 0, 0, 0, 72, 73, 74, 75, 76, 77, + 78, 79, 80, 81, 82, 83, 84, 85, 0, 0, + 400, 13, 116, 118, 117, 24, 38, 37, 399, 30, + 0, 0, 31, 32, 68, 69, 62, 42, 0, 60, + 88, 0, 63, 0, 0, 22, 61, 121, 124, 0, + 0, 0, 64, 0, 121, 124, 0, 0, 125, 123, + 0, 0, 126, 122, 0, 125, 123, 0, 86, 126, + 122, 0, 0, 0, 18, 19, 0, 0, 20, 0, + 0, 0, 0, 0, 87, 0, 0, 0, 0, 72, + 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, + 83, 84, 85, 0, 0, 0, 13, 0, 0, 220, + 24, 38, 37, 0, 30, 0, 325, 31, 32, 68, + 69, 62, 0, 0, 60, 88, 0, 63, 121, 124, + 22, 61, 0, 0, 0, 0, 0, 64, 0, 125, + 123, 0, 0, 126, 122, 0, 0, 0, 0, 0, + 121, 124, 0, 86, 0, 0, 0, 0, 0, 18, + 19, 125, 123, 20, 0, 126, 122, 0, 0, 87, + 0, 0, 0, 0, 72, 73, 74, 75, 76, 77, + 78, 79, 80, 81, 82, 83, 84, 85, 17, 39, + 0, 13, 0, 0, 22, 24, 38, 37, 0, 30, + 340, 0, 31, 32, 68, 69, 0, 339, 0, 0, + 0, 343, 344, 342, 349, 351, 348, 350, 345, 346, + 347, 352, 241, 18, 19, 0, 194, 20, 0, 244, + 0, 0, 0, 247, 0, 0, 193, 0, 11, 12, + 14, 15, 16, 21, 23, 25, 26, 27, 28, 29, + 33, 34, 0, 0, 120, 13, 0, 0, 195, 24, + 38, 37, 219, 30, 0, 0, 31, 32, 35, 36, + 0, 0, 0, 120, 196, 0, 0, 0, 0, 0, + 0, 100, 102, 103, 0, 104, 105, 106, 107, 108, + 109, 110, 111, 112, 113, 0, 114, 115, 119, 101, + 100, 102, 103, 0, 104, 105, 106, 107, 108, 109, + 110, 111, 112, 113, 198, 114, 115, 119, 101, 120, + 0, 62, 0, 116, 118, 117, 0, 185, 176, 0, + 0, 61, 0, 0, 0, 62, 0, 0, 0, 0, + 0, 185, 116, 118, 117, 61, 100, 102, 103, 0, + 104, 105, 106, 86, 108, 109, 110, 111, 112, 113, + 0, 114, 115, 119, 101, 0, 0, 86, 0, 87, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 184, + 183, 0, 0, 87, 0, 0, 0, 0, 116, 118, + 117, 0, 0, 184, 183, 409, 0, 0, 0, 0, + 0, 0, 0, 0, 202, 203, 343, 344, 342, 349, + 351, 348, 350, 345, 346, 347, 352, 0, 179, 180, } var yyPact = [...]int16{ - 64, 165, 844, 844, 632, 780, -1000, -1000, -1000, 202, + 55, 269, 806, 806, 657, 12, -1000, -1000, -1000, 267, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, - -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 370, -1000, - 266, -1000, 906, -1000, -1000, -1000, -1000, -1000, -1000, -1000, - -1000, -1000, -1000, -1000, -1000, -1000, -1000, -3, 19, 126, - -1000, -1000, 716, -1000, 716, 166, -1000, 86, 137, -1000, - -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, - -1000, -1000, -1000, -1000, -1000, -1000, 321, -1000, -1000, 488, - -1000, -1000, 291, 181, -1000, -1000, 21, -1000, -54, -54, - -54, -54, -54, -54, -54, -54, -54, -54, -54, -54, - -54, -54, -54, -54, 978, -1000, -1000, 335, 169, 275, - 275, 275, 275, 275, 275, 126, -57, -1000, 193, 193, - 548, -1000, 26, 612, 33, -15, -1000, 42, 275, 476, - -1000, -1000, 216, 157, -1000, -1000, 262, -1000, 179, -1000, - 112, 222, 716, -1000, -51, -44, -1000, 716, 716, 716, - 716, 716, 716, 716, 716, 716, 716, 716, 716, 716, - 716, 716, -1000, -1000, -1000, 484, 125, 92, -3, -1000, - -1000, 275, -1000, 87, -1000, -1000, -1000, -1000, -1000, -1000, - -1000, 161, 161, 332, -1000, -3, -1000, 275, 86, -10, - -10, -15, -15, -15, -15, -1000, -1000, -1000, 464, -1000, - -1000, 81, -1000, 906, -1000, -1000, -1000, 390, -1000, 88, - -1000, 103, -1000, -1000, -1000, -1000, -1000, 102, -1000, -1000, - -1000, -1000, -1000, -1000, -1000, 32, 55, 3, -1000, -1000, - -1000, 715, 980, 193, 193, 193, 193, 33, 33, 545, - 545, 545, 971, 925, 545, 545, 971, 33, 33, 545, - 33, 980, -1000, 84, 80, 275, -15, 40, 275, 612, - 39, -1000, -1000, -1000, 669, -1000, 167, -1000, -1000, -1000, + -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 488, + -1000, 329, -1000, 889, -1000, -1000, -1000, -1000, -1000, -1000, + -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -4, 27, + 222, -1000, -1000, 742, -1000, 742, 263, -1000, 172, 165, + 230, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, + -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 426, -1000, + -1000, 495, -1000, -1000, 345, 326, -1000, -1000, 31, -1000, + -58, -58, -58, -58, -58, -58, -58, -58, -58, -58, + -58, -58, -58, -58, -58, -58, 956, -1000, -1000, 176, + 942, 324, 324, 324, 324, 324, 324, 222, -52, -1000, + 266, 266, 572, -1000, 870, 717, 126, -13, -1000, 141, + 139, 324, 494, -1000, -1000, 168, 188, -1000, -1000, 417, + -1000, 189, -1000, 116, 847, 742, -1000, -46, -63, -1000, + 742, 742, 742, 742, 742, 742, 742, 742, 742, 742, + 742, 742, 742, 742, 742, -1000, -1000, -1000, 507, 219, + 214, 213, -4, -1000, -1000, 324, -1000, 190, -1000, -1000, + -1000, -1000, -1000, -1000, -1000, 101, 101, 276, -1000, -4, + -1000, 324, 172, 165, 59, 59, -13, -13, -13, -13, + -1000, -1000, -1000, 487, -1000, -1000, 49, -1000, 889, -1000, + -1000, -1000, -1000, 739, -1000, 406, -1000, 88, -1000, -1000, + -1000, -1000, -1000, 48, -1000, -1000, -1000, -1000, -1000, -1000, + -1000, 21, 142, 13, -1000, -1000, -1000, 813, 9, 266, + 266, 266, 266, 126, 126, 569, 569, 569, 310, 935, + 569, 569, 310, 126, 126, 569, 126, 9, -1000, 162, + 160, 158, 324, -13, 108, 107, 324, 717, 94, -1000, + -1000, -1000, 179, -1000, 167, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, - -1000, -1000, -1000, -1000, -1000, 716, 275, -1000, -1000, -1000, - -1000, -1000, -1000, 51, 51, 31, 51, 78, 78, 346, - 35, -1000, -1000, 272, 270, 261, 259, 237, 236, 234, - 229, 217, 213, 210, -1000, -1000, -1000, -1000, -1000, 37, - 275, 465, -1000, 364, -1000, 152, -1000, -1000, -1000, 389, - -1000, 906, 382, -1000, -1000, -1000, 51, -1000, 30, 25, - 785, -1000, -1000, -1000, 36, 311, 311, 311, 161, 141, - 141, 36, 141, 36, -78, -1000, 308, -1000, 275, -1000, - -1000, -1000, -1000, -1000, -1000, 51, 51, -1000, -1000, -1000, - 51, -1000, -1000, -1000, -1000, -1000, -1000, 311, -1000, -1000, - -1000, -1000, -1000, -1000, -1000, -1000, -1000, 275, 172, -1000, - -1000, -1000, 162, -1000, 150, -1000, 483, -1000, -1000, -1000, - -1000, -1000, + -1000, -1000, -1000, -1000, 742, 324, -1000, -1000, -1000, -1000, + -1000, -1000, 53, 53, 20, 53, 155, 155, 201, 150, + -1000, -1000, 323, 322, 321, 317, 315, 298, 295, 294, + 292, 290, 281, -1000, -1000, -1000, -1000, -1000, 87, 36, + 324, 636, -1000, -1000, 643, -1000, 98, -1000, -1000, -1000, + 402, -1000, 889, 476, -1000, -1000, -1000, 53, -1000, 19, + 18, 1008, -1000, -1000, -1000, 50, 284, 284, 284, 101, + 234, 234, 50, 234, 50, -65, -1000, -1000, 233, -1000, + 324, -1000, -1000, -1000, -1000, -1000, -1000, 53, 53, -1000, + -1000, -1000, 53, -1000, -1000, -1000, -1000, -1000, -1000, 284, + -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 324, + 403, -1000, -1000, -1000, 217, -1000, 174, -1000, 313, -1000, + -1000, -1000, -1000, -1000, } var yyPgo = [...]int16{ - 0, 444, 13, 433, 6, 15, 430, 318, 23, 427, - 10, 422, 14, 222, 355, 419, 16, 415, 28, 12, - 413, 410, 7, 408, 9, 5, 396, 3, 2, 4, - 394, 25, 1, 393, 384, 33, 200, 381, 375, 86, - 373, 358, 27, 357, 26, 356, 11, 347, 345, 339, - 331, 325, 324, 319, 299, 285, 0, 297, 8, 296, - 288, 281, + 0, 478, 13, 463, 6, 15, 462, 371, 22, 458, + 9, 457, 14, 292, 378, 454, 16, 448, 19, 12, + 447, 443, 7, 439, 4, 5, 436, 3, 2, 10, + 435, 21, 1, 434, 433, 26, 204, 432, 422, 88, + 409, 407, 28, 406, 41, 405, 11, 403, 402, 387, + 385, 384, 379, 376, 373, 340, 0, 358, 8, 357, + 350, 342, } var yyR1 = [...]int8{ @@ -610,22 +612,22 @@ var yyR1 = [...]int8{ 2, 2, 2, 2, 2, 14, 14, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, - 7, 7, 7, 7, 13, 13, 13, 13, 15, 15, - 15, 16, 16, 16, 16, 16, 16, 16, 61, 21, - 21, 21, 21, 20, 20, 20, 20, 20, 20, 20, - 20, 20, 30, 30, 30, 22, 22, 22, 22, 23, - 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, - 24, 24, 24, 25, 25, 26, 26, 26, 11, 11, - 11, 11, 3, 3, 3, 3, 3, 3, 3, 3, - 3, 3, 3, 3, 3, 3, 6, 6, 6, 6, + 7, 7, 7, 7, 7, 13, 13, 13, 13, 15, + 15, 15, 16, 16, 16, 16, 16, 16, 16, 61, + 21, 21, 21, 21, 20, 20, 20, 20, 20, 20, + 20, 20, 20, 30, 30, 30, 22, 22, 22, 22, + 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, + 24, 24, 24, 24, 25, 25, 26, 26, 26, 11, + 11, 11, 11, 3, 3, 3, 3, 3, 3, 3, + 3, 3, 3, 3, 3, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, - 6, 6, 6, 6, 6, 6, 6, 6, 8, 8, - 5, 5, 5, 5, 46, 46, 29, 29, 31, 31, - 32, 32, 28, 27, 27, 52, 10, 19, 19, 59, - 59, 59, 59, 59, 59, 59, 59, 12, 12, 56, - 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, - 57, + 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, + 8, 8, 5, 5, 5, 5, 46, 46, 29, 29, + 31, 31, 32, 32, 28, 27, 27, 52, 10, 19, + 19, 59, 59, 59, 59, 59, 59, 59, 59, 59, + 59, 12, 12, 56, 56, 56, 56, 56, 56, 56, + 56, 56, 56, 56, 56, 57, } var yyR2 = [...]int8{ @@ -642,116 +644,118 @@ var yyR2 = [...]int8{ 1, 3, 3, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, 3, 4, 2, 0, 3, 1, - 2, 3, 3, 1, 3, 3, 2, 1, 2, 0, - 3, 2, 1, 1, 3, 1, 3, 4, 1, 3, - 5, 5, 1, 1, 1, 4, 3, 3, 2, 3, - 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, - 3, 3, 3, 4, 3, 3, 1, 2, 1, 1, + 1, 1, 1, 1, 1, 3, 4, 2, 0, 3, + 1, 2, 3, 3, 1, 3, 3, 2, 1, 2, + 0, 3, 2, 1, 1, 3, 1, 3, 4, 1, + 3, 5, 5, 1, 1, 1, 4, 3, 3, 2, + 3, 1, 2, 3, 3, 3, 3, 3, 3, 3, + 3, 3, 3, 3, 4, 3, 3, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, - 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, - 1, 1, 1, 2, 1, 1, 1, 0, 1, 1, - 2, 3, 4, 6, 7, 4, 1, 1, 1, 1, - 2, 3, 3, 3, 3, 3, 3, 3, 6, 1, - 3, + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, + 2, 2, 1, 1, 1, 2, 1, 1, 1, 0, + 1, 1, 2, 3, 3, 4, 4, 6, 7, 4, + 1, 1, 1, 1, 2, 3, 3, 3, 3, 3, + 3, 3, 3, 6, 1, 3, } var yyChk = [...]int16{ - -1000, -60, 101, 102, 103, 104, 2, 10, -14, -7, + -1000, -60, 102, 103, 104, 105, 2, 10, -14, -7, -13, 62, 63, 79, 64, 65, 66, 12, 47, 48, 51, 67, 18, 68, 83, 69, 70, 71, 72, 73, - 87, 90, 91, 74, 75, 92, 85, 84, 13, -61, - -14, 10, -39, -34, -37, -40, -45, -46, -47, -48, - -49, -51, -52, -53, -54, -55, -33, -56, -3, 12, - 19, 9, 15, 25, -8, -7, -44, 92, -12, -57, - 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, - 72, 73, 74, 75, 41, 57, 13, -55, -13, -15, - 20, -16, 12, -10, 2, 25, -21, 2, 41, 59, - 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, - 53, 54, 56, 57, 83, 85, 84, 58, 14, 41, - 57, 53, 42, 52, 56, -35, -42, 2, 79, 87, - 15, -42, -39, -56, -39, -56, -44, 15, 15, -1, - 20, -2, 12, -10, 2, 20, 7, 2, 4, 2, - 4, 24, -36, -43, -38, -50, 78, -36, -36, -36, + 87, 90, 91, 74, 75, 92, 93, 85, 84, 13, + -61, -14, 10, -39, -34, -37, -40, -45, -46, -47, + -48, -49, -51, -52, -53, -54, -55, -33, -56, -3, + 12, 19, 9, 15, 25, -8, -7, -44, 92, 93, + -12, -57, 62, 63, 64, 65, 66, 67, 68, 69, + 70, 71, 72, 73, 74, 75, 41, 57, 13, -55, + -13, -15, 20, -16, 12, -10, 2, 25, -21, 2, + 41, 59, 42, 43, 45, 46, 47, 48, 49, 50, + 51, 52, 53, 54, 56, 57, 83, 85, 84, 58, + 14, 41, 57, 53, 42, 52, 56, -35, -42, 2, + 79, 87, 15, -42, -39, -56, -39, -56, -44, 15, + 15, 15, -1, 20, -2, 12, -10, 2, 20, 7, + 2, 4, 2, 4, 24, -36, -43, -38, -50, 78, -36, -36, -36, -36, -36, -36, -36, -36, -36, -36, - -36, -36, -59, 2, -46, -8, 92, -12, -56, 68, - 67, 15, -32, -9, 2, -29, -31, 90, 91, 19, - 9, 41, 57, -58, 2, -56, -46, -8, 92, -56, - -56, -56, -56, -56, -56, -42, -35, -18, 15, 2, - -18, -41, 22, -39, 22, 22, 22, -56, 20, 7, - 2, -5, 2, 4, 54, 44, 55, -5, 20, -16, - 25, 2, 25, 2, -20, 5, -30, -22, 12, -29, - -31, 16, -39, 82, 86, 80, 81, -39, -39, -39, - -39, -39, -39, -39, -39, -39, -39, -39, -39, -39, - -39, -39, -46, 92, -12, 15, -56, 15, 15, -56, - 15, -29, -29, 21, 6, 2, -17, 22, -4, -6, - 25, 2, 62, 78, 63, 79, 64, 65, 66, 80, - 81, 12, 82, 47, 48, 51, 67, 18, 68, 83, - 86, 69, 70, 71, 72, 73, 90, 91, 59, 74, - 75, 92, 85, 84, 22, 7, 7, 20, -2, 25, - 2, 25, 2, 26, 26, -31, 26, 41, 57, -23, - 24, 17, -24, 30, 28, 29, 35, 36, 37, 33, - 31, 34, 32, 38, -18, -18, -19, -18, -19, 15, - 15, -56, 22, -56, 22, -58, 21, 2, 22, 7, - 2, -39, -56, -28, 19, -28, 26, -28, -22, -22, - 24, 17, 2, 17, 6, 6, 6, 6, 6, 6, - 6, 6, 6, 6, 6, 22, -56, 22, 7, 21, - 2, 22, -4, 22, -28, 26, 26, 17, -24, -27, - 57, -28, -32, -32, -32, -29, -25, 14, -25, -27, - -25, -27, -11, 95, 96, 97, 98, 7, -56, -28, - -28, -28, -26, -32, -56, 22, 24, 21, 2, 22, - 21, -32, + -36, -36, -36, -36, -36, -59, 2, -46, -8, 92, + 93, -12, -56, 68, 67, 15, -32, -9, 2, -29, + -31, 90, 91, 19, 9, 41, 57, -58, 2, -56, + -46, -8, 92, 93, -56, -56, -56, -56, -56, -56, + -42, -35, -18, 15, 2, -18, -41, 22, -39, 22, + 22, 22, 22, -56, 20, 7, 2, -5, 2, 4, + 54, 44, 55, -5, 20, -16, 25, 2, 25, 2, + -20, 5, -30, -22, 12, -29, -31, 16, -39, 82, + 86, 80, 81, -39, -39, -39, -39, -39, -39, -39, + -39, -39, -39, -39, -39, -39, -39, -39, -46, 92, + 93, -12, 15, -56, 15, 15, 15, -56, 15, -29, + -29, 21, 6, 2, -17, 22, -4, -6, 25, 2, + 62, 78, 63, 79, 64, 65, 66, 80, 81, 12, + 82, 47, 48, 51, 67, 18, 68, 83, 86, 69, + 70, 71, 72, 73, 90, 91, 59, 74, 75, 92, + 93, 85, 84, 22, 7, 7, 20, -2, 25, 2, + 25, 2, 26, 26, -31, 26, 41, 57, -23, 24, + 17, -24, 30, 28, 29, 35, 36, 37, 33, 31, + 34, 32, 38, -18, -18, -19, -18, -19, 15, 15, + 15, -56, 22, 22, -56, 22, -58, 21, 2, 22, + 7, 2, -39, -56, -28, 19, -28, 26, -28, -22, + -22, 24, 17, 2, 17, 6, 6, 6, 6, 6, + 6, 6, 6, 6, 6, 6, 22, 22, -56, 22, + 7, 21, 2, 22, -4, 22, -28, 26, 26, 17, + -24, -27, 57, -28, -32, -32, -32, -29, -25, 14, + -25, -27, -25, -27, -11, 96, 97, 98, 99, 7, + -56, -28, -28, -28, -26, -32, -56, 22, 24, 21, + 2, 22, 21, -32, } var yyDef = [...]int16{ - 0, -2, 137, 137, 0, 0, 7, 6, 1, 137, + 0, -2, 138, 138, 0, 0, 7, 6, 1, 138, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, - 126, 127, 128, 129, 130, 131, 132, 133, 0, 2, - -2, 3, 4, 8, 9, 10, 11, 12, 13, 14, - 15, 16, 17, 18, 19, 20, 21, 22, 0, 113, - 244, 245, 0, 255, 0, 90, 91, 131, 0, 279, - -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, - -2, -2, -2, -2, 238, 239, 0, 5, 105, 0, - 136, 139, 0, 143, 147, 256, 148, 152, 46, 46, + 126, 127, 128, 129, 130, 131, 132, 133, 134, 0, + 2, -2, 3, 4, 8, 9, 10, 11, 12, 13, + 14, 15, 16, 17, 18, 19, 20, 21, 22, 0, + 113, 246, 247, 0, 257, 0, 90, 91, 131, 132, + 0, 284, -2, -2, -2, -2, -2, -2, -2, -2, + -2, -2, -2, -2, -2, -2, 240, 241, 0, 5, + 105, 0, 137, 140, 0, 144, 148, 258, 149, 153, 46, 46, 46, 46, 46, 46, 46, 46, 46, 46, - 46, 46, 46, 46, 0, 74, 75, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 25, 26, 0, 0, - 0, 64, 0, 22, 88, -2, 89, 0, 0, 0, - 94, 96, 0, 100, 104, 134, 0, 140, 0, 146, - 0, 151, 0, 45, 50, 51, 47, 0, 0, 0, + 46, 46, 46, 46, 46, 46, 0, 74, 75, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 25, 26, + 0, 0, 0, 64, 0, 22, 88, -2, 89, 0, + 0, 0, 0, 94, 96, 0, 100, 104, 135, 0, + 141, 0, 147, 0, 152, 0, 45, 50, 51, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 72, 73, 259, 0, 0, 0, 266, 267, - 268, 0, 76, 0, 78, 250, 251, 79, 80, 246, - 247, 0, 0, 0, 87, 71, 269, 0, 0, 271, - 272, 273, 274, 275, 276, 23, 24, 27, 0, 57, - 28, 0, 66, 68, 70, 280, 277, 0, 92, 0, - 97, 0, 103, 240, 241, 242, 243, 0, 135, 138, - 141, 144, 142, 145, 150, 153, 155, 158, 162, 163, - 164, 0, 29, 0, 0, -2, -2, 30, 31, 32, - 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, - 43, 44, 260, 0, 0, 0, 270, 0, 0, 0, - 0, 248, 249, 81, 0, 86, 0, 56, 59, 61, - 62, 63, 206, 207, 208, 209, 210, 211, 212, 213, - 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, - 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, - 234, 235, 236, 237, 65, 69, 0, 93, 95, 98, - 102, 99, 101, 0, 0, 0, 0, 0, 0, 0, - 0, 168, 170, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 48, 49, 52, 258, 53, 0, - 0, 0, 261, 0, 77, 0, 83, 85, 54, 0, - 60, 67, 0, 154, 252, 156, 0, 159, 0, 0, - 0, 166, 171, 167, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 262, 0, 265, 0, 82, - 84, 55, 58, 278, 157, 0, 0, 165, 169, 172, - 0, 254, 173, 174, 175, 176, 177, 0, 178, 179, - 180, 181, 182, 188, 189, 190, 191, 0, 0, 160, - 161, 253, 0, 186, 0, 263, 0, 184, 187, 264, - 183, 185, + 0, 0, 0, 0, 0, 72, 73, 261, 0, 0, + 0, 0, 270, 271, 272, 0, 76, 0, 78, 252, + 253, 79, 80, 248, 249, 0, 0, 0, 87, 71, + 273, 0, 0, 0, 275, 276, 277, 278, 279, 280, + 23, 24, 27, 0, 57, 28, 0, 66, 68, 70, + 285, 281, 282, 0, 92, 0, 97, 0, 103, 242, + 243, 244, 245, 0, 136, 139, 142, 145, 143, 146, + 151, 154, 156, 159, 163, 164, 165, 0, 29, 0, + 0, -2, -2, 30, 31, 32, 33, 34, 35, 36, + 37, 38, 39, 40, 41, 42, 43, 44, 262, 0, + 0, 0, 0, 274, 0, 0, 0, 0, 0, 250, + 251, 81, 0, 86, 0, 56, 59, 61, 62, 63, + 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, + 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, + 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, + 237, 238, 239, 65, 69, 0, 93, 95, 98, 102, + 99, 101, 0, 0, 0, 0, 0, 0, 0, 0, + 169, 171, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 48, 49, 52, 260, 53, 0, 0, + 0, 0, 263, 264, 0, 77, 0, 83, 85, 54, + 0, 60, 67, 0, 155, 254, 157, 0, 160, 0, + 0, 0, 167, 172, 168, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 265, 266, 0, 269, + 0, 82, 84, 55, 58, 283, 158, 0, 0, 166, + 170, 173, 0, 256, 174, 175, 176, 177, 178, 0, + 179, 180, 181, 182, 183, 189, 190, 191, 192, 0, + 0, 161, 162, 255, 0, 187, 0, 267, 0, 185, + 188, 268, 184, 186, } var yyTok1 = [...]int8{ @@ -769,7 +773,7 @@ var yyTok2 = [...]int8{ 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, - 102, 103, 104, 105, + 102, 103, 104, 105, 106, } var yyTok3 = [...]int8{ @@ -1434,7 +1438,7 @@ yydefault: case 73: yyDollar = yyS[yypt-3 : yypt+1] { - yylex.(*parser).unexpected("offset", "number, duration, or step()") + yylex.(*parser).unexpected("offset", "number, duration, step(), or range()") yyVAL.node = yyDollar[1].node } case 74: @@ -1541,7 +1545,7 @@ yydefault: case 85: yyDollar = yyS[yypt-5 : yypt+1] { - yylex.(*parser).unexpected("subquery selector", "number, duration, or step() or \"]\"") + yylex.(*parser).unexpected("subquery selector", "number, duration, step(), range(), or \"]\"") yyVAL.node = yyDollar[1].node } case 86: @@ -1553,7 +1557,7 @@ yydefault: case 87: yyDollar = yyS[yypt-3 : yypt+1] { - yylex.(*parser).unexpected("subquery or range selector", "number, duration, or step()") + yylex.(*parser).unexpected("subquery or range selector", "number, duration, step(), or range()") yyVAL.node = yyDollar[1].node } case 88: @@ -1691,63 +1695,57 @@ yydefault: { yyVAL.labels = yyDollar[1].labels } - case 134: - yyDollar = yyS[yypt-3 : yypt+1] - { - yyVAL.labels = labels.New(yyDollar[2].lblList...) - } case 135: - yyDollar = yyS[yypt-4 : yypt+1] + yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.labels = labels.New(yyDollar[2].lblList...) } case 136: - yyDollar = yyS[yypt-2 : yypt+1] + yyDollar = yyS[yypt-4 : yypt+1] { - yyVAL.labels = labels.New() + yyVAL.labels = labels.New(yyDollar[2].lblList...) } case 137: - yyDollar = yyS[yypt-0 : yypt+1] + yyDollar = yyS[yypt-2 : yypt+1] { yyVAL.labels = labels.New() } case 138: + yyDollar = yyS[yypt-0 : yypt+1] + { + yyVAL.labels = labels.New() + } + case 139: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.lblList = append(yyDollar[1].lblList, yyDollar[3].label) } - case 139: + case 140: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.lblList = []labels.Label{yyDollar[1].label} } - case 140: + case 141: yyDollar = yyS[yypt-2 : yypt+1] { yylex.(*parser).unexpected("label set", "\",\" or \"}\"") yyVAL.lblList = yyDollar[1].lblList } - case 141: - yyDollar = yyS[yypt-3 : yypt+1] - { - yyVAL.label = labels.Label{Name: yyDollar[1].item.Val, Value: yylex.(*parser).unquoteString(yyDollar[3].item.Val)} - } case 142: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.label = labels.Label{Name: yyDollar[1].item.Val, Value: yylex.(*parser).unquoteString(yyDollar[3].item.Val)} } case 143: + yyDollar = yyS[yypt-3 : yypt+1] + { + yyVAL.label = labels.Label{Name: yyDollar[1].item.Val, Value: yylex.(*parser).unquoteString(yyDollar[3].item.Val)} + } + case 144: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.label = labels.Label{Name: labels.MetricName, Value: yyDollar[1].item.Val} } - case 144: - yyDollar = yyS[yypt-3 : yypt+1] - { - yylex.(*parser).unexpected("label set", "string") - yyVAL.label = labels.Label{} - } case 145: yyDollar = yyS[yypt-3 : yypt+1] { @@ -1755,18 +1753,24 @@ yydefault: yyVAL.label = labels.Label{} } case 146: + yyDollar = yyS[yypt-3 : yypt+1] + { + yylex.(*parser).unexpected("label set", "string") + yyVAL.label = labels.Label{} + } + case 147: yyDollar = yyS[yypt-2 : yypt+1] { yylex.(*parser).unexpected("label set", "\"=\"") yyVAL.label = labels.Label{} } - case 147: + case 148: yyDollar = yyS[yypt-1 : yypt+1] { yylex.(*parser).unexpected("label set", "identifier or \"}\"") yyVAL.label = labels.Label{} } - case 148: + case 149: yyDollar = yyS[yypt-2 : yypt+1] { yylex.(*parser).generatedParserResult = &seriesDescription{ @@ -1774,33 +1778,33 @@ yydefault: values: yyDollar[2].series, } } - case 149: + case 150: yyDollar = yyS[yypt-0 : yypt+1] { yyVAL.series = []SequenceValue{} } - case 150: + case 151: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.series = append(yyDollar[1].series, yyDollar[3].series...) } - case 151: + case 152: yyDollar = yyS[yypt-2 : yypt+1] { yyVAL.series = yyDollar[1].series } - case 152: + case 153: yyDollar = yyS[yypt-1 : yypt+1] { yylex.(*parser).unexpected("series values", "") yyVAL.series = nil } - case 153: + case 154: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.series = []SequenceValue{{Omitted: true}} } - case 154: + case 155: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.series = []SequenceValue{} @@ -1808,12 +1812,12 @@ yydefault: yyVAL.series = append(yyVAL.series, SequenceValue{Omitted: true}) } } - case 155: + case 156: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.series = []SequenceValue{{Value: yyDollar[1].float}} } - case 156: + case 157: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.series = []SequenceValue{} @@ -1822,7 +1826,7 @@ yydefault: yyVAL.series = append(yyVAL.series, SequenceValue{Value: yyDollar[1].float}) } } - case 157: + case 158: yyDollar = yyS[yypt-4 : yypt+1] { yyVAL.series = []SequenceValue{} @@ -1832,12 +1836,12 @@ yydefault: yyDollar[1].float += yyDollar[2].float } } - case 158: + case 159: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.series = []SequenceValue{{Histogram: yyDollar[1].histogram}} } - case 159: + case 160: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.series = []SequenceValue{} @@ -1847,7 +1851,7 @@ yydefault: //$1 += $2 } } - case 160: + case 161: yyDollar = yyS[yypt-5 : yypt+1] { val, err := yylex.(*parser).histogramsIncreaseSeries(yyDollar[1].histogram, yyDollar[3].histogram, yyDollar[5].uint) @@ -1856,7 +1860,7 @@ yydefault: } yyVAL.series = val } - case 161: + case 162: yyDollar = yyS[yypt-5 : yypt+1] { val, err := yylex.(*parser).histogramsDecreaseSeries(yyDollar[1].histogram, yyDollar[3].histogram, yyDollar[5].uint) @@ -1865,7 +1869,7 @@ yydefault: } yyVAL.series = val } - case 162: + case 163: yyDollar = yyS[yypt-1 : yypt+1] { if yyDollar[1].item.Val != "stale" { @@ -1873,130 +1877,130 @@ yydefault: } yyVAL.float = math.Float64frombits(value.StaleNaN) } - case 165: - yyDollar = yyS[yypt-4 : yypt+1] - { - yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&yyDollar[2].descriptors) - } case 166: - yyDollar = yyS[yypt-3 : yypt+1] + yyDollar = yyS[yypt-4 : yypt+1] { yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&yyDollar[2].descriptors) } case 167: yyDollar = yyS[yypt-3 : yypt+1] { - m := yylex.(*parser).newMap() - yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&m) + yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&yyDollar[2].descriptors) } case 168: - yyDollar = yyS[yypt-2 : yypt+1] + yyDollar = yyS[yypt-3 : yypt+1] { m := yylex.(*parser).newMap() yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&m) } case 169: + yyDollar = yyS[yypt-2 : yypt+1] + { + m := yylex.(*parser).newMap() + yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&m) + } + case 170: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = *(yylex.(*parser).mergeMaps(&yyDollar[1].descriptors, &yyDollar[3].descriptors)) } - case 170: + case 171: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.descriptors = yyDollar[1].descriptors } - case 171: + case 172: yyDollar = yyS[yypt-2 : yypt+1] { yylex.(*parser).unexpected("histogram description", "histogram description key, e.g. buckets:[5 10 7]") } - case 172: - yyDollar = yyS[yypt-3 : yypt+1] - { - yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["schema"] = yyDollar[3].int - } case 173: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["sum"] = yyDollar[3].float + yyVAL.descriptors["schema"] = yyDollar[3].int } case 174: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["count"] = yyDollar[3].float + yyVAL.descriptors["sum"] = yyDollar[3].float } case 175: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["z_bucket"] = yyDollar[3].float + yyVAL.descriptors["count"] = yyDollar[3].float } case 176: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["z_bucket_w"] = yyDollar[3].float + yyVAL.descriptors["z_bucket"] = yyDollar[3].float } case 177: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["custom_values"] = yyDollar[3].bucket_set + yyVAL.descriptors["z_bucket_w"] = yyDollar[3].float } case 178: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["buckets"] = yyDollar[3].bucket_set + yyVAL.descriptors["custom_values"] = yyDollar[3].bucket_set } case 179: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["offset"] = yyDollar[3].int + yyVAL.descriptors["buckets"] = yyDollar[3].bucket_set } case 180: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["n_buckets"] = yyDollar[3].bucket_set + yyVAL.descriptors["offset"] = yyDollar[3].int } case 181: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["n_offset"] = yyDollar[3].int + yyVAL.descriptors["n_buckets"] = yyDollar[3].bucket_set } case 182: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.descriptors = yylex.(*parser).newMap() - yyVAL.descriptors["counter_reset_hint"] = yyDollar[3].item + yyVAL.descriptors["n_offset"] = yyDollar[3].int } case 183: - yyDollar = yyS[yypt-4 : yypt+1] + yyDollar = yyS[yypt-3 : yypt+1] { - yyVAL.bucket_set = yyDollar[2].bucket_set + yyVAL.descriptors = yylex.(*parser).newMap() + yyVAL.descriptors["counter_reset_hint"] = yyDollar[3].item } case 184: - yyDollar = yyS[yypt-3 : yypt+1] + yyDollar = yyS[yypt-4 : yypt+1] { yyVAL.bucket_set = yyDollar[2].bucket_set } case 185: yyDollar = yyS[yypt-3 : yypt+1] { - yyVAL.bucket_set = append(yyDollar[1].bucket_set, yyDollar[3].float) + yyVAL.bucket_set = yyDollar[2].bucket_set } case 186: + yyDollar = yyS[yypt-3 : yypt+1] + { + yyVAL.bucket_set = append(yyDollar[1].bucket_set, yyDollar[3].float) + } + case 187: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.bucket_set = []float64{yyDollar[1].float} } - case 244: + case 246: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.node = &NumberLiteral{ @@ -2004,7 +2008,7 @@ yydefault: PosRange: yyDollar[1].item.PositionRange(), } } - case 245: + case 247: yyDollar = yyS[yypt-1 : yypt+1] { var err error @@ -2019,12 +2023,12 @@ yydefault: Duration: true, } } - case 246: + case 248: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.float = yylex.(*parser).number(yyDollar[1].item.Val) } - case 247: + case 249: yyDollar = yyS[yypt-1 : yypt+1] { var err error @@ -2035,17 +2039,17 @@ yydefault: } yyVAL.float = dur.Seconds() } - case 248: + case 250: yyDollar = yyS[yypt-2 : yypt+1] { yyVAL.float = yyDollar[2].float } - case 249: + case 251: yyDollar = yyS[yypt-2 : yypt+1] { yyVAL.float = -yyDollar[2].float } - case 252: + case 254: yyDollar = yyS[yypt-1 : yypt+1] { var err error @@ -2054,17 +2058,17 @@ yydefault: yylex.(*parser).addParseErrf(yyDollar[1].item.PositionRange(), "invalid repetition in series values: %s", err) } } - case 253: + case 255: yyDollar = yyS[yypt-2 : yypt+1] { yyVAL.int = -int64(yyDollar[2].uint) } - case 254: + case 256: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.int = int64(yyDollar[1].uint) } - case 255: + case 257: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.node = &StringLiteral{ @@ -2072,7 +2076,7 @@ yydefault: PosRange: yyDollar[1].item.PositionRange(), } } - case 256: + case 258: yyDollar = yyS[yypt-1 : yypt+1] { yyVAL.item = Item{ @@ -2081,12 +2085,12 @@ yydefault: Val: yylex.(*parser).unquoteString(yyDollar[1].item.Val), } } - case 257: + case 259: yyDollar = yyS[yypt-0 : yypt+1] { yyVAL.strings = nil } - case 259: + case 261: yyDollar = yyS[yypt-1 : yypt+1] { nl := yyDollar[1].node.(*NumberLiteral) @@ -2097,7 +2101,7 @@ yydefault: } yyVAL.node = nl } - case 260: + case 262: yyDollar = yyS[yypt-2 : yypt+1] { nl := yyDollar[2].node.(*NumberLiteral) @@ -2112,7 +2116,7 @@ yydefault: nl.PosRange.Start = yyDollar[1].item.Pos yyVAL.node = nl } - case 261: + case 263: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.node = &DurationExpr{ @@ -2121,7 +2125,16 @@ yydefault: EndPos: yyDollar[3].item.PositionRange().End, } } - case 262: + case 264: + yyDollar = yyS[yypt-3 : yypt+1] + { + yyVAL.node = &DurationExpr{ + Op: RANGE, + StartPos: yyDollar[1].item.PositionRange().Start, + EndPos: yyDollar[3].item.PositionRange().End, + } + } + case 265: yyDollar = yyS[yypt-4 : yypt+1] { yyVAL.node = &DurationExpr{ @@ -2134,7 +2147,20 @@ yydefault: StartPos: yyDollar[1].item.Pos, } } - case 263: + case 266: + yyDollar = yyS[yypt-4 : yypt+1] + { + yyVAL.node = &DurationExpr{ + Op: yyDollar[1].item.Typ, + RHS: &DurationExpr{ + Op: RANGE, + StartPos: yyDollar[2].item.PositionRange().Start, + EndPos: yyDollar[4].item.PositionRange().End, + }, + StartPos: yyDollar[1].item.Pos, + } + } + case 267: yyDollar = yyS[yypt-6 : yypt+1] { yyVAL.node = &DurationExpr{ @@ -2145,7 +2171,7 @@ yydefault: RHS: yyDollar[5].node.(Expr), } } - case 264: + case 268: yyDollar = yyS[yypt-7 : yypt+1] { yyVAL.node = &DurationExpr{ @@ -2161,7 +2187,7 @@ yydefault: }, } } - case 265: + case 269: yyDollar = yyS[yypt-4 : yypt+1] { de := yyDollar[3].node.(*DurationExpr) @@ -2176,7 +2202,7 @@ yydefault: } yyVAL.node = yyDollar[3].node } - case 269: + case 273: yyDollar = yyS[yypt-1 : yypt+1] { nl := yyDollar[1].node.(*NumberLiteral) @@ -2187,7 +2213,7 @@ yydefault: } yyVAL.node = nl } - case 270: + case 274: yyDollar = yyS[yypt-2 : yypt+1] { switch expr := yyDollar[2].node.(type) { @@ -2220,25 +2246,25 @@ yydefault: break } } - case 271: + case 275: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr)) yyVAL.node = &DurationExpr{Op: ADD, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)} } - case 272: + case 276: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr)) yyVAL.node = &DurationExpr{Op: SUB, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)} } - case 273: + case 277: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr)) yyVAL.node = &DurationExpr{Op: MUL, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)} } - case 274: + case 278: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr)) @@ -2249,7 +2275,7 @@ yydefault: } yyVAL.node = &DurationExpr{Op: DIV, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)} } - case 275: + case 279: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr)) @@ -2260,13 +2286,13 @@ yydefault: } yyVAL.node = &DurationExpr{Op: MOD, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)} } - case 276: + case 280: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr)) yyVAL.node = &DurationExpr{Op: POW, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)} } - case 277: + case 281: yyDollar = yyS[yypt-3 : yypt+1] { yyVAL.node = &DurationExpr{ @@ -2275,7 +2301,16 @@ yydefault: EndPos: yyDollar[3].item.PositionRange().End, } } - case 278: + case 282: + yyDollar = yyS[yypt-3 : yypt+1] + { + yyVAL.node = &DurationExpr{ + Op: RANGE, + StartPos: yyDollar[1].item.PositionRange().Start, + EndPos: yyDollar[3].item.PositionRange().End, + } + } + case 283: yyDollar = yyS[yypt-6 : yypt+1] { yyVAL.node = &DurationExpr{ @@ -2286,7 +2321,7 @@ yydefault: RHS: yyDollar[5].node.(Expr), } } - case 280: + case 285: yyDollar = yyS[yypt-3 : yypt+1] { yylex.(*parser).experimentalDurationExpr(yyDollar[2].node.(Expr)) diff --git a/promql/parser/lex.go b/promql/parser/lex.go index 296b91d1ae..ad4b685150 100644 --- a/promql/parser/lex.go +++ b/promql/parser/lex.go @@ -143,6 +143,7 @@ var key = map[string]ItemType{ "start": START, "end": END, "step": STEP, + "range": RANGE, } var histogramDesc = map[string]ItemType{ @@ -915,6 +916,9 @@ func (l *Lexer) scanDurationKeyword() bool { case "step": l.emit(STEP) return true + case "range": + l.emit(RANGE) + return true case "min": l.emit(MIN) return true @@ -1175,7 +1179,7 @@ func lexDurationExpr(l *Lexer) stateFn { case r == ',': l.emit(COMMA) return lexDurationExpr - case r == 's' || r == 'S' || r == 'm' || r == 'M': + case r == 's' || r == 'S' || r == 'm' || r == 'M' || r == 'r' || r == 'R': if l.scanDurationKeyword() { return lexDurationExpr } diff --git a/promql/parser/parse_test.go b/promql/parser/parse_test.go index b5d7c288d1..62349efd93 100644 --- a/promql/parser/parse_test.go +++ b/promql/parser/parse_test.go @@ -2708,7 +2708,7 @@ var testExpr = []struct { errors: ParseErrors{ ParseErr{ PositionRange: posrange.PositionRange{Start: 4, End: 5}, - Err: errors.New("unexpected \"]\" in subquery or range selector, expected number, duration, or step()"), + Err: errors.New("unexpected \"]\" in subquery or range selector, expected number, duration, step(), or range()"), Query: `foo[]`, }, }, @@ -2741,7 +2741,7 @@ var testExpr = []struct { errors: ParseErrors{ ParseErr{ PositionRange: posrange.PositionRange{Start: 22, End: 22}, - Err: errors.New("unexpected end of input in offset, expected number, duration, or step()"), + Err: errors.New("unexpected end of input in offset, expected number, duration, step(), or range()"), Query: `some_metric[5m] OFFSET`, }, }, @@ -4698,6 +4698,100 @@ var testExpr = []struct { }, }, }, + { + input: `foo[range()]`, + expected: &MatrixSelector{ + VectorSelector: &VectorSelector{ + Name: "foo", + LabelMatchers: []*labels.Matcher{ + MustLabelMatcher(labels.MatchEqual, model.MetricNameLabel, "foo"), + }, + PosRange: posrange.PositionRange{Start: 0, End: 3}, + }, + RangeExpr: &DurationExpr{ + Op: RANGE, + StartPos: 4, + EndPos: 11, + }, + EndPos: 12, + }, + }, + { + input: `foo[-range()]`, + expected: &MatrixSelector{ + VectorSelector: &VectorSelector{ + Name: "foo", + LabelMatchers: []*labels.Matcher{ + MustLabelMatcher(labels.MatchEqual, model.MetricNameLabel, "foo"), + }, + PosRange: posrange.PositionRange{Start: 0, End: 3}, + }, + RangeExpr: &DurationExpr{ + Op: SUB, + StartPos: 4, + RHS: &DurationExpr{Op: RANGE, StartPos: 5, EndPos: 12}, + }, + EndPos: 13, + }, + }, + { + input: `foo offset range()`, + expected: &VectorSelector{ + Name: "foo", + LabelMatchers: []*labels.Matcher{ + MustLabelMatcher(labels.MatchEqual, model.MetricNameLabel, "foo"), + }, + PosRange: posrange.PositionRange{Start: 0, End: 18}, + OriginalOffsetExpr: &DurationExpr{ + Op: RANGE, + StartPos: 11, + EndPos: 18, + }, + }, + }, + { + input: `foo offset -range()`, + expected: &VectorSelector{ + Name: "foo", + LabelMatchers: []*labels.Matcher{ + MustLabelMatcher(labels.MatchEqual, model.MetricNameLabel, "foo"), + }, + PosRange: posrange.PositionRange{Start: 0, End: 19}, + OriginalOffsetExpr: &DurationExpr{ + Op: SUB, + RHS: &DurationExpr{Op: RANGE, StartPos: 12, EndPos: 19}, + StartPos: 11, + }, + }, + }, + { + input: `foo[max(range(),5s)]`, + expected: &MatrixSelector{ + VectorSelector: &VectorSelector{ + Name: "foo", + LabelMatchers: []*labels.Matcher{ + MustLabelMatcher(labels.MatchEqual, model.MetricNameLabel, "foo"), + }, + PosRange: posrange.PositionRange{Start: 0, End: 3}, + }, + RangeExpr: &DurationExpr{ + Op: MAX, + LHS: &DurationExpr{ + Op: RANGE, + StartPos: 8, + EndPos: 15, + }, + RHS: &NumberLiteral{ + Val: 5, + Duration: true, + PosRange: posrange.PositionRange{Start: 16, End: 18}, + }, + StartPos: 4, + EndPos: 19, + }, + EndPos: 20, + }, + }, { input: `foo[4s+4s:1s*2] offset (5s-8)`, expected: &SubqueryExpr{ @@ -4942,7 +5036,7 @@ var testExpr = []struct { errors: ParseErrors{ ParseErr{ PositionRange: posrange.PositionRange{Start: 8, End: 9}, - Err: errors.New(`unexpected "]" in subquery or range selector, expected number, duration, or step()`), + Err: errors.New(`unexpected "]" in subquery or range selector, expected number, duration, step(), or range()`), Query: `foo[step]`, }, }, diff --git a/promql/parser/printer.go b/promql/parser/printer.go index 961167428b..2531bb6272 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -182,6 +182,8 @@ func (node *DurationExpr) writeTo(b *bytes.Buffer) { switch { case node.Op == STEP: b.WriteString("step()") + case node.Op == RANGE: + b.WriteString("range()") case node.Op == MIN: b.WriteString("min(") b.WriteString(node.LHS.String()) diff --git a/promql/parser/printer_test.go b/promql/parser/printer_test.go index b28da988da..b7fa3e6ccb 100644 --- a/promql/parser/printer_test.go +++ b/promql/parser/printer_test.go @@ -266,6 +266,21 @@ func TestExprString(t *testing.T) { { in: "foo[200 - min(step() + 10s, -max(step() ^ 2, 3))]", }, + { + in: "foo[range()]", + }, + { + in: "foo[-range()]", + }, + { + in: "foo offset range()", + }, + { + in: "foo offset -range()", + }, + { + in: "foo[max(range(), 5s)]", + }, { in: `predict_linear(foo[1h], 3000)`, }, diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go index d4a11b9e50..83e47f1915 100644 --- a/promql/promqltest/test.go +++ b/promql/promqltest/test.go @@ -1519,6 +1519,10 @@ func (t *test) runInstantQuery(iq atModifierTestCase, cmd *evalCmd, engine promq // Check query returns same result in range mode, // by checking against the middle step. + // Skip this check for queries containing range() since it would resolve differently. + if strings.Contains(iq.expr, "range()") { + return nil + } q, err = engine.NewRangeQuery(t.context, t.storage, nil, iq.expr, iq.evalTime.Add(-time.Minute), iq.evalTime.Add(time.Minute), time.Minute) if err != nil { return fmt.Errorf("error creating range query for %q (line %d): %w", cmd.expr, cmd.line, err) diff --git a/promql/promqltest/testdata/duration_expression.test b/promql/promqltest/testdata/duration_expression.test index db8253777b..e58b34131b 100644 --- a/promql/promqltest/testdata/duration_expression.test +++ b/promql/promqltest/testdata/duration_expression.test @@ -225,4 +225,27 @@ eval range from 50s to 60s step 5s metric1_total offset max(3s,min(step(), 1s))+ {} 8047 8052 8057 eval range from 50s to 60s step 5s metric1_total offset -(min(step(), 2s)-5)+8000 - {} 8047 8052 8057 \ No newline at end of file + {} 8047 8052 8057 + +# Test range() function - resolves to query range (end - start). +# For a range query from 50s to 60s, range() = 10s. +eval range from 50s to 60s step 10s count_over_time(metric1_total[range()]) + {} 10 10 + +eval range from 50s to 60s step 5s count_over_time(metric1_total[range()]) + {} 10 10 10 + +eval range from 50s to 60s step 5s metric1_total offset range() + metric1_total{} 40 45 50 + +eval range from 50s to 60s step 5s metric1_total offset min(range(), 8s) + metric1_total{} 42 47 52 + +clear + +load 1s + metric1_total 0+1x100 + +# For an instant query (start == end), range() = 0s, offset 0s. +eval instant at 50s metric1_total offset range() + metric1_total{} 50 From 952efe77ad15e99e0bc835971f4f48a1856ae102 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?H=C3=A9lia=20Barroso?= <66432275+heliapb@users.noreply.github.com> Date: Thu, 11 Dec 2025 08:11:42 +0000 Subject: [PATCH 125/439] Merge pull request #17667 from heliapb/fix/changelog fix: add correct pr in unified AWS SD in prometheus changelog --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 15b620bfa6..f84115bb71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ * [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #17287 * [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #17592 * [FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. #16483 -* [FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. #17046 +* [FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. #17406 * [FEATURE] Native histograms are now a stable, but optional feature, use the `scrape_native_histogram` config setting. #17232 #17315 * [FEATURE] UI: Support anchored and smoothed keyword in promql editor. #17239 * [FEATURE] UI: Show detailed relabeling steps for each discovered target. #17337 From 583bc01cc9f37bed237880083d44cab0d01e056a Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Thu, 11 Dec 2025 08:14:35 +0000 Subject: [PATCH 126/439] Refactor: TSDB: small improvement to Postings (#17661) * [TESTS] TSDB: Check that ListPostings are sorted `newListPostings` is a convenient place to do this; move it into the test file because it's not needed for anything else. Also simplify the existing `SliceIsSorted` check. Signed-off-by: Bryan Boreham * [COMMENTS] TSDB: Document that Postings are ordered. The description of `Seek()` implies they are, but it's better to be explicit. Signed-off-by: Bryan Boreham * [REFACTOR] Unexport ListPostings type It's only used within the one package, and it would be surprising if some downstream code did rely on this type, given all of its members are unexported. Signed-off-by: Bryan Boreham --------- Signed-off-by: Bryan Boreham --- tsdb/index/postings.go | 44 +++++++++++------------ tsdb/index/postings_test.go | 70 ++++++++++++++++++++----------------- 2 files changed, 59 insertions(+), 55 deletions(-) diff --git a/tsdb/index/postings.go b/tsdb/index/postings.go index 665a241c34..0185f58819 100644 --- a/tsdb/index/postings.go +++ b/tsdb/index/postings.go @@ -391,7 +391,7 @@ func (p *MemPostings) Iter(f func(labels.Label, Postings) error) error { for n, e := range p.m { for v, p := range e { - if err := f(labels.Label{Name: n, Value: v}, newListPostings(p...)); err != nil { + if err := f(labels.Label{Name: n, Value: v}, NewListPostings(p)); err != nil { return err } } @@ -478,8 +478,8 @@ func (p *MemPostings) PostingsForLabelMatching(ctx context.Context, name string, } // Now `vals` only contains the values that matched, get their postings. - its := make([]*ListPostings, 0, len(vals)) - lps := make([]ListPostings, len(vals)) + its := make([]*listPostings, 0, len(vals)) + lps := make([]listPostings, len(vals)) p.mtx.RLock() e := p.m[name] for i, v := range vals { @@ -488,7 +488,7 @@ func (p *MemPostings) PostingsForLabelMatching(ctx context.Context, name string, // If we didn't let the mutex go, we'd have these postings here, but they would be pointing nowhere // because there would be a `MemPostings.Delete()` call waiting for the lock to delete these labels, // because the series were deleted already. - lps[i] = ListPostings{list: refs} + lps[i] = listPostings{list: refs} its = append(its, &lps[i]) } } @@ -500,13 +500,13 @@ func (p *MemPostings) PostingsForLabelMatching(ctx context.Context, name string, // Postings returns a postings iterator for the given label values. func (p *MemPostings) Postings(ctx context.Context, name string, values ...string) Postings { - res := make([]*ListPostings, 0, len(values)) - lps := make([]ListPostings, len(values)) + res := make([]*listPostings, 0, len(values)) + lps := make([]listPostings, len(values)) p.mtx.RLock() postingsMapForName := p.m[name] for i, value := range values { if lp := postingsMapForName[value]; lp != nil { - lps[i] = ListPostings{list: lp} + lps[i] = listPostings{list: lp} res = append(res, &lps[i]) } } @@ -518,12 +518,12 @@ func (p *MemPostings) PostingsForAllLabelValues(ctx context.Context, name string p.mtx.RLock() e := p.m[name] - its := make([]*ListPostings, 0, len(e)) - lps := make([]ListPostings, len(e)) + its := make([]*listPostings, 0, len(e)) + lps := make([]listPostings, len(e)) i := 0 for _, refs := range e { if len(refs) > 0 { - lps[i] = ListPostings{list: refs} + lps[i] = listPostings{list: refs} its = append(its, &lps[i]) } i++ @@ -542,7 +542,7 @@ func ExpandPostings(p Postings) (res []storage.SeriesRef, err error) { return res, p.Err() } -// Postings provides iterative access over a postings list. +// Postings provides iterative access over an ordered list of SeriesRef. type Postings interface { // Next advances the iterator and returns true if another value was found. Next() bool @@ -827,25 +827,23 @@ func (rp *removedPostings) Err() error { return rp.remove.Err() } -// ListPostings implements the Postings interface over a plain list. -type ListPostings struct { +// listPostings implements the Postings interface over a plain list. +type listPostings struct { list []storage.SeriesRef cur storage.SeriesRef } +// NewListPostings creates a Postings from the supplied SeriesRefs, which must be in order. +// The list slice passed in is retained. func NewListPostings(list []storage.SeriesRef) Postings { - return newListPostings(list...) + return &listPostings{list: list} } -func newListPostings(list ...storage.SeriesRef) *ListPostings { - return &ListPostings{list: list} -} - -func (it *ListPostings) At() storage.SeriesRef { +func (it *listPostings) At() storage.SeriesRef { return it.cur } -func (it *ListPostings) Next() bool { +func (it *listPostings) Next() bool { if len(it.list) > 0 { it.cur = it.list[0] it.list = it.list[1:] @@ -855,7 +853,7 @@ func (it *ListPostings) Next() bool { return false } -func (it *ListPostings) Seek(x storage.SeriesRef) bool { +func (it *listPostings) Seek(x storage.SeriesRef) bool { // If the current value satisfies, then return. if it.cur >= x { return true @@ -877,12 +875,12 @@ func (it *ListPostings) Seek(x storage.SeriesRef) bool { return true } -func (*ListPostings) Err() error { +func (*listPostings) Err() error { return nil } // Len returns the remaining number of postings in the list. -func (it *ListPostings) Len() int { +func (it *listPostings) Len() int { return len(it.list) } diff --git a/tsdb/index/postings_test.go b/tsdb/index/postings_test.go index 56c0f02455..0fbe7a58a2 100644 --- a/tsdb/index/postings_test.go +++ b/tsdb/index/postings_test.go @@ -20,6 +20,7 @@ import ( "errors" "fmt" "math/rand" + "slices" "sort" "strconv" "strings" @@ -62,9 +63,7 @@ func TestMemPostings_ensureOrder(t *testing.T) { for _, e := range p.m { for _, l := range e { - ok := sort.SliceIsSorted(l, func(i, j int) bool { - return l[i] < l[j] - }) + ok := slices.IsSorted(l) require.True(t, ok, "postings list %v is not sorted", l) } } @@ -285,9 +284,16 @@ func consumePostings(p Postings) error { return p.Err() } +func newListPostings(list ...storage.SeriesRef) *listPostings { + if !slices.IsSorted(list) { + panic("newListPostings: list is not sorted") + } + return &listPostings{list: list} +} + // Create ListPostings for a benchmark, collecting the original sets of references // so they can be reset without additional memory allocations. -func createPostings(lps *[]*ListPostings, refs *[][]storage.SeriesRef, params ...storage.SeriesRef) { +func createPostings(lps *[]*listPostings, refs *[][]storage.SeriesRef, params ...storage.SeriesRef) { var temp []storage.SeriesRef for i := 0; i < len(params); i += 3 { for j := params[i]; j < params[i+1]; j += params[i+2] { @@ -299,7 +305,7 @@ func createPostings(lps *[]*ListPostings, refs *[][]storage.SeriesRef, params .. } // Reset the ListPostings to their original values each time round the benchmark loop. -func resetPostings(its []Postings, lps []*ListPostings, refs [][]storage.SeriesRef) { +func resetPostings(its []Postings, lps []*listPostings, refs [][]storage.SeriesRef) { for j := range refs { lps[j].list = refs[j] its[j] = lps[j] @@ -308,7 +314,7 @@ func resetPostings(its []Postings, lps []*ListPostings, refs [][]storage.SeriesR func BenchmarkIntersect(t *testing.B) { t.Run("LongPostings1", func(bench *testing.B) { - var lps []*ListPostings + var lps []*listPostings var refs [][]storage.SeriesRef createPostings(&lps, &refs, 0, 10000000, 2) createPostings(&lps, &refs, 5000000, 5000100, 4, 5090000, 5090600, 4) @@ -327,7 +333,7 @@ func BenchmarkIntersect(t *testing.B) { }) t.Run("LongPostings2", func(bench *testing.B) { - var lps []*ListPostings + var lps []*listPostings var refs [][]storage.SeriesRef createPostings(&lps, &refs, 0, 12500000, 1) createPostings(&lps, &refs, 7500000, 12500000, 1) @@ -346,7 +352,7 @@ func BenchmarkIntersect(t *testing.B) { }) t.Run("ManyPostings", func(bench *testing.B) { - var lps []*ListPostings + var lps []*listPostings var refs [][]storage.SeriesRef for range 100 { createPostings(&lps, &refs, 1, 100, 1) @@ -365,7 +371,7 @@ func BenchmarkIntersect(t *testing.B) { } func BenchmarkMerge(t *testing.B) { - var lps []*ListPostings + var lps []*listPostings var refs [][]storage.SeriesRef // Create 100000 matchers(k=100000), making sure all memory allocation is done before starting the loop. @@ -378,7 +384,7 @@ func BenchmarkMerge(t *testing.B) { refs = append(refs, temp) } - its := make([]*ListPostings, len(refs)) + its := make([]*listPostings, len(refs)) for _, nSeries := range []int{1, 10, 10000, 100000} { t.Run(strconv.Itoa(nSeries), func(bench *testing.B) { ctx := context.Background() @@ -1229,78 +1235,78 @@ func TestPostingsWithIndexHeap(t *testing.T) { func TestListPostings(t *testing.T) { t.Run("empty list", func(t *testing.T) { p := NewListPostings(nil) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) require.False(t, p.Next()) require.False(t, p.Seek(10)) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("one posting", func(t *testing.T) { t.Run("next", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10}) - require.Equal(t, 1, p.(*ListPostings).Len()) + require.Equal(t, 1, p.(*listPostings).Len()) require.True(t, p.Next()) require.Equal(t, storage.SeriesRef(10), p.At()) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek less", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10}) - require.Equal(t, 1, p.(*ListPostings).Len()) + require.Equal(t, 1, p.(*listPostings).Len()) require.True(t, p.Seek(5)) require.Equal(t, storage.SeriesRef(10), p.At()) require.True(t, p.Seek(5)) require.Equal(t, storage.SeriesRef(10), p.At()) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek equal", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10}) - require.Equal(t, 1, p.(*ListPostings).Len()) + require.Equal(t, 1, p.(*listPostings).Len()) require.True(t, p.Seek(10)) require.Equal(t, storage.SeriesRef(10), p.At()) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek more", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10}) - require.Equal(t, 1, p.(*ListPostings).Len()) + require.Equal(t, 1, p.(*listPostings).Len()) require.False(t, p.Seek(15)) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek after next", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10}) - require.Equal(t, 1, p.(*ListPostings).Len()) + require.Equal(t, 1, p.(*listPostings).Len()) require.True(t, p.Next()) require.False(t, p.Seek(15)) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) }) t.Run("multiple postings", func(t *testing.T) { t.Run("next", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10, 20}) - require.Equal(t, 2, p.(*ListPostings).Len()) + require.Equal(t, 2, p.(*listPostings).Len()) require.True(t, p.Next()) require.Equal(t, storage.SeriesRef(10), p.At()) require.True(t, p.Next()) require.Equal(t, storage.SeriesRef(20), p.At()) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10, 20}) - require.Equal(t, 2, p.(*ListPostings).Len()) + require.Equal(t, 2, p.(*listPostings).Len()) require.True(t, p.Seek(5)) require.Equal(t, storage.SeriesRef(10), p.At()) require.True(t, p.Seek(5)) @@ -1315,30 +1321,30 @@ func TestListPostings(t *testing.T) { require.Equal(t, storage.SeriesRef(20), p.At()) require.False(t, p.Next()) require.NoError(t, p.Err()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek lest than last", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10, 20, 30, 40, 50}) - require.Equal(t, 5, p.(*ListPostings).Len()) + require.Equal(t, 5, p.(*listPostings).Len()) require.True(t, p.Seek(45)) require.Equal(t, storage.SeriesRef(50), p.At()) require.False(t, p.Next()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek exactly last", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10, 20, 30, 40, 50}) - require.Equal(t, 5, p.(*ListPostings).Len()) + require.Equal(t, 5, p.(*listPostings).Len()) require.True(t, p.Seek(50)) require.Equal(t, storage.SeriesRef(50), p.At()) require.False(t, p.Next()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) t.Run("seek more than last", func(t *testing.T) { p := NewListPostings([]storage.SeriesRef{10, 20, 30, 40, 50}) - require.Equal(t, 5, p.(*ListPostings).Len()) + require.Equal(t, 5, p.(*listPostings).Len()) require.False(t, p.Seek(60)) require.False(t, p.Next()) - require.Equal(t, 0, p.(*ListPostings).Len()) + require.Equal(t, 0, p.(*listPostings).Len()) }) }) From c2b86775b6f51fe4383f0e6b6a3f460a88263f99 Mon Sep 17 00:00:00 2001 From: Rushabh Mehta <139112780+RushabhMehta2005@users.noreply.github.com> Date: Fri, 12 Dec 2025 18:31:57 +0530 Subject: [PATCH 127/439] scrape: Fix potential goroutine leak in scrapeAndReport (#17554) * [scrape] Fix potential goroutine leak in scrape loop Signed-off-by: Rushabh Mehta * Use correct error var Signed-off-by: Rushabh Mehta * Add regression test Signed-off-by: Rushabh Mehta --------- Signed-off-by: Rushabh Mehta --- scrape/scrape.go | 10 ++++++++-- scrape/scrape_test.go | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/scrape/scrape.go b/scrape/scrape.go index bbb93c8801..fc406a6811 100644 --- a/scrape/scrape.go +++ b/scrape/scrape.go @@ -1459,7 +1459,10 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er sl.l.Warn("Append failed", "err", err) } if errc != nil { - errc <- forcedErr + select { + case errc <- forcedErr: + case <-sl.ctx.Done(): + } } return start @@ -1496,7 +1499,10 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er } sl.scrapeFailureLoggerMtx.RUnlock() if errc != nil { - errc <- scrapeErr + select { + case errc <- scrapeErr: + case <-sl.ctx.Done(): + } } if errors.Is(scrapeErr, errBodySizeLimit) { bytesRead = -1 diff --git a/scrape/scrape_test.go b/scrape/scrape_test.go index 5ccdb80019..eab1499158 100644 --- a/scrape/scrape_test.go +++ b/scrape/scrape_test.go @@ -51,6 +51,7 @@ import ( "go.opentelemetry.io/otel/propagation" sdktrace "go.opentelemetry.io/otel/sdk/trace" "go.uber.org/atomic" + "go.uber.org/goleak" "github.com/prometheus/prometheus/config" "github.com/prometheus/prometheus/discovery" @@ -1258,6 +1259,45 @@ func TestScrapeLoopForcedErr(t *testing.T) { } } +func TestScrapeLoopRun_ContextCancelTerminatesBlockedSend(t *testing.T) { + // Regression test for issue #17553 + defer goleak.VerifyNone(t) + + var ( + signal = make(chan struct{}) + errc = make(chan error) + scraper = &testScraper{} + app = func(context.Context) storage.Appender { return &nopAppender{} } + ) + + ctx, cancel := context.WithCancel(context.Background()) + + sl := newBasicScrapeLoop(t, ctx, scraper, app, 100*time.Millisecond) + + forcedErr := errors.New("forced err") + sl.setForcedError(forcedErr) + + scraper.scrapeFunc = func(context.Context, io.Writer) error { + return nil + } + + go func() { + sl.run(errc) + close(signal) + }() + + time.Sleep(50 * time.Millisecond) + + cancel() + + select { + case <-signal: + // success case + case <-time.After(3 * time.Second): + require.FailNow(t, "Scrape loop failed to exit on context cancellation (goroutine leak detected)") + } +} + func TestScrapeLoopMetadata(t *testing.T) { var ( signal = make(chan struct{}) From 41665a4a559bbc834c7465bd4683e1ddba2608c2 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Fri, 12 Dec 2025 13:44:13 +0000 Subject: [PATCH 128/439] [CHORE] TSDB: Remove unused LabelValueFor function The last use of this was removed 4 years ago. Signed-off-by: Bryan Boreham --- tsdb/block.go | 10 -------- tsdb/head_read.go | 15 ------------ tsdb/index/index.go | 57 ------------------------------------------- tsdb/ooo_head_read.go | 4 --- tsdb/querier_test.go | 12 --------- 5 files changed, 98 deletions(-) diff --git a/tsdb/block.go b/tsdb/block.go index 44c6ef5053..dcbb172e72 100644 --- a/tsdb/block.go +++ b/tsdb/block.go @@ -102,11 +102,6 @@ type IndexReader interface { // LabelNames returns all the unique label names present in the index in sorted order. LabelNames(ctx context.Context, matchers ...*labels.Matcher) ([]string, error) - // LabelValueFor returns label value for the given label name in the series referred to by ID. - // If the series couldn't be found or the series doesn't have the requested label a - // storage.ErrNotFound is returned as error. - LabelValueFor(ctx context.Context, id storage.SeriesRef, label string) (string, error) - // LabelNamesFor returns all the label names for the series referred to by the postings. // The names returned are sorted. LabelNamesFor(ctx context.Context, postings index.Postings) ([]string, error) @@ -551,11 +546,6 @@ func (r blockIndexReader) Close() error { return nil } -// LabelValueFor returns label value for the given label name in the series referred to by ID. -func (r blockIndexReader) LabelValueFor(ctx context.Context, id storage.SeriesRef, label string) (string, error) { - return r.ir.LabelValueFor(ctx, id, label) -} - // LabelNamesFor returns all the label names for the series referred to by the postings. // The names returned are sorted. func (r blockIndexReader) LabelNamesFor(ctx context.Context, postings index.Postings) ([]string, error) { diff --git a/tsdb/head_read.go b/tsdb/head_read.go index 8485d65435..f2681accc0 100644 --- a/tsdb/head_read.go +++ b/tsdb/head_read.go @@ -261,21 +261,6 @@ func unpackHeadChunkRef(ref chunks.ChunkRef) (seriesID chunks.HeadSeriesRef, chu return sid, (cid & (oooChunkIDMask - 1)), (cid & oooChunkIDMask) != 0 } -// LabelValueFor returns label value for the given label name in the series referred to by ID. -func (h *headIndexReader) LabelValueFor(_ context.Context, id storage.SeriesRef, label string) (string, error) { - memSeries := h.head.series.getByID(chunks.HeadSeriesRef(id)) - if memSeries == nil { - return "", storage.ErrNotFound - } - - value := memSeries.labels().Get(label) - if value == "" { - return "", storage.ErrNotFound - } - - return value, nil -} - // LabelNamesFor returns all the label names for the series referred to by the postings. // The names returned are sorted. func (h *headIndexReader) LabelNamesFor(ctx context.Context, series index.Postings) ([]string, error) { diff --git a/tsdb/index/index.go b/tsdb/index/index.go index 28eacd7c00..253a515815 100644 --- a/tsdb/index/index.go +++ b/tsdb/index/index.go @@ -1447,32 +1447,6 @@ func (r *Reader) LabelNamesFor(ctx context.Context, postings Postings) ([]string return names, nil } -// LabelValueFor returns label value for the given label name in the series referred to by ID. -func (r *Reader) LabelValueFor(ctx context.Context, id storage.SeriesRef, label string) (string, error) { - offset := id - // In version 2 series IDs are no longer exact references but series are 16-byte padded - // and the ID is the multiple of 16 of the actual position. - if r.version != FormatV1 { - offset = id * seriesByteAlign - } - d := encoding.NewDecbufUvarintAt(r.b, int(offset), castagnoliTable) - buf := d.Get() - if d.Err() != nil { - return "", fmt.Errorf("label values for: %w", d.Err()) - } - - value, err := r.dec.LabelValueFor(ctx, buf, label) - if err != nil { - return "", storage.ErrNotFound - } - - if value == "" { - return "", storage.ErrNotFound - } - - return value, nil -} - // Series reads the series with the given ID and writes its labels and chunks into builder and chks. func (r *Reader) Series(id storage.SeriesRef, builder *labels.ScratchBuilder, chks *[]chunks.Meta) error { offset := id @@ -1809,37 +1783,6 @@ func (*Decoder) LabelNamesOffsetsFor(b []byte) ([]uint32, error) { return offsets, d.Err() } -// LabelValueFor decodes a label for a given series. -func (dec *Decoder) LabelValueFor(ctx context.Context, b []byte, label string) (string, error) { - d := encoding.Decbuf{B: b} - k := d.Uvarint() - - for range k { - lno := uint32(d.Uvarint()) - lvo := uint32(d.Uvarint()) - - if d.Err() != nil { - return "", fmt.Errorf("read series label offsets: %w", d.Err()) - } - - ln, err := dec.LookupSymbol(ctx, lno) - if err != nil { - return "", fmt.Errorf("lookup label name: %w", err) - } - - if ln == label { - lv, err := dec.LookupSymbol(ctx, lvo) - if err != nil { - return "", fmt.Errorf("lookup label value: %w", err) - } - - return lv, nil - } - } - - return "", d.Err() -} - // Series decodes a series entry from the given byte slice into builder and chks. // Previous contents of builder can be overwritten - make sure you copy before retaining. // Skips reading chunks metadata if chks is nil. diff --git a/tsdb/ooo_head_read.go b/tsdb/ooo_head_read.go index af8f9b1f83..4cecb9fd6c 100644 --- a/tsdb/ooo_head_read.go +++ b/tsdb/ooo_head_read.go @@ -500,10 +500,6 @@ func (*OOOCompactionHeadIndexReader) LabelNames(context.Context, ...*labels.Matc return nil, errors.New("not implemented") } -func (*OOOCompactionHeadIndexReader) LabelValueFor(context.Context, storage.SeriesRef, string) (string, error) { - return "", errors.New("not implemented") -} - func (*OOOCompactionHeadIndexReader) LabelNamesFor(context.Context, index.Postings) ([]string, error) { return nil, errors.New("not implemented") } diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go index 6c3e37792f..4fe21c31ff 100644 --- a/tsdb/querier_test.go +++ b/tsdb/querier_test.go @@ -2294,10 +2294,6 @@ func (m mockIndex) LabelValues(_ context.Context, name string, hints *storage.La return values, nil } -func (m mockIndex) LabelValueFor(_ context.Context, id storage.SeriesRef, label string) (string, error) { - return m.series[id].l.Get(label), nil -} - func (m mockIndex) LabelNamesFor(_ context.Context, postings index.Postings) ([]string, error) { namesMap := make(map[string]bool) for postings.Next() { @@ -3315,10 +3311,6 @@ func (mockMatcherIndex) LabelValues(context.Context, string, *storage.LabelHints return []string{}, errors.New("label values called") } -func (mockMatcherIndex) LabelValueFor(context.Context, storage.SeriesRef, string) (string, error) { - return "", errors.New("label value for called") -} - func (mockMatcherIndex) LabelNamesFor(context.Context, index.Postings) ([]string, error) { return nil, errors.New("label names for called") } @@ -3739,10 +3731,6 @@ func (mockReaderOfLabels) LabelValues(context.Context, string, *storage.LabelHin return make([]string, mockReaderOfLabelsSeriesCount), nil } -func (mockReaderOfLabels) LabelValueFor(context.Context, storage.SeriesRef, string) (string, error) { - panic("LabelValueFor called") -} - func (mockReaderOfLabels) SortedLabelValues(context.Context, string, *storage.LabelHints, ...*labels.Matcher) ([]string, error) { panic("SortedLabelValues called") } From 6efbb873c720b3f1af6a8a401fc738773663bb4e Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Fri, 12 Dec 2025 14:48:29 +0100 Subject: [PATCH 129/439] promql: Fix collision error with delayed name removal for non-overlapping series Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- promql/engine.go | 99 ++++++++++++++++++- .../testdata/name_label_dropping.test | 10 ++ 2 files changed, 104 insertions(+), 5 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 6ba6008b19..37c4e12cd9 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -1175,7 +1175,7 @@ func (ev *evaluator) Eval(ctx context.Context, expr parser.Expr) (v parser.Value v, ws = ev.eval(ctx, expr) if ev.enableDelayedNameRemoval { - ev.cleanupMetricLabels(v) + v = ev.cleanupMetricLabels(v) } return v, ws, nil } @@ -3832,7 +3832,7 @@ func (*evaluator) aggregationCountValues(e *parser.AggregateExpr, grouping []str return enh.Out, nil } -func (ev *evaluator) cleanupMetricLabels(v parser.Value) { +func (ev *evaluator) cleanupMetricLabels(v parser.Value) parser.Value { if v.Type() == parser.ValueTypeMatrix { mat := v.(Matrix) for i := range mat { @@ -3840,9 +3840,7 @@ func (ev *evaluator) cleanupMetricLabels(v parser.Value) { mat[i].Metric = mat[i].Metric.DropReserved(schema.IsMetadataLabel) } } - if mat.ContainsSameLabelset() { - ev.errorf("vector cannot contain metrics with the same labelset") - } + return ev.mergeSeriesWithSameLabelset(mat) } else if v.Type() == parser.ValueTypeVector { vec := v.(Vector) for i := range vec { @@ -3853,7 +3851,98 @@ func (ev *evaluator) cleanupMetricLabels(v parser.Value) { if vec.ContainsSameLabelset() { ev.errorf("vector cannot contain metrics with the same labelset") } + return vec } + return v +} + +// mergeSeriesWithSameLabelset merges series in a matrix that have the same labelset +// after __name__ label removal. This happens when delayed name removal is enabled and +// operations like OR combine series that originally had different names but end up +// with the same labelset after dropping the name. If series with the same labelset +// have overlapping timestamps, an error is returned. +func (ev *evaluator) mergeSeriesWithSameLabelset(mat Matrix) Matrix { + if len(mat) <= 1 { + return mat + } + + // Group series by their labelset hash. + seriesByHash := make(map[uint64][]int) + for i := range mat { + hash := mat[i].Metric.Hash() + seriesByHash[hash] = append(seriesByHash[hash], i) + } + + // Check if any merging is needed. + needsMerge := false + for _, indices := range seriesByHash { + if len(indices) > 1 { + needsMerge = true + break + } + } + + if !needsMerge { + return mat + } + + // Merge series with the same labelset. + merged := make(Matrix, 0, len(seriesByHash)) + for _, indices := range seriesByHash { + base := mat[indices[0]] + + if len(indices) == 1 { + // No collision, add as-is. + merged = append(merged, base) + continue + } + + // Multiple series with the same labelset - check for overlaps and merge. + // Build a set of timestamps to detect overlaps. + timestamps := make(map[int64]struct{}, len(base.Floats)+len(base.Histograms)) + for _, p := range base.Floats { + timestamps[p.T] = struct{}{} + } + for _, p := range base.Histograms { + timestamps[p.T] = struct{}{} + } + + // Merge remaining series, checking for timestamp overlaps. + for _, idx := range indices[1:] { + series := mat[idx] + + // Check floats for overlaps. + for _, p := range series.Floats { + if _, exists := timestamps[p.T]; exists { + ev.errorf("vector cannot contain metrics with the same labelset") + } + timestamps[p.T] = struct{}{} + } + // Check histograms for overlaps. + for _, p := range series.Histograms { + if _, exists := timestamps[p.T]; exists { + ev.errorf("vector cannot contain metrics with the same labelset") + } + timestamps[p.T] = struct{}{} + } + + // No overlaps, merge the samples. + base.Floats = append(base.Floats, series.Floats...) + base.Histograms = append(base.Histograms, series.Histograms...) + } + + // Sort merged samples by timestamp. + sort.Slice(base.Floats, func(i, j int) bool { + return base.Floats[i].T < base.Floats[j].T + }) + sort.Slice(base.Histograms, func(i, j int) bool { + return base.Histograms[i].T < base.Histograms[j].T + }) + + merged = append(merged, base) + } + + return merged } func addToSeries(ss *Series, ts int64, f float64, h *histogram.FloatHistogram, numSteps int) { diff --git a/promql/promqltest/testdata/name_label_dropping.test b/promql/promqltest/testdata/name_label_dropping.test index 3a6f4098df..e0180c7ffe 100644 --- a/promql/promqltest/testdata/name_label_dropping.test +++ b/promql/promqltest/testdata/name_label_dropping.test @@ -126,3 +126,13 @@ eval instant at 10m sum by (__name__) (metric_total{env="3"} or rate(metric_tota # Same as above, but with reversed order. eval instant at 10m sum by (__name__) (rate(metric_total{env="3"}[5m]) or metric_total{env="1"}) metric_total 10 + +clear + +# Test delayed name removal with range queries and OR operator. +load 10m + metric_a 1 _ + metric_b 3 4 + +eval range from 0 to 20m step 10m -metric_a or -metric_b + {} -1 -4 _ From 763b935b456738cf51cffbcfaa8073d80c785d4c Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 12 Dec 2025 11:15:10 +0000 Subject: [PATCH 130/439] refactor(tsdb/agent): 1:1 copy of db.go and db_test.go for starting point Signed-off-by: bwplotka --- tsdb/agent/db_append_v2.go | 1292 ++++++++++++++++++++++++++++ tsdb/agent/db_append_v2_test.go | 1396 +++++++++++++++++++++++++++++++ 2 files changed, 2688 insertions(+) create mode 100644 tsdb/agent/db_append_v2.go create mode 100644 tsdb/agent/db_append_v2_test.go diff --git a/tsdb/agent/db_append_v2.go b/tsdb/agent/db_append_v2.go new file mode 100644 index 0000000000..5c9774cd58 --- /dev/null +++ b/tsdb/agent/db_append_v2.go @@ -0,0 +1,1292 @@ +// Copyright 2021 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package agent + +import ( + "context" + "errors" + "fmt" + "log/slog" + "math" + "path/filepath" + "sync" + "time" + "unicode/utf8" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/common/model" + "go.uber.org/atomic" + + "github.com/prometheus/prometheus/model/exemplar" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/model/metadata" + "github.com/prometheus/prometheus/model/timestamp" + "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/storage/remote" + "github.com/prometheus/prometheus/tsdb" + "github.com/prometheus/prometheus/tsdb/chunks" + tsdb_errors "github.com/prometheus/prometheus/tsdb/errors" + "github.com/prometheus/prometheus/tsdb/record" + "github.com/prometheus/prometheus/tsdb/tsdbutil" + "github.com/prometheus/prometheus/tsdb/wlog" + "github.com/prometheus/prometheus/util/compression" + "github.com/prometheus/prometheus/util/zeropool" +) + +const ( + sampleMetricTypeFloat = "float" + sampleMetricTypeHistogram = "histogram" +) + +var ErrUnsupported = errors.New("unsupported operation with WAL-only storage") + +// Default values for options. +var ( + DefaultTruncateFrequency = 2 * time.Hour + DefaultMinWALTime = int64(5 * time.Minute / time.Millisecond) + DefaultMaxWALTime = int64(4 * time.Hour / time.Millisecond) +) + +// Options of the WAL storage. +type Options struct { + // Segments (wal files) max size. + // WALSegmentSize <= 0, segment size is default size. + // WALSegmentSize > 0, segment size is WALSegmentSize. + WALSegmentSize int + + // WALCompression configures the compression type to use on records in the WAL. + WALCompression compression.Type + + // StripeSize is the size (power of 2) in entries of the series hash map. Reducing the size will save memory but impact performance. + StripeSize int + + // TruncateFrequency determines how frequently to truncate data from the WAL. + TruncateFrequency time.Duration + + // Shortest and longest amount of time data can exist in the WAL before being + // deleted. + MinWALTime, MaxWALTime int64 + + // NoLockfile disables creation and consideration of a lock file. + NoLockfile bool + + // OutOfOrderTimeWindow specifies how much out of order is allowed, if any. + OutOfOrderTimeWindow int64 +} + +// DefaultOptions used for the WAL storage. They are reasonable for setups using +// millisecond-precision timestamps. +func DefaultOptions() *Options { + return &Options{ + WALSegmentSize: wlog.DefaultSegmentSize, + WALCompression: compression.None, + StripeSize: tsdb.DefaultStripeSize, + TruncateFrequency: DefaultTruncateFrequency, + MinWALTime: DefaultMinWALTime, + MaxWALTime: DefaultMaxWALTime, + NoLockfile: false, + OutOfOrderTimeWindow: 0, + } +} + +type dbMetrics struct { + r prometheus.Registerer + + numActiveSeries prometheus.Gauge + numWALSeriesPendingDeletion prometheus.Gauge + totalAppendedSamples *prometheus.CounterVec + totalAppendedExemplars prometheus.Counter + totalOutOfOrderSamples prometheus.Counter + walTruncateDuration prometheus.Summary + walCorruptionsTotal prometheus.Counter + walTotalReplayDuration prometheus.Gauge + checkpointDeleteFail prometheus.Counter + checkpointDeleteTotal prometheus.Counter + checkpointCreationFail prometheus.Counter + checkpointCreationTotal prometheus.Counter +} + +func newDBMetrics(r prometheus.Registerer) *dbMetrics { + m := dbMetrics{r: r} + m.numActiveSeries = prometheus.NewGauge(prometheus.GaugeOpts{ + Name: "prometheus_agent_active_series", + Help: "Number of active series being tracked by the WAL storage", + }) + + m.numWALSeriesPendingDeletion = prometheus.NewGauge(prometheus.GaugeOpts{ + Name: "prometheus_agent_deleted_series", + Help: "Number of series pending deletion from the WAL", + }) + + m.totalAppendedSamples = prometheus.NewCounterVec(prometheus.CounterOpts{ + Name: "prometheus_agent_samples_appended_total", + Help: "Total number of samples appended to the storage", + }, []string{"type"}) + + m.totalAppendedExemplars = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_exemplars_appended_total", + Help: "Total number of exemplars appended to the storage", + }) + + m.totalOutOfOrderSamples = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_out_of_order_samples_total", + Help: "Total number of out of order samples ingestion failed attempts.", + }) + + m.walTruncateDuration = prometheus.NewSummary(prometheus.SummaryOpts{ + Name: "prometheus_agent_truncate_duration_seconds", + Help: "Duration of WAL truncation.", + }) + + m.walCorruptionsTotal = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_corruptions_total", + Help: "Total number of WAL corruptions.", + }) + + m.walTotalReplayDuration = prometheus.NewGauge(prometheus.GaugeOpts{ + Name: "prometheus_agent_data_replay_duration_seconds", + Help: "Time taken to replay the data on disk.", + }) + + m.checkpointDeleteFail = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_checkpoint_deletions_failed_total", + Help: "Total number of checkpoint deletions that failed.", + }) + + m.checkpointDeleteTotal = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_checkpoint_deletions_total", + Help: "Total number of checkpoint deletions attempted.", + }) + + m.checkpointCreationFail = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_checkpoint_creations_failed_total", + Help: "Total number of checkpoint creations that failed.", + }) + + m.checkpointCreationTotal = prometheus.NewCounter(prometheus.CounterOpts{ + Name: "prometheus_agent_checkpoint_creations_total", + Help: "Total number of checkpoint creations attempted.", + }) + + if r != nil { + r.MustRegister( + m.numActiveSeries, + m.numWALSeriesPendingDeletion, + m.totalAppendedSamples, + m.totalAppendedExemplars, + m.totalOutOfOrderSamples, + m.walTruncateDuration, + m.walCorruptionsTotal, + m.walTotalReplayDuration, + m.checkpointDeleteFail, + m.checkpointDeleteTotal, + m.checkpointCreationFail, + m.checkpointCreationTotal, + ) + } + + return &m +} + +func (m *dbMetrics) Unregister() { + if m.r == nil { + return + } + cs := []prometheus.Collector{ + m.numActiveSeries, + m.numWALSeriesPendingDeletion, + m.totalAppendedSamples, + m.totalAppendedExemplars, + m.totalOutOfOrderSamples, + m.walTruncateDuration, + m.walCorruptionsTotal, + m.walTotalReplayDuration, + m.checkpointDeleteFail, + m.checkpointDeleteTotal, + m.checkpointCreationFail, + m.checkpointCreationTotal, + } + for _, c := range cs { + m.r.Unregister(c) + } +} + +// DB represents a WAL-only storage. It implements storage.DB. +type DB struct { + mtx sync.RWMutex + logger *slog.Logger + opts *Options + rs *remote.Storage + + wal *wlog.WL + locker *tsdbutil.DirLocker + + appenderPool sync.Pool + bufPool sync.Pool + + // These pools are only used during WAL replay and are reset at the end. + // NOTE: Adjust resetWALReplayResources() upon changes to the pools. + walReplaySeriesPool zeropool.Pool[[]record.RefSeries] + walReplaySamplesPool zeropool.Pool[[]record.RefSample] + walReplayHistogramsPool zeropool.Pool[[]record.RefHistogramSample] + walReplayFloatHistogramsPool zeropool.Pool[[]record.RefFloatHistogramSample] + + nextRef *atomic.Uint64 + series *stripeSeries + // deleted is a map of (ref IDs that should be deleted from WAL) to (the WAL segment they + // must be kept around to). + deleted map[chunks.HeadSeriesRef]int + + donec chan struct{} + stopc chan struct{} + + writeNotified wlog.WriteNotified + + metrics *dbMetrics +} + +// Open returns a new agent.DB in the given directory. +func Open(l *slog.Logger, reg prometheus.Registerer, rs *remote.Storage, dir string, opts *Options) (*DB, error) { + opts = validateOptions(opts) + + locker, err := tsdbutil.NewDirLocker(dir, "agent", l, reg) + if err != nil { + return nil, err + } + if !opts.NoLockfile { + if err := locker.Lock(); err != nil { + return nil, err + } + } + + // remote_write expects WAL to be stored in a "wal" subdirectory of the main storage. + dir = filepath.Join(dir, "wal") + + w, err := wlog.NewSize(l, reg, dir, opts.WALSegmentSize, opts.WALCompression) + if err != nil { + return nil, fmt.Errorf("creating WAL: %w", err) + } + + db := &DB{ + logger: l, + opts: opts, + rs: rs, + + wal: w, + locker: locker, + + nextRef: atomic.NewUint64(0), + series: newStripeSeries(opts.StripeSize), + deleted: make(map[chunks.HeadSeriesRef]int), + + donec: make(chan struct{}), + stopc: make(chan struct{}), + + metrics: newDBMetrics(reg), + } + + db.bufPool.New = func() any { + return make([]byte, 0, 1024) + } + + db.appenderPool.New = func() any { + return &appender{ + DB: db, + pendingSeries: make([]record.RefSeries, 0, 100), + pendingSamples: make([]record.RefSample, 0, 100), + pendingHistograms: make([]record.RefHistogramSample, 0, 100), + pendingFloatHistograms: make([]record.RefFloatHistogramSample, 0, 100), + pendingExamplars: make([]record.RefExemplar, 0, 10), + } + } + + if err := db.replayWAL(); err != nil { + db.logger.Warn("encountered WAL read error, attempting repair", "err", err) + if err := w.Repair(err); err != nil { + return nil, fmt.Errorf("repair corrupted WAL: %w", err) + } + db.logger.Info("successfully repaired WAL") + } + + go db.run() + return db, nil +} + +// SetWriteNotified allows to set an instance to notify when a write happens. +// It must be used during initialization. It is not safe to use it during execution. +func (db *DB) SetWriteNotified(wn wlog.WriteNotified) { + db.writeNotified = wn +} + +func validateOptions(opts *Options) *Options { + if opts == nil { + opts = DefaultOptions() + } + if opts.WALSegmentSize <= 0 { + opts.WALSegmentSize = wlog.DefaultSegmentSize + } + + if opts.WALCompression == "" { + opts.WALCompression = compression.None + } + + // Revert StripeSize to DefaultStripeSize if StripeSize is either 0 or not a power of 2. + if opts.StripeSize <= 0 || ((opts.StripeSize & (opts.StripeSize - 1)) != 0) { + opts.StripeSize = tsdb.DefaultStripeSize + } + if opts.TruncateFrequency <= 0 { + opts.TruncateFrequency = DefaultTruncateFrequency + } + if opts.MinWALTime <= 0 { + opts.MinWALTime = DefaultMinWALTime + } + if opts.MaxWALTime <= 0 { + opts.MaxWALTime = DefaultMaxWALTime + } + if opts.MinWALTime > opts.MaxWALTime { + opts.MaxWALTime = opts.MinWALTime + } + + if t := int64(opts.TruncateFrequency / time.Millisecond); opts.MaxWALTime < t { + opts.MaxWALTime = t + } + return opts +} + +func (db *DB) replayWAL() error { + db.logger.Info("replaying WAL, this may take a while", "dir", db.wal.Dir()) + defer db.resetWALReplayResources() + start := time.Now() + + dir, startFrom, err := wlog.LastCheckpoint(db.wal.Dir()) + if err != nil && !errors.Is(err, record.ErrNotFound) { + return fmt.Errorf("find last checkpoint: %w", err) + } + + multiRef := map[chunks.HeadSeriesRef]chunks.HeadSeriesRef{} + + if err == nil { + sr, err := wlog.NewSegmentsReader(dir) + if err != nil { + return fmt.Errorf("open checkpoint: %w", err) + } + defer func() { + if err := sr.Close(); err != nil { + db.logger.Warn("error while closing the wal segments reader", "err", err) + } + }() + + // A corrupted checkpoint is a hard error for now and requires user + // intervention. There's likely little data that can be recovered anyway. + if err := db.loadWAL(wlog.NewReader(sr), multiRef); err != nil { + return fmt.Errorf("backfill checkpoint: %w", err) + } + startFrom++ + db.logger.Info("WAL checkpoint loaded") + } + + // Find the last segment. + _, last, err := wlog.Segments(db.wal.Dir()) + if err != nil { + return fmt.Errorf("finding WAL segments: %w", err) + } + + // Backfill segments from the most recent checkpoint onwards. + for i := startFrom; i <= last; i++ { + seg, err := wlog.OpenReadSegment(wlog.SegmentName(db.wal.Dir(), i)) + if err != nil { + return fmt.Errorf("open WAL segment: %d: %w", i, err) + } + + sr := wlog.NewSegmentBufReader(seg) + err = db.loadWAL(wlog.NewReader(sr), multiRef) + if err := sr.Close(); err != nil { + db.logger.Warn("error while closing the wal segments reader", "err", err) + } + if err != nil { + return err + } + db.logger.Info("WAL segment loaded", "segment", i, "maxSegment", last) + } + + walReplayDuration := time.Since(start) + db.metrics.walTotalReplayDuration.Set(walReplayDuration.Seconds()) + + return nil +} + +func (db *DB) resetWALReplayResources() { + db.walReplaySeriesPool = zeropool.Pool[[]record.RefSeries]{} + db.walReplaySamplesPool = zeropool.Pool[[]record.RefSample]{} + db.walReplayHistogramsPool = zeropool.Pool[[]record.RefHistogramSample]{} + db.walReplayFloatHistogramsPool = zeropool.Pool[[]record.RefFloatHistogramSample]{} +} + +func (db *DB) loadWAL(r *wlog.Reader, multiRef map[chunks.HeadSeriesRef]chunks.HeadSeriesRef) (err error) { + var ( + syms = labels.NewSymbolTable() // One table for the whole WAL. + dec = record.NewDecoder(syms, db.logger) + lastRef = chunks.HeadSeriesRef(db.nextRef.Load()) + + decoded = make(chan any, 10) + errCh = make(chan error, 1) + ) + + go func() { + defer close(decoded) + var err error + for r.Next() { + rec := r.Record() + switch dec.Type(rec) { + case record.Series: + series := db.walReplaySeriesPool.Get()[:0] + series, err = dec.Series(rec, series) + if err != nil { + errCh <- &wlog.CorruptionErr{ + Err: fmt.Errorf("decode series: %w", err), + Segment: r.Segment(), + Offset: r.Offset(), + } + return + } + decoded <- series + case record.Samples: + samples := db.walReplaySamplesPool.Get()[:0] + samples, err = dec.Samples(rec, samples) + if err != nil { + errCh <- &wlog.CorruptionErr{ + Err: fmt.Errorf("decode samples: %w", err), + Segment: r.Segment(), + Offset: r.Offset(), + } + return + } + decoded <- samples + case record.HistogramSamples, record.CustomBucketsHistogramSamples: + histograms := db.walReplayHistogramsPool.Get()[:0] + histograms, err = dec.HistogramSamples(rec, histograms) + if err != nil { + errCh <- &wlog.CorruptionErr{ + Err: fmt.Errorf("decode histogram samples: %w", err), + Segment: r.Segment(), + Offset: r.Offset(), + } + return + } + decoded <- histograms + case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: + floatHistograms := db.walReplayFloatHistogramsPool.Get()[:0] + floatHistograms, err = dec.FloatHistogramSamples(rec, floatHistograms) + if err != nil { + errCh <- &wlog.CorruptionErr{ + Err: fmt.Errorf("decode float histogram samples: %w", err), + Segment: r.Segment(), + Offset: r.Offset(), + } + return + } + decoded <- floatHistograms + case record.Tombstones, record.Exemplars: + // We don't care about tombstones or exemplars during replay. + // TODO: If decide to decode exemplars, we should make sure to prepopulate + // stripeSeries.exemplars in the next block by using setLatestExemplar. + continue + default: + errCh <- &wlog.CorruptionErr{ + Err: fmt.Errorf("invalid record type %v", dec.Type(rec)), + Segment: r.Segment(), + Offset: r.Offset(), + } + } + } + }() + + var nonExistentSeriesRefs atomic.Uint64 + + for d := range decoded { + switch v := d.(type) { + case []record.RefSeries: + for _, entry := range v { + // If this is a new series, create it in memory. If we never read in a + // sample for this series, its timestamp will remain at 0 and it will + // be deleted at the next GC. + if db.series.GetByID(entry.Ref) == nil { + series := &memSeries{ref: entry.Ref, lset: entry.Labels, lastTs: 0} + db.series.Set(entry.Labels.Hash(), series) + multiRef[entry.Ref] = series.ref + db.metrics.numActiveSeries.Inc() + if entry.Ref > lastRef { + lastRef = entry.Ref + } + } + } + db.walReplaySeriesPool.Put(v) + case []record.RefSample: + for _, entry := range v { + // Update the lastTs for the series based + ref, ok := multiRef[entry.Ref] + if !ok { + nonExistentSeriesRefs.Inc() + continue + } + series := db.series.GetByID(ref) + if entry.T > series.lastTs { + series.lastTs = entry.T + } + } + db.walReplaySamplesPool.Put(v) + case []record.RefHistogramSample: + for _, entry := range v { + // Update the lastTs for the series based + ref, ok := multiRef[entry.Ref] + if !ok { + nonExistentSeriesRefs.Inc() + continue + } + series := db.series.GetByID(ref) + if entry.T > series.lastTs { + series.lastTs = entry.T + } + } + db.walReplayHistogramsPool.Put(v) + case []record.RefFloatHistogramSample: + for _, entry := range v { + // Update the lastTs for the series based + ref, ok := multiRef[entry.Ref] + if !ok { + nonExistentSeriesRefs.Inc() + continue + } + series := db.series.GetByID(ref) + if entry.T > series.lastTs { + series.lastTs = entry.T + } + } + db.walReplayFloatHistogramsPool.Put(v) + default: + panic(fmt.Errorf("unexpected decoded type: %T", d)) + } + } + + if v := nonExistentSeriesRefs.Load(); v > 0 { + db.logger.Warn("found sample referencing non-existing series", "skipped_series", v) + } + + db.nextRef.Store(uint64(lastRef)) + + select { + case err := <-errCh: + return err + default: + if r.Err() != nil { + return fmt.Errorf("read records: %w", r.Err()) + } + return nil + } +} + +func (db *DB) run() { + defer close(db.donec) + +Loop: + for { + select { + case <-db.stopc: + break Loop + case <-time.After(db.opts.TruncateFrequency): + // The timestamp ts is used to determine which series are not receiving + // samples and may be deleted from the WAL. Their most recent append + // timestamp is compared to ts, and if that timestamp is older then ts, + // they are considered inactive and may be deleted. + // + // Subtracting a duration from ts will add a buffer for when series are + // considered inactive and safe for deletion. + ts := max(db.rs.LowestSentTimestamp()-db.opts.MinWALTime, 0) + + // Network issues can prevent the result of getRemoteWriteTimestamp from + // changing. We don't want data in the WAL to grow forever, so we set a cap + // on the maximum age data can be. If our ts is older than this cutoff point, + // we'll shift it forward to start deleting very stale data. + if maxTS := timestamp.FromTime(time.Now()) - db.opts.MaxWALTime; ts < maxTS { + ts = maxTS + } + + db.logger.Debug("truncating the WAL", "ts", ts) + if err := db.truncate(ts); err != nil { + db.logger.Warn("failed to truncate WAL", "err", err) + } + } + } +} + +// keepSeriesInWALCheckpointFn returns a function that is used to determine whether a series record should be kept in the checkpoint. +// last is the last WAL segment that was considered for checkpointing. +// NOTE: the agent implementation here is different from the Prometheus implementation, in that it uses WAL segment numbers instead of timestamps. +func (db *DB) keepSeriesInWALCheckpointFn(last int) func(id chunks.HeadSeriesRef) bool { + return func(id chunks.HeadSeriesRef) bool { + // Keep the record if the series exists in the db. + if db.series.GetByID(id) != nil { + return true + } + + // Keep the record if the series was recently deleted. + seg, ok := db.deleted[id] + return ok && seg > last + } +} + +func (db *DB) truncate(mint int64) error { + db.logger.Info("series GC started") + db.mtx.RLock() + defer db.mtx.RUnlock() + + start := time.Now() + + db.gc(mint) + db.logger.Info("series GC completed", "duration", time.Since(start)) + + first, last, err := wlog.Segments(db.wal.Dir()) + if err != nil { + return fmt.Errorf("get segment range: %w", err) + } + + // Start a new segment so low ingestion volume instances don't have more WAL + // than needed. + if _, err := db.wal.NextSegment(); err != nil { + return fmt.Errorf("next segment: %w", err) + } + + last-- // Never consider most recent segment for checkpoint + if last < 0 { + return nil // no segments yet + } + + // The lower two-thirds of segments should contain mostly obsolete samples. + // If we have less than two segments, it's not worth checkpointing yet. + last = first + (last-first)*2/3 + if last <= first { + return nil + } + + db.metrics.checkpointCreationTotal.Inc() + + if _, err = wlog.Checkpoint(db.logger, db.wal, first, last, db.keepSeriesInWALCheckpointFn(last), mint); err != nil { + db.metrics.checkpointCreationFail.Inc() + var cerr *wlog.CorruptionErr + if errors.As(err, &cerr) { + db.metrics.walCorruptionsTotal.Inc() + } + return fmt.Errorf("create checkpoint: %w", err) + } + if err := db.wal.Truncate(last + 1); err != nil { + // If truncating fails, we'll just try it again at the next checkpoint. + // Leftover segments will still just be ignored in the future if there's a + // checkpoint that supersedes them. + db.logger.Error("truncating segments failed", "err", err) + } + + // The checkpoint is written and segments before it are truncated, so we + // no longer need to track deleted series that were being kept around. + for ref, segment := range db.deleted { + if segment <= last { + delete(db.deleted, ref) + } + } + db.metrics.checkpointDeleteTotal.Inc() + db.metrics.numWALSeriesPendingDeletion.Set(float64(len(db.deleted))) + + if err := wlog.DeleteCheckpoints(db.wal.Dir(), last); err != nil { + // Leftover old checkpoints do not cause problems down the line beyond + // occupying disk space. They will just be ignored since a newer checkpoint + // exists. + db.logger.Error("delete old checkpoints", "err", err) + db.metrics.checkpointDeleteFail.Inc() + } + + db.metrics.walTruncateDuration.Observe(time.Since(start).Seconds()) + + db.logger.Info("WAL checkpoint complete", "first", first, "last", last, "duration", time.Since(start)) + return nil +} + +// gc marks ref IDs that have not received a sample since mint as deleted in +// s.deleted, along with the segment where they originally got deleted. +func (db *DB) gc(mint int64) { + deleted := db.series.GC(mint) + db.metrics.numActiveSeries.Sub(float64(len(deleted))) + + _, last, _ := wlog.Segments(db.wal.Dir()) + + // We want to keep series records for any newly deleted series + // until we've passed the last recorded segment. This prevents + // the WAL having samples for series records that no longer exist. + for ref := range deleted { + db.deleted[ref] = last + } + + db.metrics.numWALSeriesPendingDeletion.Set(float64(len(db.deleted))) +} + +// StartTime implements the Storage interface. +func (*DB) StartTime() (int64, error) { + return int64(model.Latest), nil +} + +// Querier implements the Storage interface. +func (*DB) Querier(int64, int64) (storage.Querier, error) { + return nil, ErrUnsupported +} + +// ChunkQuerier implements the Storage interface. +func (*DB) ChunkQuerier(int64, int64) (storage.ChunkQuerier, error) { + return nil, ErrUnsupported +} + +// ExemplarQuerier implements the Storage interface. +func (*DB) ExemplarQuerier(context.Context) (storage.ExemplarQuerier, error) { + return nil, ErrUnsupported +} + +// Appender implements storage.Storage. +func (db *DB) Appender(context.Context) storage.Appender { + return db.appenderPool.Get().(storage.Appender) +} + +// Close implements the Storage interface. +func (db *DB) Close() error { + db.mtx.Lock() + defer db.mtx.Unlock() + + close(db.stopc) + <-db.donec + + db.metrics.Unregister() + + return tsdb_errors.NewMulti(db.locker.Release(), db.wal.Close()).Err() +} + +type appender struct { + *DB + hints *storage.AppendOptions + + pendingSeries []record.RefSeries + pendingSamples []record.RefSample + pendingHistograms []record.RefHistogramSample + pendingFloatHistograms []record.RefFloatHistogramSample + pendingExamplars []record.RefExemplar + + // Pointers to the series referenced by each element of pendingSamples. + // Series lock is not held on elements. + sampleSeries []*memSeries + + // Pointers to the series referenced by each element of pendingHistograms. + // Series lock is not held on elements. + histogramSeries []*memSeries + + // Pointers to the series referenced by each element of pendingFloatHistograms. + // Series lock is not held on elements. + floatHistogramSeries []*memSeries +} + +func (a *appender) SetOptions(opts *storage.AppendOptions) { + a.hints = opts +} + +func (a *appender) Append(ref storage.SeriesRef, l labels.Labels, t int64, v float64) (storage.SeriesRef, error) { + // series references and chunk references are identical for agent mode. + headRef := chunks.HeadSeriesRef(ref) + + series := a.series.GetByID(headRef) + if series == nil { + // Ensure no empty or duplicate labels have gotten through. This mirrors the + // equivalent validation code in the TSDB's headAppender. + l = l.WithoutEmpty() + if l.IsEmpty() { + return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + } + + if lbl, dup := l.HasDuplicateLabelNames(); dup { + return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) + } + + var created bool + series, created = a.getOrCreate(l) + if created { + a.pendingSeries = append(a.pendingSeries, record.RefSeries{ + Ref: series.ref, + Labels: l, + }) + + a.metrics.numActiveSeries.Inc() + } + } + + series.Lock() + defer series.Unlock() + + if t <= a.minValidTime(series.lastTs) { + a.metrics.totalOutOfOrderSamples.Inc() + return 0, storage.ErrOutOfOrderSample + } + + // NOTE: always modify pendingSamples and sampleSeries together. + a.pendingSamples = append(a.pendingSamples, record.RefSample{ + Ref: series.ref, + T: t, + V: v, + }) + a.sampleSeries = append(a.sampleSeries, series) + + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + return storage.SeriesRef(series.ref), nil +} + +func (a *appender) getOrCreate(l labels.Labels) (series *memSeries, created bool) { + hash := l.Hash() + + series = a.series.GetByHash(hash, l) + if series != nil { + return series, false + } + + ref := chunks.HeadSeriesRef(a.nextRef.Inc()) + series = &memSeries{ref: ref, lset: l, lastTs: math.MinInt64} + a.series.Set(hash, series) + return series, true +} + +func (a *appender) AppendExemplar(ref storage.SeriesRef, _ labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { + // Series references and chunk references are identical for agent mode. + headRef := chunks.HeadSeriesRef(ref) + + s := a.series.GetByID(headRef) + if s == nil { + return 0, fmt.Errorf("unknown series ref when trying to add exemplar: %d", ref) + } + + // Ensure no empty labels have gotten through. + e.Labels = e.Labels.WithoutEmpty() + + if lbl, dup := e.Labels.HasDuplicateLabelNames(); dup { + return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidExemplar) + } + + // Exemplar label length does not include chars involved in text rendering such as quotes + // equals sign, or commas. See definition of const ExemplarMaxLabelLength. + labelSetLen := 0 + err := e.Labels.Validate(func(l labels.Label) error { + labelSetLen += utf8.RuneCountInString(l.Name) + labelSetLen += utf8.RuneCountInString(l.Value) + + if labelSetLen > exemplar.ExemplarMaxLabelSetLength { + return storage.ErrExemplarLabelLength + } + return nil + }) + if err != nil { + return 0, err + } + + // Check for duplicate vs last stored exemplar for this series, and discard those. + // Otherwise, record the current exemplar as the latest. + // Prometheus' TSDB returns 0 when encountering duplicates, so we do the same here. + prevExemplar := a.series.GetLatestExemplar(s.ref) + if prevExemplar != nil && prevExemplar.Equals(e) { + // Duplicate, don't return an error but don't accept the exemplar. + return 0, nil + } + a.series.SetLatestExemplar(s.ref, &e) + + a.pendingExamplars = append(a.pendingExamplars, record.RefExemplar{ + Ref: s.ref, + T: e.Ts, + V: e.Value, + Labels: e.Labels, + }) + + a.metrics.totalAppendedExemplars.Inc() + return storage.SeriesRef(s.ref), nil +} + +func (a *appender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + if h != nil { + if err := h.Validate(); err != nil { + return 0, err + } + } + + if fh != nil { + if err := fh.Validate(); err != nil { + return 0, err + } + } + + // series references and chunk references are identical for agent mode. + headRef := chunks.HeadSeriesRef(ref) + + series := a.series.GetByID(headRef) + if series == nil { + // Ensure no empty or duplicate labels have gotten through. This mirrors the + // equivalent validation code in the TSDB's headAppender. + l = l.WithoutEmpty() + if l.IsEmpty() { + return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + } + + if lbl, dup := l.HasDuplicateLabelNames(); dup { + return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) + } + + var created bool + series, created = a.getOrCreate(l) + if created { + a.pendingSeries = append(a.pendingSeries, record.RefSeries{ + Ref: series.ref, + Labels: l, + }) + + a.metrics.numActiveSeries.Inc() + } + } + + series.Lock() + defer series.Unlock() + + if t <= a.minValidTime(series.lastTs) { + a.metrics.totalOutOfOrderSamples.Inc() + return 0, storage.ErrOutOfOrderSample + } + + switch { + case h != nil: + // NOTE: always modify pendingHistograms and histogramSeries together + a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{ + Ref: series.ref, + T: t, + H: h, + }) + a.histogramSeries = append(a.histogramSeries, series) + case fh != nil: + // NOTE: always modify pendingFloatHistograms and floatHistogramSeries together + a.pendingFloatHistograms = append(a.pendingFloatHistograms, record.RefFloatHistogramSample{ + Ref: series.ref, + T: t, + FH: fh, + }) + a.floatHistogramSeries = append(a.floatHistogramSeries, series) + } + + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + return storage.SeriesRef(series.ref), nil +} + +func (*appender) UpdateMetadata(storage.SeriesRef, labels.Labels, metadata.Metadata) (storage.SeriesRef, error) { + // TODO: Wire metadata in the Agent's appender. + return 0, nil +} + +func (a *appender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { + if h != nil { + if err := h.Validate(); err != nil { + return 0, err + } + } + if fh != nil { + if err := fh.Validate(); err != nil { + return 0, err + } + } + if st >= t { + return 0, storage.ErrSTNewerThanSample + } + + series := a.series.GetByID(chunks.HeadSeriesRef(ref)) + if series == nil { + // Ensure no empty labels have gotten through. + l = l.WithoutEmpty() + if l.IsEmpty() { + return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + } + + if lbl, dup := l.HasDuplicateLabelNames(); dup { + return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) + } + + var created bool + series, created = a.getOrCreate(l) + if created { + a.pendingSeries = append(a.pendingSeries, record.RefSeries{ + Ref: series.ref, + Labels: l, + }) + a.metrics.numActiveSeries.Inc() + } + } + + series.Lock() + defer series.Unlock() + + if st <= a.minValidTime(series.lastTs) { + return 0, storage.ErrOutOfOrderST + } + + if st <= series.lastTs { + // discard the sample if it's out of order. + return 0, storage.ErrOutOfOrderST + } + series.lastTs = st + + switch { + case h != nil: + zeroHistogram := &histogram.Histogram{} + a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{ + Ref: series.ref, + T: st, + H: zeroHistogram, + }) + a.histogramSeries = append(a.histogramSeries, series) + case fh != nil: + a.pendingFloatHistograms = append(a.pendingFloatHistograms, record.RefFloatHistogramSample{ + Ref: series.ref, + T: st, + FH: &histogram.FloatHistogram{}, + }) + a.floatHistogramSeries = append(a.floatHistogramSeries, series) + } + + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + return storage.SeriesRef(series.ref), nil +} + +func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64) (storage.SeriesRef, error) { + if st >= t { + return 0, storage.ErrSTNewerThanSample + } + + series := a.series.GetByID(chunks.HeadSeriesRef(ref)) + if series == nil { + l = l.WithoutEmpty() + if l.IsEmpty() { + return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + } + + if lbl, dup := l.HasDuplicateLabelNames(); dup { + return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) + } + + newSeries, created := a.getOrCreate(l) + if created { + a.pendingSeries = append(a.pendingSeries, record.RefSeries{ + Ref: newSeries.ref, + Labels: l, + }) + a.metrics.numActiveSeries.Inc() + } + + series = newSeries + } + + series.Lock() + defer series.Unlock() + + if t <= a.minValidTime(series.lastTs) { + a.metrics.totalOutOfOrderSamples.Inc() + return 0, storage.ErrOutOfOrderSample + } + + if st <= series.lastTs { + // discard the sample if it's out of order. + return 0, storage.ErrOutOfOrderST + } + series.lastTs = st + + // NOTE: always modify pendingSamples and sampleSeries together. + a.pendingSamples = append(a.pendingSamples, record.RefSample{ + Ref: series.ref, + T: st, + V: 0, + }) + a.sampleSeries = append(a.sampleSeries, series) + + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeFloat).Inc() + + return storage.SeriesRef(series.ref), nil +} + +// Commit submits the collected samples and purges the batch. +func (a *appender) Commit() error { + if err := a.log(); err != nil { + return err + } + + a.clearData() + a.appenderPool.Put(a) + + if a.writeNotified != nil { + a.writeNotified.Notify() + } + return nil +} + +// log logs all pending data to the WAL. +func (a *appender) log() error { + a.mtx.RLock() + defer a.mtx.RUnlock() + + var encoder record.Encoder + buf := a.bufPool.Get().([]byte) + defer func() { + a.bufPool.Put(buf) //nolint:staticcheck + }() + + if len(a.pendingSeries) > 0 { + buf = encoder.Series(a.pendingSeries, buf) + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + + if len(a.pendingSamples) > 0 { + buf = encoder.Samples(a.pendingSamples, buf) + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + + if len(a.pendingHistograms) > 0 { + var customBucketsHistograms []record.RefHistogramSample + buf, customBucketsHistograms = encoder.HistogramSamples(a.pendingHistograms, buf) + if len(buf) > 0 { + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + if len(customBucketsHistograms) > 0 { + buf = encoder.CustomBucketsHistogramSamples(customBucketsHistograms, nil) + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + } + + if len(a.pendingFloatHistograms) > 0 { + var customBucketsFloatHistograms []record.RefFloatHistogramSample + buf, customBucketsFloatHistograms = encoder.FloatHistogramSamples(a.pendingFloatHistograms, buf) + if len(buf) > 0 { + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + if len(customBucketsFloatHistograms) > 0 { + buf = encoder.CustomBucketsFloatHistogramSamples(customBucketsFloatHistograms, nil) + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + } + + if len(a.pendingExamplars) > 0 { + buf = encoder.Exemplars(a.pendingExamplars, buf) + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + + var series *memSeries + for i, s := range a.pendingSamples { + series = a.sampleSeries[i] + if !series.updateTimestamp(s.T) { + a.metrics.totalOutOfOrderSamples.Inc() + } + } + for i, s := range a.pendingHistograms { + series = a.histogramSeries[i] + if !series.updateTimestamp(s.T) { + a.metrics.totalOutOfOrderSamples.Inc() + } + } + for i, s := range a.pendingFloatHistograms { + series = a.floatHistogramSeries[i] + if !series.updateTimestamp(s.T) { + a.metrics.totalOutOfOrderSamples.Inc() + } + } + + return nil +} + +// clearData clears all pending data. +func (a *appender) clearData() { + a.pendingSeries = a.pendingSeries[:0] + a.pendingSamples = a.pendingSamples[:0] + a.pendingHistograms = a.pendingHistograms[:0] + a.pendingFloatHistograms = a.pendingFloatHistograms[:0] + a.pendingExamplars = a.pendingExamplars[:0] + a.sampleSeries = a.sampleSeries[:0] + a.histogramSeries = a.histogramSeries[:0] + a.floatHistogramSeries = a.floatHistogramSeries[:0] +} + +func (a *appender) Rollback() error { + // Series are created in-memory regardless of rollback. This means we must + // log them to the WAL, otherwise subsequent commits may reference a series + // which was never written to the WAL. + if err := a.logSeries(); err != nil { + return err + } + + a.clearData() + a.appenderPool.Put(a) + return nil +} + +// logSeries logs only pending series records to the WAL. +func (a *appender) logSeries() error { + a.mtx.RLock() + defer a.mtx.RUnlock() + + if len(a.pendingSeries) > 0 { + buf := a.bufPool.Get().([]byte) + defer func() { + a.bufPool.Put(buf) //nolint:staticcheck + }() + + var encoder record.Encoder + buf = encoder.Series(a.pendingSeries, buf) + if err := a.wal.Log(buf); err != nil { + return err + } + buf = buf[:0] + } + + return nil +} + +// minValidTime returns the minimum timestamp that a sample can have +// and is needed for preventing underflow. +func (a *appender) minValidTime(lastTs int64) int64 { + if lastTs < math.MinInt64+a.opts.OutOfOrderTimeWindow { + return math.MinInt64 + } + + return lastTs - a.opts.OutOfOrderTimeWindow +} diff --git a/tsdb/agent/db_append_v2_test.go b/tsdb/agent/db_append_v2_test.go new file mode 100644 index 0000000000..7409f79ec5 --- /dev/null +++ b/tsdb/agent/db_append_v2_test.go @@ -0,0 +1,1396 @@ +// Copyright 2021 The Prometheus Authors +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package agent + +import ( + "context" + "errors" + "fmt" + "io" + "math" + "path/filepath" + "strconv" + "testing" + "time" + + "github.com/prometheus/client_golang/prometheus" + dto "github.com/prometheus/client_model/go" + "github.com/prometheus/common/model" + "github.com/prometheus/common/promslog" + "github.com/stretchr/testify/require" + + "github.com/prometheus/prometheus/model/exemplar" + "github.com/prometheus/prometheus/model/histogram" + "github.com/prometheus/prometheus/model/labels" + "github.com/prometheus/prometheus/storage" + "github.com/prometheus/prometheus/storage/remote" + "github.com/prometheus/prometheus/tsdb" + "github.com/prometheus/prometheus/tsdb/chunks" + "github.com/prometheus/prometheus/tsdb/record" + "github.com/prometheus/prometheus/tsdb/tsdbutil" + "github.com/prometheus/prometheus/tsdb/wlog" + "github.com/prometheus/prometheus/util/testutil" +) + +func TestDB_InvalidSeries(t *testing.T) { + s := createTestAgentDB(t, nil, DefaultOptions()) + defer s.Close() + + app := s.Appender(context.Background()) + + t.Run("Samples", func(t *testing.T) { + _, err := app.Append(0, labels.Labels{}, 0, 0) + require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject empty labels") + + _, err = app.Append(0, labels.FromStrings("a", "1", "a", "2"), 0, 0) + require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject duplicate labels") + }) + + t.Run("Histograms", func(t *testing.T) { + _, err := app.AppendHistogram(0, labels.Labels{}, 0, tsdbutil.GenerateTestHistograms(1)[0], nil) + require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject empty labels") + + _, err = app.AppendHistogram(0, labels.FromStrings("a", "1", "a", "2"), 0, tsdbutil.GenerateTestHistograms(1)[0], nil) + require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject duplicate labels") + }) + + t.Run("Exemplars", func(t *testing.T) { + sRef, err := app.Append(0, labels.FromStrings("a", "1"), 0, 0) + require.NoError(t, err, "should not reject valid series") + + _, err = app.AppendExemplar(0, labels.EmptyLabels(), exemplar.Exemplar{}) + require.EqualError(t, err, "unknown series ref when trying to add exemplar: 0") + + e := exemplar.Exemplar{Labels: labels.FromStrings("a", "1", "a", "2")} + _, err = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + require.ErrorIs(t, err, tsdb.ErrInvalidExemplar, "should reject duplicate labels") + + e = exemplar.Exemplar{Labels: labels.FromStrings("a_somewhat_long_trace_id", "nYJSNtFrFTY37VR7mHzEE/LIDt7cdAQcuOzFajgmLDAdBSRHYPDzrxhMA4zz7el8naI/AoXFv9/e/G0vcETcIoNUi3OieeLfaIRQci2oa")} + _, err = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + require.ErrorIs(t, err, storage.ErrExemplarLabelLength, "should reject too long label length") + + // Inverse check + e = exemplar.Exemplar{Labels: labels.FromStrings("a", "1"), Value: 20, Ts: 10, HasTs: true} + _, err = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + require.NoError(t, err, "should not reject valid exemplars") + }) +} + +func createTestAgentDB(t testing.TB, reg prometheus.Registerer, opts *Options) *DB { + t.Helper() + + dbDir := t.TempDir() + rs := remote.NewStorage(promslog.NewNopLogger(), reg, startTime, dbDir, time.Second*30, nil, false) + t.Cleanup(func() { + require.NoError(t, rs.Close()) + }) + + db, err := Open(promslog.NewNopLogger(), reg, rs, dbDir, opts) + require.NoError(t, err) + return db +} + +func TestUnsupportedFunctions(t *testing.T) { + s := createTestAgentDB(t, nil, DefaultOptions()) + defer s.Close() + + t.Run("Querier", func(t *testing.T) { + _, err := s.Querier(0, 0) + require.Equal(t, err, ErrUnsupported) + }) + + t.Run("ChunkQuerier", func(t *testing.T) { + _, err := s.ChunkQuerier(0, 0) + require.Equal(t, err, ErrUnsupported) + }) + + t.Run("ExemplarQuerier", func(t *testing.T) { + _, err := s.ExemplarQuerier(context.TODO()) + require.Equal(t, err, ErrUnsupported) + }) +} + +func TestCommit(t *testing.T) { + const ( + numDatapoints = 1000 + numHistograms = 100 + numSeries = 8 + ) + + s := createTestAgentDB(t, nil, DefaultOptions()) + app := s.Appender(context.TODO()) + + lbls := labelsForTest(t.Name(), numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for i := range numDatapoints { + sample := chunks.GenerateSamples(0, 1) + ref, err := app.Append(0, lset, sample[0].T(), sample[0].F()) + require.NoError(t, err) + + e := exemplar.Exemplar{ + Labels: lset, + Ts: sample[0].T() + int64(i), + Value: sample[0].F(), + HasTs: true, + } + _, err = app.AppendExemplar(ref, lset, e) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + customBucketHistograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), customBucketHistograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + customBucketFloatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), nil, customBucketFloatHistograms[i]) + require.NoError(t, err) + } + } + + require.NoError(t, app.Commit()) + require.NoError(t, s.Close()) + + sr, err := wlog.NewSegmentsReader(s.wal.Dir()) + require.NoError(t, err) + defer func() { + require.NoError(t, sr.Close()) + }() + + // Read records from WAL and check for expected count of series, samples, and exemplars. + var ( + r = wlog.NewReader(sr) + dec = record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + + walSeriesCount, walSamplesCount, walExemplarsCount, walHistogramCount, walFloatHistogramCount int + ) + for r.Next() { + rec := r.Record() + switch dec.Type(rec) { + case record.Series: + var series []record.RefSeries + series, err = dec.Series(rec, series) + require.NoError(t, err) + walSeriesCount += len(series) + + case record.Samples: + var samples []record.RefSample + samples, err = dec.Samples(rec, samples) + require.NoError(t, err) + walSamplesCount += len(samples) + + case record.HistogramSamples, record.CustomBucketsHistogramSamples: + var histograms []record.RefHistogramSample + histograms, err = dec.HistogramSamples(rec, histograms) + require.NoError(t, err) + walHistogramCount += len(histograms) + + case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: + var floatHistograms []record.RefFloatHistogramSample + floatHistograms, err = dec.FloatHistogramSamples(rec, floatHistograms) + require.NoError(t, err) + walFloatHistogramCount += len(floatHistograms) + + case record.Exemplars: + var exemplars []record.RefExemplar + exemplars, err = dec.Exemplars(rec, exemplars) + require.NoError(t, err) + walExemplarsCount += len(exemplars) + + default: + } + } + + // Check that the WAL contained the same number of committed series/samples/exemplars. + require.Equal(t, numSeries*5, walSeriesCount, "unexpected number of series") + require.Equal(t, numSeries*numDatapoints, walSamplesCount, "unexpected number of samples") + require.Equal(t, numSeries*numDatapoints, walExemplarsCount, "unexpected number of exemplars") + require.Equal(t, numSeries*numHistograms*2, walHistogramCount, "unexpected number of histograms") + require.Equal(t, numSeries*numHistograms*2, walFloatHistogramCount, "unexpected number of float histograms") +} + +func TestRollback(t *testing.T) { + const ( + numDatapoints = 1000 + numHistograms = 100 + numSeries = 8 + ) + + s := createTestAgentDB(t, nil, DefaultOptions()) + app := s.Appender(context.TODO()) + + lbls := labelsForTest(t.Name(), numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for range numDatapoints { + sample := chunks.GenerateSamples(0, 1) + _, err := app.Append(0, lset, sample[0].T(), sample[0].F()) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + // Do a rollback, which should clear uncommitted data. A followup call to + // commit should persist nothing to the WAL. + require.NoError(t, app.Rollback()) + require.NoError(t, app.Commit()) + require.NoError(t, s.Close()) + + sr, err := wlog.NewSegmentsReader(s.wal.Dir()) + require.NoError(t, err) + defer func() { + require.NoError(t, sr.Close()) + }() + + // Read records from WAL and check for expected count of series and samples. + var ( + r = wlog.NewReader(sr) + dec = record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + + walSeriesCount, walSamplesCount, walHistogramCount, walFloatHistogramCount, walExemplarsCount int + ) + for r.Next() { + rec := r.Record() + switch dec.Type(rec) { + case record.Series: + var series []record.RefSeries + series, err = dec.Series(rec, series) + require.NoError(t, err) + walSeriesCount += len(series) + + case record.Samples: + var samples []record.RefSample + samples, err = dec.Samples(rec, samples) + require.NoError(t, err) + walSamplesCount += len(samples) + + case record.Exemplars: + var exemplars []record.RefExemplar + exemplars, err = dec.Exemplars(rec, exemplars) + require.NoError(t, err) + walExemplarsCount += len(exemplars) + + case record.HistogramSamples, record.CustomBucketsHistogramSamples: + var histograms []record.RefHistogramSample + histograms, err = dec.HistogramSamples(rec, histograms) + require.NoError(t, err) + walHistogramCount += len(histograms) + + case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: + var floatHistograms []record.RefFloatHistogramSample + floatHistograms, err = dec.FloatHistogramSamples(rec, floatHistograms) + require.NoError(t, err) + walFloatHistogramCount += len(floatHistograms) + + default: + } + } + + // Check that only series get stored after calling Rollback. + require.Equal(t, numSeries*5, walSeriesCount, "series should have been written to WAL") + require.Equal(t, 0, walSamplesCount, "samples should not have been written to WAL") + require.Equal(t, 0, walExemplarsCount, "exemplars should not have been written to WAL") + require.Equal(t, 0, walHistogramCount, "histograms should not have been written to WAL") + require.Equal(t, 0, walFloatHistogramCount, "float histograms should not have been written to WAL") +} + +func TestFullTruncateWAL(t *testing.T) { + const ( + numDatapoints = 1000 + numHistograms = 100 + numSeries = 800 + lastTs = 500 + ) + + reg := prometheus.NewRegistry() + opts := DefaultOptions() + opts.TruncateFrequency = time.Minute * 2 + + s := createTestAgentDB(t, reg, opts) + defer func() { + require.NoError(t, s.Close()) + }() + app := s.Appender(context.TODO()) + + lbls := labelsForTest(t.Name(), numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for range numDatapoints { + _, err := app.Append(0, lset, int64(lastTs), 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(lastTs), histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(lastTs), histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(lastTs), nil, floatHistograms[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, int64(lastTs), nil, floatHistograms[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Truncate WAL with mint to GC all the samples. + s.truncate(lastTs + 1) + + m := gatherFamily(t, reg, "prometheus_agent_deleted_series") + require.Equal(t, float64(numSeries*5), m.Metric[0].Gauge.GetValue(), "agent wal truncate mismatch of deleted series count") +} + +func TestPartialTruncateWAL(t *testing.T) { + const ( + numDatapoints = 1000 + numSeries = 800 + ) + + opts := DefaultOptions() + + reg := prometheus.NewRegistry() + s := createTestAgentDB(t, reg, opts) + defer func() { + require.NoError(t, s.Close()) + }() + app := s.Appender(context.TODO()) + + // Create first batch of 800 series with 1000 data-points with a fixed lastTs as 500. + var lastTs int64 = 500 + lbls := labelsForTest(t.Name()+"batch-1", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for range numDatapoints { + _, err := app.Append(0, lset, lastTs, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_histogram_batch-1", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram_batch-1", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_float_histogram_batch-1", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram_batch-1", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Create second batch of 800 series with 1000 data-points with a fixed lastTs as 600. + lastTs = 600 + lbls = labelsForTest(t.Name()+"batch-2", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for range numDatapoints { + _, err := app.Append(0, lset, lastTs, 0) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_histogram_batch-2", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram_batch-2", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_float_histogram_batch-2", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram_batch-2", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numDatapoints) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + } + + // Truncate WAL with mint to GC only the first batch of 800 series and retaining 2nd batch of 800 series. + s.truncate(lastTs - 1) + + m := gatherFamily(t, reg, "prometheus_agent_deleted_series") + require.Len(t, m.Metric, 1) + require.Equal(t, float64(numSeries*5), m.Metric[0].Gauge.GetValue(), "agent wal truncate mismatch of deleted series count") +} + +func TestWALReplay(t *testing.T) { + const ( + numDatapoints = 1000 + numHistograms = 100 + numSeries = 8 + lastTs = 500 + ) + + s := createTestAgentDB(t, nil, DefaultOptions()) + app := s.Appender(context.TODO()) + + lbls := labelsForTest(t.Name(), numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for range numDatapoints { + _, err := app.Append(0, lset, lastTs, 0) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) + + for i := range numHistograms { + _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + require.NoError(t, app.Commit()) + require.NoError(t, s.Close()) + + // Hack: s.wal.Dir() is the /wal subdirectory of the original storage path. + // We need the original directory so we can recreate the storage for replay. + storageDir := filepath.Dir(s.wal.Dir()) + + reg := prometheus.NewRegistry() + replayStorage, err := Open(s.logger, reg, nil, storageDir, s.opts) + if err != nil { + t.Fatalf("unable to create storage for the agent: %v", err) + } + defer func() { + require.NoError(t, replayStorage.Close()) + }() + + // Check if all the series are retrieved back from the WAL. + m := gatherFamily(t, reg, "prometheus_agent_active_series") + require.Equal(t, float64(numSeries*5), m.Metric[0].Gauge.GetValue(), "agent wal replay mismatch of active series count") + + // Check if lastTs of the samples retrieved from the WAL is retained. + metrics := replayStorage.series.series + for i := range metrics { + mp := metrics[i] + for _, v := range mp { + require.Equal(t, v.lastTs, int64(lastTs)) + } + } +} + +func TestLockfile(t *testing.T) { + tsdbutil.TestDirLockerUsage(t, func(t *testing.T, data string, createLock bool) (*tsdbutil.DirLocker, testutil.Closer) { + logger := promslog.NewNopLogger() + reg := prometheus.NewRegistry() + rs := remote.NewStorage(logger, reg, startTime, data, time.Second*30, nil, false) + t.Cleanup(func() { + require.NoError(t, rs.Close()) + }) + + opts := DefaultOptions() + opts.NoLockfile = !createLock + + // Create the DB. This should create lockfile and its metrics. + db, err := Open(logger, nil, rs, data, opts) + require.NoError(t, err) + + return db.locker, testutil.NewCallbackCloser(func() { + require.NoError(t, db.Close()) + }) + }) +} + +func Test_ExistingWAL_NextRef(t *testing.T) { + dbDir := t.TempDir() + rs := remote.NewStorage(promslog.NewNopLogger(), nil, startTime, dbDir, time.Second*30, nil, false) + defer func() { + require.NoError(t, rs.Close()) + }() + + db, err := Open(promslog.NewNopLogger(), nil, rs, dbDir, DefaultOptions()) + require.NoError(t, err) + + seriesCount := 10 + + // Append series + app := db.Appender(context.Background()) + for i := range seriesCount { + lset := labels.FromStrings(model.MetricNameLabel, fmt.Sprintf("series_%d", i)) + _, err := app.Append(0, lset, 0, 100) + require.NoError(t, err) + } + + histogramCount := 10 + histograms := tsdbutil.GenerateTestHistograms(histogramCount) + // Append series + for i := range histogramCount { + lset := labels.FromStrings(model.MetricNameLabel, fmt.Sprintf("histogram_%d", i)) + _, err := app.AppendHistogram(0, lset, 0, histograms[i], nil) + require.NoError(t, err) + } + require.NoError(t, app.Commit()) + + // Truncate the WAL to force creation of a new segment. + require.NoError(t, db.truncate(0)) + require.NoError(t, db.Close()) + + // Create a new storage and see what nextRef is initialized to. + db, err = Open(promslog.NewNopLogger(), nil, rs, dbDir, DefaultOptions()) + require.NoError(t, err) + defer func() { + require.NoError(t, db.Close()) + }() + + require.Equal(t, uint64(seriesCount+histogramCount), db.nextRef.Load(), "nextRef should be equal to the number of series written across the entire WAL") +} + +func Test_validateOptions(t *testing.T) { + t.Run("Apply defaults to zero values", func(t *testing.T) { + opts := validateOptions(&Options{}) + require.Equal(t, DefaultOptions(), opts) + }) + + t.Run("Defaults are already valid", func(t *testing.T) { + require.Equal(t, DefaultOptions(), validateOptions(nil)) + }) + + t.Run("MaxWALTime should not be lower than TruncateFrequency", func(t *testing.T) { + opts := validateOptions(&Options{ + MaxWALTime: int64(time.Hour / time.Millisecond), + TruncateFrequency: 2 * time.Hour, + }) + require.Equal(t, int64(2*time.Hour/time.Millisecond), opts.MaxWALTime) + }) +} + +func startTime() (int64, error) { + return time.Now().Unix() * 1000, nil +} + +// Create series for tests. +func labelsForTest(lName string, seriesCount int) [][]labels.Label { + var series [][]labels.Label + + for i := range seriesCount { + lset := []labels.Label{ + {Name: "a", Value: lName}, + {Name: "instance", Value: "localhost" + strconv.Itoa(i)}, + {Name: "job", Value: "prometheus"}, + } + series = append(series, lset) + } + + return series +} + +func gatherFamily(t *testing.T, reg prometheus.Gatherer, familyName string) *dto.MetricFamily { + t.Helper() + + families, err := reg.Gather() + require.NoError(t, err, "failed to gather metrics") + + for _, f := range families { + if f.GetName() == familyName { + return f + } + } + + t.Fatalf("could not find family %s", familyName) + + return nil +} + +func TestStorage_DuplicateExemplarsIgnored(t *testing.T) { + s := createTestAgentDB(t, nil, DefaultOptions()) + app := s.Appender(context.Background()) + defer s.Close() + + sRef, err := app.Append(0, labels.FromStrings("a", "1"), 0, 0) + require.NoError(t, err, "should not reject valid series") + + // Write a few exemplars to our appender and call Commit(). + // If the Labels, Value or Timestamp are different than the last exemplar, + // then a new one should be appended; Otherwise, it should be skipped. + e := exemplar.Exemplar{Labels: labels.FromStrings("a", "1"), Value: 20, Ts: 10, HasTs: true} + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + + e.Labels = labels.FromStrings("b", "2") + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + + e.Value = 42 + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + + e.Ts = 25 + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + + require.NoError(t, app.Commit()) + + // Read back what was written to the WAL. + var walExemplarsCount int + sr, err := wlog.NewSegmentsReader(s.wal.Dir()) + require.NoError(t, err) + defer sr.Close() + r := wlog.NewReader(sr) + + dec := record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + for r.Next() { + rec := r.Record() + if dec.Type(rec) == record.Exemplars { + var exemplars []record.RefExemplar + exemplars, err = dec.Exemplars(rec, exemplars) + require.NoError(t, err) + walExemplarsCount += len(exemplars) + } + } + + // We had 9 calls to AppendExemplar but only 4 of those should have gotten through. + require.Equal(t, 4, walExemplarsCount) +} + +func TestDBAllowOOOSamples(t *testing.T) { + const ( + numDatapoints = 5 + numHistograms = 5 + numSeries = 4 + offset = 100 + ) + + reg := prometheus.NewRegistry() + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = math.MaxInt64 + s := createTestAgentDB(t, reg, opts) + app := s.Appender(context.TODO()) + + // Let's add some samples in the [offset, offset+numDatapoints) range. + lbls := labelsForTest(t.Name(), numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + for i := offset; i < numDatapoints+offset; i++ { + ref, err := app.Append(0, lset, int64(i), float64(i)) + require.NoError(t, err) + + e := exemplar.Exemplar{ + Labels: lset, + Ts: int64(i) * 2, + Value: float64(i), + HasTs: true, + } + _, err = app.AppendExemplar(ref, lset, e) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numHistograms) + + for i := offset; i < numDatapoints+offset; i++ { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i-offset], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) + + for i := offset; i < numDatapoints+offset; i++ { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i-offset], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) + + for i := offset; i < numDatapoints+offset; i++ { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i-offset]) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram", numSeries) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) + + for i := offset; i < numDatapoints+offset; i++ { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i-offset]) + require.NoError(t, err) + } + } + + require.NoError(t, app.Commit()) + m := gatherFamily(t, reg, "prometheus_agent_samples_appended_total") + require.Equal(t, float64(20), m.Metric[0].Counter.GetValue(), "agent wal mismatch of total appended samples") + require.Equal(t, float64(80), m.Metric[1].Counter.GetValue(), "agent wal mismatch of total appended histograms") + require.NoError(t, s.Close()) + + // Hack: s.wal.Dir() is the /wal subdirectory of the original storage path. + // We need the original directory so we can recreate the storage for replay. + storageDir := filepath.Dir(s.wal.Dir()) + + // Replay the storage so that the lastTs for each series is recorded. + reg2 := prometheus.NewRegistry() + db, err := Open(s.logger, reg2, nil, storageDir, s.opts) + if err != nil { + t.Fatalf("unable to create storage for the agent: %v", err) + } + + app = db.Appender(context.Background()) + + // Now the lastTs will have been recorded successfully. + // Let's try appending twice as many OOO samples in the [0, numDatapoints) range. + lbls = labelsForTest(t.Name()+"_histogram", numSeries*2) + for _, l := range lbls { + lset := labels.New(l...) + + for i := range numDatapoints { + ref, err := app.Append(0, lset, int64(i), float64(i)) + require.NoError(t, err) + + e := exemplar.Exemplar{ + Labels: lset, + Ts: int64(i) * 2, + Value: float64(i), + HasTs: true, + } + _, err = app.AppendExemplar(ref, lset, e) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_histogram", numSeries*2) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestHistograms(numHistograms) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_histogram", numSeries*2) + for _, l := range lbls { + lset := labels.New(l...) + + histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_float_histogram", numSeries*2) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + lbls = labelsForTest(t.Name()+"_custom_buckets_float_histogram", numSeries*2) + for _, l := range lbls { + lset := labels.New(l...) + + floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) + + for i := range numDatapoints { + _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + require.NoError(t, err) + } + } + + require.NoError(t, app.Commit()) + m = gatherFamily(t, reg2, "prometheus_agent_samples_appended_total") + require.Equal(t, float64(40), m.Metric[0].Counter.GetValue(), "agent wal mismatch of total appended samples") + require.Equal(t, float64(160), m.Metric[1].Counter.GetValue(), "agent wal mismatch of total appended histograms") + require.NoError(t, db.Close()) +} + +func TestDBOutOfOrderTimeWindow(t *testing.T) { + tc := []struct { + outOfOrderTimeWindow, firstTs, secondTs int64 + expectedError error + }{ + {0, 100, 101, nil}, + {0, 100, 100, storage.ErrOutOfOrderSample}, + {0, 100, 99, storage.ErrOutOfOrderSample}, + {100, 100, 1, nil}, + {100, 100, 0, storage.ErrOutOfOrderSample}, + } + + for _, c := range tc { + t.Run(fmt.Sprintf("outOfOrderTimeWindow=%d, firstTs=%d, secondTs=%d, expectedError=%s", c.outOfOrderTimeWindow, c.firstTs, c.secondTs, c.expectedError), func(t *testing.T) { + reg := prometheus.NewRegistry() + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = c.outOfOrderTimeWindow + s := createTestAgentDB(t, reg, opts) + app := s.Appender(context.TODO()) + + lbls := labelsForTest(t.Name()+"_histogram", 1) + lset := labels.New(lbls[0]...) + _, err := app.AppendHistogram(0, lset, c.firstTs, tsdbutil.GenerateTestHistograms(1)[0], nil) + require.NoError(t, err) + err = app.Commit() + require.NoError(t, err) + _, err = app.AppendHistogram(0, lset, c.secondTs, tsdbutil.GenerateTestHistograms(1)[0], nil) + require.ErrorIs(t, err, c.expectedError) + + lbls = labelsForTest(t.Name(), 1) + lset = labels.New(lbls[0]...) + _, err = app.Append(0, lset, c.firstTs, 0) + require.NoError(t, err) + err = app.Commit() + require.NoError(t, err) + _, err = app.Append(0, lset, c.secondTs, 0) + require.ErrorIs(t, err, c.expectedError) + + expectedAppendedSamples := float64(2) + if c.expectedError != nil { + expectedAppendedSamples = 1 + } + m := gatherFamily(t, reg, "prometheus_agent_samples_appended_total") + require.Equal(t, expectedAppendedSamples, m.Metric[0].Counter.GetValue(), "agent wal mismatch of total appended samples") + require.Equal(t, expectedAppendedSamples, m.Metric[1].Counter.GetValue(), "agent wal mismatch of total appended histograms") + require.NoError(t, s.Close()) + }) + } +} + +type walSample struct { + t int64 + f float64 + h *histogram.Histogram + lbls labels.Labels + ref storage.SeriesRef +} + +func TestDBStartTimestampSamplesIngestion(t *testing.T) { + t.Parallel() + + type appendableSample struct { + t int64 + st int64 + v float64 + lbls labels.Labels + h *histogram.Histogram + expectsError bool + } + + testHistogram := tsdbutil.GenerateTestHistograms(1)[0] + zeroHistogram := &histogram.Histogram{} + + lbls := labelsForTest(t.Name(), 1) + defLbls := labels.New(lbls[0]...) + + testCases := []struct { + name string + inputSamples []appendableSample + expectedSamples []*walSample + expectedSeriesCount int + }{ + { + name: "in order ct+normal sample/floatSamples", + inputSamples: []appendableSample{ + {t: 100, st: 1, v: 10, lbls: defLbls}, + {t: 101, st: 1, v: 10, lbls: defLbls}, + }, + expectedSamples: []*walSample{ + {t: 1, f: 0, lbls: defLbls}, + {t: 100, f: 10, lbls: defLbls}, + {t: 101, f: 10, lbls: defLbls}, + }, + }, + { + name: "ST+float && ST+histogram samples", + inputSamples: []appendableSample{ + { + t: 100, + st: 30, + v: 20, + lbls: defLbls, + }, + { + t: 300, + st: 230, + h: testHistogram, + lbls: defLbls, + }, + }, + expectedSamples: []*walSample{ + {t: 30, f: 0, lbls: defLbls}, + {t: 100, f: 20, lbls: defLbls}, + {t: 230, h: zeroHistogram, lbls: defLbls}, + {t: 300, h: testHistogram, lbls: defLbls}, + }, + expectedSeriesCount: 1, + }, + { + name: "ST+float && ST+histogram samples with error", + inputSamples: []appendableSample{ + { + // invalid ST + t: 100, + st: 100, + v: 10, + lbls: defLbls, + expectsError: true, + }, + { + // invalid ST histogram + t: 300, + st: 300, + h: testHistogram, + lbls: defLbls, + expectsError: true, + }, + }, + expectedSamples: []*walSample{ + {t: 100, f: 10, lbls: defLbls}, + {t: 300, h: testHistogram, lbls: defLbls}, + }, + expectedSeriesCount: 0, + }, + { + name: "In order ct+normal sample/histogram", + inputSamples: []appendableSample{ + {t: 100, h: testHistogram, st: 1, lbls: defLbls}, + {t: 101, h: testHistogram, st: 1, lbls: defLbls}, + }, + expectedSamples: []*walSample{ + {t: 1, h: &histogram.Histogram{}}, + {t: 100, h: testHistogram}, + {t: 101, h: &histogram.Histogram{CounterResetHint: histogram.NotCounterReset}}, + }, + }, + { + name: "ct+normal then OOO sample/float", + inputSamples: []appendableSample{ + {t: 60_000, st: 40_000, v: 10, lbls: defLbls}, + {t: 120_000, st: 40_000, v: 10, lbls: defLbls}, + {t: 180_000, st: 40_000, v: 10, lbls: defLbls}, + {t: 50_000, st: 40_000, v: 10, lbls: defLbls}, + }, + expectedSamples: []*walSample{ + {t: 40_000, f: 0, lbls: defLbls}, + {t: 50_000, f: 10, lbls: defLbls}, + {t: 60_000, f: 10, lbls: defLbls}, + {t: 120_000, f: 10, lbls: defLbls}, + {t: 180_000, f: 10, lbls: defLbls}, + }, + }, + } + + for _, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + + reg := prometheus.NewRegistry() + opts := DefaultOptions() + opts.OutOfOrderTimeWindow = 360_000 + s := createTestAgentDB(t, reg, opts) + app := s.Appender(context.TODO()) + + for _, sample := range tc.inputSamples { + // We supposed to write a Histogram to the WAL + if sample.h != nil { + _, err := app.AppendHistogramSTZeroSample(0, sample.lbls, sample.t, sample.st, zeroHistogram, nil) + if !errors.Is(err, storage.ErrOutOfOrderST) { + require.Equal(t, sample.expectsError, err != nil, "expected error: %v, got: %v", sample.expectsError, err) + } + + _, err = app.AppendHistogram(0, sample.lbls, sample.t, sample.h, nil) + require.NoError(t, err) + } else { + // We supposed to write a float sample to the WAL + _, err := app.AppendSTZeroSample(0, sample.lbls, sample.t, sample.st) + if !errors.Is(err, storage.ErrOutOfOrderST) { + require.Equal(t, sample.expectsError, err != nil, "expected error: %v, got: %v", sample.expectsError, err) + } + + _, err = app.Append(0, sample.lbls, sample.t, sample.v) + require.NoError(t, err) + } + } + + require.NoError(t, app.Commit()) + // Close the DB to ensure all data is flushed to the WAL + require.NoError(t, s.Close()) + + // Check that we dont have any OOO samples in the WAL by checking metrics + families, err := reg.Gather() + require.NoError(t, err, "failed to gather metrics") + for _, f := range families { + if f.GetName() == "prometheus_agent_out_of_order_samples_total" { + t.Fatalf("unexpected metric %s", f.GetName()) + } + } + + outputSamples := readWALSamples(t, s.wal.Dir()) + + require.Len(t, outputSamples, len(tc.expectedSamples), "Expected %d samples", len(tc.expectedSamples)) + + for i, expectedSample := range tc.expectedSamples { + for _, sample := range outputSamples { + if sample.t == expectedSample.t && sample.lbls.String() == expectedSample.lbls.String() { + if expectedSample.h != nil { + require.Equal(t, expectedSample.h, sample.h, "histogram value mismatch (sample index %d)", i) + } else { + require.Equal(t, expectedSample.f, sample.f, "value mismatch (sample index %d)", i) + } + } + } + } + }) + } +} + +func readWALSamples(t *testing.T, walDir string) []*walSample { + t.Helper() + sr, err := wlog.NewSegmentsReader(walDir) + require.NoError(t, err) + defer func(sr io.ReadCloser) { + err := sr.Close() + require.NoError(t, err) + }(sr) + + r := wlog.NewReader(sr) + dec := record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) + + var ( + samples []record.RefSample + histograms []record.RefHistogramSample + + lastSeries record.RefSeries + outputSamples = make([]*walSample, 0) + ) + + for r.Next() { + rec := r.Record() + switch dec.Type(rec) { + case record.Series: + series, err := dec.Series(rec, nil) + require.NoError(t, err) + lastSeries = series[0] + case record.Samples: + samples, err = dec.Samples(rec, samples[:0]) + require.NoError(t, err) + for _, s := range samples { + outputSamples = append(outputSamples, &walSample{ + t: s.T, + f: s.V, + lbls: lastSeries.Labels.Copy(), + ref: storage.SeriesRef(lastSeries.Ref), + }) + } + case record.HistogramSamples: + histograms, err = dec.HistogramSamples(rec, histograms[:0]) + require.NoError(t, err) + for _, h := range histograms { + outputSamples = append(outputSamples, &walSample{ + t: h.T, + h: h.H, + lbls: lastSeries.Labels.Copy(), + ref: storage.SeriesRef(lastSeries.Ref), + }) + } + } + } + + return outputSamples +} + +func BenchmarkCreateSeries(b *testing.B) { + s := createTestAgentDB(b, nil, DefaultOptions()) + defer s.Close() + + app := s.Appender(context.Background()).(*appender) + lbls := make([]labels.Labels, b.N) + + for i, l := range labelsForTest("benchmark", b.N) { + lbls[i] = labels.New(l...) + } + + b.ResetTimer() + + for _, l := range lbls { + app.getOrCreate(l) + } +} From cd98ded6ea2b821c8b14c53372e73a8f3f42b1d2 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 12 Dec 2025 22:43:51 +0000 Subject: [PATCH 131/439] test: add regression test against remote write handler bad response stats Signed-off-by: bwplotka --- storage/remote/client.go | 3 ++ storage/remote/write_handler.go | 4 ++ storage/remote/write_handler_test.go | 72 ++++++++++++++++++++++++++++ 3 files changed, 79 insertions(+) diff --git a/storage/remote/client.go b/storage/remote/client.go index c535ea3425..0f2b5ddca6 100644 --- a/storage/remote/client.go +++ b/storage/remote/client.go @@ -301,6 +301,9 @@ func (c *Client) Store(ctx context.Context, req []byte, attempt int) (WriteRespo _ = httpResp.Body.Close() }() + // NOTE(bwplotka): Only PRW2 spec defines response HTTP headers. However, spec does not block + // PRW1 from sending them too for reliability. Support this case. + // // TODO(bwplotka): Pass logger and emit debug on error? // Parsing error means there were some response header values we can't parse, // we can continue handling. diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go index f8296b4a80..2bc65e8286 100644 --- a/storage/remote/write_handler.go +++ b/storage/remote/write_handler.go @@ -96,6 +96,10 @@ func isHistogramValidationError(err error) bool { } // Store implements remoteapi.writeStorage interface. +// TODO(bwplotka): Improve remoteapi.Store API. Right now it's confusing if PRWv1 flows should use WriteResponse or not. +// If it's not filled, it will be "confirmed zero" which caused partial error reporting on client side in the past. +// Temporary fix was done to only care about WriteResponse stats for PRW2 (see https://github.com/prometheus/client_golang/pull/1927 +// but better approach would be to only confirm if explicit stats were injected. func (h *writeHandler) Store(r *http.Request, msgType remoteapi.WriteMessageType) (*remoteapi.WriteResponse, error) { // Store receives request with decompressed content in body. body, err := io.ReadAll(r.Body) diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go index f1c064c64d..2610142db9 100644 --- a/storage/remote/write_handler_test.go +++ b/storage/remote/write_handler_test.go @@ -1510,3 +1510,75 @@ func TestHistogramsReduction(t *testing.T) { }) } } + +// Regression test for https://github.com/prometheus/prometheus/issues/17659 +func TestRemoteWriteHandler_ResponseStats(t *testing.T) { + payloadV1, _, _, err := buildWriteRequest(nil, writeRequestFixture.Timeseries, nil, nil, nil, nil, "snappy") + require.NoError(t, err) + payloadV2, _, _, err := buildV2WriteRequest(nil, writeV2RequestFixture.Timeseries, writeV2RequestFixture.Symbols, nil, nil, nil, "snappy") + require.NoError(t, err) + + for _, tt := range []struct { + msgType remoteapi.WriteMessageType + forceInjectHeaders bool + expectHeaders bool + }{ + { + msgType: remoteapi.WriteV1MessageType, + }, + { + msgType: remoteapi.WriteV1MessageType, + forceInjectHeaders: true, + expectHeaders: true, + }, + { + msgType: remoteapi.WriteV2MessageType, + expectHeaders: true, + }, + } { + t.Run(fmt.Sprintf("msg=%v/force-inject-headers=%v", tt.msgType, tt.forceInjectHeaders), func(t *testing.T) { + // Setup server side. + appendable := &mockAppendable{} + handler := NewWriteHandler( + promslog.NewNopLogger(), + nil, + appendable, + []remoteapi.WriteMessageType{remoteapi.WriteV1MessageType, remoteapi.WriteV2MessageType}, + false, + false, + false, + ) + + if tt.forceInjectHeaders { + base := handler + handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + base.ServeHTTP(w, r) + + // Inject response header. This simulates PRWv1 server that uses PRWv2 response headers + // for confirmation of samples. This is not against spec and we support it. + w.Header().Set(rw20WrittenSamplesHeader, fmt.Sprintf("%d", len(appendable.samples))) + }) + } + + srv := httptest.NewServer(handler) + + // Send message and do the parse response flow. + c := &Client{Client: srv.Client(), urlString: srv.URL, timeout: 5 * time.Minute, writeProtoMsg: tt.msgType} + + payload := payloadV2 + if tt.msgType == remoteapi.WriteV1MessageType { + payload = payloadV1 + } + stats, err := c.Store(t.Context(), payload, 0) + require.NoError(t, err) + + fmt.Println(stats) + if tt.expectHeaders { + require.True(t, stats.Confirmed) + require.Equal(t, len(appendable.samples), stats.Samples) + } else { + require.False(t, stats.Confirmed) + } + }) + } +} From ad367b504b0b088a197d759aa7f01caeeeb6ee50 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 12 Dec 2025 11:12:28 +0000 Subject: [PATCH 132/439] refactor(tsdb/agent)[PART3]: add AppenderV2 support to agent Signed-off-by: bwplotka --- cmd/prometheus/main.go | 6 +- storage/interface.go | 4 +- storage/interface_append.go | 4 +- tsdb/agent/db.go | 127 ++- tsdb/agent/db_append_v2.go | 1342 ++++--------------------------- tsdb/agent/db_append_v2_test.go | 590 +++++--------- tsdb/agent/db_test.go | 92 +-- tsdb/head_append_v2.go | 7 +- 8 files changed, 455 insertions(+), 1717 deletions(-) diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go index 53379dc940..e903b87beb 100644 --- a/cmd/prometheus/main.go +++ b/cmd/prometheus/main.go @@ -265,6 +265,7 @@ func (c *flagConfig) setFeatureListOptions(logger *slog.Logger) error { case "created-timestamp-zero-ingestion": c.scrape.EnableStartTimestampZeroIngestion = true c.web.STZeroIngestionEnabled = true + c.agent.EnableSTAsZeroSample = true // Change relevant global variables. Hacky, but it's hard to pass a new option or default to unmarshallers. config.DefaultConfig.GlobalConfig.ScrapeProtocols = config.DefaultProtoFirstScrapeProtocols config.DefaultGlobalConfig.ScrapeProtocols = config.DefaultProtoFirstScrapeProtocols @@ -1409,6 +1410,7 @@ func main() { "MinWALTime", cfg.agent.MinWALTime, "MaxWALTime", cfg.agent.MaxWALTime, "OutOfOrderTimeWindow", cfg.agent.OutOfOrderTimeWindow, + "EnableSTAsZeroSample", cfg.agent.EnableSTAsZeroSample, ) localStorage.Set(db, 0) @@ -1947,7 +1949,8 @@ type agentOptions struct { TruncateFrequency model.Duration MinWALTime, MaxWALTime model.Duration NoLockfile bool - OutOfOrderTimeWindow int64 + OutOfOrderTimeWindow int64 // TODO(bwplotka): Unused option, fix it or remove. + EnableSTAsZeroSample bool } func (opts agentOptions) ToAgentOptions(outOfOrderTimeWindow int64) agent.Options { @@ -1963,6 +1966,7 @@ func (opts agentOptions) ToAgentOptions(outOfOrderTimeWindow int64) agent.Option MaxWALTime: durationToInt64Millis(time.Duration(opts.MaxWALTime)), NoLockfile: opts.NoLockfile, OutOfOrderTimeWindow: outOfOrderTimeWindow, + EnableSTAsZeroSample: opts.EnableSTAsZeroSample, } } diff --git a/storage/interface.go b/storage/interface.go index f7d7953de4..ae8bec033e 100644 --- a/storage/interface.go +++ b/storage/interface.go @@ -274,8 +274,8 @@ type AppendOptions struct { // // Operations on the Appender interface are not goroutine-safe. // -// The order of samples appended via the Appender is preserved within each -// series. I.e. samples are not reordered per timestamp, or by float/histogram +// The order of samples appended via the Appender is preserved within each series. +// I.e. timestamp order within batch is not validated, samples are not reordered per timestamp or by float/histogram // type. // // WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026). diff --git a/storage/interface_append.go b/storage/interface_append.go index 880e57f194..cc7045dbd5 100644 --- a/storage/interface_append.go +++ b/storage/interface_append.go @@ -103,8 +103,8 @@ var _ error = &AppendPartialError{} // // Operations on the AppenderV2 interface are not goroutine-safe. // -// The order of samples appended via the AppenderV2 is preserved within each -// series. I.e. samples are not reordered per timestamp, or by float/histogram +// The order of samples appended via the AppenderV2 is preserved within each series. +// I.e. timestamp order within batch is not validated, samples are not reordered per timestamp or by float/histogram // type. type AppenderV2 interface { AppenderTransaction diff --git a/tsdb/agent/db.go b/tsdb/agent/db.go index 5c9774cd58..7b3e74f51a 100644 --- a/tsdb/agent/db.go +++ b/tsdb/agent/db.go @@ -84,6 +84,15 @@ type Options struct { // OutOfOrderTimeWindow specifies how much out of order is allowed, if any. OutOfOrderTimeWindow int64 + + // EnableSTAsZeroSample represents 'created-timestamp-zero-ingestion' feature flag. + // If true, ST, if non-empty and earlier than sample timestamp, will be stored + // as a zero sample before the actual sample. + // + // The zero sample is best-effort, only debug log on failure is emitted. + // NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 + // is implemented. + EnableSTAsZeroSample bool } // DefaultOptions used for the WAL storage. They are reasonable for setups using @@ -233,8 +242,9 @@ type DB struct { wal *wlog.WL locker *tsdbutil.DirLocker - appenderPool sync.Pool - bufPool sync.Pool + appenderPool sync.Pool + appenderV2Pool sync.Pool + bufPool sync.Pool // These pools are only used during WAL replay and are reset at the end. // NOTE: Adjust resetWALReplayResources() upon changes to the pools. @@ -303,12 +313,26 @@ func Open(l *slog.Logger, reg prometheus.Registerer, rs *remote.Storage, dir str db.appenderPool.New = func() any { return &appender{ - DB: db, - pendingSeries: make([]record.RefSeries, 0, 100), - pendingSamples: make([]record.RefSample, 0, 100), - pendingHistograms: make([]record.RefHistogramSample, 0, 100), - pendingFloatHistograms: make([]record.RefFloatHistogramSample, 0, 100), - pendingExamplars: make([]record.RefExemplar, 0, 10), + appenderBase: appenderBase{ + DB: db, + pendingSeries: make([]record.RefSeries, 0, 100), + pendingSamples: make([]record.RefSample, 0, 100), + pendingHistograms: make([]record.RefHistogramSample, 0, 100), + pendingFloatHistograms: make([]record.RefFloatHistogramSample, 0, 100), + pendingExamplars: make([]record.RefExemplar, 0, 10), + }, + } + } + db.appenderV2Pool.New = func() any { + return &appenderV2{ + appenderBase: appenderBase{ + DB: db, + pendingSeries: make([]record.RefSeries, 0, 100), + pendingSamples: make([]record.RefSample, 0, 100), + pendingHistograms: make([]record.RefHistogramSample, 0, 100), + pendingFloatHistograms: make([]record.RefFloatHistogramSample, 0, 100), + pendingExamplars: make([]record.RefExemplar, 0, 10), + }, } } @@ -777,9 +801,8 @@ func (db *DB) Close() error { return tsdb_errors.NewMulti(db.locker.Release(), db.wal.Close()).Err() } -type appender struct { +type appenderBase struct { *DB - hints *storage.AppendOptions pendingSeries []record.RefSeries pendingSamples []record.RefSample @@ -800,6 +823,12 @@ type appender struct { floatHistogramSeries []*memSeries } +type appender struct { + appenderBase + + hints *storage.AppendOptions +} + func (a *appender) SetOptions(opts *storage.AppendOptions) { a.hints = opts } @@ -853,7 +882,7 @@ func (a *appender) Append(ref storage.SeriesRef, l labels.Labels, t int64, v flo return storage.SeriesRef(series.ref), nil } -func (a *appender) getOrCreate(l labels.Labels) (series *memSeries, created bool) { +func (a *appenderBase) getOrCreate(l labels.Labels) (series *memSeries, created bool) { hash := l.Hash() series = a.series.GetByHash(hash, l) @@ -879,47 +908,53 @@ func (a *appender) AppendExemplar(ref storage.SeriesRef, _ labels.Labels, e exem // Ensure no empty labels have gotten through. e.Labels = e.Labels.WithoutEmpty() - if lbl, dup := e.Labels.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidExemplar) - } - - // Exemplar label length does not include chars involved in text rendering such as quotes - // equals sign, or commas. See definition of const ExemplarMaxLabelLength. - labelSetLen := 0 - err := e.Labels.Validate(func(l labels.Label) error { - labelSetLen += utf8.RuneCountInString(l.Name) - labelSetLen += utf8.RuneCountInString(l.Value) - - if labelSetLen > exemplar.ExemplarMaxLabelSetLength { - return storage.ErrExemplarLabelLength + if err := a.validateExemplar(s.ref, e); err != nil { + if errors.Is(err, storage.ErrDuplicateExemplar) { + // Duplicate, don't return an error but don't accept the exemplar. + return 0, nil } - return nil - }) - if err != nil { return 0, err } - // Check for duplicate vs last stored exemplar for this series, and discard those. - // Otherwise, record the current exemplar as the latest. - // Prometheus' TSDB returns 0 when encountering duplicates, so we do the same here. - prevExemplar := a.series.GetLatestExemplar(s.ref) - if prevExemplar != nil && prevExemplar.Equals(e) { - // Duplicate, don't return an error but don't accept the exemplar. - return 0, nil - } a.series.SetLatestExemplar(s.ref, &e) - a.pendingExamplars = append(a.pendingExamplars, record.RefExemplar{ Ref: s.ref, T: e.Ts, V: e.Value, Labels: e.Labels, }) - a.metrics.totalAppendedExemplars.Inc() return storage.SeriesRef(s.ref), nil } +func (a *appenderBase) validateExemplar(ref chunks.HeadSeriesRef, e exemplar.Exemplar) error { + if lbl, dup := e.Labels.HasDuplicateLabelNames(); dup { + return fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidExemplar) + } + + // Exemplar label length does not include chars involved in text rendering such as quotes + // equals sign, or commas. See definition of const ExemplarMaxLabelLength. + labelSetLen := 0 + if err := e.Labels.Validate(func(l labels.Label) error { + labelSetLen += utf8.RuneCountInString(l.Name) + labelSetLen += utf8.RuneCountInString(l.Value) + if labelSetLen > exemplar.ExemplarMaxLabelSetLength { + return storage.ErrExemplarLabelLength + } + return nil + }); err != nil { + return err + } + // Check for duplicate vs last stored exemplar for this series, and discard those. + // Otherwise, record the current exemplar as the latest. + // Prometheus' TSDB returns 0 when encountering duplicates, so we do the same here. + prevExemplar := a.series.GetLatestExemplar(ref) + if prevExemplar != nil && prevExemplar.Equals(e) { + return storage.ErrDuplicateExemplar + } + return nil +} + func (a *appender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { if h != nil { if err := h.Validate(); err != nil { @@ -1046,6 +1081,9 @@ func (a *appender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.L // discard the sample if it's out of order. return 0, storage.ErrOutOfOrderST } + // NOTE(bwplotka): This is a bug, as we "commit" pending sample TS as the WAL last TS. It was likely done + // to satisfy incorrect TestDBStartTimestampSamplesIngestion test. We are leaving it as-is given the planned removal + // of AppenderV1 as per https://github.com/prometheus/prometheus/issues/17632. series.lastTs = st switch { @@ -1110,6 +1148,9 @@ func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, // discard the sample if it's out of order. return 0, storage.ErrOutOfOrderST } + // NOTE(bwplotka): This is a bug, as we "commit" pending sample TS as the WAL last TS. It was likely done + // to satisfy incorrect TestDBStartTimestampSamplesIngestion test. We are leaving it as-is given the planned removal + // of AppenderV1 as per https://github.com/prometheus/prometheus/issues/17632. series.lastTs = st // NOTE: always modify pendingSamples and sampleSeries together. @@ -1126,7 +1167,7 @@ func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, } // Commit submits the collected samples and purges the batch. -func (a *appender) Commit() error { +func (a *appenderBase) Commit() error { if err := a.log(); err != nil { return err } @@ -1141,7 +1182,7 @@ func (a *appender) Commit() error { } // log logs all pending data to the WAL. -func (a *appender) log() error { +func (a *appenderBase) log() error { a.mtx.RLock() defer a.mtx.RUnlock() @@ -1235,7 +1276,7 @@ func (a *appender) log() error { } // clearData clears all pending data. -func (a *appender) clearData() { +func (a *appenderBase) clearData() { a.pendingSeries = a.pendingSeries[:0] a.pendingSamples = a.pendingSamples[:0] a.pendingHistograms = a.pendingHistograms[:0] @@ -1246,7 +1287,7 @@ func (a *appender) clearData() { a.floatHistogramSeries = a.floatHistogramSeries[:0] } -func (a *appender) Rollback() error { +func (a *appenderBase) Rollback() error { // Series are created in-memory regardless of rollback. This means we must // log them to the WAL, otherwise subsequent commits may reference a series // which was never written to the WAL. @@ -1260,7 +1301,7 @@ func (a *appender) Rollback() error { } // logSeries logs only pending series records to the WAL. -func (a *appender) logSeries() error { +func (a *appenderBase) logSeries() error { a.mtx.RLock() defer a.mtx.RUnlock() @@ -1283,7 +1324,7 @@ func (a *appender) logSeries() error { // minValidTime returns the minimum timestamp that a sample can have // and is needed for preventing underflow. -func (a *appender) minValidTime(lastTs int64) int64 { +func (a *appenderBase) minValidTime(lastTs int64) int64 { if lastTs < math.MinInt64+a.opts.OutOfOrderTimeWindow { return math.MinInt64 } diff --git a/tsdb/agent/db_append_v2.go b/tsdb/agent/db_append_v2.go index 5c9774cd58..ae4e3a4a84 100644 --- a/tsdb/agent/db_append_v2.go +++ b/tsdb/agent/db_append_v2.go @@ -1,4 +1,4 @@ -// Copyright 2021 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -17,927 +17,51 @@ import ( "context" "errors" "fmt" - "log/slog" - "math" - "path/filepath" - "sync" - "time" - "unicode/utf8" - - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/common/model" - "go.uber.org/atomic" "github.com/prometheus/prometheus/model/exemplar" "github.com/prometheus/prometheus/model/histogram" "github.com/prometheus/prometheus/model/labels" - "github.com/prometheus/prometheus/model/metadata" - "github.com/prometheus/prometheus/model/timestamp" + "github.com/prometheus/prometheus/model/value" "github.com/prometheus/prometheus/storage" - "github.com/prometheus/prometheus/storage/remote" "github.com/prometheus/prometheus/tsdb" "github.com/prometheus/prometheus/tsdb/chunks" - tsdb_errors "github.com/prometheus/prometheus/tsdb/errors" "github.com/prometheus/prometheus/tsdb/record" - "github.com/prometheus/prometheus/tsdb/tsdbutil" - "github.com/prometheus/prometheus/tsdb/wlog" - "github.com/prometheus/prometheus/util/compression" - "github.com/prometheus/prometheus/util/zeropool" ) -const ( - sampleMetricTypeFloat = "float" - sampleMetricTypeHistogram = "histogram" -) - -var ErrUnsupported = errors.New("unsupported operation with WAL-only storage") - -// Default values for options. -var ( - DefaultTruncateFrequency = 2 * time.Hour - DefaultMinWALTime = int64(5 * time.Minute / time.Millisecond) - DefaultMaxWALTime = int64(4 * time.Hour / time.Millisecond) -) - -// Options of the WAL storage. -type Options struct { - // Segments (wal files) max size. - // WALSegmentSize <= 0, segment size is default size. - // WALSegmentSize > 0, segment size is WALSegmentSize. - WALSegmentSize int - - // WALCompression configures the compression type to use on records in the WAL. - WALCompression compression.Type - - // StripeSize is the size (power of 2) in entries of the series hash map. Reducing the size will save memory but impact performance. - StripeSize int - - // TruncateFrequency determines how frequently to truncate data from the WAL. - TruncateFrequency time.Duration - - // Shortest and longest amount of time data can exist in the WAL before being - // deleted. - MinWALTime, MaxWALTime int64 - - // NoLockfile disables creation and consideration of a lock file. - NoLockfile bool - - // OutOfOrderTimeWindow specifies how much out of order is allowed, if any. - OutOfOrderTimeWindow int64 +// AppenderV2 implements storage.AppenderV2. +func (db *DB) AppenderV2(context.Context) storage.AppenderV2 { + return db.appenderV2Pool.Get().(storage.AppenderV2) } -// DefaultOptions used for the WAL storage. They are reasonable for setups using -// millisecond-precision timestamps. -func DefaultOptions() *Options { - return &Options{ - WALSegmentSize: wlog.DefaultSegmentSize, - WALCompression: compression.None, - StripeSize: tsdb.DefaultStripeSize, - TruncateFrequency: DefaultTruncateFrequency, - MinWALTime: DefaultMinWALTime, - MaxWALTime: DefaultMaxWALTime, - NoLockfile: false, - OutOfOrderTimeWindow: 0, - } +type appenderV2 struct { + appenderBase } -type dbMetrics struct { - r prometheus.Registerer - - numActiveSeries prometheus.Gauge - numWALSeriesPendingDeletion prometheus.Gauge - totalAppendedSamples *prometheus.CounterVec - totalAppendedExemplars prometheus.Counter - totalOutOfOrderSamples prometheus.Counter - walTruncateDuration prometheus.Summary - walCorruptionsTotal prometheus.Counter - walTotalReplayDuration prometheus.Gauge - checkpointDeleteFail prometheus.Counter - checkpointDeleteTotal prometheus.Counter - checkpointCreationFail prometheus.Counter - checkpointCreationTotal prometheus.Counter -} - -func newDBMetrics(r prometheus.Registerer) *dbMetrics { - m := dbMetrics{r: r} - m.numActiveSeries = prometheus.NewGauge(prometheus.GaugeOpts{ - Name: "prometheus_agent_active_series", - Help: "Number of active series being tracked by the WAL storage", - }) - - m.numWALSeriesPendingDeletion = prometheus.NewGauge(prometheus.GaugeOpts{ - Name: "prometheus_agent_deleted_series", - Help: "Number of series pending deletion from the WAL", - }) - - m.totalAppendedSamples = prometheus.NewCounterVec(prometheus.CounterOpts{ - Name: "prometheus_agent_samples_appended_total", - Help: "Total number of samples appended to the storage", - }, []string{"type"}) - - m.totalAppendedExemplars = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_exemplars_appended_total", - Help: "Total number of exemplars appended to the storage", - }) - - m.totalOutOfOrderSamples = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_out_of_order_samples_total", - Help: "Total number of out of order samples ingestion failed attempts.", - }) - - m.walTruncateDuration = prometheus.NewSummary(prometheus.SummaryOpts{ - Name: "prometheus_agent_truncate_duration_seconds", - Help: "Duration of WAL truncation.", - }) - - m.walCorruptionsTotal = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_corruptions_total", - Help: "Total number of WAL corruptions.", - }) - - m.walTotalReplayDuration = prometheus.NewGauge(prometheus.GaugeOpts{ - Name: "prometheus_agent_data_replay_duration_seconds", - Help: "Time taken to replay the data on disk.", - }) - - m.checkpointDeleteFail = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_checkpoint_deletions_failed_total", - Help: "Total number of checkpoint deletions that failed.", - }) - - m.checkpointDeleteTotal = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_checkpoint_deletions_total", - Help: "Total number of checkpoint deletions attempted.", - }) - - m.checkpointCreationFail = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_checkpoint_creations_failed_total", - Help: "Total number of checkpoint creations that failed.", - }) - - m.checkpointCreationTotal = prometheus.NewCounter(prometheus.CounterOpts{ - Name: "prometheus_agent_checkpoint_creations_total", - Help: "Total number of checkpoint creations attempted.", - }) - - if r != nil { - r.MustRegister( - m.numActiveSeries, - m.numWALSeriesPendingDeletion, - m.totalAppendedSamples, - m.totalAppendedExemplars, - m.totalOutOfOrderSamples, - m.walTruncateDuration, - m.walCorruptionsTotal, - m.walTotalReplayDuration, - m.checkpointDeleteFail, - m.checkpointDeleteTotal, - m.checkpointCreationFail, - m.checkpointCreationTotal, - ) - } - - return &m -} - -func (m *dbMetrics) Unregister() { - if m.r == nil { - return - } - cs := []prometheus.Collector{ - m.numActiveSeries, - m.numWALSeriesPendingDeletion, - m.totalAppendedSamples, - m.totalAppendedExemplars, - m.totalOutOfOrderSamples, - m.walTruncateDuration, - m.walCorruptionsTotal, - m.walTotalReplayDuration, - m.checkpointDeleteFail, - m.checkpointDeleteTotal, - m.checkpointCreationFail, - m.checkpointCreationTotal, - } - for _, c := range cs { - m.r.Unregister(c) - } -} - -// DB represents a WAL-only storage. It implements storage.DB. -type DB struct { - mtx sync.RWMutex - logger *slog.Logger - opts *Options - rs *remote.Storage - - wal *wlog.WL - locker *tsdbutil.DirLocker - - appenderPool sync.Pool - bufPool sync.Pool - - // These pools are only used during WAL replay and are reset at the end. - // NOTE: Adjust resetWALReplayResources() upon changes to the pools. - walReplaySeriesPool zeropool.Pool[[]record.RefSeries] - walReplaySamplesPool zeropool.Pool[[]record.RefSample] - walReplayHistogramsPool zeropool.Pool[[]record.RefHistogramSample] - walReplayFloatHistogramsPool zeropool.Pool[[]record.RefFloatHistogramSample] - - nextRef *atomic.Uint64 - series *stripeSeries - // deleted is a map of (ref IDs that should be deleted from WAL) to (the WAL segment they - // must be kept around to). - deleted map[chunks.HeadSeriesRef]int - - donec chan struct{} - stopc chan struct{} - - writeNotified wlog.WriteNotified - - metrics *dbMetrics -} - -// Open returns a new agent.DB in the given directory. -func Open(l *slog.Logger, reg prometheus.Registerer, rs *remote.Storage, dir string, opts *Options) (*DB, error) { - opts = validateOptions(opts) - - locker, err := tsdbutil.NewDirLocker(dir, "agent", l, reg) - if err != nil { - return nil, err - } - if !opts.NoLockfile { - if err := locker.Lock(); err != nil { - return nil, err - } - } - - // remote_write expects WAL to be stored in a "wal" subdirectory of the main storage. - dir = filepath.Join(dir, "wal") - - w, err := wlog.NewSize(l, reg, dir, opts.WALSegmentSize, opts.WALCompression) - if err != nil { - return nil, fmt.Errorf("creating WAL: %w", err) - } - - db := &DB{ - logger: l, - opts: opts, - rs: rs, - - wal: w, - locker: locker, - - nextRef: atomic.NewUint64(0), - series: newStripeSeries(opts.StripeSize), - deleted: make(map[chunks.HeadSeriesRef]int), - - donec: make(chan struct{}), - stopc: make(chan struct{}), - - metrics: newDBMetrics(reg), - } - - db.bufPool.New = func() any { - return make([]byte, 0, 1024) - } - - db.appenderPool.New = func() any { - return &appender{ - DB: db, - pendingSeries: make([]record.RefSeries, 0, 100), - pendingSamples: make([]record.RefSample, 0, 100), - pendingHistograms: make([]record.RefHistogramSample, 0, 100), - pendingFloatHistograms: make([]record.RefFloatHistogramSample, 0, 100), - pendingExamplars: make([]record.RefExemplar, 0, 10), - } - } - - if err := db.replayWAL(); err != nil { - db.logger.Warn("encountered WAL read error, attempting repair", "err", err) - if err := w.Repair(err); err != nil { - return nil, fmt.Errorf("repair corrupted WAL: %w", err) - } - db.logger.Info("successfully repaired WAL") - } - - go db.run() - return db, nil -} - -// SetWriteNotified allows to set an instance to notify when a write happens. -// It must be used during initialization. It is not safe to use it during execution. -func (db *DB) SetWriteNotified(wn wlog.WriteNotified) { - db.writeNotified = wn -} - -func validateOptions(opts *Options) *Options { - if opts == nil { - opts = DefaultOptions() - } - if opts.WALSegmentSize <= 0 { - opts.WALSegmentSize = wlog.DefaultSegmentSize - } - - if opts.WALCompression == "" { - opts.WALCompression = compression.None - } - - // Revert StripeSize to DefaultStripeSize if StripeSize is either 0 or not a power of 2. - if opts.StripeSize <= 0 || ((opts.StripeSize & (opts.StripeSize - 1)) != 0) { - opts.StripeSize = tsdb.DefaultStripeSize - } - if opts.TruncateFrequency <= 0 { - opts.TruncateFrequency = DefaultTruncateFrequency - } - if opts.MinWALTime <= 0 { - opts.MinWALTime = DefaultMinWALTime - } - if opts.MaxWALTime <= 0 { - opts.MaxWALTime = DefaultMaxWALTime - } - if opts.MinWALTime > opts.MaxWALTime { - opts.MaxWALTime = opts.MinWALTime - } - - if t := int64(opts.TruncateFrequency / time.Millisecond); opts.MaxWALTime < t { - opts.MaxWALTime = t - } - return opts -} - -func (db *DB) replayWAL() error { - db.logger.Info("replaying WAL, this may take a while", "dir", db.wal.Dir()) - defer db.resetWALReplayResources() - start := time.Now() - - dir, startFrom, err := wlog.LastCheckpoint(db.wal.Dir()) - if err != nil && !errors.Is(err, record.ErrNotFound) { - return fmt.Errorf("find last checkpoint: %w", err) - } - - multiRef := map[chunks.HeadSeriesRef]chunks.HeadSeriesRef{} - - if err == nil { - sr, err := wlog.NewSegmentsReader(dir) - if err != nil { - return fmt.Errorf("open checkpoint: %w", err) - } - defer func() { - if err := sr.Close(); err != nil { - db.logger.Warn("error while closing the wal segments reader", "err", err) - } - }() - - // A corrupted checkpoint is a hard error for now and requires user - // intervention. There's likely little data that can be recovered anyway. - if err := db.loadWAL(wlog.NewReader(sr), multiRef); err != nil { - return fmt.Errorf("backfill checkpoint: %w", err) - } - startFrom++ - db.logger.Info("WAL checkpoint loaded") - } - - // Find the last segment. - _, last, err := wlog.Segments(db.wal.Dir()) - if err != nil { - return fmt.Errorf("finding WAL segments: %w", err) - } - - // Backfill segments from the most recent checkpoint onwards. - for i := startFrom; i <= last; i++ { - seg, err := wlog.OpenReadSegment(wlog.SegmentName(db.wal.Dir(), i)) - if err != nil { - return fmt.Errorf("open WAL segment: %d: %w", i, err) - } - - sr := wlog.NewSegmentBufReader(seg) - err = db.loadWAL(wlog.NewReader(sr), multiRef) - if err := sr.Close(); err != nil { - db.logger.Warn("error while closing the wal segments reader", "err", err) - } - if err != nil { - return err - } - db.logger.Info("WAL segment loaded", "segment", i, "maxSegment", last) - } - - walReplayDuration := time.Since(start) - db.metrics.walTotalReplayDuration.Set(walReplayDuration.Seconds()) - - return nil -} - -func (db *DB) resetWALReplayResources() { - db.walReplaySeriesPool = zeropool.Pool[[]record.RefSeries]{} - db.walReplaySamplesPool = zeropool.Pool[[]record.RefSample]{} - db.walReplayHistogramsPool = zeropool.Pool[[]record.RefHistogramSample]{} - db.walReplayFloatHistogramsPool = zeropool.Pool[[]record.RefFloatHistogramSample]{} -} - -func (db *DB) loadWAL(r *wlog.Reader, multiRef map[chunks.HeadSeriesRef]chunks.HeadSeriesRef) (err error) { +// Append appends pending sample to agent's DB. +// TODO: Wire metadata in the Agent's appender. +func (a *appenderV2) Append(ref storage.SeriesRef, l labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) { var ( - syms = labels.NewSymbolTable() // One table for the whole WAL. - dec = record.NewDecoder(syms, db.logger) - lastRef = chunks.HeadSeriesRef(db.nextRef.Load()) - - decoded = make(chan any, 10) - errCh = make(chan error, 1) + // Avoid shadowing err variables for reliability. + valErr, partialErr error + sampleMetricType = sampleMetricTypeFloat + isStale bool ) - - go func() { - defer close(decoded) - var err error - for r.Next() { - rec := r.Record() - switch dec.Type(rec) { - case record.Series: - series := db.walReplaySeriesPool.Get()[:0] - series, err = dec.Series(rec, series) - if err != nil { - errCh <- &wlog.CorruptionErr{ - Err: fmt.Errorf("decode series: %w", err), - Segment: r.Segment(), - Offset: r.Offset(), - } - return - } - decoded <- series - case record.Samples: - samples := db.walReplaySamplesPool.Get()[:0] - samples, err = dec.Samples(rec, samples) - if err != nil { - errCh <- &wlog.CorruptionErr{ - Err: fmt.Errorf("decode samples: %w", err), - Segment: r.Segment(), - Offset: r.Offset(), - } - return - } - decoded <- samples - case record.HistogramSamples, record.CustomBucketsHistogramSamples: - histograms := db.walReplayHistogramsPool.Get()[:0] - histograms, err = dec.HistogramSamples(rec, histograms) - if err != nil { - errCh <- &wlog.CorruptionErr{ - Err: fmt.Errorf("decode histogram samples: %w", err), - Segment: r.Segment(), - Offset: r.Offset(), - } - return - } - decoded <- histograms - case record.FloatHistogramSamples, record.CustomBucketsFloatHistogramSamples: - floatHistograms := db.walReplayFloatHistogramsPool.Get()[:0] - floatHistograms, err = dec.FloatHistogramSamples(rec, floatHistograms) - if err != nil { - errCh <- &wlog.CorruptionErr{ - Err: fmt.Errorf("decode float histogram samples: %w", err), - Segment: r.Segment(), - Offset: r.Offset(), - } - return - } - decoded <- floatHistograms - case record.Tombstones, record.Exemplars: - // We don't care about tombstones or exemplars during replay. - // TODO: If decide to decode exemplars, we should make sure to prepopulate - // stripeSeries.exemplars in the next block by using setLatestExemplar. - continue - default: - errCh <- &wlog.CorruptionErr{ - Err: fmt.Errorf("invalid record type %v", dec.Type(rec)), - Segment: r.Segment(), - Offset: r.Offset(), - } - } - } - }() - - var nonExistentSeriesRefs atomic.Uint64 - - for d := range decoded { - switch v := d.(type) { - case []record.RefSeries: - for _, entry := range v { - // If this is a new series, create it in memory. If we never read in a - // sample for this series, its timestamp will remain at 0 and it will - // be deleted at the next GC. - if db.series.GetByID(entry.Ref) == nil { - series := &memSeries{ref: entry.Ref, lset: entry.Labels, lastTs: 0} - db.series.Set(entry.Labels.Hash(), series) - multiRef[entry.Ref] = series.ref - db.metrics.numActiveSeries.Inc() - if entry.Ref > lastRef { - lastRef = entry.Ref - } - } - } - db.walReplaySeriesPool.Put(v) - case []record.RefSample: - for _, entry := range v { - // Update the lastTs for the series based - ref, ok := multiRef[entry.Ref] - if !ok { - nonExistentSeriesRefs.Inc() - continue - } - series := db.series.GetByID(ref) - if entry.T > series.lastTs { - series.lastTs = entry.T - } - } - db.walReplaySamplesPool.Put(v) - case []record.RefHistogramSample: - for _, entry := range v { - // Update the lastTs for the series based - ref, ok := multiRef[entry.Ref] - if !ok { - nonExistentSeriesRefs.Inc() - continue - } - series := db.series.GetByID(ref) - if entry.T > series.lastTs { - series.lastTs = entry.T - } - } - db.walReplayHistogramsPool.Put(v) - case []record.RefFloatHistogramSample: - for _, entry := range v { - // Update the lastTs for the series based - ref, ok := multiRef[entry.Ref] - if !ok { - nonExistentSeriesRefs.Inc() - continue - } - series := db.series.GetByID(ref) - if entry.T > series.lastTs { - series.lastTs = entry.T - } - } - db.walReplayFloatHistogramsPool.Put(v) - default: - panic(fmt.Errorf("unexpected decoded type: %T", d)) - } + // Fail fast on incorrect histograms. + switch { + case fh != nil: + sampleMetricType = sampleMetricTypeHistogram + valErr = fh.Validate() + case h != nil: + sampleMetricType = sampleMetricTypeHistogram + valErr = h.Validate() + } + if valErr != nil { + return 0, valErr } - if v := nonExistentSeriesRefs.Load(); v > 0 { - db.logger.Warn("found sample referencing non-existing series", "skipped_series", v) - } - - db.nextRef.Store(uint64(lastRef)) - - select { - case err := <-errCh: - return err - default: - if r.Err() != nil { - return fmt.Errorf("read records: %w", r.Err()) - } - return nil - } -} - -func (db *DB) run() { - defer close(db.donec) - -Loop: - for { - select { - case <-db.stopc: - break Loop - case <-time.After(db.opts.TruncateFrequency): - // The timestamp ts is used to determine which series are not receiving - // samples and may be deleted from the WAL. Their most recent append - // timestamp is compared to ts, and if that timestamp is older then ts, - // they are considered inactive and may be deleted. - // - // Subtracting a duration from ts will add a buffer for when series are - // considered inactive and safe for deletion. - ts := max(db.rs.LowestSentTimestamp()-db.opts.MinWALTime, 0) - - // Network issues can prevent the result of getRemoteWriteTimestamp from - // changing. We don't want data in the WAL to grow forever, so we set a cap - // on the maximum age data can be. If our ts is older than this cutoff point, - // we'll shift it forward to start deleting very stale data. - if maxTS := timestamp.FromTime(time.Now()) - db.opts.MaxWALTime; ts < maxTS { - ts = maxTS - } - - db.logger.Debug("truncating the WAL", "ts", ts) - if err := db.truncate(ts); err != nil { - db.logger.Warn("failed to truncate WAL", "err", err) - } - } - } -} - -// keepSeriesInWALCheckpointFn returns a function that is used to determine whether a series record should be kept in the checkpoint. -// last is the last WAL segment that was considered for checkpointing. -// NOTE: the agent implementation here is different from the Prometheus implementation, in that it uses WAL segment numbers instead of timestamps. -func (db *DB) keepSeriesInWALCheckpointFn(last int) func(id chunks.HeadSeriesRef) bool { - return func(id chunks.HeadSeriesRef) bool { - // Keep the record if the series exists in the db. - if db.series.GetByID(id) != nil { - return true - } - - // Keep the record if the series was recently deleted. - seg, ok := db.deleted[id] - return ok && seg > last - } -} - -func (db *DB) truncate(mint int64) error { - db.logger.Info("series GC started") - db.mtx.RLock() - defer db.mtx.RUnlock() - - start := time.Now() - - db.gc(mint) - db.logger.Info("series GC completed", "duration", time.Since(start)) - - first, last, err := wlog.Segments(db.wal.Dir()) - if err != nil { - return fmt.Errorf("get segment range: %w", err) - } - - // Start a new segment so low ingestion volume instances don't have more WAL - // than needed. - if _, err := db.wal.NextSegment(); err != nil { - return fmt.Errorf("next segment: %w", err) - } - - last-- // Never consider most recent segment for checkpoint - if last < 0 { - return nil // no segments yet - } - - // The lower two-thirds of segments should contain mostly obsolete samples. - // If we have less than two segments, it's not worth checkpointing yet. - last = first + (last-first)*2/3 - if last <= first { - return nil - } - - db.metrics.checkpointCreationTotal.Inc() - - if _, err = wlog.Checkpoint(db.logger, db.wal, first, last, db.keepSeriesInWALCheckpointFn(last), mint); err != nil { - db.metrics.checkpointCreationFail.Inc() - var cerr *wlog.CorruptionErr - if errors.As(err, &cerr) { - db.metrics.walCorruptionsTotal.Inc() - } - return fmt.Errorf("create checkpoint: %w", err) - } - if err := db.wal.Truncate(last + 1); err != nil { - // If truncating fails, we'll just try it again at the next checkpoint. - // Leftover segments will still just be ignored in the future if there's a - // checkpoint that supersedes them. - db.logger.Error("truncating segments failed", "err", err) - } - - // The checkpoint is written and segments before it are truncated, so we - // no longer need to track deleted series that were being kept around. - for ref, segment := range db.deleted { - if segment <= last { - delete(db.deleted, ref) - } - } - db.metrics.checkpointDeleteTotal.Inc() - db.metrics.numWALSeriesPendingDeletion.Set(float64(len(db.deleted))) - - if err := wlog.DeleteCheckpoints(db.wal.Dir(), last); err != nil { - // Leftover old checkpoints do not cause problems down the line beyond - // occupying disk space. They will just be ignored since a newer checkpoint - // exists. - db.logger.Error("delete old checkpoints", "err", err) - db.metrics.checkpointDeleteFail.Inc() - } - - db.metrics.walTruncateDuration.Observe(time.Since(start).Seconds()) - - db.logger.Info("WAL checkpoint complete", "first", first, "last", last, "duration", time.Since(start)) - return nil -} - -// gc marks ref IDs that have not received a sample since mint as deleted in -// s.deleted, along with the segment where they originally got deleted. -func (db *DB) gc(mint int64) { - deleted := db.series.GC(mint) - db.metrics.numActiveSeries.Sub(float64(len(deleted))) - - _, last, _ := wlog.Segments(db.wal.Dir()) - - // We want to keep series records for any newly deleted series - // until we've passed the last recorded segment. This prevents - // the WAL having samples for series records that no longer exist. - for ref := range deleted { - db.deleted[ref] = last - } - - db.metrics.numWALSeriesPendingDeletion.Set(float64(len(db.deleted))) -} - -// StartTime implements the Storage interface. -func (*DB) StartTime() (int64, error) { - return int64(model.Latest), nil -} - -// Querier implements the Storage interface. -func (*DB) Querier(int64, int64) (storage.Querier, error) { - return nil, ErrUnsupported -} - -// ChunkQuerier implements the Storage interface. -func (*DB) ChunkQuerier(int64, int64) (storage.ChunkQuerier, error) { - return nil, ErrUnsupported -} - -// ExemplarQuerier implements the Storage interface. -func (*DB) ExemplarQuerier(context.Context) (storage.ExemplarQuerier, error) { - return nil, ErrUnsupported -} - -// Appender implements storage.Storage. -func (db *DB) Appender(context.Context) storage.Appender { - return db.appenderPool.Get().(storage.Appender) -} - -// Close implements the Storage interface. -func (db *DB) Close() error { - db.mtx.Lock() - defer db.mtx.Unlock() - - close(db.stopc) - <-db.donec - - db.metrics.Unregister() - - return tsdb_errors.NewMulti(db.locker.Release(), db.wal.Close()).Err() -} - -type appender struct { - *DB - hints *storage.AppendOptions - - pendingSeries []record.RefSeries - pendingSamples []record.RefSample - pendingHistograms []record.RefHistogramSample - pendingFloatHistograms []record.RefFloatHistogramSample - pendingExamplars []record.RefExemplar - - // Pointers to the series referenced by each element of pendingSamples. - // Series lock is not held on elements. - sampleSeries []*memSeries - - // Pointers to the series referenced by each element of pendingHistograms. - // Series lock is not held on elements. - histogramSeries []*memSeries - - // Pointers to the series referenced by each element of pendingFloatHistograms. - // Series lock is not held on elements. - floatHistogramSeries []*memSeries -} - -func (a *appender) SetOptions(opts *storage.AppendOptions) { - a.hints = opts -} - -func (a *appender) Append(ref storage.SeriesRef, l labels.Labels, t int64, v float64) (storage.SeriesRef, error) { // series references and chunk references are identical for agent mode. - headRef := chunks.HeadSeriesRef(ref) - - series := a.series.GetByID(headRef) - if series == nil { - // Ensure no empty or duplicate labels have gotten through. This mirrors the - // equivalent validation code in the TSDB's headAppender. - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) - } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - var created bool - series, created = a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: series.ref, - Labels: l, - }) - - a.metrics.numActiveSeries.Inc() - } - } - - series.Lock() - defer series.Unlock() - - if t <= a.minValidTime(series.lastTs) { - a.metrics.totalOutOfOrderSamples.Inc() - return 0, storage.ErrOutOfOrderSample - } - - // NOTE: always modify pendingSamples and sampleSeries together. - a.pendingSamples = append(a.pendingSamples, record.RefSample{ - Ref: series.ref, - T: t, - V: v, - }) - a.sampleSeries = append(a.sampleSeries, series) - - a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeFloat).Inc() - return storage.SeriesRef(series.ref), nil -} - -func (a *appender) getOrCreate(l labels.Labels) (series *memSeries, created bool) { - hash := l.Hash() - - series = a.series.GetByHash(hash, l) - if series != nil { - return series, false - } - - ref := chunks.HeadSeriesRef(a.nextRef.Inc()) - series = &memSeries{ref: ref, lset: l, lastTs: math.MinInt64} - a.series.Set(hash, series) - return series, true -} - -func (a *appender) AppendExemplar(ref storage.SeriesRef, _ labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { - // Series references and chunk references are identical for agent mode. - headRef := chunks.HeadSeriesRef(ref) - - s := a.series.GetByID(headRef) + s := a.series.GetByID(chunks.HeadSeriesRef(ref)) if s == nil { - return 0, fmt.Errorf("unknown series ref when trying to add exemplar: %d", ref) - } - - // Ensure no empty labels have gotten through. - e.Labels = e.Labels.WithoutEmpty() - - if lbl, dup := e.Labels.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidExemplar) - } - - // Exemplar label length does not include chars involved in text rendering such as quotes - // equals sign, or commas. See definition of const ExemplarMaxLabelLength. - labelSetLen := 0 - err := e.Labels.Validate(func(l labels.Label) error { - labelSetLen += utf8.RuneCountInString(l.Name) - labelSetLen += utf8.RuneCountInString(l.Value) - - if labelSetLen > exemplar.ExemplarMaxLabelSetLength { - return storage.ErrExemplarLabelLength - } - return nil - }) - if err != nil { - return 0, err - } - - // Check for duplicate vs last stored exemplar for this series, and discard those. - // Otherwise, record the current exemplar as the latest. - // Prometheus' TSDB returns 0 when encountering duplicates, so we do the same here. - prevExemplar := a.series.GetLatestExemplar(s.ref) - if prevExemplar != nil && prevExemplar.Equals(e) { - // Duplicate, don't return an error but don't accept the exemplar. - return 0, nil - } - a.series.SetLatestExemplar(s.ref, &e) - - a.pendingExamplars = append(a.pendingExamplars, record.RefExemplar{ - Ref: s.ref, - T: e.Ts, - V: e.Value, - Labels: e.Labels, - }) - - a.metrics.totalAppendedExemplars.Inc() - return storage.SeriesRef(s.ref), nil -} - -func (a *appender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - if h != nil { - if err := h.Validate(); err != nil { - return 0, err - } - } - - if fh != nil { - if err := fh.Validate(); err != nil { - return 0, err - } - } - - // series references and chunk references are identical for agent mode. - headRef := chunks.HeadSeriesRef(ref) - - series := a.series.GetByID(headRef) - if series == nil { // Ensure no empty or duplicate labels have gotten through. This mirrors the // equivalent validation code in the TSDB's headAppender. l = l.WithoutEmpty() @@ -950,10 +74,10 @@ func (a *appender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int } var created bool - series, created = a.getOrCreate(l) + s, created = a.getOrCreate(l) if created { a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: series.ref, + Ref: s.ref, Labels: l, }) @@ -961,332 +85,140 @@ func (a *appender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int } } - series.Lock() - defer series.Unlock() + s.Lock() + lastTS := s.lastTs + s.Unlock() - if t <= a.minValidTime(series.lastTs) { + // TODO(bwplotka): Handle ST natively (as per PROM-60). + if a.opts.EnableSTAsZeroSample && st != 0 { + a.bestEffortAppendSTZeroSample(s, lastTS, st, t, h, fh) + } + + if t <= a.minValidTime(lastTS) { a.metrics.totalOutOfOrderSamples.Inc() return 0, storage.ErrOutOfOrderSample } switch { - case h != nil: - // NOTE: always modify pendingHistograms and histogramSeries together - a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{ - Ref: series.ref, - T: t, - H: h, - }) - a.histogramSeries = append(a.histogramSeries, series) case fh != nil: + isStale = value.IsStaleNaN(fh.Sum) // NOTE: always modify pendingFloatHistograms and floatHistogramSeries together a.pendingFloatHistograms = append(a.pendingFloatHistograms, record.RefFloatHistogramSample{ - Ref: series.ref, + Ref: s.ref, T: t, FH: fh, }) - a.floatHistogramSeries = append(a.floatHistogramSeries, series) + a.floatHistogramSeries = append(a.floatHistogramSeries, s) + case h != nil: + isStale = value.IsStaleNaN(h.Sum) + // NOTE: always modify pendingHistograms and histogramSeries together + a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{ + Ref: s.ref, + T: t, + H: h, + }) + a.histogramSeries = append(a.histogramSeries, s) + default: + isStale = value.IsStaleNaN(v) + + // NOTE: always modify pendingSamples and sampleSeries together. + a.pendingSamples = append(a.pendingSamples, record.RefSample{ + Ref: s.ref, + T: t, + V: v, + }) + a.sampleSeries = append(a.sampleSeries, s) + } + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricType).Inc() + if isStale { + // For stale values we never attempt to process metadata/exemplars, claim the success. + return storage.SeriesRef(s.ref), nil } - a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - return storage.SeriesRef(series.ref), nil + // Append exemplars if any and if storage was configured for it. + // TODO(bwplotka): Agent does not have equivalent of a.head.opts.EnableExemplarStorage && a.head.opts.MaxExemplars.Load() > 0 ? + if len(opts.Exemplars) > 0 { + // Currently only exemplars can return partial errors. + partialErr = a.appendExemplars(s, opts.Exemplars) + } + return storage.SeriesRef(s.ref), partialErr } -func (*appender) UpdateMetadata(storage.SeriesRef, labels.Labels, metadata.Metadata) (storage.SeriesRef, error) { - // TODO: Wire metadata in the Agent's appender. - return 0, nil -} - -func (a *appender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) { - if h != nil { - if err := h.Validate(); err != nil { - return 0, err - } - } - if fh != nil { - if err := fh.Validate(); err != nil { - return 0, err - } - } - if st >= t { - return 0, storage.ErrSTNewerThanSample - } - - series := a.series.GetByID(chunks.HeadSeriesRef(ref)) - if series == nil { +func (a *appenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemplar) error { + var errs []error + for _, e := range exemplar { // Ensure no empty labels have gotten through. - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + e.Labels = e.Labels.WithoutEmpty() + + if err := a.validateExemplar(s.ref, e); err != nil { + if !errors.Is(err, storage.ErrDuplicateExemplar) { + // Except duplicates, return partial errors. + errs = append(errs, err) + continue + } + if !errors.Is(err, storage.ErrOutOfOrderExemplar) { + a.logger.Debug("Error while adding an exemplar on AppendSample", "exemplars", fmt.Sprintf("%+v", e), "err", e) + } + continue } - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - var created bool - series, created = a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: series.ref, - Labels: l, - }) - a.metrics.numActiveSeries.Inc() - } + a.series.SetLatestExemplar(s.ref, &e) + a.pendingExamplars = append(a.pendingExamplars, record.RefExemplar{ + Ref: s.ref, + T: e.Ts, + V: e.Value, + Labels: e.Labels, + }) + a.metrics.totalAppendedExemplars.Inc() } - - series.Lock() - defer series.Unlock() - - if st <= a.minValidTime(series.lastTs) { - return 0, storage.ErrOutOfOrderST + if len(errs) > 0 { + return &storage.AppendPartialError{ExemplarErrors: errs} } + return nil +} - if st <= series.lastTs { - // discard the sample if it's out of order. - return 0, storage.ErrOutOfOrderST +// NOTE(bwplotka): This feature might be deprecated and removed once PROM-60 +// is implemented. +// +// ST is an experimental feature, we don't fail the append on errors, just debug log. +func (a *appenderV2) bestEffortAppendSTZeroSample(s *memSeries, lastTS, st, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) { + if st >= t { + a.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrSTNewerThanSample) + return + } + if st <= lastTS { + a.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrOutOfOrderST) + return } - series.lastTs = st switch { - case h != nil: - zeroHistogram := &histogram.Histogram{} - a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{ - Ref: series.ref, - T: st, - H: zeroHistogram, - }) - a.histogramSeries = append(a.histogramSeries, series) case fh != nil: - a.pendingFloatHistograms = append(a.pendingFloatHistograms, record.RefFloatHistogramSample{ - Ref: series.ref, - T: st, - FH: &histogram.FloatHistogram{}, - }) - a.floatHistogramSeries = append(a.floatHistogramSeries, series) + zeroFloatHistogram := &histogram.FloatHistogram{ + // The STZeroSample represents a counter reset by definition. + CounterResetHint: histogram.CounterReset, + // Replicate other fields to avoid needless chunk creation. + Schema: fh.Schema, + ZeroThreshold: fh.ZeroThreshold, + CustomValues: fh.CustomValues, + } + a.pendingFloatHistograms = append(a.pendingFloatHistograms, record.RefFloatHistogramSample{Ref: s.ref, T: st, FH: zeroFloatHistogram}) + a.floatHistogramSeries = append(a.floatHistogramSeries, s) + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + case h != nil: + zeroHistogram := &histogram.Histogram{ + // The STZeroSample represents a counter reset by definition. + CounterResetHint: histogram.CounterReset, + // Replicate other fields to avoid needless chunk creation. + Schema: h.Schema, + ZeroThreshold: h.ZeroThreshold, + CustomValues: h.CustomValues, + } + a.pendingHistograms = append(a.pendingHistograms, record.RefHistogramSample{Ref: s.ref, T: st, H: zeroHistogram}) + a.histogramSeries = append(a.histogramSeries, s) + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() + default: + a.pendingSamples = append(a.pendingSamples, record.RefSample{Ref: s.ref, T: st, V: 0}) + a.sampleSeries = append(a.sampleSeries, s) + a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeFloat).Inc() } - - a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeHistogram).Inc() - return storage.SeriesRef(series.ref), nil -} - -func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, st int64) (storage.SeriesRef, error) { - if st >= t { - return 0, storage.ErrSTNewerThanSample - } - - series := a.series.GetByID(chunks.HeadSeriesRef(ref)) - if series == nil { - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) - } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - newSeries, created := a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: newSeries.ref, - Labels: l, - }) - a.metrics.numActiveSeries.Inc() - } - - series = newSeries - } - - series.Lock() - defer series.Unlock() - - if t <= a.minValidTime(series.lastTs) { - a.metrics.totalOutOfOrderSamples.Inc() - return 0, storage.ErrOutOfOrderSample - } - - if st <= series.lastTs { - // discard the sample if it's out of order. - return 0, storage.ErrOutOfOrderST - } - series.lastTs = st - - // NOTE: always modify pendingSamples and sampleSeries together. - a.pendingSamples = append(a.pendingSamples, record.RefSample{ - Ref: series.ref, - T: st, - V: 0, - }) - a.sampleSeries = append(a.sampleSeries, series) - - a.metrics.totalAppendedSamples.WithLabelValues(sampleMetricTypeFloat).Inc() - - return storage.SeriesRef(series.ref), nil -} - -// Commit submits the collected samples and purges the batch. -func (a *appender) Commit() error { - if err := a.log(); err != nil { - return err - } - - a.clearData() - a.appenderPool.Put(a) - - if a.writeNotified != nil { - a.writeNotified.Notify() - } - return nil -} - -// log logs all pending data to the WAL. -func (a *appender) log() error { - a.mtx.RLock() - defer a.mtx.RUnlock() - - var encoder record.Encoder - buf := a.bufPool.Get().([]byte) - defer func() { - a.bufPool.Put(buf) //nolint:staticcheck - }() - - if len(a.pendingSeries) > 0 { - buf = encoder.Series(a.pendingSeries, buf) - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - - if len(a.pendingSamples) > 0 { - buf = encoder.Samples(a.pendingSamples, buf) - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - - if len(a.pendingHistograms) > 0 { - var customBucketsHistograms []record.RefHistogramSample - buf, customBucketsHistograms = encoder.HistogramSamples(a.pendingHistograms, buf) - if len(buf) > 0 { - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - if len(customBucketsHistograms) > 0 { - buf = encoder.CustomBucketsHistogramSamples(customBucketsHistograms, nil) - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - } - - if len(a.pendingFloatHistograms) > 0 { - var customBucketsFloatHistograms []record.RefFloatHistogramSample - buf, customBucketsFloatHistograms = encoder.FloatHistogramSamples(a.pendingFloatHistograms, buf) - if len(buf) > 0 { - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - if len(customBucketsFloatHistograms) > 0 { - buf = encoder.CustomBucketsFloatHistogramSamples(customBucketsFloatHistograms, nil) - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - } - - if len(a.pendingExamplars) > 0 { - buf = encoder.Exemplars(a.pendingExamplars, buf) - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - - var series *memSeries - for i, s := range a.pendingSamples { - series = a.sampleSeries[i] - if !series.updateTimestamp(s.T) { - a.metrics.totalOutOfOrderSamples.Inc() - } - } - for i, s := range a.pendingHistograms { - series = a.histogramSeries[i] - if !series.updateTimestamp(s.T) { - a.metrics.totalOutOfOrderSamples.Inc() - } - } - for i, s := range a.pendingFloatHistograms { - series = a.floatHistogramSeries[i] - if !series.updateTimestamp(s.T) { - a.metrics.totalOutOfOrderSamples.Inc() - } - } - - return nil -} - -// clearData clears all pending data. -func (a *appender) clearData() { - a.pendingSeries = a.pendingSeries[:0] - a.pendingSamples = a.pendingSamples[:0] - a.pendingHistograms = a.pendingHistograms[:0] - a.pendingFloatHistograms = a.pendingFloatHistograms[:0] - a.pendingExamplars = a.pendingExamplars[:0] - a.sampleSeries = a.sampleSeries[:0] - a.histogramSeries = a.histogramSeries[:0] - a.floatHistogramSeries = a.floatHistogramSeries[:0] -} - -func (a *appender) Rollback() error { - // Series are created in-memory regardless of rollback. This means we must - // log them to the WAL, otherwise subsequent commits may reference a series - // which was never written to the WAL. - if err := a.logSeries(); err != nil { - return err - } - - a.clearData() - a.appenderPool.Put(a) - return nil -} - -// logSeries logs only pending series records to the WAL. -func (a *appender) logSeries() error { - a.mtx.RLock() - defer a.mtx.RUnlock() - - if len(a.pendingSeries) > 0 { - buf := a.bufPool.Get().([]byte) - defer func() { - a.bufPool.Put(buf) //nolint:staticcheck - }() - - var encoder record.Encoder - buf = encoder.Series(a.pendingSeries, buf) - if err := a.wal.Log(buf); err != nil { - return err - } - buf = buf[:0] - } - - return nil -} - -// minValidTime returns the minimum timestamp that a sample can have -// and is needed for preventing underflow. -func (a *appender) minValidTime(lastTs int64) int64 { - if lastTs < math.MinInt64+a.opts.OutOfOrderTimeWindow { - return math.MinInt64 - } - - return lastTs - a.opts.OutOfOrderTimeWindow } diff --git a/tsdb/agent/db_append_v2_test.go b/tsdb/agent/db_append_v2_test.go index 7409f79ec5..ec92cfa630 100644 --- a/tsdb/agent/db_append_v2_test.go +++ b/tsdb/agent/db_append_v2_test.go @@ -1,4 +1,4 @@ -// Copyright 2021 The Prometheus Authors +// Copyright The Prometheus Authors // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at @@ -15,17 +15,14 @@ package agent import ( "context" - "errors" "fmt" - "io" "math" "path/filepath" - "strconv" "testing" "time" + "github.com/google/go-cmp/cmp" "github.com/prometheus/client_golang/prometheus" - dto "github.com/prometheus/client_model/go" "github.com/prometheus/common/model" "github.com/prometheus/common/promslog" "github.com/stretchr/testify/require" @@ -43,85 +40,56 @@ import ( "github.com/prometheus/prometheus/util/testutil" ) -func TestDB_InvalidSeries(t *testing.T) { +func TestDB_InvalidSeries_AppendV2(t *testing.T) { s := createTestAgentDB(t, nil, DefaultOptions()) defer s.Close() - app := s.Appender(context.Background()) - + app := s.AppenderV2(context.Background()) t.Run("Samples", func(t *testing.T) { - _, err := app.Append(0, labels.Labels{}, 0, 0) + _, err := app.Append(0, labels.Labels{}, 0, 0, 0, nil, nil, storage.AOptions{}) require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject empty labels") - _, err = app.Append(0, labels.FromStrings("a", "1", "a", "2"), 0, 0) + _, err = app.Append(0, labels.FromStrings("a", "1", "a", "2"), 0, 0, 0, nil, nil, storage.AOptions{}) require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject duplicate labels") }) t.Run("Histograms", func(t *testing.T) { - _, err := app.AppendHistogram(0, labels.Labels{}, 0, tsdbutil.GenerateTestHistograms(1)[0], nil) + _, err := app.Append(0, labels.Labels{}, 0, 0, 0, tsdbutil.GenerateTestHistograms(1)[0], nil, storage.AOptions{}) require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject empty labels") - _, err = app.AppendHistogram(0, labels.FromStrings("a", "1", "a", "2"), 0, tsdbutil.GenerateTestHistograms(1)[0], nil) + _, err = app.Append(0, labels.FromStrings("a", "1", "a", "2"), 0, 0, 0, tsdbutil.GenerateTestHistograms(1)[0], nil, storage.AOptions{}) require.ErrorIs(t, err, tsdb.ErrInvalidSample, "should reject duplicate labels") }) t.Run("Exemplars", func(t *testing.T) { - sRef, err := app.Append(0, labels.FromStrings("a", "1"), 0, 0) - require.NoError(t, err, "should not reject valid series") - - _, err = app.AppendExemplar(0, labels.EmptyLabels(), exemplar.Exemplar{}) - require.EqualError(t, err, "unknown series ref when trying to add exemplar: 0") - e := exemplar.Exemplar{Labels: labels.FromStrings("a", "1", "a", "2")} - _, err = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - require.ErrorIs(t, err, tsdb.ErrInvalidExemplar, "should reject duplicate labels") + _, err := app.Append(0, labels.FromStrings("a", "1"), 0, 0, 0, nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{e}, + }) + partErr := &storage.AppendPartialError{} + require.ErrorAs(t, err, &partErr) + require.Len(t, partErr.ExemplarErrors, 1) + require.ErrorIs(t, partErr.ExemplarErrors[0], tsdb.ErrInvalidExemplar, "should reject duplicate labels") e = exemplar.Exemplar{Labels: labels.FromStrings("a_somewhat_long_trace_id", "nYJSNtFrFTY37VR7mHzEE/LIDt7cdAQcuOzFajgmLDAdBSRHYPDzrxhMA4zz7el8naI/AoXFv9/e/G0vcETcIoNUi3OieeLfaIRQci2oa")} - _, err = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - require.ErrorIs(t, err, storage.ErrExemplarLabelLength, "should reject too long label length") + _, err = app.Append(0, labels.FromStrings("a", "2"), 0, 0, 0, nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{e}, + }) + partErr = &storage.AppendPartialError{} + require.ErrorAs(t, err, &partErr) + require.Len(t, partErr.ExemplarErrors, 1) + require.ErrorIs(t, partErr.ExemplarErrors[0], storage.ErrExemplarLabelLength, "should reject too long label length") - // Inverse check + // Inverse check. e = exemplar.Exemplar{Labels: labels.FromStrings("a", "1"), Value: 20, Ts: 10, HasTs: true} - _, err = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + _, err = app.Append(0, labels.FromStrings("a", "1"), 0, 0, 0, nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{e}, + }) require.NoError(t, err, "should not reject valid exemplars") }) } -func createTestAgentDB(t testing.TB, reg prometheus.Registerer, opts *Options) *DB { - t.Helper() - - dbDir := t.TempDir() - rs := remote.NewStorage(promslog.NewNopLogger(), reg, startTime, dbDir, time.Second*30, nil, false) - t.Cleanup(func() { - require.NoError(t, rs.Close()) - }) - - db, err := Open(promslog.NewNopLogger(), reg, rs, dbDir, opts) - require.NoError(t, err) - return db -} - -func TestUnsupportedFunctions(t *testing.T) { - s := createTestAgentDB(t, nil, DefaultOptions()) - defer s.Close() - - t.Run("Querier", func(t *testing.T) { - _, err := s.Querier(0, 0) - require.Equal(t, err, ErrUnsupported) - }) - - t.Run("ChunkQuerier", func(t *testing.T) { - _, err := s.ChunkQuerier(0, 0) - require.Equal(t, err, ErrUnsupported) - }) - - t.Run("ExemplarQuerier", func(t *testing.T) { - _, err := s.ExemplarQuerier(context.TODO()) - require.Equal(t, err, ErrUnsupported) - }) -} - -func TestCommit(t *testing.T) { +func TestCommit_AppendV2(t *testing.T) { const ( numDatapoints = 1000 numHistograms = 100 @@ -129,7 +97,7 @@ func TestCommit(t *testing.T) { ) s := createTestAgentDB(t, nil, DefaultOptions()) - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) lbls := labelsForTest(t.Name(), numSeries) for _, l := range lbls { @@ -137,16 +105,14 @@ func TestCommit(t *testing.T) { for i := range numDatapoints { sample := chunks.GenerateSamples(0, 1) - ref, err := app.Append(0, lset, sample[0].T(), sample[0].F()) - require.NoError(t, err) - - e := exemplar.Exemplar{ - Labels: lset, - Ts: sample[0].T() + int64(i), - Value: sample[0].F(), - HasTs: true, - } - _, err = app.AppendExemplar(ref, lset, e) + _, err := app.Append(0, lset, 0, sample[0].T(), sample[0].F(), nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{{ + Labels: lset, + Ts: sample[0].T() + int64(i), + Value: sample[0].F(), + HasTs: true, + }}, + }) require.NoError(t, err) } } @@ -158,7 +124,7 @@ func TestCommit(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -170,7 +136,7 @@ func TestCommit(t *testing.T) { customBucketHistograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), customBucketHistograms[i], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, customBucketHistograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -182,7 +148,7 @@ func TestCommit(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -194,7 +160,7 @@ func TestCommit(t *testing.T) { customBucketFloatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), nil, customBucketFloatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, customBucketFloatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -260,7 +226,7 @@ func TestCommit(t *testing.T) { require.Equal(t, numSeries*numHistograms*2, walFloatHistogramCount, "unexpected number of float histograms") } -func TestRollback(t *testing.T) { +func TestRollback_AppendV2(t *testing.T) { const ( numDatapoints = 1000 numHistograms = 100 @@ -268,7 +234,7 @@ func TestRollback(t *testing.T) { ) s := createTestAgentDB(t, nil, DefaultOptions()) - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) lbls := labelsForTest(t.Name(), numSeries) for _, l := range lbls { @@ -276,7 +242,7 @@ func TestRollback(t *testing.T) { for range numDatapoints { sample := chunks.GenerateSamples(0, 1) - _, err := app.Append(0, lset, sample[0].T(), sample[0].F()) + _, err := app.Append(0, lset, 0, sample[0].T(), sample[0].F(), nil, nil, storage.AOptions{}) require.NoError(t, err) } } @@ -288,7 +254,7 @@ func TestRollback(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -300,7 +266,7 @@ func TestRollback(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -312,7 +278,7 @@ func TestRollback(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -324,7 +290,7 @@ func TestRollback(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -393,7 +359,7 @@ func TestRollback(t *testing.T) { require.Equal(t, 0, walFloatHistogramCount, "float histograms should not have been written to WAL") } -func TestFullTruncateWAL(t *testing.T) { +func TestFullTruncateWAL_AppendV2(t *testing.T) { const ( numDatapoints = 1000 numHistograms = 100 @@ -409,14 +375,14 @@ func TestFullTruncateWAL(t *testing.T) { defer func() { require.NoError(t, s.Close()) }() - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) lbls := labelsForTest(t.Name(), numSeries) for _, l := range lbls { lset := labels.New(l...) for range numDatapoints { - _, err := app.Append(0, lset, int64(lastTs), 0) + _, err := app.Append(0, lset, 0, int64(lastTs), 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -429,7 +395,7 @@ func TestFullTruncateWAL(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(lastTs), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(lastTs), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -442,7 +408,7 @@ func TestFullTruncateWAL(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(lastTs), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(lastTs), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -455,7 +421,7 @@ func TestFullTruncateWAL(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(lastTs), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(lastTs), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -468,7 +434,7 @@ func TestFullTruncateWAL(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, int64(lastTs), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(lastTs), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -481,7 +447,7 @@ func TestFullTruncateWAL(t *testing.T) { require.Equal(t, float64(numSeries*5), m.Metric[0].Gauge.GetValue(), "agent wal truncate mismatch of deleted series count") } -func TestPartialTruncateWAL(t *testing.T) { +func TestPartialTruncateWAL_AppendV2(t *testing.T) { const ( numDatapoints = 1000 numSeries = 800 @@ -494,7 +460,7 @@ func TestPartialTruncateWAL(t *testing.T) { defer func() { require.NoError(t, s.Close()) }() - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) // Create first batch of 800 series with 1000 data-points with a fixed lastTs as 500. var lastTs int64 = 500 @@ -503,7 +469,7 @@ func TestPartialTruncateWAL(t *testing.T) { lset := labels.New(l...) for range numDatapoints { - _, err := app.Append(0, lset, lastTs, 0) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -516,7 +482,7 @@ func TestPartialTruncateWAL(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + _, err := app.Append(0, lset, 0, lastTs, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -529,7 +495,7 @@ func TestPartialTruncateWAL(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + _, err := app.Append(0, lset, 0, lastTs, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -542,7 +508,7 @@ func TestPartialTruncateWAL(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -555,7 +521,7 @@ func TestPartialTruncateWAL(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -568,7 +534,7 @@ func TestPartialTruncateWAL(t *testing.T) { lset := labels.New(l...) for range numDatapoints { - _, err := app.Append(0, lset, lastTs, 0) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -581,7 +547,7 @@ func TestPartialTruncateWAL(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + _, err := app.Append(0, lset, 0, lastTs, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -594,7 +560,7 @@ func TestPartialTruncateWAL(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + _, err := app.Append(0, lset, 0, lastTs, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -607,7 +573,7 @@ func TestPartialTruncateWAL(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -620,7 +586,7 @@ func TestPartialTruncateWAL(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numDatapoints) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -634,7 +600,7 @@ func TestPartialTruncateWAL(t *testing.T) { require.Equal(t, float64(numSeries*5), m.Metric[0].Gauge.GetValue(), "agent wal truncate mismatch of deleted series count") } -func TestWALReplay(t *testing.T) { +func TestWALReplay_AppendV2(t *testing.T) { const ( numDatapoints = 1000 numHistograms = 100 @@ -643,14 +609,14 @@ func TestWALReplay(t *testing.T) { ) s := createTestAgentDB(t, nil, DefaultOptions()) - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) lbls := labelsForTest(t.Name(), numSeries) for _, l := range lbls { lset := labels.New(l...) for range numDatapoints { - _, err := app.Append(0, lset, lastTs, 0) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) } } @@ -662,7 +628,7 @@ func TestWALReplay(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + _, err := app.Append(0, lset, 0, lastTs, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -674,7 +640,7 @@ func TestWALReplay(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, lastTs, histograms[i], nil) + _, err := app.Append(0, lset, 0, lastTs, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -686,7 +652,7 @@ func TestWALReplay(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -698,7 +664,7 @@ func TestWALReplay(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) for i := range numHistograms { - _, err := app.AppendHistogram(0, lset, lastTs, nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, lastTs, 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -733,29 +699,7 @@ func TestWALReplay(t *testing.T) { } } -func TestLockfile(t *testing.T) { - tsdbutil.TestDirLockerUsage(t, func(t *testing.T, data string, createLock bool) (*tsdbutil.DirLocker, testutil.Closer) { - logger := promslog.NewNopLogger() - reg := prometheus.NewRegistry() - rs := remote.NewStorage(logger, reg, startTime, data, time.Second*30, nil, false) - t.Cleanup(func() { - require.NoError(t, rs.Close()) - }) - - opts := DefaultOptions() - opts.NoLockfile = !createLock - - // Create the DB. This should create lockfile and its metrics. - db, err := Open(logger, nil, rs, data, opts) - require.NoError(t, err) - - return db.locker, testutil.NewCallbackCloser(func() { - require.NoError(t, db.Close()) - }) - }) -} - -func Test_ExistingWAL_NextRef(t *testing.T) { +func Test_ExistingWAL_NextRef_AppendV2(t *testing.T) { dbDir := t.TempDir() rs := remote.NewStorage(promslog.NewNopLogger(), nil, startTime, dbDir, time.Second*30, nil, false) defer func() { @@ -768,10 +712,10 @@ func Test_ExistingWAL_NextRef(t *testing.T) { seriesCount := 10 // Append series - app := db.Appender(context.Background()) + app := db.AppenderV2(context.Background()) for i := range seriesCount { lset := labels.FromStrings(model.MetricNameLabel, fmt.Sprintf("series_%d", i)) - _, err := app.Append(0, lset, 0, 100) + _, err := app.Append(0, lset, 0, 0, 100, nil, nil, storage.AOptions{}) require.NoError(t, err) } @@ -780,7 +724,7 @@ func Test_ExistingWAL_NextRef(t *testing.T) { // Append series for i := range histogramCount { lset := labels.FromStrings(model.MetricNameLabel, fmt.Sprintf("histogram_%d", i)) - _, err := app.AppendHistogram(0, lset, 0, histograms[i], nil) + _, err := app.Append(0, lset, 0, 0, 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } require.NoError(t, app.Commit()) @@ -799,90 +743,23 @@ func Test_ExistingWAL_NextRef(t *testing.T) { require.Equal(t, uint64(seriesCount+histogramCount), db.nextRef.Load(), "nextRef should be equal to the number of series written across the entire WAL") } -func Test_validateOptions(t *testing.T) { - t.Run("Apply defaults to zero values", func(t *testing.T) { - opts := validateOptions(&Options{}) - require.Equal(t, DefaultOptions(), opts) - }) - - t.Run("Defaults are already valid", func(t *testing.T) { - require.Equal(t, DefaultOptions(), validateOptions(nil)) - }) - - t.Run("MaxWALTime should not be lower than TruncateFrequency", func(t *testing.T) { - opts := validateOptions(&Options{ - MaxWALTime: int64(time.Hour / time.Millisecond), - TruncateFrequency: 2 * time.Hour, - }) - require.Equal(t, int64(2*time.Hour/time.Millisecond), opts.MaxWALTime) - }) -} - -func startTime() (int64, error) { - return time.Now().Unix() * 1000, nil -} - -// Create series for tests. -func labelsForTest(lName string, seriesCount int) [][]labels.Label { - var series [][]labels.Label - - for i := range seriesCount { - lset := []labels.Label{ - {Name: "a", Value: lName}, - {Name: "instance", Value: "localhost" + strconv.Itoa(i)}, - {Name: "job", Value: "prometheus"}, - } - series = append(series, lset) - } - - return series -} - -func gatherFamily(t *testing.T, reg prometheus.Gatherer, familyName string) *dto.MetricFamily { - t.Helper() - - families, err := reg.Gather() - require.NoError(t, err, "failed to gather metrics") - - for _, f := range families { - if f.GetName() == familyName { - return f - } - } - - t.Fatalf("could not find family %s", familyName) - - return nil -} - -func TestStorage_DuplicateExemplarsIgnored(t *testing.T) { +func TestStorage_DuplicateExemplarsIgnored_AppendV2(t *testing.T) { s := createTestAgentDB(t, nil, DefaultOptions()) - app := s.Appender(context.Background()) + app := s.AppenderV2(context.Background()) defer s.Close() - sRef, err := app.Append(0, labels.FromStrings("a", "1"), 0, 0) - require.NoError(t, err, "should not reject valid series") - // Write a few exemplars to our appender and call Commit(). // If the Labels, Value or Timestamp are different than the last exemplar, // then a new one should be appended; Otherwise, it should be skipped. - e := exemplar.Exemplar{Labels: labels.FromStrings("a", "1"), Value: 20, Ts: 10, HasTs: true} - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - - e.Labels = labels.FromStrings("b", "2") - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - - e.Value = 42 - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - - e.Ts = 25 - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) - _, _ = app.AppendExemplar(sRef, labels.EmptyLabels(), e) + e1 := exemplar.Exemplar{Labels: labels.FromStrings("a", "1"), Value: 20, Ts: 10, HasTs: true} + e2 := exemplar.Exemplar{Labels: labels.FromStrings("b", "2"), Value: 20, Ts: 10, HasTs: true} + e3 := exemplar.Exemplar{Labels: labels.FromStrings("b", "2"), Value: 42, Ts: 10, HasTs: true} + e4 := exemplar.Exemplar{Labels: labels.FromStrings("b", "2"), Value: 42, Ts: 25, HasTs: true} + _, err := app.Append(0, labels.FromStrings("a", "1"), 0, 0, 0, nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{e1, e1, e2, e2, e2, e3, e3, e4, e4}, + }) + require.NoError(t, err, "should not reject valid series") require.NoError(t, app.Commit()) // Read back what was written to the WAL. @@ -907,7 +784,7 @@ func TestStorage_DuplicateExemplarsIgnored(t *testing.T) { require.Equal(t, 4, walExemplarsCount) } -func TestDBAllowOOOSamples(t *testing.T) { +func TestDBAllowOOOSamples_AppendV2(t *testing.T) { const ( numDatapoints = 5 numHistograms = 5 @@ -919,7 +796,7 @@ func TestDBAllowOOOSamples(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = math.MaxInt64 s := createTestAgentDB(t, reg, opts) - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) // Let's add some samples in the [offset, offset+numDatapoints) range. lbls := labelsForTest(t.Name(), numSeries) @@ -927,16 +804,14 @@ func TestDBAllowOOOSamples(t *testing.T) { lset := labels.New(l...) for i := offset; i < numDatapoints+offset; i++ { - ref, err := app.Append(0, lset, int64(i), float64(i)) - require.NoError(t, err) - - e := exemplar.Exemplar{ - Labels: lset, - Ts: int64(i) * 2, - Value: float64(i), - HasTs: true, - } - _, err = app.AppendExemplar(ref, lset, e) + _, err := app.Append(0, lset, 0, int64(i), float64(i), nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{{ + Labels: lset, + Ts: int64(i) * 2, + Value: float64(i), + HasTs: true, + }}, + }) require.NoError(t, err) } } @@ -948,7 +823,7 @@ func TestDBAllowOOOSamples(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numHistograms) for i := offset; i < numDatapoints+offset; i++ { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i-offset], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i-offset], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -960,7 +835,7 @@ func TestDBAllowOOOSamples(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) for i := offset; i < numDatapoints+offset; i++ { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i-offset], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i-offset], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -972,7 +847,7 @@ func TestDBAllowOOOSamples(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) for i := offset; i < numDatapoints+offset; i++ { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i-offset]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i-offset], storage.AOptions{}) require.NoError(t, err) } } @@ -984,7 +859,7 @@ func TestDBAllowOOOSamples(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) for i := offset; i < numDatapoints+offset; i++ { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i-offset]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i-offset], storage.AOptions{}) require.NoError(t, err) } } @@ -1006,7 +881,7 @@ func TestDBAllowOOOSamples(t *testing.T) { t.Fatalf("unable to create storage for the agent: %v", err) } - app = db.Appender(context.Background()) + app = db.AppenderV2(context.Background()) // Now the lastTs will have been recorded successfully. // Let's try appending twice as many OOO samples in the [0, numDatapoints) range. @@ -1015,16 +890,14 @@ func TestDBAllowOOOSamples(t *testing.T) { lset := labels.New(l...) for i := range numDatapoints { - ref, err := app.Append(0, lset, int64(i), float64(i)) - require.NoError(t, err) - - e := exemplar.Exemplar{ - Labels: lset, - Ts: int64(i) * 2, - Value: float64(i), - HasTs: true, - } - _, err = app.AppendExemplar(ref, lset, e) + _, err := app.Append(0, lset, 0, int64(i), float64(i), nil, nil, storage.AOptions{ + Exemplars: []exemplar.Exemplar{{ + Labels: lset, + Ts: int64(i) * 2, + Value: float64(i), + HasTs: true, + }}, + }) require.NoError(t, err) } } @@ -1036,7 +909,7 @@ func TestDBAllowOOOSamples(t *testing.T) { histograms := tsdbutil.GenerateTestHistograms(numHistograms) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -1048,7 +921,7 @@ func TestDBAllowOOOSamples(t *testing.T) { histograms := tsdbutil.GenerateTestCustomBucketsHistograms(numHistograms) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, int64(i), histograms[i], nil) + _, err := app.Append(0, lset, 0, int64(i), 0, histograms[i], nil, storage.AOptions{}) require.NoError(t, err) } } @@ -1060,7 +933,7 @@ func TestDBAllowOOOSamples(t *testing.T) { floatHistograms := tsdbutil.GenerateTestFloatHistograms(numHistograms) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -1072,7 +945,7 @@ func TestDBAllowOOOSamples(t *testing.T) { floatHistograms := tsdbutil.GenerateTestCustomBucketsFloatHistograms(numHistograms) for i := range numDatapoints { - _, err := app.AppendHistogram(0, lset, int64(i), nil, floatHistograms[i]) + _, err := app.Append(0, lset, 0, int64(i), 0, nil, floatHistograms[i], storage.AOptions{}) require.NoError(t, err) } } @@ -1084,7 +957,7 @@ func TestDBAllowOOOSamples(t *testing.T) { require.NoError(t, db.Close()) } -func TestDBOutOfOrderTimeWindow(t *testing.T) { +func TestDBOutOfOrderTimeWindow_AppendV2(t *testing.T) { tc := []struct { outOfOrderTimeWindow, firstTs, secondTs int64 expectedError error @@ -1102,24 +975,24 @@ func TestDBOutOfOrderTimeWindow(t *testing.T) { opts := DefaultOptions() opts.OutOfOrderTimeWindow = c.outOfOrderTimeWindow s := createTestAgentDB(t, reg, opts) - app := s.Appender(context.TODO()) + app := s.AppenderV2(context.TODO()) lbls := labelsForTest(t.Name()+"_histogram", 1) lset := labels.New(lbls[0]...) - _, err := app.AppendHistogram(0, lset, c.firstTs, tsdbutil.GenerateTestHistograms(1)[0], nil) + _, err := app.Append(0, lset, 0, c.firstTs, 0, tsdbutil.GenerateTestHistograms(1)[0], nil, storage.AOptions{}) require.NoError(t, err) err = app.Commit() require.NoError(t, err) - _, err = app.AppendHistogram(0, lset, c.secondTs, tsdbutil.GenerateTestHistograms(1)[0], nil) + _, err = app.Append(0, lset, 0, c.secondTs, 0, tsdbutil.GenerateTestHistograms(1)[0], nil, storage.AOptions{}) require.ErrorIs(t, err, c.expectedError) lbls = labelsForTest(t.Name(), 1) lset = labels.New(lbls[0]...) - _, err = app.Append(0, lset, c.firstTs, 0) + _, err = app.Append(0, lset, 0, c.firstTs, 0, nil, nil, storage.AOptions{}) require.NoError(t, err) err = app.Commit() require.NoError(t, err) - _, err = app.Append(0, lset, c.secondTs, 0) + _, err = app.Append(0, lset, 0, c.secondTs, 0, nil, nil, storage.AOptions{}) require.ErrorIs(t, err, c.expectedError) expectedAppendedSamples := float64(2) @@ -1134,28 +1007,26 @@ func TestDBOutOfOrderTimeWindow(t *testing.T) { } } -type walSample struct { - t int64 - f float64 - h *histogram.Histogram - lbls labels.Labels - ref storage.SeriesRef -} - -func TestDBStartTimestampSamplesIngestion(t *testing.T) { +func TestDB_EnableSTZeroInjection_AppendV2(t *testing.T) { t.Parallel() + // NOTE: Eventually wal sample and appendable sample should be the same. type appendableSample struct { - t int64 - st int64 - v float64 - lbls labels.Labels - h *histogram.Histogram - expectsError bool + st, t int64 + v float64 + lbls labels.Labels + h *histogram.Histogram } - testHistogram := tsdbutil.GenerateTestHistograms(1)[0] - zeroHistogram := &histogram.Histogram{} + testHistograms := tsdbutil.GenerateTestHistograms(2) + zeroHistogram := &histogram.Histogram{ + // The STZeroSample represents a counter reset by definition. + CounterResetHint: histogram.CounterReset, + // Replicate other fields to avoid needless chunk creation. + Schema: testHistograms[0].Schema, + ZeroThreshold: testHistograms[0].ZeroThreshold, + CustomValues: testHistograms[0].CustomValues, + } lbls := labelsForTest(t.Name(), 1) defLbls := labels.New(lbls[0]...) @@ -1163,7 +1034,7 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { testCases := []struct { name string inputSamples []appendableSample - expectedSamples []*walSample + expectedSamples []walSample expectedSeriesCount int }{ { @@ -1172,10 +1043,10 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { {t: 100, st: 1, v: 10, lbls: defLbls}, {t: 101, st: 1, v: 10, lbls: defLbls}, }, - expectedSamples: []*walSample{ - {t: 1, f: 0, lbls: defLbls}, - {t: 100, f: 10, lbls: defLbls}, - {t: 101, f: 10, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 1, f: 0, lbls: defLbls, ref: 1}, + {t: 100, f: 10, lbls: defLbls, ref: 1}, + {t: 101, f: 10, lbls: defLbls, ref: 1}, }, }, { @@ -1190,54 +1061,52 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { { t: 300, st: 230, - h: testHistogram, + h: testHistograms[0], lbls: defLbls, }, }, - expectedSamples: []*walSample{ - {t: 30, f: 0, lbls: defLbls}, - {t: 100, f: 20, lbls: defLbls}, - {t: 230, h: zeroHistogram, lbls: defLbls}, - {t: 300, h: testHistogram, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 30, f: 0, lbls: defLbls, ref: 1}, + {t: 100, f: 20, lbls: defLbls, ref: 1}, + {t: 230, h: zeroHistogram, lbls: defLbls, ref: 1}, + {t: 300, h: testHistograms[0], lbls: defLbls, ref: 1}, }, expectedSeriesCount: 1, }, { - name: "ST+float && ST+histogram samples with error", + name: "ST+float && ST+histogram samples with error; should be ignored", inputSamples: []appendableSample{ { // invalid ST - t: 100, - st: 100, - v: 10, - lbls: defLbls, - expectsError: true, + t: 100, + st: 100, + v: 10, + lbls: defLbls, }, { // invalid ST histogram - t: 300, - st: 300, - h: testHistogram, - lbls: defLbls, - expectsError: true, + t: 300, + st: 300, + h: testHistograms[0], + lbls: defLbls, }, }, - expectedSamples: []*walSample{ - {t: 100, f: 10, lbls: defLbls}, - {t: 300, h: testHistogram, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 100, f: 10, lbls: defLbls, ref: 1}, + {t: 300, h: testHistograms[0], lbls: defLbls, ref: 1}, }, expectedSeriesCount: 0, }, { name: "In order ct+normal sample/histogram", inputSamples: []appendableSample{ - {t: 100, h: testHistogram, st: 1, lbls: defLbls}, - {t: 101, h: testHistogram, st: 1, lbls: defLbls}, + {t: 100, h: testHistograms[0], st: 1, lbls: defLbls}, + {t: 101, h: testHistograms[1], st: 1, lbls: defLbls}, }, - expectedSamples: []*walSample{ - {t: 1, h: &histogram.Histogram{}}, - {t: 100, h: testHistogram}, - {t: 101, h: &histogram.Histogram{CounterResetHint: histogram.NotCounterReset}}, + expectedSamples: []walSample{ + {t: 1, h: zeroHistogram, lbls: defLbls, ref: 1}, + {t: 100, h: testHistograms[0], lbls: defLbls, ref: 1}, + {t: 101, h: testHistograms[1], lbls: defLbls, ref: 1}, }, }, { @@ -1248,12 +1117,12 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { {t: 180_000, st: 40_000, v: 10, lbls: defLbls}, {t: 50_000, st: 40_000, v: 10, lbls: defLbls}, }, - expectedSamples: []*walSample{ - {t: 40_000, f: 0, lbls: defLbls}, - {t: 50_000, f: 10, lbls: defLbls}, - {t: 60_000, f: 10, lbls: defLbls}, - {t: 120_000, f: 10, lbls: defLbls}, - {t: 180_000, f: 10, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 40_000, f: 0, lbls: defLbls, ref: 1}, + {t: 60_000, f: 10, lbls: defLbls, ref: 1}, + {t: 120_000, f: 10, lbls: defLbls, ref: 1}, + {t: 180_000, f: 10, lbls: defLbls, ref: 1}, + {t: 50_000, f: 10, lbls: defLbls, ref: 1}, // OOO sample. }, }, } @@ -1265,36 +1134,21 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { reg := prometheus.NewRegistry() opts := DefaultOptions() opts.OutOfOrderTimeWindow = 360_000 + opts.EnableSTAsZeroSample = true s := createTestAgentDB(t, reg, opts) - app := s.Appender(context.TODO()) for _, sample := range tc.inputSamples { - // We supposed to write a Histogram to the WAL - if sample.h != nil { - _, err := app.AppendHistogramSTZeroSample(0, sample.lbls, sample.t, sample.st, zeroHistogram, nil) - if !errors.Is(err, storage.ErrOutOfOrderST) { - require.Equal(t, sample.expectsError, err != nil, "expected error: %v, got: %v", sample.expectsError, err) - } - - _, err = app.AppendHistogram(0, sample.lbls, sample.t, sample.h, nil) - require.NoError(t, err) - } else { - // We supposed to write a float sample to the WAL - _, err := app.AppendSTZeroSample(0, sample.lbls, sample.t, sample.st) - if !errors.Is(err, storage.ErrOutOfOrderST) { - require.Equal(t, sample.expectsError, err != nil, "expected error: %v, got: %v", sample.expectsError, err) - } - - _, err = app.Append(0, sample.lbls, sample.t, sample.v) - require.NoError(t, err) - } + // Simulate one sample per series logic we have in all our ingestion paths in Prometheus. + app := s.AppenderV2(t.Context()) + _, err := app.Append(0, sample.lbls, sample.st, sample.t, sample.v, sample.h, nil, storage.AOptions{}) + require.NoError(t, err) + require.NoError(t, app.Commit()) } - require.NoError(t, app.Commit()) // Close the DB to ensure all data is flushed to the WAL require.NoError(t, s.Close()) - // Check that we dont have any OOO samples in the WAL by checking metrics + // Check that we don't have any OOO samples in the WAL by checking metrics families, err := reg.Gather() require.NoError(t, err, "failed to gather metrics") for _, f := range families { @@ -1303,94 +1157,8 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { } } - outputSamples := readWALSamples(t, s.wal.Dir()) - - require.Len(t, outputSamples, len(tc.expectedSamples), "Expected %d samples", len(tc.expectedSamples)) - - for i, expectedSample := range tc.expectedSamples { - for _, sample := range outputSamples { - if sample.t == expectedSample.t && sample.lbls.String() == expectedSample.lbls.String() { - if expectedSample.h != nil { - require.Equal(t, expectedSample.h, sample.h, "histogram value mismatch (sample index %d)", i) - } else { - require.Equal(t, expectedSample.f, sample.f, "value mismatch (sample index %d)", i) - } - } - } - } + got := readWALSamples(t, s.wal.Dir()) + testutil.RequireEqualWithOptions(t, tc.expectedSamples, got, cmp.Options{cmp.AllowUnexported(walSample{})}) }) } } - -func readWALSamples(t *testing.T, walDir string) []*walSample { - t.Helper() - sr, err := wlog.NewSegmentsReader(walDir) - require.NoError(t, err) - defer func(sr io.ReadCloser) { - err := sr.Close() - require.NoError(t, err) - }(sr) - - r := wlog.NewReader(sr) - dec := record.NewDecoder(labels.NewSymbolTable(), promslog.NewNopLogger()) - - var ( - samples []record.RefSample - histograms []record.RefHistogramSample - - lastSeries record.RefSeries - outputSamples = make([]*walSample, 0) - ) - - for r.Next() { - rec := r.Record() - switch dec.Type(rec) { - case record.Series: - series, err := dec.Series(rec, nil) - require.NoError(t, err) - lastSeries = series[0] - case record.Samples: - samples, err = dec.Samples(rec, samples[:0]) - require.NoError(t, err) - for _, s := range samples { - outputSamples = append(outputSamples, &walSample{ - t: s.T, - f: s.V, - lbls: lastSeries.Labels.Copy(), - ref: storage.SeriesRef(lastSeries.Ref), - }) - } - case record.HistogramSamples: - histograms, err = dec.HistogramSamples(rec, histograms[:0]) - require.NoError(t, err) - for _, h := range histograms { - outputSamples = append(outputSamples, &walSample{ - t: h.T, - h: h.H, - lbls: lastSeries.Labels.Copy(), - ref: storage.SeriesRef(lastSeries.Ref), - }) - } - } - } - - return outputSamples -} - -func BenchmarkCreateSeries(b *testing.B) { - s := createTestAgentDB(b, nil, DefaultOptions()) - defer s.Close() - - app := s.Appender(context.Background()).(*appender) - lbls := make([]labels.Labels, b.N) - - for i, l := range labelsForTest("benchmark", b.N) { - lbls[i] = labels.New(l...) - } - - b.ResetTimer() - - for _, l := range lbls { - app.getOrCreate(l) - } -} diff --git a/tsdb/agent/db_test.go b/tsdb/agent/db_test.go index 7409f79ec5..94e84fa2eb 100644 --- a/tsdb/agent/db_test.go +++ b/tsdb/agent/db_test.go @@ -24,6 +24,7 @@ import ( "testing" "time" + "github.com/google/go-cmp/cmp" "github.com/prometheus/client_golang/prometheus" dto "github.com/prometheus/client_model/go" "github.com/prometheus/common/model" @@ -1142,6 +1143,10 @@ type walSample struct { ref storage.SeriesRef } +// NOTE(bwplotka): This test is testing behaviour of storage.Appender interface against its invariants (see +// storage.Appender comment) around validation of the order of samples within a single Appender. This results +// in a slight bug in AppendSTZero* methods. We are leaving it as-is given the planned removal of AppenderV1 as +// per https://github.com/prometheus/prometheus/issues/17632. func TestDBStartTimestampSamplesIngestion(t *testing.T) { t.Parallel() @@ -1154,7 +1159,7 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { expectsError bool } - testHistogram := tsdbutil.GenerateTestHistograms(1)[0] + testHistograms := tsdbutil.GenerateTestHistograms(2) zeroHistogram := &histogram.Histogram{} lbls := labelsForTest(t.Name(), 1) @@ -1163,7 +1168,7 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { testCases := []struct { name string inputSamples []appendableSample - expectedSamples []*walSample + expectedSamples []walSample expectedSeriesCount int }{ { @@ -1172,10 +1177,10 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { {t: 100, st: 1, v: 10, lbls: defLbls}, {t: 101, st: 1, v: 10, lbls: defLbls}, }, - expectedSamples: []*walSample{ - {t: 1, f: 0, lbls: defLbls}, - {t: 100, f: 10, lbls: defLbls}, - {t: 101, f: 10, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 1, f: 0, lbls: defLbls, ref: 1}, + {t: 100, f: 10, lbls: defLbls, ref: 1}, + {t: 101, f: 10, lbls: defLbls, ref: 1}, }, }, { @@ -1190,15 +1195,15 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { { t: 300, st: 230, - h: testHistogram, + h: testHistograms[0], lbls: defLbls, }, }, - expectedSamples: []*walSample{ - {t: 30, f: 0, lbls: defLbls}, - {t: 100, f: 20, lbls: defLbls}, - {t: 230, h: zeroHistogram, lbls: defLbls}, - {t: 300, h: testHistogram, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 30, f: 0, lbls: defLbls, ref: 1}, + {t: 100, f: 20, lbls: defLbls, ref: 1}, + {t: 230, h: zeroHistogram, lbls: defLbls, ref: 1}, + {t: 300, h: testHistograms[0], lbls: defLbls, ref: 1}, }, expectedSeriesCount: 1, }, @@ -1217,27 +1222,27 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { // invalid ST histogram t: 300, st: 300, - h: testHistogram, + h: testHistograms[0], lbls: defLbls, expectsError: true, }, }, - expectedSamples: []*walSample{ - {t: 100, f: 10, lbls: defLbls}, - {t: 300, h: testHistogram, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 100, f: 10, lbls: defLbls, ref: 1}, + {t: 300, h: testHistograms[0], lbls: defLbls, ref: 1}, }, expectedSeriesCount: 0, }, { name: "In order ct+normal sample/histogram", inputSamples: []appendableSample{ - {t: 100, h: testHistogram, st: 1, lbls: defLbls}, - {t: 101, h: testHistogram, st: 1, lbls: defLbls}, + {t: 100, h: testHistograms[0], st: 1, lbls: defLbls}, + {t: 101, h: testHistograms[1], st: 1, lbls: defLbls}, }, - expectedSamples: []*walSample{ - {t: 1, h: &histogram.Histogram{}}, - {t: 100, h: testHistogram}, - {t: 101, h: &histogram.Histogram{CounterResetHint: histogram.NotCounterReset}}, + expectedSamples: []walSample{ + {t: 1, h: &histogram.Histogram{}, lbls: defLbls, ref: 1}, + {t: 100, h: testHistograms[0], lbls: defLbls, ref: 1}, + {t: 101, h: testHistograms[1], lbls: defLbls, ref: 1}, }, }, { @@ -1248,12 +1253,12 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { {t: 180_000, st: 40_000, v: 10, lbls: defLbls}, {t: 50_000, st: 40_000, v: 10, lbls: defLbls}, }, - expectedSamples: []*walSample{ - {t: 40_000, f: 0, lbls: defLbls}, - {t: 50_000, f: 10, lbls: defLbls}, - {t: 60_000, f: 10, lbls: defLbls}, - {t: 120_000, f: 10, lbls: defLbls}, - {t: 180_000, f: 10, lbls: defLbls}, + expectedSamples: []walSample{ + {t: 40_000, f: 0, lbls: defLbls, ref: 1}, + {t: 60_000, f: 10, lbls: defLbls, ref: 1}, + {t: 120_000, f: 10, lbls: defLbls, ref: 1}, + {t: 180_000, f: 10, lbls: defLbls, ref: 1}, + {t: 50_000, f: 10, lbls: defLbls, ref: 1}, // OOO sample. }, }, } @@ -1294,7 +1299,7 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { // Close the DB to ensure all data is flushed to the WAL require.NoError(t, s.Close()) - // Check that we dont have any OOO samples in the WAL by checking metrics + // Check that we don't have any OOO samples in the WAL by checking metrics families, err := reg.Gather() require.NoError(t, err, "failed to gather metrics") for _, f := range families { @@ -1303,26 +1308,13 @@ func TestDBStartTimestampSamplesIngestion(t *testing.T) { } } - outputSamples := readWALSamples(t, s.wal.Dir()) - - require.Len(t, outputSamples, len(tc.expectedSamples), "Expected %d samples", len(tc.expectedSamples)) - - for i, expectedSample := range tc.expectedSamples { - for _, sample := range outputSamples { - if sample.t == expectedSample.t && sample.lbls.String() == expectedSample.lbls.String() { - if expectedSample.h != nil { - require.Equal(t, expectedSample.h, sample.h, "histogram value mismatch (sample index %d)", i) - } else { - require.Equal(t, expectedSample.f, sample.f, "value mismatch (sample index %d)", i) - } - } - } - } + got := readWALSamples(t, s.wal.Dir()) + testutil.RequireEqualWithOptions(t, tc.expectedSamples, got, cmp.Options{cmp.AllowUnexported(walSample{})}) }) } } -func readWALSamples(t *testing.T, walDir string) []*walSample { +func readWALSamples(t *testing.T, walDir string) []walSample { t.Helper() sr, err := wlog.NewSegmentsReader(walDir) require.NoError(t, err) @@ -1339,7 +1331,7 @@ func readWALSamples(t *testing.T, walDir string) []*walSample { histograms []record.RefHistogramSample lastSeries record.RefSeries - outputSamples = make([]*walSample, 0) + outputSamples = make([]walSample, 0) ) for r.Next() { @@ -1353,7 +1345,7 @@ func readWALSamples(t *testing.T, walDir string) []*walSample { samples, err = dec.Samples(rec, samples[:0]) require.NoError(t, err) for _, s := range samples { - outputSamples = append(outputSamples, &walSample{ + outputSamples = append(outputSamples, walSample{ t: s.T, f: s.V, lbls: lastSeries.Labels.Copy(), @@ -1364,7 +1356,7 @@ func readWALSamples(t *testing.T, walDir string) []*walSample { histograms, err = dec.HistogramSamples(rec, histograms[:0]) require.NoError(t, err) for _, h := range histograms { - outputSamples = append(outputSamples, &walSample{ + outputSamples = append(outputSamples, walSample{ t: h.T, h: h.H, lbls: lastSeries.Labels.Copy(), @@ -1373,14 +1365,14 @@ func readWALSamples(t *testing.T, walDir string) []*walSample { } } } - return outputSamples } -func BenchmarkCreateSeries(b *testing.B) { +func BenchmarkGetOrCreate(b *testing.B) { s := createTestAgentDB(b, nil, DefaultOptions()) defer s.Close() + // NOTE: This benchmarks appenderBase, so it does not matter if it's V1 or V2. app := s.Appender(context.Background()).(*appender) lbls := make([]labels.Labels, b.N) diff --git a/tsdb/head_append_v2.go b/tsdb/head_append_v2.go index c5ed9898e9..95118528eb 100644 --- a/tsdb/head_append_v2.go +++ b/tsdb/head_append_v2.go @@ -167,11 +167,11 @@ func (a *headAppenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t i // an optimization for the more likely case. switch a.typesInBatch[s.ref] { case stHistogram, stCustomBucketHistogram: - return a.Append(ref, ls, st, t, 0, &histogram.Histogram{Sum: v}, nil, storage.AOptions{ + return a.Append(storage.SeriesRef(s.ref), ls, st, t, 0, &histogram.Histogram{Sum: v}, nil, storage.AOptions{ RejectOutOfOrder: opts.RejectOutOfOrder, }) case stFloatHistogram, stCustomBucketFloatHistogram: - return a.Append(ref, ls, st, t, 0, nil, &histogram.FloatHistogram{Sum: v}, storage.AOptions{ + return a.Append(storage.SeriesRef(s.ref), ls, st, t, 0, nil, &histogram.FloatHistogram{Sum: v}, storage.AOptions{ RejectOutOfOrder: opts.RejectOutOfOrder, }) } @@ -202,7 +202,7 @@ func (a *headAppenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t i if isStale { // For stale values we never attempt to process metadata/exemplars, claim the success. - return ref, nil + return storage.SeriesRef(s.ref), nil } // Append exemplars if any and if storage was configured for it. @@ -324,6 +324,7 @@ func (a *headAppenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemp if !errors.Is(err, storage.ErrDuplicateExemplar) && !errors.Is(err, storage.ErrExemplarsDisabled) { // Except duplicates, return partial errors. errs = append(errs, err) + continue } if !errors.Is(err, storage.ErrOutOfOrderExemplar) { a.head.logger.Debug("Error while adding an exemplar on AppendSample", "exemplars", fmt.Sprintf("%+v", e), "err", e) From 109f9409ed8df64561bd01870ad16165f036558b Mon Sep 17 00:00:00 2001 From: Aditya Tiwari <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Sun, 14 Dec 2025 23:11:19 +0530 Subject: [PATCH 133/439] [BugFix] UI : autocomplete metadata for OpenMetrics counter _total metrics (#17682) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: autocomplete metadata for OpenMetrics counter _total metrics Signed-off-by: ADITYATIWARI342005 <142050150+ADITYATIWARI342005@users.noreply.github.com> * fix/lint: properly indent the test file Signed-off-by: ADITYA TIWARI * fix/test: exclude expexcted block Signed-off-by: ADITYA TIWARI * fix/test: refacttoor comment for relevance Signed-off-by: ADITYA TIWARI * fix: add openmetrics _total metadata support to autocomplete by extending hybridcomplete’s suffix handling and adding a Jest test that covers the base-name-only metadata scenario for _total counters Signed-off-by: ADITYA TIWARI * fix: break long-comment to separate line, re-trigger workflow Signed-off-by: ADITYA TIWARI * fix: also strip _total when resolving metric metadata in Selector and MetricsExplorer Signed-off-by: ADITYA TIWARI --------- Signed-off-by: ADITYATIWARI342005 <142050150+ADITYATIWARI342005@users.noreply.github.com> Signed-off-by: ADITYA TIWARI --- .../src/pages/query/ExplainViews/Selector.tsx | 2 +- .../query/MetricsExplorer/MetricsExplorer.tsx | 2 +- .../src/complete/hybrid.test.ts | 33 +++++++++++++++++++ .../codemirror-promql/src/complete/hybrid.ts | 9 +++-- 4 files changed, 39 insertions(+), 7 deletions(-) diff --git a/web/ui/mantine-ui/src/pages/query/ExplainViews/Selector.tsx b/web/ui/mantine-ui/src/pages/query/ExplainViews/Selector.tsx index 2c564d3a4a..a83a0141d5 100644 --- a/web/ui/mantine-ui/src/pages/query/ExplainViews/Selector.tsx +++ b/web/ui/mantine-ui/src/pages/query/ExplainViews/Selector.tsx @@ -126,7 +126,7 @@ const matchingCriteriaList = ( }; const SelectorExplainView: FC = ({ node }) => { - const baseMetricName = node.name.replace(/(_count|_sum|_bucket)$/, ""); + const baseMetricName = node.name.replace(/(_count|_sum|_bucket|_total)$/, ""); const { lookbackDelta } = useSettings(); // Try to get metadata for the full unchanged metric name first. diff --git a/web/ui/mantine-ui/src/pages/query/MetricsExplorer/MetricsExplorer.tsx b/web/ui/mantine-ui/src/pages/query/MetricsExplorer/MetricsExplorer.tsx index 9c33a3df75..c351984698 100644 --- a/web/ui/mantine-ui/src/pages/query/MetricsExplorer/MetricsExplorer.tsx +++ b/web/ui/mantine-ui/src/pages/query/MetricsExplorer/MetricsExplorer.tsx @@ -73,7 +73,7 @@ const MetricsExplorer: FC = ({ // histogram/summary suffixes, it may be a metric that is not following naming // conventions, see https://github.com/prometheus/prometheus/issues/16907). data.data[m] ?? - data.data[m.replace(/(_count|_sum|_bucket)$/, "")] ?? [ + data.data[m.replace(/(_count|_sum|_bucket|_total)$/, "")] ?? [ { help: "unknown", type: "unknown", unit: "unknown" }, ] ); diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts index 587e9c5304..1f3985af63 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts @@ -29,6 +29,7 @@ import { import { EqlSingle, Neq } from '@prometheus-io/lezer-promql'; import { syntaxTree } from '@codemirror/language'; import { newCompleteStrategy } from './index'; +import nock from 'nock'; describe('analyzeCompletion test', () => { const testCases = [ @@ -1453,4 +1454,36 @@ describe('autocomplete promQL test', () => { expect(value.expectedResult).toEqual(result); }); }); + + it('online autocomplete of openmetrics counter', async () => { + const metricName = 'direct_notifications_total'; + const baseMetricName = 'direct_notifications'; + nock('http://localhost:8080') + .get('/api/v1/label/__name__/values') + .query(true) + .reply(200, { status: 'success', data: [metricName] }); + nock('http://localhost:8080') + .get('/api/v1/metadata') + .query(true) + .reply(200, { + status: 'success', + data: { + [baseMetricName]: [ + { + type: 'counter', + help: 'Number of direct notifications.', + unit: '', + }, + ], + }, + }); + const state = createEditorState(metricName); + const context = new CompletionContext(state, metricName.length, true); + const completion = newCompleteStrategy({ remote: { url: 'http://localhost:8080' } }); + const result = await completion.promQL(context); + // nock only mocks the HTTP endpoints; this test just ensures remote completion works + // when metadata for an OpenMetrics _total counter is stored under its base metric name. + expect(result).not.toBeNull(); + expect((result as NonNullable).options.length).toBeGreaterThan(0); + }); }); diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts index 814147e532..bb5f4d9d36 100644 --- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts +++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts @@ -689,11 +689,10 @@ export class HybridComplete implements CompleteStrategy { .then((metricMetadata) => { if (metricMetadata) { for (const [metricName, node] of metricCompletion) { - // First check if the full metric name has metadata (even if it has one of the - // histogram/summary suffixes, it may be a metric that is not following naming - // conventions, see https://github.com/prometheus/prometheus/issues/16907). - // Then fall back to the base metric name if full metadata doesn't exist. - const metadata = metricMetadata[metricName] ?? metricMetadata[metricName.replace(/(_count|_sum|_bucket)$/, '')]; + // First check if the full metric name has metadata (even if it has one of the histogram/summary/openmetrics suffixes + // it may be a metric that is not following naming conventions) + // Then fall back to the base metric name if full metadata doesn't exist + const metadata = metricMetadata[metricName] ?? metricMetadata[metricName.replace(/(_count|_sum|_bucket|_total)$/, '')]; if (metadata) { if (metadata.length > 1) { // it means the metricName has different possible helper and type From c94101d023462a0657b2ced8a3334fc69d1c0f80 Mon Sep 17 00:00:00 2001 From: NamanParlecha Date: Mon, 15 Dec 2025 14:01:17 +0530 Subject: [PATCH 134/439] TSDB: Option to configure TSDB Block Reload Interval (#16728) Add --storage.tsdb.block-reload-interval flag to configure TSDB block reload interval. --------- Signed-off-by: Naman-B-Parlecha Signed-off-by: NamanParlecha Co-authored-by: Arve Knudsen --- cmd/prometheus/main.go | 10 ++++++++++ tsdb/db.go | 9 ++++++++- tsdb/db_test.go | 39 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 1 deletion(-) diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go index 53379dc940..9d6d864971 100644 --- a/cmd/prometheus/main.go +++ b/cmd/prometheus/main.go @@ -486,6 +486,9 @@ func main() { serverOnlyFlag(a, "storage.tsdb.delay-compact-file.path", "Path to a JSON file with uploaded TSDB blocks e.g. Thanos shipper meta file. If set TSDB will only compact 1 level blocks that are marked as uploaded in that file, improving external storage integrations e.g. with Thanos sidecar. 1+ level compactions won't be delayed."). Default("").StringVar(&tsdbDelayCompactFilePath) + serverOnlyFlag(a, "storage.tsdb.block-reload-interval", "Interval at which to check for new or removed blocks in storage. Users who manually backfill or drop blocks must wait up to this duration before changes become available."). + Default("1m").Hidden().SetValue(&cfg.tsdb.BlockReloadInterval) + agentOnlyFlag(a, "storage.agent.path", "Base path for metrics storage."). Default("data-agent/").StringVar(&cfg.agentStoragePath) @@ -677,6 +680,10 @@ func main() { } cfg.tsdb.MaxExemplars = cfgFile.StorageConfig.ExemplarsConfig.MaxExemplars } + if cfg.tsdb.BlockReloadInterval < model.Duration(1*time.Second) { + logger.Warn("The option --storage.tsdb.block-reload-interval is set to a value less than 1s. Setting it to 1s to avoid overload.") + cfg.tsdb.BlockReloadInterval = model.Duration(1 * time.Second) + } if cfgFile.StorageConfig.TSDBConfig != nil { cfg.tsdb.OutOfOrderTimeWindow = cfgFile.StorageConfig.TSDBConfig.OutOfOrderTimeWindow if cfgFile.StorageConfig.TSDBConfig.Retention != nil { @@ -1353,6 +1360,7 @@ func main() { "RetentionDuration", cfg.tsdb.RetentionDuration, "WALSegmentSize", cfg.tsdb.WALSegmentSize, "WALCompressionType", cfg.tsdb.WALCompressionType, + "BlockReloadInterval", cfg.tsdb.BlockReloadInterval, ) startTimeMargin := int64(2 * time.Duration(cfg.tsdb.MinBlockDuration).Seconds() * 1000) @@ -1910,6 +1918,7 @@ type tsdbOptions struct { EnableOverlappingCompaction bool UseUncachedIO bool BlockCompactionExcludeFunc tsdb.BlockExcludeFilterFunc + BlockReloadInterval model.Duration } func (opts tsdbOptions) ToTSDBOptions() tsdb.Options { @@ -1934,6 +1943,7 @@ func (opts tsdbOptions) ToTSDBOptions() tsdb.Options { EnableOverlappingCompaction: opts.EnableOverlappingCompaction, UseUncachedIO: opts.UseUncachedIO, BlockCompactionExcludeFunc: opts.BlockCompactionExcludeFunc, + BlockReloadInterval: time.Duration(opts.BlockReloadInterval), FeatureRegistry: features.DefaultRegistry, } } diff --git a/tsdb/db.go b/tsdb/db.go index cd1a090686..f765710dd7 100644 --- a/tsdb/db.go +++ b/tsdb/db.go @@ -94,6 +94,7 @@ func DefaultOptions() *Options { CompactionDelayMaxPercent: DefaultCompactionDelayMaxPercent, CompactionDelay: time.Duration(0), PostingsDecoderFactory: DefaultPostingsDecoderFactory, + BlockReloadInterval: 1 * time.Minute, } } @@ -239,6 +240,9 @@ type Options struct { // It's passed down to the TSDB compactor. BlockCompactionExcludeFunc BlockExcludeFilterFunc + // BlockReloadInterval is the interval at which blocks are reloaded. + BlockReloadInterval time.Duration + // FeatureRegistry is used to register TSDB features. FeatureRegistry features.Collector } @@ -844,6 +848,9 @@ func validateOpts(opts *Options, rngs []int64) (*Options, []int64) { if opts.OutOfOrderTimeWindow < 0 { opts.OutOfOrderTimeWindow = 0 } + if opts.BlockReloadInterval < 1*time.Second { + opts.BlockReloadInterval = 1 * time.Second + } if len(rngs) == 0 { // Start with smallest block duration and create exponential buckets until the exceed the @@ -1131,7 +1138,7 @@ func (db *DB) run(ctx context.Context) { } select { - case <-time.After(1 * time.Minute): + case <-time.After(db.opts.BlockReloadInterval): db.cmtx.Lock() if err := db.reloadBlocks(); err != nil { db.logger.Error("reloadBlocks", "err", err) diff --git a/tsdb/db_test.go b/tsdb/db_test.go index 4e084ef0d8..4612eace3b 100644 --- a/tsdb/db_test.go +++ b/tsdb/db_test.go @@ -9284,3 +9284,42 @@ func TestBlockClosingBlockedDuringRemoteRead(t *testing.T) { case <-blockClosed: } } + +func TestBlockReloadInterval(t *testing.T) { + t.Parallel() + + cases := []struct { + name string + reloadInterval time.Duration + expectedReloads float64 + }{ + { + name: "extremely small interval", + reloadInterval: 1 * time.Millisecond, + expectedReloads: 5, + }, + { + name: "one second interval", + reloadInterval: 1 * time.Second, + expectedReloads: 5, + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + t.Parallel() + db := newTestDB(t, withOpts(&Options{ + BlockReloadInterval: c.reloadInterval, + })) + if c.reloadInterval < 1*time.Second { + require.Equal(t, 1*time.Second, db.opts.BlockReloadInterval, "interval should be clamped to minimum of 1 second") + } + require.Equal(t, float64(1), prom_testutil.ToFloat64(db.metrics.reloads), "there should be one initial reload") + require.Eventually(t, func() bool { + return prom_testutil.ToFloat64(db.metrics.reloads) == c.expectedReloads + }, + 5*time.Second, + 100*time.Millisecond, + ) + }) + } +} From 29878f7b91cf485448ccd571c7ac3d35ec43dabc Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Mon, 15 Dec 2025 11:56:34 +0100 Subject: [PATCH 135/439] promql: Optimize mergeSeriesWithSameLabelset for common case Add fast path that returns early when no duplicate labelsets exist, avoiding allocations in the common case. For the merge case, simplify collision detection by checking for duplicate timestamps after sorting instead of building a timestamp map, reducing memory overhead. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- promql/engine.go | 71 ++++++++++++++++-------------------------------- 1 file changed, 24 insertions(+), 47 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 37c4e12cd9..07fb03d66c 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -3866,6 +3866,13 @@ func (ev *evaluator) mergeSeriesWithSameLabelset(mat Matrix) Matrix { return mat } + // Fast path: check if there are any duplicate labelsets without allocating. + // This is the common case and we want to avoid allocations. + if !mat.ContainsSameLabelset() { + return mat + } + + // Slow path: there are duplicates, so we need to merge series with non-overlapping timestamps. // Group series by their labelset hash. seriesByHash := make(map[uint64][]int) for i := range mat { @@ -3873,62 +3880,20 @@ func (ev *evaluator) mergeSeriesWithSameLabelset(mat Matrix) Matrix { seriesByHash[hash] = append(seriesByHash[hash], i) } - // Check if any merging is needed. - needsMerge := false - for _, indices := range seriesByHash { - if len(indices) > 1 { - needsMerge = true - break - } - } - - if !needsMerge { - return mat - } - // Merge series with the same labelset. merged := make(Matrix, 0, len(seriesByHash)) for _, indices := range seriesByHash { - base := mat[indices[0]] - if len(indices) == 1 { // No collision, add as-is. - merged = append(merged, base) + merged = append(merged, mat[indices[0]]) continue } - // Multiple series with the same labelset - check for overlaps and merge. - // Build a set of timestamps to detect overlaps. - timestamps := make(map[int64]struct{}, len(base.Floats)+len(base.Histograms)) - for _, p := range base.Floats { - timestamps[p.T] = struct{}{} - } - for _, p := range base.Histograms { - timestamps[p.T] = struct{}{} - } - - // Merge remaining series, checking for timestamp overlaps. + // Multiple series with the same labelset - merge all samples. + base := mat[indices[0]] for _, idx := range indices[1:] { - series := mat[idx] - - // Check floats for overlaps. - for _, p := range series.Floats { - if _, exists := timestamps[p.T]; exists { - ev.errorf("vector cannot contain metrics with the same labelset") - } - timestamps[p.T] = struct{}{} - } - // Check histograms for overlaps. - for _, p := range series.Histograms { - if _, exists := timestamps[p.T]; exists { - ev.errorf("vector cannot contain metrics with the same labelset") - } - timestamps[p.T] = struct{}{} - } - - // No overlaps, merge the samples. - base.Floats = append(base.Floats, series.Floats...) - base.Histograms = append(base.Histograms, series.Histograms...) + base.Floats = append(base.Floats, mat[idx].Floats...) + base.Histograms = append(base.Histograms, mat[idx].Histograms...) } // Sort merged samples by timestamp. @@ -3939,6 +3904,18 @@ func (ev *evaluator) mergeSeriesWithSameLabelset(mat Matrix) Matrix { return base.Histograms[i].T < base.Histograms[j].T }) + // Check for duplicate timestamps in sorted samples. + for i := 1; i < len(base.Floats); i++ { + if base.Floats[i].T == base.Floats[i-1].T { + ev.errorf("vector cannot contain metrics with the same labelset") + } + } + for i := 1; i < len(base.Histograms); i++ { + if base.Histograms[i].T == base.Histograms[i-1].T { + ev.errorf("vector cannot contain metrics with the same labelset") + } + } + merged = append(merged, base) } From 872980e3bf311e1631ae596167a8ba04f272f317 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Mon, 15 Dec 2025 12:46:58 +0100 Subject: [PATCH 136/439] chore(deps): update module golang.org/x/crypto to v0.45.0 [security] (#17690) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- go.mod | 16 ++++++++-------- go.sum | 32 ++++++++++++++++---------------- internal/tools/go.mod | 16 ++++++++-------- internal/tools/go.sum | 32 ++++++++++++++++---------------- 4 files changed, 48 insertions(+), 48 deletions(-) diff --git a/go.mod b/go.mod index 9cf136eb39..7d830e86a2 100644 --- a/go.mod +++ b/go.mod @@ -87,9 +87,9 @@ require ( go.uber.org/multierr v1.11.0 go.yaml.in/yaml/v2 v2.4.3 golang.org/x/oauth2 v0.32.0 - golang.org/x/sync v0.17.0 - golang.org/x/sys v0.37.0 - golang.org/x/text v0.30.0 + golang.org/x/sync v0.18.0 + golang.org/x/sys v0.38.0 + golang.org/x/text v0.31.0 google.golang.org/api v0.252.0 google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 google.golang.org/grpc v1.76.0 @@ -219,13 +219,13 @@ require ( go.opentelemetry.io/collector/pipeline v1.45.0 // indirect go.opentelemetry.io/proto/otlp v1.7.1 // indirect go.uber.org/zap v1.27.0 // indirect - golang.org/x/crypto v0.43.0 // indirect + golang.org/x/crypto v0.45.0 // indirect golang.org/x/exp v0.0.0-20250808145144-a408d31f581a // indirect - golang.org/x/mod v0.28.0 // indirect - golang.org/x/net v0.46.0 // indirect - golang.org/x/term v0.36.0 // indirect + golang.org/x/mod v0.29.0 // indirect + golang.org/x/net v0.47.0 // indirect + golang.org/x/term v0.37.0 // indirect golang.org/x/time v0.13.0 // indirect - golang.org/x/tools v0.37.0 // indirect + golang.org/x/tools v0.38.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797 // indirect gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect diff --git a/go.sum b/go.sum index 579e86ca58..70720765e7 100644 --- a/go.sum +++ b/go.sum @@ -594,14 +594,14 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04= -golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0= +golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= +golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= golang.org/x/exp v0.0.0-20250808145144-a408d31f581a h1:Y+7uR/b1Mw2iSXZ3G//1haIiSElDQZ8KWh0h+sZPG90= golang.org/x/exp v0.0.0-20250808145144-a408d31f581a/go.mod h1:rT6SFzZ7oxADUDx58pcaKFTcZ+inxAa9fTrYx/uVYwg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.28.0 h1:gQBtGhjxykdjY9YhZpSlZIsbnaE2+PgjfLWUQTnoZ1U= -golang.org/x/mod v0.28.0/go.mod h1:yfB/L0NOf/kmEbXjzCPOx1iK1fRutOydrCMsqRhEBxI= +golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA= +golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= @@ -612,8 +612,8 @@ golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLL golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy7fQ90B1CfIiPueXVOjqfkSzI8= -golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4= -golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210= +golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= +golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -622,8 +622,8 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= -golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I= +golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -649,17 +649,17 @@ golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ= -golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= +golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.36.0 h1:zMPR+aF8gfksFprF/Nc/rd1wRS1EI6nDBGyWAvDzx2Q= -golang.org/x/term v0.36.0/go.mod h1:Qu394IJq6V6dCBRgwqshf3mPF85AqzYEzofzRdZkWss= +golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU= +golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k= -golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM= +golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= +golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -668,8 +668,8 @@ golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtn golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE= -golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w= +golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ= +golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs= golang.org/x/tools/godoc v0.1.0-deprecated h1:o+aZ1BOj6Hsx/GBdJO/s815sqftjSnrZZwyYTHODvtk= golang.org/x/tools/godoc v0.1.0-deprecated/go.mod h1:qM63CriJ961IHWmnWa9CjZnBndniPt4a3CK0PVB9bIg= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= diff --git a/internal/tools/go.mod b/internal/tools/go.mod index a343a56834..e4817a35cd 100644 --- a/internal/tools/go.mod +++ b/internal/tools/go.mod @@ -89,15 +89,15 @@ require ( go.uber.org/multierr v1.11.0 // indirect go.uber.org/zap v1.27.0 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect - golang.org/x/crypto v0.42.0 // indirect + golang.org/x/crypto v0.45.0 // indirect golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b // indirect - golang.org/x/mod v0.28.0 // indirect - golang.org/x/net v0.44.0 // indirect - golang.org/x/sync v0.17.0 // indirect - golang.org/x/sys v0.36.0 // indirect - golang.org/x/term v0.35.0 // indirect - golang.org/x/text v0.29.0 // indirect - golang.org/x/tools v0.37.0 // indirect + golang.org/x/mod v0.29.0 // indirect + golang.org/x/net v0.47.0 // indirect + golang.org/x/sync v0.18.0 // indirect + golang.org/x/sys v0.38.0 // indirect + golang.org/x/term v0.37.0 // indirect + golang.org/x/text v0.31.0 // indirect + golang.org/x/tools v0.38.0 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4 // indirect google.golang.org/grpc v1.75.1 // indirect diff --git a/internal/tools/go.sum b/internal/tools/go.sum index 3a2788f200..26df5c98a2 100644 --- a/internal/tools/go.sum +++ b/internal/tools/go.sum @@ -221,25 +221,25 @@ go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.42.0 h1:chiH31gIWm57EkTXpwnqf8qeuMUi0yekh6mT2AvFlqI= -golang.org/x/crypto v0.42.0/go.mod h1:4+rDnOTJhQCx2q7/j6rAN5XDw8kPjeaXEUR2eL94ix8= +golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= +golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b h1:DXr+pvt3nC887026GRP39Ej11UATqWDmWuS99x26cD0= golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b/go.mod h1:4QTo5u+SEIbbKW1RacMZq1YEfOBqeXa19JeshGi+zc4= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.28.0 h1:gQBtGhjxykdjY9YhZpSlZIsbnaE2+PgjfLWUQTnoZ1U= -golang.org/x/mod v0.28.0/go.mod h1:yfB/L0NOf/kmEbXjzCPOx1iK1fRutOydrCMsqRhEBxI= +golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA= +golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.44.0 h1:evd8IRDyfNBMBTTY5XRF1vaZlD+EmWx6x8PkhR04H/I= -golang.org/x/net v0.44.0/go.mod h1:ECOoLqd5U3Lhyeyo/QDCEVQ4sNgYsqvCZ722XogGieY= +golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= +golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= -golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I= +golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -247,22 +247,22 @@ golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k= -golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= -golang.org/x/term v0.35.0 h1:bZBVKBudEyhRcajGcNc3jIfWPqV4y/Kt2XcoigOWtDQ= -golang.org/x/term v0.35.0/go.mod h1:TPGtkTLesOwf2DE8CgVYiZinHAOuy5AYUYT1lENIZnA= +golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= +golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU= +golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk= -golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4= +golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= +golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs= golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE= -golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w= +golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ= +golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= From da253bddf507f54ff2cf452efb7201e9515f2676 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Fri, 12 Dec 2025 22:47:03 +0000 Subject: [PATCH 137/439] fix: ensure remote PRWv1 write handler does not send false 0 response headers Signed-off-by: bwplotka --- go.mod | 4 ++-- go.sum | 8 ++++---- storage/remote/write_handler_test.go | 17 ++++++++--------- 3 files changed, 14 insertions(+), 15 deletions(-) diff --git a/go.mod b/go.mod index 9cf136eb39..35904d311b 100644 --- a/go.mod +++ b/go.mod @@ -43,7 +43,7 @@ require ( github.com/hetznercloud/hcloud-go/v2 v2.29.0 github.com/ionos-cloud/sdk-go/v6 v6.3.4 github.com/json-iterator/go v1.1.12 - github.com/klauspost/compress v1.18.1 + github.com/klauspost/compress v1.18.2 github.com/kolo/xmlrpc v0.0.0-20220921171641-a4b6fa1dd06b github.com/linode/linodego v1.60.0 github.com/miekg/dns v1.1.68 @@ -56,7 +56,7 @@ require ( github.com/ovh/go-ovh v1.9.0 github.com/prometheus/alertmanager v0.28.1 github.com/prometheus/client_golang v1.23.2 - github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a + github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.67.4 github.com/prometheus/common/assets v0.2.0 diff --git a/go.sum b/go.sum index 579e86ca58..0aa41a311b 100644 --- a/go.sum +++ b/go.sum @@ -302,8 +302,8 @@ github.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRt github.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.18.1 h1:bcSGx7UbpBqMChDtsF28Lw6v/G94LPrrbMbdC3JH2co= -github.com/klauspost/compress v1.18.1/go.mod h1:ZQFFVG+MdnR0P+l6wpXgIL4NTtwiKIdBnrBd8Nrxr+0= +github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= +github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo= github.com/knadh/koanf/maps v0.1.2/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI= github.com/knadh/koanf/providers/confmap v1.0.0 h1:mHKLJTE7iXEys6deO5p6olAiZdG5zwp8Aebir+/EaRE= @@ -438,8 +438,8 @@ github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5Fsn github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU= github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= -github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a h1:RF1vfKM34/3DbGNis22BGd6sDDY3XBi0eM7pYqmOEO0= -github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a/go.mod h1:FGJuwvfcPY0V5enm+w8zF1RNS062yugQtPPQp1c4Io4= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca h1:BOxmsLoL2ymn8lXJtorca7N/m+2vDQUDoEtPjf0iAxA= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca/go.mod h1:gndBHh3ZdjBozGcGrjUYjN3UJLRS3l2drALtu4lUt+k= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go index 2610142db9..82cb000be7 100644 --- a/storage/remote/write_handler_test.go +++ b/storage/remote/write_handler_test.go @@ -1520,19 +1520,23 @@ func TestRemoteWriteHandler_ResponseStats(t *testing.T) { for _, tt := range []struct { msgType remoteapi.WriteMessageType + payload []byte forceInjectHeaders bool expectHeaders bool }{ { msgType: remoteapi.WriteV1MessageType, + payload: payloadV1, }, { msgType: remoteapi.WriteV1MessageType, + payload: payloadV1, forceInjectHeaders: true, expectHeaders: true, }, { msgType: remoteapi.WriteV2MessageType, + payload: payloadV2, expectHeaders: true, }, } { @@ -1552,11 +1556,11 @@ func TestRemoteWriteHandler_ResponseStats(t *testing.T) { if tt.forceInjectHeaders { base := handler handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - base.ServeHTTP(w, r) - // Inject response header. This simulates PRWv1 server that uses PRWv2 response headers // for confirmation of samples. This is not against spec and we support it. - w.Header().Set(rw20WrittenSamplesHeader, fmt.Sprintf("%d", len(appendable.samples))) + w.Header().Set(rw20WrittenSamplesHeader, "2") + + base.ServeHTTP(w, r) }) } @@ -1565,14 +1569,9 @@ func TestRemoteWriteHandler_ResponseStats(t *testing.T) { // Send message and do the parse response flow. c := &Client{Client: srv.Client(), urlString: srv.URL, timeout: 5 * time.Minute, writeProtoMsg: tt.msgType} - payload := payloadV2 - if tt.msgType == remoteapi.WriteV1MessageType { - payload = payloadV1 - } - stats, err := c.Store(t.Context(), payload, 0) + stats, err := c.Store(t.Context(), tt.payload, 0) require.NoError(t, err) - fmt.Println(stats) if tt.expectHeaders { require.True(t, stats.Confirmed) require.Equal(t, len(appendable.samples), stats.Samples) From 2e296c11ecdab0c96d2aecafbc221711e1aa9a8b Mon Sep 17 00:00:00 2001 From: bwplotka Date: Mon, 15 Dec 2025 15:41:53 +0000 Subject: [PATCH 138/439] chore: cut 3.8.1 Signed-off-by: bwplotka --- CHANGELOG.md | 6 +++++- VERSION | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3304339867..2da2d46a0c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,9 +2,13 @@ ## main / unreleased +## 3.8.1 / 2025-12-16 + +* [BUGFIX] remote: Fix Remote Write receiver, so it does not send wrong response headers for v1 flow and cause Prometheus senders to emit false partial error log and metrics. #17683 + ## 3.8.0 / 2025-11-28 -* [CHANGE] Remote-write 2 (receiving): Update to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md). "created timestamp" (CT) is now called "start timestamp" (ST). #17411 +* [CHANGE] remote: Update Remote Write receiving to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md). "created timestamp" (CT) is now called "start timestamp" (ST). #17411 * [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #17287 * [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #17592 * [FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. #16483 diff --git a/VERSION b/VERSION index 19811903a7..f280719674 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.8.0 +3.8.1 From 089ed0b083d7c4234e880cf612fa019d409e8137 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Mon, 15 Dec 2025 17:10:18 +0100 Subject: [PATCH 139/439] chore: Update OSS-Fuzz CIFuzz actions to latest version Update google/oss-fuzz/infra/cifuzz actions from cafd7a0e to 537c8005. Prior to this PR, the OSS-Fuzz builder environment was updated and now produces binaries that require GLIBC 2.32 or newer. However, the fuzzing runtime was based on Ubuntu 20.04 (GLIBC 2.31), while the builder itself runs in a more recent environment. This mismatch caused compatibility issues that this PR solves. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- .github/workflows/fuzzing.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/fuzzing.yml b/.github/workflows/fuzzing.yml index 3d3aa82d1c..24702c2920 100644 --- a/.github/workflows/fuzzing.yml +++ b/.github/workflows/fuzzing.yml @@ -10,12 +10,12 @@ jobs: steps: - name: Build Fuzzers id: build - uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@cafd7a0eb8ecb4e007c56897996a9b65c49c972f # master + uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@537c8005ba4c9de026b2fa3550663280d25d6175 # master with: oss-fuzz-project-name: "prometheus" dry-run: false - name: Run Fuzzers - uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@cafd7a0eb8ecb4e007c56897996a9b65c49c972f # master + uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@537c8005ba4c9de026b2fa3550663280d25d6175 # master # Note: Regularly check for updates to the pinned commit hash at: # https://github.com/google/oss-fuzz/tree/master/infra/cifuzz/actions/run_fuzzers with: From 7739353f5d8176ebc4c46668aea5de88518e3302 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Mon, 15 Dec 2025 16:41:13 +0000 Subject: [PATCH 140/439] chore: upgrade npm Signed-off-by: bwplotka --- web/ui/mantine-ui/package.json | 4 ++-- web/ui/module/codemirror-promql/package.json | 4 ++-- web/ui/module/lezer-promql/package.json | 2 +- web/ui/package-lock.json | 4 ++-- web/ui/package.json | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json index 7ec13b1b8d..baf47d6f6b 100644 --- a/web/ui/mantine-ui/package.json +++ b/web/ui/mantine-ui/package.json @@ -1,7 +1,7 @@ { "name": "@prometheus-io/mantine-ui", "private": true, - "version": "0.308.0", + "version": "0.308.1", "type": "module", "scripts": { "start": "vite", @@ -28,7 +28,7 @@ "@microsoft/fetch-event-source": "^2.0.1", "@nexucis/fuzzy": "^0.5.1", "@nexucis/kvsearch": "^0.9.1", - "@prometheus-io/codemirror-promql": "0.308.0", + "@prometheus-io/codemirror-promql": "0.308.1", "@reduxjs/toolkit": "^2.10.1", "@tabler/icons-react": "^3.35.0", "@tanstack/react-query": "^5.90.7", diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json index ee7bcc045f..5f632320bd 100644 --- a/web/ui/module/codemirror-promql/package.json +++ b/web/ui/module/codemirror-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/codemirror-promql", - "version": "0.308.0", + "version": "0.308.1", "description": "a CodeMirror mode for the PromQL language", "types": "dist/esm/index.d.ts", "module": "dist/esm/index.js", @@ -29,7 +29,7 @@ }, "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md", "dependencies": { - "@prometheus-io/lezer-promql": "0.308.0", + "@prometheus-io/lezer-promql": "0.308.1", "lru-cache": "^11.2.2" }, "devDependencies": { diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json index 034ead9741..85cc4c50ed 100644 --- a/web/ui/module/lezer-promql/package.json +++ b/web/ui/module/lezer-promql/package.json @@ -1,6 +1,6 @@ { "name": "@prometheus-io/lezer-promql", - "version": "0.308.0", + "version": "0.308.1", "description": "lezer-based PromQL grammar", "main": "dist/index.cjs", "type": "module", diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json index 7f2961784b..883ee7aaee 100644 --- a/web/ui/package-lock.json +++ b/web/ui/package-lock.json @@ -1,12 +1,12 @@ { "name": "prometheus-io", - "version": "0.308.0", + "version": "0.308.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "prometheus-io", - "version": "0.308.0", + "version": "0.308.1", "workspaces": [ "mantine-ui", "module/*" diff --git a/web/ui/package.json b/web/ui/package.json index 5023d1d21b..44d0b52ce0 100644 --- a/web/ui/package.json +++ b/web/ui/package.json @@ -1,7 +1,7 @@ { "name": "prometheus-io", "description": "Monorepo for the Prometheus UI", - "version": "0.308.0", + "version": "0.308.1", "private": true, "scripts": { "build": "bash build_ui.sh --all", From bf552e66c07b722e16cef43bf8584580e6d3cda4 Mon Sep 17 00:00:00 2001 From: Bartlomiej Plotka Date: Mon, 15 Dec 2025 20:41:23 +0000 Subject: [PATCH 141/439] Merge pull request #17695 from roidelapluie/roidelapluie/fixfuzzing chore: Update OSS-Fuzz CIFuzz actions to latest version --- .github/workflows/fuzzing.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/fuzzing.yml b/.github/workflows/fuzzing.yml index 3d3aa82d1c..24702c2920 100644 --- a/.github/workflows/fuzzing.yml +++ b/.github/workflows/fuzzing.yml @@ -10,12 +10,12 @@ jobs: steps: - name: Build Fuzzers id: build - uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@cafd7a0eb8ecb4e007c56897996a9b65c49c972f # master + uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@537c8005ba4c9de026b2fa3550663280d25d6175 # master with: oss-fuzz-project-name: "prometheus" dry-run: false - name: Run Fuzzers - uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@cafd7a0eb8ecb4e007c56897996a9b65c49c972f # master + uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@537c8005ba4c9de026b2fa3550663280d25d6175 # master # Note: Regularly check for updates to the pinned commit hash at: # https://github.com/google/oss-fuzz/tree/master/infra/cifuzz/actions/run_fuzzers with: From 39c7fca0e9374ab3c033963dde98c8ce1371feae Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Tue, 16 Dec 2025 12:40:45 +0100 Subject: [PATCH 142/439] Move CODEOWNERS from .github to root (#17664) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Move CODEOWNERS from .github to root The default maintainers are added to each line to make sure that prometheus team members can still approve if: - specific code owner is not responding - we require code owner review (not currently) Signed-off-by: György Krajcsovits * Apply suggestions from code review Co-authored-by: Bartlomiej Plotka Signed-off-by: George Krajcsovits * Update CODEOWNERS Co-authored-by: Bartlomiej Plotka Signed-off-by: George Krajcsovits * add notice about keeping the files in sync Signed-off-by: György Krajcsovits --------- Signed-off-by: György Krajcsovits Signed-off-by: George Krajcsovits Co-authored-by: Bartlomiej Plotka --- .github/CODEOWNERS | 10 ---------- CODEOWNERS | 26 ++++++++++++++++++++++---- MAINTAINERS.md | 2 ++ 3 files changed, 24 insertions(+), 14 deletions(-) delete mode 100644 .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS deleted file mode 100644 index 7f7cec9cda..0000000000 --- a/.github/CODEOWNERS +++ /dev/null @@ -1,10 +0,0 @@ -/web/ui @juliusv -/web/ui/module @juliusv @nexucis -/storage/remote @cstyan @bwplotka @tomwilkie -/storage/remote/otlptranslator @aknuds1 @jesusvazquez -/discovery/kubernetes @brancz -/tsdb @jesusvazquez -/promql @roidelapluie -/cmd/promtool @dgl -/documentation/prometheus-mixin @metalmatze - diff --git a/CODEOWNERS b/CODEOWNERS index c5b7f25349..7dda7dc1a4 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -2,9 +2,27 @@ # They are code owners by default for the whole repo. * @prometheus/default-maintainers -# Example adding a dedicated maintainer for AWS SD, and also "default -# maintainers" so that they do not need to bypass codeowners check to merge -# something. -# Example comes from +# +# Please keep this file in sync with the MAINTAINERS.md file! +# + +# Subsystems. +/Makefile @simonpasquier @SuperQ @prometheus/default-maintainers +/cmd/promtool @dgl @prometheus/default-maintainers +/documentation/prometheus-mixin @metalmatze @prometheus/default-maintainers +/model/histogram @beorn7 @krajorama @prometheus/default-maintainers +/web/ui @juliusv @prometheus/default-maintainers +/web/ui/module @juliusv @nexucis @prometheus/default-maintainers +/promql @roidelapluie @prometheus/default-maintainers +/storage/remote @cstyan @bwplotka @tomwilkie @npazosmendez @alexgreenbank @prometheus/default-maintainers +/storage/remote/otlptranslator @aknuds1 @jesusvazquez @ArthurSens @prometheus/default-maintainers +/tsdb @jesusvazquez @codesome @bwplotka @krajorama @prometheus/default-maintainers + +# Service discovery. +/discovery/kubernetes @brancz @prometheus/default-maintainers +/discovery/stackit @jkroepke @prometheus/default-maintainers +# Pending # https://github.com/prometheus/prometheus/pull/17105#issuecomment-3248209452 # /discovery/aws/ @matt-gp @prometheus/default-maintainers +# https://github.com/prometheus/prometheus/pull/15212#issuecomment-3575225179 +# /discovery/aliyun @KeyOfSpectator @prometheus/default-maintainers diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 8d107b9774..c91b270bc6 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -1,5 +1,7 @@ # Maintainers +## Please keep this file in sync with the CODEOWNERS file! + General maintainers: * Bryan Boreham (bjboreham@gmail.com / @bboreham) * Ayoub Mrini (ayoubmrini424@gmail.com / @machine424) From 5b299ef99eb14b6b99d3fbc2af828c85eb763e4d Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Sun, 14 Dec 2025 20:18:06 +0000 Subject: [PATCH 143/439] fix/promql/parser: Fix utf-8 label quoting in format_query endpoint Signed-off-by: ADITYA TIWARI --- promql/parser/printer.go | 19 ++++++++++++--- promql/parser/printer_test.go | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/promql/parser/printer.go b/promql/parser/printer.go index 961167428b..67b13eaf12 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -109,7 +109,7 @@ func writeLabels(b *bytes.Buffer, ss []string) { if i > 0 { b.WriteString(", ") } - if !model.LegacyValidation.IsValidMetricName(s) { + if !model.LegacyValidation.IsValidLabelName(s) { b.Write(strconv.AppendQuote(b.AvailableBuffer(), s)) } else { b.WriteString(s) @@ -145,6 +145,19 @@ func (node *BinaryExpr) ShortString() string { return node.Op.String() + node.returnBool() + node.getMatchingStr() } +// joinLabels joins label names, quoting them if they are not valid legacy label names. +func joinLabels(labels []string) string { + quoted := make([]string, 0, len(labels)) + for _, label := range labels { + if model.LegacyValidation.IsValidLabelName(label) { + quoted = append(quoted, label) + } else { + quoted = append(quoted, strconv.Quote(label)) + } + } + return strings.Join(quoted, ", ") +} + func (node *BinaryExpr) getMatchingStr() string { matching := "" vm := node.VectorMatching @@ -154,7 +167,7 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.On { vmTag = "on" } - matching = fmt.Sprintf(" %s (%s)", vmTag, strings.Join(vm.MatchingLabels, ", ")) + matching = fmt.Sprintf(" %s (%s)", vmTag, joinLabels(vm.MatchingLabels)) } if vm.Card == CardManyToOne || vm.Card == CardOneToMany { @@ -162,7 +175,7 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.Card == CardManyToOne { vmCard = "left" } - matching += fmt.Sprintf(" group_%s (%s)", vmCard, strings.Join(vm.Include, ", ")) + matching += fmt.Sprintf(" group_%s (%s)", vmCard, joinLabels(vm.Include)) } } return matching diff --git a/promql/parser/printer_test.go b/promql/parser/printer_test.go index b28da988da..c9a3cb35e8 100644 --- a/promql/parser/printer_test.go +++ b/promql/parser/printer_test.go @@ -269,6 +269,10 @@ func TestExprString(t *testing.T) { { in: `predict_linear(foo[1h], 3000)`, }, + { + in: `sum by("üüü") (foo)`, + out: `sum by ("üüü") (foo)`, + }, } EnableExtendedRangeSelectors = true @@ -394,3 +398,45 @@ func TestVectorSelector_String(t *testing.T) { }) } } + +func TestBinaryExprUTF8Labels(t *testing.T) { + testCases := []struct { + name string + input string + expected string + }{ + { + name: "UTF-8 labels in on clause", + input: `foo / on("äää") bar`, + expected: `foo / on ("äää") bar`, + }, + { + name: "UTF-8 labels in group_left clause", + input: `foo / on("äää") group_left("ööö") bar`, + expected: `foo / on ("äää") group_left ("ööö") bar`, + }, + { + name: "Mixed legacy and UTF-8 labels", + input: `foo / on(legacy, "üüü") bar`, + expected: `foo / on (legacy, "üüü") bar`, + }, + { + name: "Legacy labels only (should not quote)", + input: `foo / on(job, instance) bar`, + expected: `foo / on (job, instance) bar`, + }, + } + + for _, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + expr, err := ParseExpr(tc.input) + if err != nil { + t.Fatalf("Failed to parse: %v", err) + } + result := expr.String() + if result != tc.expected { + t.Errorf("Expected: %s\nGot: %s", tc.expected, result) + } + }) + } +} From 2e4f5e8cfc966fe29cac87dac337acd70ac3e15c Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Mon, 15 Dec 2025 12:38:12 +0000 Subject: [PATCH 144/439] promql/parser: consolidate label quoting logic refactors binary expression formatting to reuse writeLabels() instead of maintaining separate joinLabels() function. adds comprehensive UTF-8 label tests for all expression types Signed-off-by: ADITYA TIWARI --- promql/parser/printer.go | 32 +++++++++++++++++--------------- promql/parser/printer_test.go | 18 ++++++++++++++++++ 2 files changed, 35 insertions(+), 15 deletions(-) diff --git a/promql/parser/printer.go b/promql/parser/printer.go index 67b13eaf12..cca04ae222 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -145,21 +145,9 @@ func (node *BinaryExpr) ShortString() string { return node.Op.String() + node.returnBool() + node.getMatchingStr() } -// joinLabels joins label names, quoting them if they are not valid legacy label names. -func joinLabels(labels []string) string { - quoted := make([]string, 0, len(labels)) - for _, label := range labels { - if model.LegacyValidation.IsValidLabelName(label) { - quoted = append(quoted, label) - } else { - quoted = append(quoted, strconv.Quote(label)) - } - } - return strings.Join(quoted, ", ") -} - func (node *BinaryExpr) getMatchingStr() string { matching := "" + var b bytes.Buffer vm := node.VectorMatching if vm != nil { if len(vm.MatchingLabels) > 0 || vm.On || vm.Card == CardManyToOne || vm.Card == CardOneToMany { @@ -167,7 +155,14 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.On { vmTag = "on" } - matching = fmt.Sprintf(" %s (%s)", vmTag, joinLabels(vm.MatchingLabels)) + // Use writeLabels() instead of joinLabels() + b.Reset() + b.WriteString(" ") + b.WriteString(vmTag) + b.WriteString(" (") + writeLabels(&b, vm.MatchingLabels) + b.WriteString(")") + matching = b.String() } if vm.Card == CardManyToOne || vm.Card == CardOneToMany { @@ -175,7 +170,14 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.Card == CardManyToOne { vmCard = "left" } - matching += fmt.Sprintf(" group_%s (%s)", vmCard, joinLabels(vm.Include)) + // Use writeLabels() instead of joinLabels() + b.Reset() + b.WriteString(" group_") + b.WriteString(vmCard) + b.WriteString(" (") + writeLabels(&b, vm.Include) + b.WriteString(")") + matching += b.String() } } return matching diff --git a/promql/parser/printer_test.go b/promql/parser/printer_test.go index c9a3cb35e8..31e707ee96 100644 --- a/promql/parser/printer_test.go +++ b/promql/parser/printer_test.go @@ -273,6 +273,14 @@ func TestExprString(t *testing.T) { in: `sum by("üüü") (foo)`, out: `sum by ("üüü") (foo)`, }, + { + in: `sum without("äää") (foo)`, + out: `sum without ("äää") (foo)`, + }, + { + in: `count by("ööö", job) (foo)`, + out: `count by ("ööö", job) (foo)`, + }, } EnableExtendedRangeSelectors = true @@ -410,11 +418,21 @@ func TestBinaryExprUTF8Labels(t *testing.T) { input: `foo / on("äää") bar`, expected: `foo / on ("äää") bar`, }, + { + name: "UTF-8 labels in ignoring clause", + input: `foo / ignoring("üüü") bar`, + expected: `foo / ignoring ("üüü") bar`, + }, { name: "UTF-8 labels in group_left clause", input: `foo / on("äää") group_left("ööö") bar`, expected: `foo / on ("äää") group_left ("ööö") bar`, }, + { + name: "UTF-8 labels in group_right clause", + input: `foo / on("äää") group_right("ööö") bar`, + expected: `foo / on ("äää") group_right ("ööö") bar`, + }, { name: "Mixed legacy and UTF-8 labels", input: `foo / on(legacy, "üüü") bar`, From 301b9eff446800abdb0de2ee9776621c9708e75d Mon Sep 17 00:00:00 2001 From: Aditya Tiwari <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Mon, 15 Dec 2025 18:25:09 +0530 Subject: [PATCH 145/439] Update comments to clarify label formatting method Signed-off-by: Aditya Tiwari <142050150+ADITYATIWARI342005@users.noreply.github.com> --- promql/parser/printer.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/promql/parser/printer.go b/promql/parser/printer.go index cca04ae222..9869cc5a30 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -155,7 +155,7 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.On { vmTag = "on" } - // Use writeLabels() instead of joinLabels() + // Format labels with proper UTF-8 quoting using writeLabels() b.Reset() b.WriteString(" ") b.WriteString(vmTag) @@ -170,7 +170,7 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.Card == CardManyToOne { vmCard = "left" } - // Use writeLabels() instead of joinLabels() + // Format labels with proper UTF-8 quoting using writeLabels() b.Reset() b.WriteString(" group_") b.WriteString(vmCard) From 3a82dcc6c5dc3f98fd4b646872d2448a6b53695a Mon Sep 17 00:00:00 2001 From: ADITYA TIWARI Date: Mon, 15 Dec 2025 13:39:15 +0000 Subject: [PATCH 146/439] promql/parser: simplify BinaryExpr label formatting Signed-off-by: ADITYA TIWARI --- promql/parser/printer.go | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/promql/parser/printer.go b/promql/parser/printer.go index 9869cc5a30..f30ad5a778 100644 --- a/promql/parser/printer.go +++ b/promql/parser/printer.go @@ -155,11 +155,7 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.On { vmTag = "on" } - // Format labels with proper UTF-8 quoting using writeLabels() - b.Reset() - b.WriteString(" ") - b.WriteString(vmTag) - b.WriteString(" (") + b.WriteString(" " + vmTag + " (") writeLabels(&b, vm.MatchingLabels) b.WriteString(")") matching = b.String() @@ -170,11 +166,8 @@ func (node *BinaryExpr) getMatchingStr() string { if vm.Card == CardManyToOne { vmCard = "left" } - // Format labels with proper UTF-8 quoting using writeLabels() b.Reset() - b.WriteString(" group_") - b.WriteString(vmCard) - b.WriteString(" (") + b.WriteString(" group_" + vmCard + " (") writeLabels(&b, vm.Include) b.WriteString(")") matching += b.String() From c818ad5a8ff25354d8a3d4e666450efb9bb695f4 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Tue, 16 Dec 2025 15:09:46 +0000 Subject: [PATCH 147/439] Propose Bryan Boreham as release shepherd for 3.9 Signed-off-by: Bryan Boreham --- RELEASE.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/RELEASE.md b/RELEASE.md index 952f9f010d..c7375b35aa 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -18,7 +18,8 @@ Please see [the v2.55 RELEASE.md](https://github.com/prometheus/prometheus/blob/ | v3.6 | 2025-08-01 | Ayoub Mrini (Github: @machine424) | | v3.7 | 2025-09-25 | Arthur Sens and George Krajcsovits (Github: @ArthurSens and @krajorama)| | v3.8 | 2025-11-06 | Jan Fajerski (GitHub: @jan--f) | -| v3.9 | 2025-12-18 | **volunteer welcome** | +| v3.9 | 2025-12-18 | Bryan Boreham (GitHub: @bboreham) | +| v3.10 | 2026-02-05 | **volunteer welcome** | If you are interested in volunteering please create a pull request against the [prometheus/prometheus](https://github.com/prometheus/prometheus) repository and propose yourself for the release series of your choice. From b336889d8fbfa863ceb9175b458ce5db539fefe9 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Tue, 16 Dec 2025 14:50:16 +0100 Subject: [PATCH 148/439] PromQL: Fix collision in label_join and label_replace with non-overlapping series. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- promql/functions.go | 10 ++----- promql/promqltest/testdata/functions.test | 35 +++++++++++++++++++++++ 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/promql/functions.go b/promql/functions.go index 925ae83ae5..f844bf5ada 100644 --- a/promql/functions.go +++ b/promql/functions.go @@ -1859,11 +1859,8 @@ func (ev *evaluator) evalLabelReplace(ctx context.Context, args parser.Expressio } } } - if matrix.ContainsSameLabelset() { - ev.errorf("vector cannot contain metrics with the same labelset") - } - return matrix, ws + return ev.mergeSeriesWithSameLabelset(matrix), ws } // === Vector(s Scalar) (Vector, Annotations) === @@ -1913,11 +1910,8 @@ func (ev *evaluator) evalLabelJoin(ctx context.Context, args parser.Expressions) matrix[i].DropName = el.DropName } } - if matrix.ContainsSameLabelset() { - ev.errorf("vector cannot contain metrics with the same labelset") - } - return matrix, ws + return ev.mergeSeriesWithSameLabelset(matrix), ws } // Common code for date related functions. diff --git a/promql/promqltest/testdata/functions.test b/promql/promqltest/testdata/functions.test index ba3df76ff6..7bc4bcb624 100644 --- a/promql/promqltest/testdata/functions.test +++ b/promql/promqltest/testdata/functions.test @@ -2014,3 +2014,38 @@ eval instant at 0m scalar({type="histogram"}) # One float in the vector. eval instant at 0m scalar({l="x"}) 1 + +clear +load 20m + series{label="a", idx="1"} 2 _ + series{label="a", idx="2"} _ 4 + +eval instant at 0 label_replace(series, "idx", "replaced", "idx", ".*") + series{label="a", idx="replaced"} 2 + +eval instant at 20m label_replace(series, "idx", "replaced", "idx", ".*") + series{label="a", idx="replaced"} 4 + +eval range from 0 to 20m step 20m label_replace(series, "idx", "replaced", "idx", ".*") + series{label="a", idx="replaced"} 2 4 + +# Test label_join with non-overlapping series. +eval instant at 0 label_join(series, "idx", ",", "label", "label") + series{label="a", idx="a,a"} 2 + +eval instant at 20m label_join(series, "idx", ",", "label", "label") + series{label="a", idx="a,a"} 4 + +eval range from 0 to 20m step 20m label_join(series, "idx", ",", "label", "label") + series{label="a", idx="a,a"} 2 4 + +# Test label_replace failure with overlapping timestamps (same labelset at same time). +clear +load 1m + overlap{label="a", idx="1"} 1 + overlap{label="a", idx="2"} 2 + +eval_fail instant at 0 label_replace(overlap, "idx", "same", "idx", ".*") + +# Test label_join failure with overlapping timestamps (same labelset at same time). +eval_fail instant at 0 label_join(overlap, "idx", ",", "label", "label") From 9ab52f9211c4d443c928021cb2f96ef588ef1ccf Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Tue, 16 Dec 2025 19:01:14 +0100 Subject: [PATCH 149/439] Do not notify the prometheus/default-maintainers team for code reviews (#17705) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove default-maintainers as the backup for all code. Everybody in that team is a prometheus team member, but not everybody wants to get notified for all review requests. Downside is that if we enable require codeowners approval, the team member cannot approve any PR anymore. Signed-off-by: György Krajcsovits --- CODEOWNERS | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/CODEOWNERS b/CODEOWNERS index 7dda7dc1a4..7a7ec8f215 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -1,28 +1,24 @@ -# Prometheus team members are members of the "default maintainers" github team. -# They are code owners by default for the whole repo. -* @prometheus/default-maintainers - # # Please keep this file in sync with the MAINTAINERS.md file! # # Subsystems. -/Makefile @simonpasquier @SuperQ @prometheus/default-maintainers -/cmd/promtool @dgl @prometheus/default-maintainers -/documentation/prometheus-mixin @metalmatze @prometheus/default-maintainers -/model/histogram @beorn7 @krajorama @prometheus/default-maintainers -/web/ui @juliusv @prometheus/default-maintainers -/web/ui/module @juliusv @nexucis @prometheus/default-maintainers -/promql @roidelapluie @prometheus/default-maintainers -/storage/remote @cstyan @bwplotka @tomwilkie @npazosmendez @alexgreenbank @prometheus/default-maintainers -/storage/remote/otlptranslator @aknuds1 @jesusvazquez @ArthurSens @prometheus/default-maintainers -/tsdb @jesusvazquez @codesome @bwplotka @krajorama @prometheus/default-maintainers +/Makefile @simonpasquier @SuperQ +/cmd/promtool @dgl +/documentation/prometheus-mixin @metalmatze +/model/histogram @beorn7 @krajorama +/web/ui @juliusv +/web/ui/module @juliusv @nexucis +/promql @roidelapluie +/storage/remote @cstyan @bwplotka @tomwilkie @npazosmendez @alexgreenbank +/storage/remote/otlptranslator @aknuds1 @jesusvazquez @ArthurSens +/tsdb @jesusvazquez @codesome @bwplotka @krajorama # Service discovery. -/discovery/kubernetes @brancz @prometheus/default-maintainers -/discovery/stackit @jkroepke @prometheus/default-maintainers +/discovery/kubernetes @brancz +/discovery/stackit @jkroepke # Pending # https://github.com/prometheus/prometheus/pull/17105#issuecomment-3248209452 -# /discovery/aws/ @matt-gp @prometheus/default-maintainers +# /discovery/aws/ @matt-gp # https://github.com/prometheus/prometheus/pull/15212#issuecomment-3575225179 -# /discovery/aliyun @KeyOfSpectator @prometheus/default-maintainers +# /discovery/aliyun @KeyOfSpectator From cdc31d96f976bbc2737139c824f2cc882aa9a92a Mon Sep 17 00:00:00 2001 From: Jorge Turrado Ferrero Date: Tue, 16 Dec 2025 22:50:51 +0100 Subject: [PATCH 150/439] feat: Document how to authenticate STACKIT Service Accounts using RFC7523 (#17645) --- docs/configuration/configuration.md | 29 +++++++++++++++++-- documentation/examples/prometheus-stackit.yml | 22 +++++++++++--- 2 files changed, 44 insertions(+), 7 deletions(-) diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md index 0b944008ef..8f6e4d9b87 100644 --- a/docs/configuration/configuration.md +++ b/docs/configuration/configuration.md @@ -2554,12 +2554,35 @@ project: [ ] ``` -A Service Account Token can be set through `http_config`. +A [Service Account Key](https://docs.stackit.cloud/platform/access-and-identity/service-accounts/how-tos/manage-service-account-keys/) can be set through `http_config`. This can be done mapping values from STACKIT Service Account json into oauth2 configuration. + +From a given Service Account json +```json +{ + //.... + "credentials": { + "kid": "6a7c3b36-xxxxxxxx", + "iss": "xxxx@sa.stackit.cloud", + "sub": "af2c2336-xxxxxxxx", + "aud": "https://stackit-service-account-prod.apps.01.cf.eu01.stackit.cloud", + "privateKey": "-----BEGIN PRIVATE KEY-----xxxx" + } +} +``` + +properties can be mapped as: ```yaml stackit_sd_config: -- authorization: - credentials: +- oauth2: + client_id: + client_certificate_key: + client_certificate_key_id: + iss: + audience: + grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer" + token_url: "https://service-account.api.stackit.cloud/token" + signature_algorithm: RS512 ``` ### `` diff --git a/documentation/examples/prometheus-stackit.yml b/documentation/examples/prometheus-stackit.yml index 623cb231ff..9be3f9c53a 100644 --- a/documentation/examples/prometheus-stackit.yml +++ b/documentation/examples/prometheus-stackit.yml @@ -12,8 +12,15 @@ scrape_configs: stackit_sd_configs: - project: 11111111-1111-1111-1111-111111111111 - authorization: - credentials: "" + oauth2: + client_id: + client_certificate_key: + client_certificate_key_id: + iss: + audience: + grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer" + token_url: "https://service-account.api.stackit.cloud/token" + signature_algorithm: RS512 relabel_configs: # Use the public IPv4 and port 9100 to scrape the target. - source_labels: [__meta_stackit_public_ipv4] @@ -25,8 +32,15 @@ scrape_configs: stackit_sd_configs: - project: 11111111-1111-1111-1111-111111111111 - authorization: - credentials: "" + oauth2: + client_id: + client_certificate_key: + client_certificate_key_id: + iss: + audience: + grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer" + token_url: "https://service-account.api.stackit.cloud/token" + signature_algorithm: RS512 relabel_configs: # Use the private IPv4 within the STACKIT Subnet and port 9100 to scrape the target. - source_labels: [__meta_stackit_private_ipv4_mynet] From 69aa3ac67b514bf149fbc4d0a6e64b3730bb94bf Mon Sep 17 00:00:00 2001 From: Charles Korn Date: Wed, 17 Dec 2025 10:04:14 +1100 Subject: [PATCH 151/439] tsdb: correctly log error in `headAppenderV2.appendExemplars` Signed-off-by: Charles Korn --- tsdb/head_append_v2.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tsdb/head_append_v2.go b/tsdb/head_append_v2.go index c5ed9898e9..d109b4a499 100644 --- a/tsdb/head_append_v2.go +++ b/tsdb/head_append_v2.go @@ -326,7 +326,7 @@ func (a *headAppenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemp errs = append(errs, err) } if !errors.Is(err, storage.ErrOutOfOrderExemplar) { - a.head.logger.Debug("Error while adding an exemplar on AppendSample", "exemplars", fmt.Sprintf("%+v", e), "err", e) + a.head.logger.Debug("Error while adding an exemplar on AppendSample", "exemplars", fmt.Sprintf("%+v", e), "err", err) } continue } From fc019d66288f660f6ba86262a21580a8fa2223c6 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Wed, 17 Dec 2025 05:29:30 +0000 Subject: [PATCH 152/439] refactor(scrape): DRY-ed getOrCreate flow Signed-off-by: bwplotka --- tsdb/agent/db.go | 115 ++++++++++++------------------------- tsdb/agent/db_append_v2.go | 25 ++------ 2 files changed, 40 insertions(+), 100 deletions(-) diff --git a/tsdb/agent/db.go b/tsdb/agent/db.go index 7b3e74f51a..a0f7a93b6d 100644 --- a/tsdb/agent/db.go +++ b/tsdb/agent/db.go @@ -839,26 +839,10 @@ func (a *appender) Append(ref storage.SeriesRef, l labels.Labels, t int64, v flo series := a.series.GetByID(headRef) if series == nil { - // Ensure no empty or duplicate labels have gotten through. This mirrors the - // equivalent validation code in the TSDB's headAppender. - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) - } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - var created bool - series, created = a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: series.ref, - Labels: l, - }) - - a.metrics.numActiveSeries.Inc() + var err error + series, err = a.getOrCreate(l) + if err != nil { + return 0, err } } @@ -882,18 +866,35 @@ func (a *appender) Append(ref storage.SeriesRef, l labels.Labels, t int64, v flo return storage.SeriesRef(series.ref), nil } -func (a *appenderBase) getOrCreate(l labels.Labels) (series *memSeries, created bool) { +func (a *appenderBase) getOrCreate(l labels.Labels) (series *memSeries, err error) { + // Ensure no empty or duplicate labels have gotten through. This mirrors the + // equivalent validation code in the TSDB's headAppender. + l = l.WithoutEmpty() + if l.IsEmpty() { + return nil, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + } + + if lbl, dup := l.HasDuplicateLabelNames(); dup { + return nil, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) + } + hash := l.Hash() series = a.series.GetByHash(hash, l) if series != nil { - return series, false + return series, nil } ref := chunks.HeadSeriesRef(a.nextRef.Inc()) series = &memSeries{ref: ref, lset: l, lastTs: math.MinInt64} a.series.Set(hash, series) - return series, true + + a.pendingSeries = append(a.pendingSeries, record.RefSeries{ + Ref: series.ref, + Labels: l, + }) + a.metrics.numActiveSeries.Inc() + return series, nil } func (a *appender) AppendExemplar(ref storage.SeriesRef, _ labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) { @@ -973,26 +974,10 @@ func (a *appender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int series := a.series.GetByID(headRef) if series == nil { - // Ensure no empty or duplicate labels have gotten through. This mirrors the - // equivalent validation code in the TSDB's headAppender. - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) - } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - var created bool - series, created = a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: series.ref, - Labels: l, - }) - - a.metrics.numActiveSeries.Inc() + var err error + series, err = a.getOrCreate(l) + if err != nil { + return 0, err } } @@ -1049,24 +1034,10 @@ func (a *appender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.L series := a.series.GetByID(chunks.HeadSeriesRef(ref)) if series == nil { - // Ensure no empty labels have gotten through. - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) - } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - var created bool - series, created = a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: series.ref, - Labels: l, - }) - a.metrics.numActiveSeries.Inc() + var err error + series, err = a.getOrCreate(l) + if err != nil { + return 0, err } } @@ -1115,25 +1086,11 @@ func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t, series := a.series.GetByID(chunks.HeadSeriesRef(ref)) if series == nil { - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) + var err error + series, err = a.getOrCreate(l) + if err != nil { + return 0, err } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - newSeries, created := a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: newSeries.ref, - Labels: l, - }) - a.metrics.numActiveSeries.Inc() - } - - series = newSeries } series.Lock() diff --git a/tsdb/agent/db_append_v2.go b/tsdb/agent/db_append_v2.go index ae4e3a4a84..ca74c6038d 100644 --- a/tsdb/agent/db_append_v2.go +++ b/tsdb/agent/db_append_v2.go @@ -23,7 +23,6 @@ import ( "github.com/prometheus/prometheus/model/labels" "github.com/prometheus/prometheus/model/value" "github.com/prometheus/prometheus/storage" - "github.com/prometheus/prometheus/tsdb" "github.com/prometheus/prometheus/tsdb/chunks" "github.com/prometheus/prometheus/tsdb/record" ) @@ -62,26 +61,10 @@ func (a *appenderV2) Append(ref storage.SeriesRef, l labels.Labels, st, t int64, // series references and chunk references are identical for agent mode. s := a.series.GetByID(chunks.HeadSeriesRef(ref)) if s == nil { - // Ensure no empty or duplicate labels have gotten through. This mirrors the - // equivalent validation code in the TSDB's headAppender. - l = l.WithoutEmpty() - if l.IsEmpty() { - return 0, fmt.Errorf("empty labelset: %w", tsdb.ErrInvalidSample) - } - - if lbl, dup := l.HasDuplicateLabelNames(); dup { - return 0, fmt.Errorf(`label name "%s" is not unique: %w`, lbl, tsdb.ErrInvalidSample) - } - - var created bool - s, created = a.getOrCreate(l) - if created { - a.pendingSeries = append(a.pendingSeries, record.RefSeries{ - Ref: s.ref, - Labels: l, - }) - - a.metrics.numActiveSeries.Inc() + var err error + s, err = a.getOrCreate(l) + if err != nil { + return 0, err } } From bab7614d1ba035659a7b778791918978039ab71e Mon Sep 17 00:00:00 2001 From: bwplotka Date: Wed, 17 Dec 2025 05:36:14 +0000 Subject: [PATCH 153/439] fix: ensure no race on lset Signed-off-by: bwplotka --- tsdb/agent/db_append_v2.go | 13 +++++++------ tsdb/head_append_v2.go | 9 +++++---- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/tsdb/agent/db_append_v2.go b/tsdb/agent/db_append_v2.go index ca74c6038d..f356a4feae 100644 --- a/tsdb/agent/db_append_v2.go +++ b/tsdb/agent/db_append_v2.go @@ -38,7 +38,7 @@ type appenderV2 struct { // Append appends pending sample to agent's DB. // TODO: Wire metadata in the Agent's appender. -func (a *appenderV2) Append(ref storage.SeriesRef, l labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) { +func (a *appenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) { var ( // Avoid shadowing err variables for reliability. valErr, partialErr error @@ -62,7 +62,7 @@ func (a *appenderV2) Append(ref storage.SeriesRef, l labels.Labels, st, t int64, s := a.series.GetByID(chunks.HeadSeriesRef(ref)) if s == nil { var err error - s, err = a.getOrCreate(l) + s, err = a.getOrCreate(ls) if err != nil { return 0, err } @@ -74,7 +74,7 @@ func (a *appenderV2) Append(ref storage.SeriesRef, l labels.Labels, st, t int64, // TODO(bwplotka): Handle ST natively (as per PROM-60). if a.opts.EnableSTAsZeroSample && st != 0 { - a.bestEffortAppendSTZeroSample(s, lastTS, st, t, h, fh) + a.bestEffortAppendSTZeroSample(s, ls, lastTS, st, t, h, fh) } if t <= a.minValidTime(lastTS) { @@ -164,13 +164,14 @@ func (a *appenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemplar) // is implemented. // // ST is an experimental feature, we don't fail the append on errors, just debug log. -func (a *appenderV2) bestEffortAppendSTZeroSample(s *memSeries, lastTS, st, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) { +func (a *appenderV2) bestEffortAppendSTZeroSample(s *memSeries, ls labels.Labels, lastTS, st, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) { + // NOTE: Use lset instead of s.lset to avoid locking memSeries. Using s.ref is acceptable without locking. if st >= t { - a.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrSTNewerThanSample) + a.logger.Debug("Error when appending ST", "series", ls.String(), "st", st, "t", t, "err", storage.ErrSTNewerThanSample) return } if st <= lastTS { - a.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrOutOfOrderST) + a.logger.Debug("Error when appending ST", "series", ls.String(), "st", st, "t", t, "err", storage.ErrOutOfOrderST) return } diff --git a/tsdb/head_append_v2.go b/tsdb/head_append_v2.go index 95118528eb..7987a30f51 100644 --- a/tsdb/head_append_v2.go +++ b/tsdb/head_append_v2.go @@ -145,7 +145,7 @@ func (a *headAppenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t i // TODO(bwplotka): Handle ST natively (as per PROM-60). if a.head.opts.EnableSTAsZeroSample && st != 0 { - a.bestEffortAppendSTZeroSample(s, st, t, h, fh) + a.bestEffortAppendSTZeroSample(s, ls, st, t, h, fh) } switch { @@ -344,13 +344,14 @@ func (a *headAppenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemp // is implemented. // // ST is an experimental feature, we don't fail the append on errors, just debug log. -func (a *headAppenderV2) bestEffortAppendSTZeroSample(s *memSeries, st, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) { +func (a *headAppenderV2) bestEffortAppendSTZeroSample(s *memSeries, ls labels.Labels, st, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) { + // NOTE: Use lset instead of s.lset to avoid locking memSeries. Using s.ref is acceptable without locking. if st >= t { - a.head.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrSTNewerThanSample) + a.head.logger.Debug("Error when appending ST", "series", ls.String(), "st", st, "t", t, "err", storage.ErrSTNewerThanSample) return } if st < a.minValidTime { - a.head.logger.Debug("Error when appending ST", "series", s.lset.String(), "st", st, "t", t, "err", storage.ErrOutOfBounds) + a.head.logger.Debug("Error when appending ST", "series", ls.String(), "st", st, "t", t, "err", storage.ErrOutOfBounds) return } From 96ff5b8f9c3d1778a1604dc0067f4034344de475 Mon Sep 17 00:00:00 2001 From: bwplotka Date: Wed, 17 Dec 2025 05:47:28 +0000 Subject: [PATCH 154/439] addressed comment Signed-off-by: bwplotka --- tsdb/agent/db_append_v2_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/tsdb/agent/db_append_v2_test.go b/tsdb/agent/db_append_v2_test.go index ec92cfa630..6a85e93c35 100644 --- a/tsdb/agent/db_append_v2_test.go +++ b/tsdb/agent/db_append_v2_test.go @@ -1007,6 +1007,7 @@ func TestDBOutOfOrderTimeWindow_AppendV2(t *testing.T) { } } +// TestDB_EnableSTZeroInjection_AppendV2 replaces TestDBStartTimestampSamplesIngestion. func TestDB_EnableSTZeroInjection_AppendV2(t *testing.T) { t.Parallel() From 37b97a020061da6f9ca028244808fc02653b57be Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Wed, 17 Dec 2025 10:45:12 +0100 Subject: [PATCH 155/439] PromQL: Fix collision in unary negation with non-overlapping series. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- promql/engine.go | 4 +-- promql/engine_test.go | 35 +++++++++++++++++++++ promql/promqltest/testdata/operators.test | 37 +++++++++++++++++++++++ 3 files changed, 74 insertions(+), 2 deletions(-) diff --git a/promql/engine.go b/promql/engine.go index 07fb03d66c..5a08da121c 100644 --- a/promql/engine.go +++ b/promql/engine.go @@ -2191,8 +2191,8 @@ func (ev *evaluator) eval(ctx context.Context, expr parser.Expr) (parser.Value, mat[i].Histograms[j].H = mat[i].Histograms[j].H.Copy().Mul(-1) } } - if !ev.enableDelayedNameRemoval && mat.ContainsSameLabelset() { - ev.errorf("vector cannot contain metrics with the same labelset") + if !ev.enableDelayedNameRemoval { + mat = ev.mergeSeriesWithSameLabelset(mat) } } return mat, ws diff --git a/promql/engine_test.go b/promql/engine_test.go index 80bb75c945..208ac4f89d 100644 --- a/promql/engine_test.go +++ b/promql/engine_test.go @@ -3946,6 +3946,41 @@ eval instant at 1m histogram_fraction(-Inf, 0.7071067811865475, histogram_nan) {case="100% NaNs"} 0.0 {case="20% NaNs"} 0.4 +# Test unary negation with non-overlapping series that have different metric names. +# After negation, the __name__ label is dropped, so series with different names +# but same other labels should merge if they don't overlap in time. +clear +load 20m + http_requests{job="api"} 2 _ + http_errors{job="api"} _ 4 + +eval instant at 0 -{job="api"} + {job="api"} -2 + +eval instant at 20m -{job="api"} + {job="api"} -4 + +eval range from 0 to 20m step 20m -{job="api"} + {job="api"} -2 -4 + +# Test unary negation failure with overlapping timestamps (same labelset at same time). +clear +load 1m + http_requests{job="api"} 1 + http_errors{job="api"} 2 + +eval_fail instant at 0 -{job="api"} + +# Test unary negation with "or" operator combining metrics with removed names. +clear +load 10m + metric_a 1 _ + metric_b 3 4 + +# Use "-" unary operator as a simple way to remove the metric name. +eval range from 0 to 20m step 10m -metric_a or -metric_b + {} -1 -4 + `, engine) } diff --git a/promql/promqltest/testdata/operators.test b/promql/promqltest/testdata/operators.test index e570be9630..cd608b3c36 100644 --- a/promql/promqltest/testdata/operators.test +++ b/promql/promqltest/testdata/operators.test @@ -980,3 +980,40 @@ eval instant at 10m (testhistogram) and on() (vector(-1) == 1) eval range from 0 to 10m step 5m (testhistogram) and on() (vector(-1) == 1) clear + +# Test unary negation with non-overlapping series that have different metric names. +# After negation, the __name__ label is dropped, so series with different names +# but same other labels should merge if they don't overlap in time. +load 20m + http_requests{job="api"} 2 _ + http_errors{job="api"} _ 4 + +eval instant at 0 -{job="api"} + {job="api"} -2 + +eval instant at 20m -{job="api"} + {job="api"} -4 + +eval range from 0 to 20m step 20m -{job="api"} + {job="api"} -2 -4 + +# Test unary negation failure with overlapping timestamps (same labelset at same time). +clear +load 1m + http_requests{job="api"} 1 + http_errors{job="api"} 2 + +eval_fail instant at 0 -{job="api"} + +clear + +# Test unary negation with "or" operator combining metrics with removed names. +load 10m + metric_a 1 _ + metric_b 3 4 + +# Use "-" unary operator as a simple way to remove the metric name. +eval range from 0 to 20m step 10m -metric_a or -metric_b + {} -1 -4 + +clear From 962341f621d94584cbca70613c92ad8ee54b0721 Mon Sep 17 00:00:00 2001 From: George Krajcsovits Date: Wed, 17 Dec 2025 16:13:49 +0100 Subject: [PATCH 156/439] Add more potential code owners for SD (#17709) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit To enable when we work out how to give approval right to their paths only. Signed-off-by: György Krajcsovits --- CODEOWNERS | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CODEOWNERS b/CODEOWNERS index 7a7ec8f215..f28cdbf832 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -19,6 +19,8 @@ /discovery/stackit @jkroepke # Pending # https://github.com/prometheus/prometheus/pull/17105#issuecomment-3248209452 -# /discovery/aws/ @matt-gp +# /discovery/aws/ @matt-gp @sysadmind # https://github.com/prometheus/prometheus/pull/15212#issuecomment-3575225179 # /discovery/aliyun @KeyOfSpectator +# https://github.com/prometheus/prometheus/pull/14108#issuecomment-2639515421 +# /discovery/nomad @jaloren @jrasell From 146080186d963e1b91a90b3d047b3bd30e052f83 Mon Sep 17 00:00:00 2001 From: Aditya Tiwari <142050150+ADITYATIWARI342005@users.noreply.github.com> Date: Thu, 18 Dec 2025 15:49:06 +0530 Subject: [PATCH 157/439] promtool: Add --lint flag to check metrics command to allow disabling linting (#17669) * promtool: allow cardinality with metrics linting and add --lint to check metrics Signed-off-by: ADITYA TIWARI * fix/ci: Simplify test case variable declaration Remove unnecessary variable declaration in test cases. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> * promtool: avoid Tee for --lint=none Signed-off-by: ADITYA TIWARI * promtool: validate at least one feature enabled in check metrics addresses feedback to ensure the command does something useful now fails with clear error when both --lint=none and no --extended flag. Signed-off-by: ADITYA TIWARI --------- Signed-off-by: ADITYA TIWARI Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com> --- cmd/promtool/main.go | 59 +++++++++++++++------- cmd/promtool/main_test.go | 94 +++++++++++++++++++++++++++++++++++ docs/command-line/promtool.md | 17 +++++-- 3 files changed, 150 insertions(+), 20 deletions(-) diff --git a/cmd/promtool/main.go b/cmd/promtool/main.go index bc47c3b505..d379d6e587 100644 --- a/cmd/promtool/main.go +++ b/cmd/promtool/main.go @@ -162,7 +162,11 @@ func main() { checkRulesIgnoreUnknownFields := checkRulesCmd.Flag("ignore-unknown-fields", "Ignore unknown fields in the rule files. This is useful when you want to extend rule files with custom metadata. Ensure that those fields are removed before loading them into the Prometheus server as it performs strict checks by default.").Default("false").Bool() checkMetricsCmd := checkCmd.Command("metrics", checkMetricsUsage) - checkMetricsExtended := checkCmd.Flag("extended", "Print extended information related to the cardinality of the metrics.").Bool() + checkMetricsExtended := checkMetricsCmd.Flag("extended", "Print extended information related to the cardinality of the metrics.").Bool() + checkMetricsLint := checkMetricsCmd.Flag( + "lint", + "Linting checks to apply for metrics. Available options are: all, none. Use --lint=none to disable metrics linting.", + ).Default(lintOptionAll).String() agentMode := checkConfigCmd.Flag("agent", "Check config file for Prometheus in Agent mode.").Bool() queryCmd := app.Command("query", "Run query against a Prometheus server.") @@ -375,7 +379,7 @@ func main() { os.Exit(CheckRules(newRulesLintConfig(*checkRulesLint, *checkRulesLintFatal, *checkRulesIgnoreUnknownFields, model.UTF8Validation), *ruleFiles...)) case checkMetricsCmd.FullCommand(): - os.Exit(CheckMetrics(*checkMetricsExtended)) + os.Exit(CheckMetrics(*checkMetricsExtended, *checkMetricsLint)) case pushMetricsCmd.FullCommand(): os.Exit(PushMetrics(remoteWriteURL, httpRoundTripper, *pushMetricsHeaders, *pushMetricsTimeout, *pushMetricsProtoMsg, *pushMetricsLabels, *metricFiles...)) @@ -1018,36 +1022,53 @@ func ruleMetric(rule rulefmt.Rule) string { } var checkMetricsUsage = strings.TrimSpace(` -Pass Prometheus metrics over stdin to lint them for consistency and correctness. +Pass Prometheus metrics over stdin to lint them for consistency and correctness, and optionally perform cardinality analysis. examples: $ cat metrics.prom | promtool check metrics -$ curl -s http://localhost:9090/metrics | promtool check metrics +$ curl -s http://localhost:9090/metrics | promtool check metrics --extended + +$ curl -s http://localhost:9100/metrics | promtool check metrics --extended --lint=none `) // CheckMetrics performs a linting pass on input metrics. -func CheckMetrics(extended bool) int { - var buf bytes.Buffer - tee := io.TeeReader(os.Stdin, &buf) - l := promlint.New(tee) - problems, err := l.Lint() - if err != nil { - fmt.Fprintln(os.Stderr, "error while linting:", err) +func CheckMetrics(extended bool, lint string) int { + // Validate that at least one feature is enabled. + if !extended && lint == lintOptionNone { + fmt.Fprintln(os.Stderr, "error: at least one of --extended or linting must be enabled") + fmt.Fprintln(os.Stderr, "Use --extended for cardinality analysis, or remove --lint=none to enable linting") return failureExitCode } - for _, p := range problems { - fmt.Fprintln(os.Stderr, p.Metric, p.Text) + var buf bytes.Buffer + var ( + problems []promlint.Problem + reader io.Reader + err error + ) + + if lint != lintOptionNone { + tee := io.TeeReader(os.Stdin, &buf) + l := promlint.New(tee) + problems, err = l.Lint() + if err != nil { + fmt.Fprintln(os.Stderr, "error while linting:", err) + return failureExitCode + } + for _, p := range problems { + fmt.Fprintln(os.Stderr, p.Metric, p.Text) + } + reader = &buf + } else { + reader = os.Stdin } - if len(problems) > 0 { - return lintErrExitCode - } + hasLintProblems := len(problems) > 0 if extended { - stats, total, err := checkMetricsExtended(&buf) + stats, total, err := checkMetricsExtended(reader) if err != nil { fmt.Fprintln(os.Stderr, err) return failureExitCode @@ -1061,6 +1082,10 @@ func CheckMetrics(extended bool) int { w.Flush() } + if hasLintProblems { + return lintErrExitCode + } + return successExitCode } diff --git a/cmd/promtool/main_test.go b/cmd/promtool/main_test.go index a9a54f6d5f..094852a01b 100644 --- a/cmd/promtool/main_test.go +++ b/cmd/promtool/main_test.go @@ -18,6 +18,7 @@ import ( "context" "errors" "fmt" + "io" "net/http" "net/http/httptest" "net/url" @@ -402,6 +403,99 @@ func TestCheckMetricsExtended(t *testing.T) { }, stats) } +func TestCheckMetricsLintOptions(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("Skipping on windows") + } + + const testMetrics = ` +# HELP testMetric_CamelCase A test metric with camelCase +# TYPE testMetric_CamelCase gauge +testMetric_CamelCase{label="value1"} 1 +` + + tests := []struct { + name string + lint string + extended bool + wantErrCode int + wantLint bool + wantCard bool + }{ + { + name: "default_all_with_extended", + lint: lintOptionAll, + extended: true, + wantErrCode: lintErrExitCode, + wantLint: true, + wantCard: true, + }, + { + name: "lint_none_with_extended", + lint: lintOptionNone, + extended: true, + wantErrCode: successExitCode, + wantLint: false, + wantCard: true, + }, + { + name: "both_disabled_fails", + lint: lintOptionNone, + extended: false, + wantErrCode: failureExitCode, + wantLint: false, + wantCard: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r, w, err := os.Pipe() + require.NoError(t, err) + _, err = w.WriteString(testMetrics) + require.NoError(t, err) + w.Close() + + oldStdin := os.Stdin + os.Stdin = r + defer func() { os.Stdin = oldStdin }() + + oldStdout := os.Stdout + oldStderr := os.Stderr + rOut, wOut, err := os.Pipe() + require.NoError(t, err) + rErr, wErr, err := os.Pipe() + require.NoError(t, err) + os.Stdout = wOut + os.Stderr = wErr + + code := CheckMetrics(tt.extended, tt.lint) + + wOut.Close() + wErr.Close() + os.Stdout = oldStdout + os.Stderr = oldStderr + + var outBuf, errBuf bytes.Buffer + _, _ = io.Copy(&outBuf, rOut) + _, _ = io.Copy(&errBuf, rErr) + + require.Equal(t, tt.wantErrCode, code) + if tt.wantLint { + require.Contains(t, errBuf.String(), "testMetric_CamelCase") + } else { + require.NotContains(t, errBuf.String(), "testMetric_CamelCase") + } + + if tt.wantCard { + require.Contains(t, outBuf.String(), "Cardinality") + } else { + require.NotContains(t, outBuf.String(), "Cardinality") + } + }) + } +} + func TestExitCodes(t *testing.T) { if testing.Short() { t.Skip("skipping test in short mode.") diff --git a/docs/command-line/promtool.md b/docs/command-line/promtool.md index 70fa29cd1c..f6737bc37f 100644 --- a/docs/command-line/promtool.md +++ b/docs/command-line/promtool.md @@ -59,7 +59,6 @@ Check the resources for validity. | Flag | Description | Default | | --- | --- | --- | | --query.lookback-delta | The server's maximum query lookback duration. | `5m` | -| --extended | Print extended information related to the cardinality of the metrics. | | @@ -192,13 +191,25 @@ Check if the rule files are valid or not. ##### `promtool check metrics` -Pass Prometheus metrics over stdin to lint them for consistency and correctness. +Pass Prometheus metrics over stdin to lint them for consistency and correctness, and optionally perform cardinality analysis. examples: $ cat metrics.prom | promtool check metrics -$ curl -s http://localhost:9090/metrics | promtool check metrics +$ curl -s http://localhost:9090/metrics | promtool check metrics `--extended` + +$ curl -s http://localhost:9100/metrics | promtool check metrics `--extended` `--lint`=none + + + +###### Flags + +| Flag | Description | Default | +| --- | --- | --- | +| --extended | Print extended information related to the cardinality of the metrics. | | +| --lint | Linting checks to apply for metrics. Available options are: all, none. Use --lint=none to disable metrics linting. | `all` | + From 4f04aaccc30c0e84ab3a413951ca84aec7e4fbc2 Mon Sep 17 00:00:00 2001 From: anubhav21sharma Date: Thu, 18 Dec 2025 11:54:27 +0000 Subject: [PATCH 158/439] UI: Add support to duplicate query panel Signed-off-by: anubhav21sharma --- .../mantine-ui/src/pages/query/ExpressionInput.tsx | 9 +++++++++ web/ui/mantine-ui/src/pages/query/QueryPanel.tsx | 4 ++++ web/ui/mantine-ui/src/state/queryPageSlice.ts | 14 ++++++++++++++ 3 files changed, 27 insertions(+) diff --git a/web/ui/mantine-ui/src/pages/query/ExpressionInput.tsx b/web/ui/mantine-ui/src/pages/query/ExpressionInput.tsx index a4b26cd910..4c3209e53a 100644 --- a/web/ui/mantine-ui/src/pages/query/ExpressionInput.tsx +++ b/web/ui/mantine-ui/src/pages/query/ExpressionInput.tsx @@ -58,6 +58,7 @@ import { lintKeymap } from "@codemirror/lint"; import { IconAlignJustified, IconBinaryTree, + IconCopy, IconDotsVertical, IconSearch, IconTerminal, @@ -121,6 +122,7 @@ interface ExpressionInputProps { executeQuery: (expr: string) => void; treeShown: boolean; setShowTree: (showTree: boolean) => void; + duplicatePanel: (expr: string) => void; removePanel: () => void; } @@ -128,6 +130,7 @@ const ExpressionInput: FC = ({ initialExpr, metricNames, executeQuery, + duplicatePanel, removePanel, treeShown, setShowTree, @@ -250,6 +253,12 @@ const ExpressionInput: FC = ({ > {treeShown ? "Hide" : "Show"} tree view + } + onClick={() => duplicatePanel(expr)} + > + Duplicate query + } diff --git a/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx b/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx index 5e41be7bb3..fcc7648a77 100644 --- a/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx +++ b/web/ui/mantine-ui/src/pages/query/QueryPanel.tsx @@ -23,6 +23,7 @@ import { FC, Suspense, useCallback, useMemo, useState } from "react"; import { useAppDispatch, useAppSelector } from "../../state/hooks"; import { addQueryToHistory, + duplicatePanel, GraphDisplayMode, GraphResolution, removePanel, @@ -111,6 +112,9 @@ const QueryPanel: FC = ({ idx, metricNames }) => { setSelectedNode(null); } }} + duplicatePanel={(expr: string) => { + dispatch(duplicatePanel({ idx, expr })); + }} removePanel={() => { dispatch(removePanel(idx)); }} diff --git a/web/ui/mantine-ui/src/state/queryPageSlice.ts b/web/ui/mantine-ui/src/state/queryPageSlice.ts index 4cf483e2b6..7a4f7b257a 100644 --- a/web/ui/mantine-ui/src/state/queryPageSlice.ts +++ b/web/ui/mantine-ui/src/state/queryPageSlice.ts @@ -115,6 +115,19 @@ export const queryPageSlice = createSlice({ state.panels.push(newDefaultPanel()); updateURL(state.panels); }, + duplicatePanel: ( + state, + { payload }: PayloadAction<{ idx: number; expr: string }> + ) => { + const newPanel = { + ...state.panels[payload.idx], + id: randomId(), + expr: payload.expr, + }; + // Insert the duplicated panel just below the original panel. + state.panels.splice(payload.idx + 1, 0, newPanel); + updateURL(state.panels); + }, removePanel: (state, { payload }: PayloadAction) => { state.panels.splice(payload, 1); updateURL(state.panels); @@ -153,6 +166,7 @@ export const { setPanels, addPanel, removePanel, + duplicatePanel, setExpr, addQueryToHistory, setShowTree, From bcd7fd174cebe933a623ffe8f3f3d75d50a1001b Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Thu, 18 Dec 2025 11:48:37 +0100 Subject: [PATCH 159/439] Makefile: Update all Go submodules in update-all-go-deps target Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- Makefile | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/Makefile b/Makefile index 30295c56e5..834f0e3ce2 100644 --- a/Makefile +++ b/Makefile @@ -189,14 +189,19 @@ update-features-testdata: @echo ">> updating features testdata" @$(GO) test ./cmd/prometheus -run TestFeaturesAPI -update-features +GO_SUBMODULE_DIRS := documentation/examples/remote_storage internal/tools web/ui/mantine-ui/src/promql/tools + .PHONY: update-all-go-deps -update-all-go-deps: - @$(MAKE) update-go-deps - @echo ">> updating Go dependencies in ./documentation/examples/remote_storage/" - @cd ./documentation/examples/remote_storage/ && for m in $$($(GO) list -mod=readonly -m -f '{{ if and (not .Indirect) (not .Main)}}{{.Path}}{{end}}' all); do \ +update-all-go-deps: update-go-deps + $(foreach dir,$(GO_SUBMODULE_DIRS),$(MAKE) update-go-deps-in-dir DIR=$(dir);) + +.PHONY: update-go-deps-in-dir +update-go-deps-in-dir: + @echo ">> updating Go dependencies in ./$(DIR)/" + @cd ./$(DIR) && for m in $$($(GO) list -mod=readonly -m -f '{{ if and (not .Indirect) (not .Main)}}{{.Path}}{{end}}' all); do \ $(GO) get $$m; \ done - @cd ./documentation/examples/remote_storage/ && $(GO) mod tidy + @cd ./$(DIR) && $(GO) mod tidy .PHONY: check-node-version check-node-version: From 4c7377f543aa1e656ea6a68aca2e254a43eb4d45 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Thu, 18 Dec 2025 17:14:14 +0000 Subject: [PATCH 160/439] Update Go dependencies (#17711) By running `make update-all-go-deps`. `hashicorp/consul/api` must be held at v1.32.1 because later versions require Go 1.25 and we choose to ensure that Promethes builds with the last two versions of Go. Also: fix compilation errors in remote-write example. Signed-off-by: Bryan Boreham --- .../example_write_adapter/server.go | 19 +- documentation/examples/remote_storage/go.mod | 98 ++-- documentation/examples/remote_storage/go.sum | 335 +++++++------ go.mod | 198 ++++---- go.sum | 456 ++++++++++-------- internal/tools/go.mod | 54 ++- internal/tools/go.sum | 119 +++-- web/ui/mantine-ui/src/promql/tools/go.mod | 24 +- web/ui/mantine-ui/src/promql/tools/go.sum | 207 +++++--- 9 files changed, 821 insertions(+), 689 deletions(-) diff --git a/documentation/examples/remote_storage/example_write_adapter/server.go b/documentation/examples/remote_storage/example_write_adapter/server.go index 727a3056d3..21267c80e5 100644 --- a/documentation/examples/remote_storage/example_write_adapter/server.go +++ b/documentation/examples/remote_storage/example_write_adapter/server.go @@ -59,7 +59,11 @@ func main() { http.Error(w, err.Error(), http.StatusBadRequest) return } - printV2(req) + err = printV2(req) + if err != nil { + http.Error(w, err.Error(), http.StatusBadRequest) + return + } default: msg := fmt.Sprintf("Unknown remote write content type: %s", contentType) fmt.Println(msg) @@ -93,10 +97,13 @@ func printV1(req *prompb.WriteRequest) { } } -func printV2(req *writev2.Request) { +func printV2(req *writev2.Request) error { b := labels.NewScratchBuilder(0) for _, ts := range req.Timeseries { - l := ts.ToLabels(&b, req.Symbols) + l, err := ts.ToLabels(&b, req.Symbols) + if err != nil { + return err + } m := ts.ToMetadata(req.Symbols) fmt.Println(l, m) @@ -104,7 +111,10 @@ func printV2(req *writev2.Request) { fmt.Printf("\tSample: %f %d\n", s.Value, s.Timestamp) } for _, ep := range ts.Exemplars { - e := ep.ToExemplar(&b, req.Symbols) + e, err := ep.ToExemplar(&b, req.Symbols) + if err != nil { + return err + } fmt.Printf("\tExemplar: %+v %f %d\n", e.Labels, e.Value, ep.Timestamp) } for _, hp := range ts.Histograms { @@ -117,4 +127,5 @@ func printV2(req *writev2.Request) { fmt.Printf("\tHistogram: %s\n", h.String()) } } + return nil } diff --git a/documentation/examples/remote_storage/go.mod b/documentation/examples/remote_storage/go.mod index a97ad32a6a..e7f9551290 100644 --- a/documentation/examples/remote_storage/go.mod +++ b/documentation/examples/remote_storage/go.mod @@ -1,6 +1,6 @@ module github.com/prometheus/prometheus/documentation/examples/remote_storage -go 1.24.0 +go 1.24.9 require ( github.com/alecthomas/kingpin/v2 v2.4.0 @@ -8,34 +8,34 @@ require ( github.com/golang/snappy v1.0.0 github.com/influxdata/influxdb-client-go/v2 v2.14.0 github.com/prometheus/client_golang v1.23.2 - github.com/prometheus/common v0.66.1 - github.com/prometheus/prometheus v0.305.1-0.20250905124657-5c2e43f09c03 + github.com/prometheus/common v0.67.4 + github.com/prometheus/prometheus v0.308.1 github.com/stretchr/testify v1.11.1 ) require ( - cloud.google.com/go/auth v0.16.2 // indirect + cloud.google.com/go/auth v0.17.0 // indirect cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect - cloud.google.com/go/compute/metadata v0.7.0 // indirect - github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0 // indirect - github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.10.1 // indirect - github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1 // indirect - github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 // indirect + cloud.google.com/go/compute/metadata v0.9.0 // indirect + github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1 // indirect + github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0 // indirect + github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect + github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 // indirect github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect - github.com/aws/aws-sdk-go-v2 v1.37.0 // indirect - github.com/aws/aws-sdk-go-v2/config v1.29.14 // indirect - github.com/aws/aws-sdk-go-v2/credentials v1.17.67 // indirect - github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30 // indirect - github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.0 // indirect - github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.0 // indirect - github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.0 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.0 // indirect - github.com/aws/aws-sdk-go-v2/service/sso v1.25.3 // indirect - github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.1 // indirect - github.com/aws/aws-sdk-go-v2/service/sts v1.33.19 // indirect - github.com/aws/smithy-go v1.22.5 // indirect + github.com/aws/aws-sdk-go-v2 v1.39.6 // indirect + github.com/aws/aws-sdk-go-v2/config v1.31.17 // indirect + github.com/aws/aws-sdk-go-v2/credentials v1.18.21 // indirect + github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 // indirect + github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 // indirect + github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 // indirect + github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 // indirect + github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 // indirect + github.com/aws/smithy-go v1.23.2 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect @@ -43,22 +43,22 @@ require ( github.com/felixge/httpsnoop v1.0.4 // indirect github.com/go-logr/logr v1.4.3 // indirect github.com/go-logr/stdr v1.2.2 // indirect - github.com/go-viper/mapstructure/v2 v2.3.0 // indirect + github.com/go-viper/mapstructure/v2 v2.4.0 // indirect github.com/gobwas/glob v0.2.3 // indirect - github.com/golang-jwt/jwt/v5 v5.2.2 // indirect + github.com/golang-jwt/jwt/v5 v5.3.0 // indirect github.com/google/s2a-go v0.1.9 // indirect github.com/google/uuid v1.6.0 // indirect github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect - github.com/googleapis/gax-go/v2 v2.14.2 // indirect - github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc // indirect + github.com/googleapis/gax-go/v2 v2.15.0 // indirect + github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 // indirect github.com/hashicorp/go-version v1.7.0 // indirect github.com/influxdata/line-protocol v0.0.0-20200327222509-2487e7298839 // indirect github.com/jpillora/backoff v1.0.0 // indirect github.com/json-iterator/go v1.1.12 // indirect - github.com/klauspost/compress v1.18.0 // indirect + github.com/klauspost/compress v1.18.2 // indirect github.com/knadh/koanf/maps v0.1.2 // indirect github.com/knadh/koanf/providers/confmap v1.0.0 // indirect - github.com/knadh/koanf/v2 v2.2.1 // indirect + github.com/knadh/koanf/v2 v2.3.0 // indirect github.com/kylelemons/godebug v1.1.0 // indirect github.com/mitchellh/copystructure v1.2.0 // indirect github.com/mitchellh/reflectwalk v1.0.2 // indirect @@ -67,21 +67,22 @@ require ( github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f // indirect github.com/oapi-codegen/runtime v1.0.0 // indirect - github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.129.0 // indirect - github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.129.0 // indirect - github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.129.0 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.139.0 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.139.0 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.139.0 // indirect github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect + github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca // indirect github.com/prometheus/client_model v0.6.2 // indirect - github.com/prometheus/otlptranslator v0.0.2 // indirect + github.com/prometheus/otlptranslator v1.0.0 // indirect github.com/prometheus/procfs v0.16.1 // indirect - github.com/prometheus/sigv4 v0.2.0 // indirect + github.com/prometheus/sigv4 v0.3.0 // indirect github.com/puzpuzpuz/xsync/v3 v3.5.1 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect go.opentelemetry.io/auto/sdk v1.1.0 // indirect go.opentelemetry.io/collector/component v1.45.0 // indirect - go.opentelemetry.io/collector/confmap v1.35.0 // indirect - go.opentelemetry.io/collector/confmap/xconfmap v0.129.0 // indirect + go.opentelemetry.io/collector/confmap v1.45.0 // indirect + go.opentelemetry.io/collector/confmap/xconfmap v0.139.0 // indirect go.opentelemetry.io/collector/consumer v1.45.0 // indirect go.opentelemetry.io/collector/featuregate v1.45.0 // indirect go.opentelemetry.io/collector/pdata v1.45.0 // indirect @@ -96,24 +97,23 @@ require ( go.uber.org/atomic v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect go.uber.org/zap v1.27.0 // indirect - go.yaml.in/yaml/v2 v2.4.2 // indirect - golang.org/x/crypto v0.41.0 // indirect - golang.org/x/net v0.43.0 // indirect - golang.org/x/oauth2 v0.30.0 // indirect - golang.org/x/sys v0.35.0 // indirect - golang.org/x/text v0.28.0 // indirect - golang.org/x/time v0.12.0 // indirect - google.golang.org/api v0.239.0 // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b // indirect + go.yaml.in/yaml/v2 v2.4.3 // indirect + go.yaml.in/yaml/v3 v3.0.4 // indirect + golang.org/x/crypto v0.43.0 // indirect + golang.org/x/net v0.46.0 // indirect + golang.org/x/oauth2 v0.32.0 // indirect + golang.org/x/sys v0.37.0 // indirect + golang.org/x/text v0.30.0 // indirect + golang.org/x/time v0.13.0 // indirect + google.golang.org/api v0.252.0 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797 // indirect google.golang.org/grpc v1.76.0 // indirect google.golang.org/protobuf v1.36.10 // indirect - gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - k8s.io/apimachinery v0.33.5 // indirect - k8s.io/client-go v0.33.5 // indirect + k8s.io/apimachinery v0.34.1 // indirect + k8s.io/client-go v0.34.1 // indirect k8s.io/klog/v2 v2.130.1 // indirect - k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect - sigs.k8s.io/yaml v1.4.0 // indirect + k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 // indirect ) exclude ( diff --git a/documentation/examples/remote_storage/go.sum b/documentation/examples/remote_storage/go.sum index b7c633982b..692f9f5abf 100644 --- a/documentation/examples/remote_storage/go.sum +++ b/documentation/examples/remote_storage/go.sum @@ -1,25 +1,25 @@ -cloud.google.com/go/auth v0.16.2 h1:QvBAGFPLrDeoiNjyfVunhQ10HKNYuOwZ5noee0M5df4= -cloud.google.com/go/auth v0.16.2/go.mod h1:sRBas2Y1fB1vZTdurouM0AzuYQBMZinrUYL8EufhtEA= +cloud.google.com/go/auth v0.17.0 h1:74yCm7hCj2rUyyAocqnFzsAYXgJhrG26XCFimrc/Kz4= +cloud.google.com/go/auth v0.17.0/go.mod h1:6wv/t5/6rOPAX4fJiRjKkJCvswLwdet7G8+UGXt7nCQ= cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc= cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c= -cloud.google.com/go/compute/metadata v0.7.0 h1:PBWF+iiAerVNe8UCHxdOt6eHLVc3ydFeOCw78U8ytSU= -cloud.google.com/go/compute/metadata v0.7.0/go.mod h1:j5MvL9PprKL39t166CoB1uVHfQMs4tFQZZcKwksXUjo= -github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0 h1:Gt0j3wceWMwPmiazCa8MzMA0MfhmPIz0Qp0FJ6qcM0U= -github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0/go.mod h1:Ot/6aikWnKWi4l9QB7qVSwa8iMphQNqkWALMoNT3rzM= -github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.10.1 h1:B+blDbyVIG3WaikNxPnhPiJ1MThR03b3vKGtER95TP4= -github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.10.1/go.mod h1:JdM5psgjfBf5fo2uWOZhflPWyDBZ/O/CNAH9CtsuZE4= +cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs= +cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1 h1:5YTBM8QDVIBN3sxBil89WfdAAqDZbyJTgh688DSxX5w= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1/go.mod h1:YD5h/ldMsG0XiIw7PdyNhLxaM317eFh5yNLccNfGdyw= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0 h1:wL5IEG5zb7BVv1Kv0Xm92orq+5hB5Nipn3B5tn4Rqfk= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0/go.mod h1:J7MUC/wtRpfGVbQ5sIItY5/FuVWmvzlY21WAOfQnq/I= github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2 h1:yz1bePFlP5Vws5+8ez6T3HWXPmwOK7Yvq8QxDBD3SKY= github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2/go.mod h1:Pa9ZNPuoNu/GztvBSKk9J1cDJW6vk/n0zLtV4mgd8N8= -github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1 h1:FPKJS1T+clwv+OLGt13a8UjqeRuh0O4SJ3lUriThc+4= -github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1/go.mod h1:j2chePtV91HrC22tGoRX3sGY42uF13WzmmV80/OdVAA= +github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 h1:9iefClla7iYpfYWdzPCRDozdmndjTm8DXdpCzPajMgA= +github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2/go.mod h1:XtLgD3ZD34DAaVIIAyG3objl5DynM3CQ/vMcbBNJZGI= github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.7.0 h1:LkHbJbgF3YyvC53aqYGR+wWQDn2Rdp9AQdGndf9QvY4= github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.7.0/go.mod h1:QyiQdW4f4/BIfB8ZutZ2s+28RAgfa/pT+zS++ZHyM1I= github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v4 v4.3.0 h1:bXwSugBiSbgtz7rOtbfGf+woewp4f06orW9OP5BjHLA= github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v4 v4.3.0/go.mod h1:Y/HgrePTmGy9HjdSGTqZNa+apUpTVIEVKXJyARP2lrk= github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM= github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE= -github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 h1:oygO0locgZJe7PpYPXT5A29ZkwJaPqcva7BVeemZOZs= -github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI= +github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 h1:XkkQbfMyuH2jTSjQjSoihryI8GINRcs4xp8lNawg0FI= +github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk= github.com/Code-Hex/go-generics-cache v1.5.1 h1:6vhZGc5M7Y/YD8cIUcY8kcuQLB4cHR7U+0KMqAA0KcU= github.com/Code-Hex/go-generics-cache v1.5.1/go.mod h1:qxcC9kRVrct9rHeiYpFWSoW1vxyillCVzX13KZG8dl4= github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow= @@ -33,36 +33,38 @@ github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7D github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk= github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA= github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4= -github.com/aws/aws-sdk-go-v2 v1.37.0 h1:YtCOESR/pN4j5oA7cVHSfOwIcuh/KwHC4DOSXFbv5F0= -github.com/aws/aws-sdk-go-v2 v1.37.0/go.mod h1:9Q0OoGQoboYIAJyslFyF1f5K1Ryddop8gqMhWx/n4Wg= -github.com/aws/aws-sdk-go-v2/config v1.29.14 h1:f+eEi/2cKCg9pqKBoAIwRGzVb70MRKqWX4dg1BDcSJM= -github.com/aws/aws-sdk-go-v2/config v1.29.14/go.mod h1:wVPHWcIFv3WO89w0rE10gzf17ZYy+UVS1Geq8Iei34g= -github.com/aws/aws-sdk-go-v2/credentials v1.17.67 h1:9KxtdcIA/5xPNQyZRgUSpYOE6j9Bc4+D7nZua0KGYOM= -github.com/aws/aws-sdk-go-v2/credentials v1.17.67/go.mod h1:p3C44m+cfnbv763s52gCqrjaqyPikj9Sg47kUVaNZQQ= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30 h1:x793wxmUWVDhshP8WW2mlnXuFrO4cOd3HLBroh1paFw= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30/go.mod h1:Jpne2tDnYiFascUEs2AWHJL9Yp7A5ZVy3TNyxaAjD6M= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.0 h1:H2iZoqW/v2Jnrh1FnU725Bq6KJ0k2uP63yH+DcY+HUI= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.0/go.mod h1:L0FqLbwMXHvNC/7crWV1iIxUlOKYZUE8KuTIA+TozAI= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.0 h1:EDped/rNzAhFPhVY0sDGbtD16OKqksfA8OjF/kLEgw8= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.0/go.mod h1:uUI335jvzpZRPpjYx6ODc/wg1qH+NnoSTK/FwVeK0C0= -github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 h1:bIqFDwgGXXN1Kpp99pDOdKMTTb5d2KyU5X/BZxjOkRo= -github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3/go.mod h1:H5O/EsxDWyU+LP/V8i5sm8cxoZgc2fdNR9bxlOFrQTo= -github.com/aws/aws-sdk-go-v2/service/ec2 v1.237.0 h1:XHE2G+yaDQql32FZt19QmQt4WuisqQJIkMUSCxeCUl8= -github.com/aws/aws-sdk-go-v2/service/ec2 v1.237.0/go.mod h1:t11/j/nH9i6bbsPH9xc04BJOsV2nVPUqrB67/TLDsyM= -github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.0 h1:6+lZi2JeGKtCraAj1rpoZfKqnQ9SptseRZioejfUOLM= -github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.0/go.mod h1:eb3gfbVIxIoGgJsi9pGne19dhCBpK6opTYpQqAmdy44= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.0 h1:eRhU3Sh8dGbaniI6B+I48XJMrTPRkK4DKo+vqIxziOU= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.0/go.mod h1:paNLV18DZ6FnWE/bd06RIKPDIFpjuvCkGKWTG/GDBeM= -github.com/aws/aws-sdk-go-v2/service/lightsail v1.44.0 h1:QiiCqpKy0prxq+92uWfESzcb7/8Y9JAamcMOzVYLEoM= -github.com/aws/aws-sdk-go-v2/service/lightsail v1.44.0/go.mod h1:ESppxYqXQCpCY+KWl3BdkQjmsQX6zxKP39SnDtRDoU0= -github.com/aws/aws-sdk-go-v2/service/sso v1.25.3 h1:1Gw+9ajCV1jogloEv1RRnvfRFia2cL6c9cuKV2Ps+G8= -github.com/aws/aws-sdk-go-v2/service/sso v1.25.3/go.mod h1:qs4a9T5EMLl/Cajiw2TcbNt2UNo/Hqlyp+GiuG4CFDI= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.1 h1:hXmVKytPfTy5axZ+fYbR5d0cFmC3JvwLm5kM83luako= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.1/go.mod h1:MlYRNmYu/fGPoxBQVvBYr9nyr948aY/WLUvwBMBJubs= -github.com/aws/aws-sdk-go-v2/service/sts v1.33.19 h1:1XuUZ8mYJw9B6lzAkXhqHlJd/XvaX32evhproijJEZY= -github.com/aws/aws-sdk-go-v2/service/sts v1.33.19/go.mod h1:cQnB8CUnxbMU82JvlqjKR2HBOm3fe9pWorWBza6MBJ4= -github.com/aws/smithy-go v1.22.5 h1:P9ATCXPMb2mPjYBgueqJNCA5S9UfktsW0tTxi+a7eqw= -github.com/aws/smithy-go v1.22.5/go.mod h1:t1ufH5HMublsJYulve2RKmHDC15xu1f26kHCp/HgceI= +github.com/aws/aws-sdk-go-v2 v1.39.6 h1:2JrPCVgWJm7bm83BDwY5z8ietmeJUbh3O2ACnn+Xsqk= +github.com/aws/aws-sdk-go-v2 v1.39.6/go.mod h1:c9pm7VwuW0UPxAEYGyTmyurVcNrbF6Rt/wixFqDhcjE= +github.com/aws/aws-sdk-go-v2/config v1.31.17 h1:QFl8lL6RgakNK86vusim14P2k8BFSxjvUkcWLDjgz9Y= +github.com/aws/aws-sdk-go-v2/config v1.31.17/go.mod h1:V8P7ILjp/Uef/aX8TjGk6OHZN6IKPM5YW6S78QnRD5c= +github.com/aws/aws-sdk-go-v2/credentials v1.18.21 h1:56HGpsgnmD+2/KpG0ikvvR8+3v3COCwaF4r+oWwOeNA= +github.com/aws/aws-sdk-go-v2/credentials v1.18.21/go.mod h1:3YELwedmQbw7cXNaII2Wywd+YY58AmLPwX4LzARgmmA= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 h1:T1brd5dR3/fzNFAQch/iBKeX07/ffu/cLu+q+RuzEWk= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13/go.mod h1:Peg/GBAQ6JDt+RoBf4meB1wylmAipb7Kg2ZFakZTlwk= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 h1:a+8/MLcWlIxo1lF9xaGt3J/u3yOZx+CdSveSNwjhD40= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13/go.mod h1:oGnKwIYZ4XttyU2JWxFrwvhF6YKiK/9/wmE3v3Iu9K8= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 h1:HBSI2kDkMdWz4ZM7FjwE7e/pWDEZ+nR95x8Ztet1ooY= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13/go.mod h1:YE94ZoDArI7awZqJzBAZ3PDD2zSfuP7w6P2knOzIn8M= +github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk= +github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc= +github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0 h1:5qBb1XV/D18qtCHd3bmmxoVglI+fZ4QWuS/EB8kIXYQ= +github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0/go.mod h1:NDdDLLW5PtLLXN661gKcvJvqAH5OBXsfhMlmKVu1/pY= +github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2 h1:oeICOX/+D0XXV1aMYJPXVe3CO37zYr7fB6HFgxchleU= +github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2/go.mod h1:rrhqfkXfa2DSNq0RyFhnnFEAyI+yJB4+2QlZKeJvMjs= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13/go.mod h1:lmKuogqSU3HzQCwZ9ZtcqOc5XGMqtDK7OIc2+DxiUEg= +github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4 h1:/1o2AYwHJojUDeMvQNyJiKZwcWCc3e4kQuTXqRLuThc= +github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4/go.mod h1:Nn2xx6HojGuNMtUFxxz/nyNLSS+tHMRsMhe3+W3wB5k= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 h1:0JPwLz1J+5lEOfy/g0SURC9cxhbQ1lIMHMa+AHZSzz0= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.1/go.mod h1:fKvyjJcz63iL/ftA6RaM8sRCtN4r4zl4tjL3qw5ec7k= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 h1:OWs0/j2UYR5LOGi88sD5/lhN6TDLG6SfA7CqsQO9zF0= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5/go.mod h1:klO+ejMvYsB4QATfEOIXk8WAEwN4N0aBfJpvC+5SZBo= +github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 h1:mLlUgHn02ue8whiR4BmxxGJLR2gwU6s6ZzJ5wDamBUs= +github.com/aws/aws-sdk-go-v2/service/sts v1.39.1/go.mod h1:E19xDjpzPZC7LS2knI9E6BaRFDK43Eul7vd6rSq2HWk= +github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM= +github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0= github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 h1:6df1vn4bBlDDo4tARvBm7l6KA9iVMnE3NWizDeWSrps= github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3/go.mod h1:CIWtjkly68+yqLPbvwwR/fjNJA/idrtULjZWh2v1ys0= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= @@ -82,25 +84,23 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/dennwc/varint v1.0.0 h1:kGNFFSSw8ToIy3obO/kKr8U9GZYUAxQEVuix4zfDWzE= github.com/dennwc/varint v1.0.0/go.mod h1:hnItb35rvZvJrbTALZtY/iQfDs48JKRG1RPpgziApxA= -github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78= -github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc= -github.com/digitalocean/godo v1.157.0 h1:ReELaS6FxXNf8gryUiVH0wmyUmZN8/NCmBX4gXd3F0o= -github.com/digitalocean/godo v1.157.0/go.mod h1:tYeiWY5ZXVpU48YaFv0M5irUFHXGorZpDNm7zzdWMzM= +github.com/digitalocean/godo v1.168.0 h1:mlORtUcPD91LQeJoznrH3XvfvgK3t8Wvrpph9giUT/Q= +github.com/digitalocean/godo v1.168.0/go.mod h1:xQsWpVCCbkDrWisHA72hPzPlnC+4W5w/McZY5ij9uvU= github.com/distribution/reference v0.5.0 h1:/FUIFXtfc/x2gpa5/VGfiGLuOIdYa1t65IKK2OFGvA0= github.com/distribution/reference v0.5.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= -github.com/docker/docker v28.3.0+incompatible h1:ffS62aKWupCWdvcee7nBU9fhnmknOqDPaJAMtfK0ImQ= -github.com/docker/docker v28.3.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM= +github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ= github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/edsrzf/mmap-go v1.2.0 h1:hXLYlkbaPzt1SaQk+anYwKSRNhufIDCchSPkUD6dD84= github.com/edsrzf/mmap-go v1.2.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q= -github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g= -github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= +github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU= +github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= github.com/envoyproxy/go-control-plane v0.13.4 h1:zEqyPVyku6IvWCFwux4x9RxkLOMUL+1vC9xUFv5l2/M= -github.com/envoyproxy/go-control-plane/envoy v1.32.4 h1:jb83lalDRZSpPWW2Z7Mck/8kXZ5CQAFYVjQcdVIr83A= -github.com/envoyproxy/go-control-plane/envoy v1.32.4/go.mod h1:Gzjc5k8JcJswLjAx1Zm+wSYE20UrLtt7JZMWiWQXQEw= +github.com/envoyproxy/go-control-plane/envoy v1.35.0 h1:ixjkELDE+ru6idPxcHLj8LBVc2bFP7iBytj353BoHUo= +github.com/envoyproxy/go-control-plane/envoy v1.35.0/go.mod h1:09qwbGVuSWWAyN5t/b3iyVfz5+z8QWGrzkoqm/8SbEs= github.com/envoyproxy/protoc-gen-validate v1.2.1 h1:DEo3O99U8j4hBFwbJfrz9VtgcDfUKS7KJ7spH3d86P8= github.com/envoyproxy/protoc-gen-validate v1.2.1/go.mod h1:d/C80l/jxXLdfEIhX1W2TmLfsJ31lvEjwamM4DxlWXU= github.com/facette/natsort v0.0.0-20181210072756-2cd4dd1e2dcb h1:IT4JYU7k4ikYg1SCxNI1/Tieq/NFvh6dzLdgi7eu0tM= @@ -109,10 +109,10 @@ github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM= github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE= github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= -github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M= -github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= -github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E= -github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ= +github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= +github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= +github.com/fxamacker/cbor/v2 v2.9.0 h1:NpKPmjDBgUfBms6tr6JZkTHtfFGcMKsw3eGcmD/sapM= +github.com/fxamacker/cbor/v2 v2.9.0/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= @@ -126,23 +126,22 @@ github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+Gr github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ= github.com/go-resty/resty/v2 v2.16.5 h1:hBKqmWrr7uRc3euHVqmh1HTHcKn99Smr7o5spptdhTM= github.com/go-resty/resty/v2 v2.16.5/go.mod h1:hkJtXbA2iKHzJheXYvQ8snQES5ZLGKMwQ07xAwp/fiA= -github.com/go-viper/mapstructure/v2 v2.3.0 h1:27XbWsHIqhbdR5TIC911OfYvgSaW93HM+dX7970Q7jk= -github.com/go-viper/mapstructure/v2 v2.3.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= +github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs= +github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/go-zookeeper/zk v1.0.4 h1:DPzxraQx7OrPyXq2phlGlNSIyWEsAox0RJmjTseMV6I= github.com/go-zookeeper/zk v1.0.4/go.mod h1:nOB03cncLtlp4t+UAkGSV+9beXP/akpekBwL+UX1Qcw= github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y= github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= -github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8= -github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk= +github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo= +github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs= github.com/golang/snappy v1.0.0/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= -github.com/google/gnostic-models v0.6.9 h1:MU/8wDLif2qCXZmzncUQ/BOfxWfthHi63KqpoNbWqVw= -github.com/google/gnostic-models v0.6.9/go.mod h1:CiWsm0s6BSQd1hRn8/QmxqB6BesYcbSZxsz9b0KuDBw= -github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/gnostic-models v0.7.0 h1:qwTtogB15McXDaNqTZdzPJRHvaVJlAl+HVQnLmJEJxo= +github.com/google/gnostic-models v0.7.0/go.mod h1:whL5G0m6dmc5cPxKc5bdKdEN3UjI7OUGxBlw57miDrQ= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8= @@ -154,18 +153,18 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4= github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA= -github.com/googleapis/gax-go/v2 v2.14.2 h1:eBLnkZ9635krYIPD+ag1USrOAI0Nr0QYF3+/3GqO0k0= -github.com/googleapis/gax-go/v2 v2.14.2/go.mod h1:ON64QhlJkhVtSqp4v1uaK92VyZ2gmvDQsweuyLV+8+w= -github.com/gophercloud/gophercloud/v2 v2.7.0 h1:o0m4kgVcPgHlcXiWAjoVxGd8QCmvM5VU+YM71pFbn0E= -github.com/gophercloud/gophercloud/v2 v2.7.0/go.mod h1:Ki/ILhYZr/5EPebrPL9Ej+tUg4lqx71/YH2JWVeU+Qk= +github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo= +github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc= +github.com/gophercloud/gophercloud/v2 v2.8.0 h1:of2+8tT6+FbEYHfYC8GBu8TXJNsXYSNm9KuvpX7Neqo= +github.com/gophercloud/gophercloud/v2 v2.8.0/go.mod h1:Ki/ILhYZr/5EPebrPL9Ej+tUg4lqx71/YH2JWVeU+Qk= github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo= github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA= -github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc h1:GN2Lv3MGO7AS6PrRoT6yV5+wkrOpcszoIsO4+4ds248= -github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk= +github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 h1:cLN4IBkmkYZNnk7EAJ0BHIethd+J6LqxFNw5mSiI2bM= +github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk= github.com/hashicorp/consul/api v1.32.0 h1:5wp5u780Gri7c4OedGEPzmlUEzi0g2KyiPphSr6zjVg= github.com/hashicorp/consul/api v1.32.0/go.mod h1:Z8YgY0eVPukT/17ejW+l+C7zJmKwgPHtjU1q16v/Y40= -github.com/hashicorp/cronexpr v1.1.2 h1:wG/ZYIKT+RT3QkOdgYc+xsKWVRgnxJ1OJtjjy84fJ9A= -github.com/hashicorp/cronexpr v1.1.2/go.mod h1:P4wA0KBl9C5q2hABiMO7cp6jcIg96CDh1Efb3g1PWA4= +github.com/hashicorp/cronexpr v1.1.3 h1:rl5IkxXN2m681EfivTlccqIryzYJSXRGRNa0xeG7NA4= +github.com/hashicorp/cronexpr v1.1.3/go.mod h1:P4wA0KBl9C5q2hABiMO7cp6jcIg96CDh1Efb3g1PWA4= github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ= @@ -184,12 +183,12 @@ github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKe github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/golang-lru v0.6.0 h1:uL2shRDx7RTrOrTCUZEGP/wJUFiUI8QT6E7z5o8jga4= github.com/hashicorp/golang-lru v0.6.0/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= -github.com/hashicorp/nomad/api v0.0.0-20241218080744-e3ac00f30eec h1:+YBzb977VrmffaCX/OBm17dEVJUcWn5dW+eqs3aIJ/A= -github.com/hashicorp/nomad/api v0.0.0-20241218080744-e3ac00f30eec/go.mod h1:svtxn6QnrQ69P23VvIWMR34tg3vmwLz4UdUzm1dSCgE= +github.com/hashicorp/nomad/api v0.0.0-20250930071859-eaa0fe0e27af h1:ScAYf8O+9xTqTJPZH8MIlUfO+ak8cb31rW1aYJgS+jE= +github.com/hashicorp/nomad/api v0.0.0-20250930071859-eaa0fe0e27af/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE= github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY= github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4= -github.com/hetznercloud/hcloud-go/v2 v2.21.1 h1:IH3liW8/cCRjfJ4cyqYvw3s1ek+KWP8dl1roa0lD8JM= -github.com/hetznercloud/hcloud-go/v2 v2.21.1/go.mod h1:XOaYycZJ3XKMVWzmqQ24/+1V7ormJHmPdck/kxrNnQA= +github.com/hetznercloud/hcloud-go/v2 v2.29.0 h1:LzNFw5XLBfftyu3WM1sdSLjOZBlWORtz2hgGydHaYV8= +github.com/hetznercloud/hcloud-go/v2 v2.29.0/go.mod h1:XBU4+EDH2KVqu2KU7Ws0+ciZcX4ygukQl/J0L5GS8P8= github.com/influxdata/influxdb-client-go/v2 v2.14.0 h1:AjbBfJuq+QoaXNcrova8smSjwJdUHnwvfjMF71M1iI4= github.com/influxdata/influxdb-client-go/v2 v2.14.0/go.mod h1:Ahpm3QXKMJslpXl3IftVLVezreAUtBOTZssDrjZEFHI= github.com/influxdata/line-protocol v0.0.0-20200327222509-2487e7298839 h1:W9WBk7wlPfJLvMCdtV4zPulc4uCPrlywQOmbFOhgQNU= @@ -207,14 +206,14 @@ github.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRt github.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= -github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= +github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo= github.com/knadh/koanf/maps v0.1.2/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI= github.com/knadh/koanf/providers/confmap v1.0.0 h1:mHKLJTE7iXEys6deO5p6olAiZdG5zwp8Aebir+/EaRE= github.com/knadh/koanf/providers/confmap v1.0.0/go.mod h1:txHYHiI2hAtF0/0sCmcuol4IDcuQbKTybiB1nOcUo1A= -github.com/knadh/koanf/v2 v2.2.1 h1:jaleChtw85y3UdBnI0wCqcg1sj1gPoz6D3caGNHtrNE= -github.com/knadh/koanf/v2 v2.2.1/go.mod h1:PSFru3ufQgTsI7IF+95rf9s8XA1+aHxKuO/W+dPoHEY= +github.com/knadh/koanf/v2 v2.3.0 h1:Qg076dDRFHvqnKG97ZEsi9TAg2/nFTa9hCdcSa1lvlM= +github.com/knadh/koanf/v2 v2.3.0/go.mod h1:gRb40VRAbd4iJMYYD5IxZ6hfuopFcXBpc9bbQpZwo28= github.com/kolo/xmlrpc v0.0.0-20220921171641-a4b6fa1dd06b h1:udzkj9S/zlT5X367kqJis0QP7YMxobob6zhzq6Yre00= github.com/kolo/xmlrpc v0.0.0-20220921171641-a4b6fa1dd06b/go.mod h1:pcaDhQK0/NJZEvtCO0qQPPropqV0sJOJ6YW7X+9kRwM= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= @@ -223,16 +222,16 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= -github.com/linode/linodego v1.52.2 h1:N9ozU27To1LMSrDd8WvJZ5STSz1eGYdyLnxhAR/dIZg= -github.com/linode/linodego v1.52.2/go.mod h1:bI949fZaVchjWyKIA08hNyvAcV6BAS+PM2op3p7PAWA= +github.com/linode/linodego v1.60.0 h1:SgsebJFRCi+lSmYy+C40wmKZeJllGGm+W12Qw4+yVdI= +github.com/linode/linodego v1.60.0/go.mod h1:1+Bt0oTz5rBnDOJbGhccxn7LYVytXTIIfAy7QYmijDs= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA= github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= -github.com/miekg/dns v1.1.66 h1:FeZXOS3VCVsKnEAd+wBkjMC3D2K+ww66Cq3VnCINuJE= -github.com/miekg/dns v1.1.66/go.mod h1:jGFzBsSNbJw6z1HYut1RKBKHA9PBdxeHrZG8J+gC2WE= +github.com/miekg/dns v1.1.68 h1:jsSRkNozw7G/mnmXULynzMNIsgY2dHC8LO6U6Ij2JEA= +github.com/miekg/dns v1.1.68/go.mod h1:fujopn7TB3Pu3JM69XaawiU0wqjpL9/8xGop5UrTPps= github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw= github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s= github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y= @@ -258,14 +257,12 @@ github.com/oapi-codegen/runtime v1.0.0/go.mod h1:LmCUMQuPB4M/nLXilQXhHw+BLZdDb18 github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4= github.com/oklog/ulid/v2 v2.1.1 h1:suPZ4ARWLOJLegGFiZZ1dFAkqzhMjL3J1TzI+5wHz8s= github.com/oklog/ulid/v2 v2.1.1/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ= -github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.129.0 h1:2pzb6bC/AAfciC9DN+8d7Y8Rsk8ZPCfp/ACTfZu87FQ= -github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.129.0/go.mod h1:tIE4dzdxuM7HnFeYA6sj5zfLuUA/JxzQ+UDl1YrHvQw= -github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.129.0 h1:ydkfqpZ5BWZfEJEs7OUhTHW59og5aZspbUYxoGcAEok= -github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.129.0/go.mod h1:oA+49dkzmhUx0YFC9JXGuPPSBL0TOTp6jkv7qSr2n0Q= -github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.129.0 h1:AOVxBvCZfTPj0GLGqBVHpAnlC9t9pl1JXUQXymHliiY= -github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.129.0/go.mod h1:0CAJ32V/bCUBhNTEvnN9wlOG5IsyZ+Bmhe9e3Eri7CU= -github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.129.0 h1:yDLSAoIi3jNt4R/5xN4IJ9YAg1rhOShgchlO/ESv8EY= -github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.129.0/go.mod h1:IXQHbTPxqNcuu44FvkyvpYJ6Qy4wh4YsCVkKsp0Flzo= +github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.139.0 h1:D5aGQCErSCb4sKIHoZhgR4El6AzgviTRYlHUpbSFqDo= +github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.139.0/go.mod h1:ZjeRsA5oaVk89fg5D+iXStx2QncmhAvtGbdSumT07H4= +github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.139.0 h1:6/j0Ta8ZJnmAFVEoC3aZ1Hs19RB4fHzlN6kOZhsBJqM= +github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.139.0/go.mod h1:VfA8xHz4xg7Fyj5bBsCDbOO3iVYzDn9wP/QFsjcAE5c= +github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.139.0 h1:iRNX/ueuad1psOVgnNkxuQmXxvF3ze5ZZCP66xKFk/w= +github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.139.0/go.mod h1:bW09lo3WgHsPsZ1mgsJvby9wCefT5o13patM5phdfIU= github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.0.2 h1:9yCKha/T5XdGtO0q9Q9a6T5NUCsTn/DrBg0D7ufOcFM= @@ -283,31 +280,31 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca h1:BOxmsLoL2ymn8lXJtorca7N/m+2vDQUDoEtPjf0iAxA= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca/go.mod h1:gndBHh3ZdjBozGcGrjUYjN3UJLRS3l2drALtu4lUt+k= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= -github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= -github.com/prometheus/otlptranslator v0.0.2 h1:+1CdeLVrRQ6Psmhnobldo0kTp96Rj80DRXRd5OSnMEQ= -github.com/prometheus/otlptranslator v0.0.2/go.mod h1:P8AwMgdD7XEr6QRUJ2QWLpiAZTgTE2UYgjlu3svompI= +github.com/prometheus/common v0.67.4 h1:yR3NqWO1/UyO1w2PhUvXlGQs/PtFmoveVO0KZ4+Lvsc= +github.com/prometheus/common v0.67.4/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= +github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEoIwkU+A6qos= +github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= -github.com/prometheus/prometheus v0.305.1-0.20250905124657-5c2e43f09c03 h1:NIVtqQm7NTsUcxfjdHuVE7pw3GVjEgwL6a9ADLSj+Wg= -github.com/prometheus/prometheus v0.305.1-0.20250905124657-5c2e43f09c03/go.mod h1:9D9CfSEbKg087QXXz2ev+G1SoB6MqQE0ll4jCmrgCe0= -github.com/prometheus/sigv4 v0.2.0 h1:qDFKnHYFswJxdzGeRP63c4HlH3Vbn1Yf/Ao2zabtVXk= -github.com/prometheus/sigv4 v0.2.0/go.mod h1:D04rqmAaPPEUkjRQxGqjoxdyJuyCh6E0M18fZr0zBiE= +github.com/prometheus/prometheus v0.308.1 h1:ApMNI/3/es3Ze90Z7CMb+wwU2BsSYur0m5VKeqHj7h4= +github.com/prometheus/prometheus v0.308.1/go.mod h1:aHjYCDz9zKRyoUXvMWvu13K9XHOkBB12XrEqibs3e0A= +github.com/prometheus/sigv4 v0.3.0 h1:QIG7nTbu0JTnNidGI1Uwl5AGVIChWUACxn2B/BQ1kms= +github.com/prometheus/sigv4 v0.3.0/go.mod h1:fKtFYDus2M43CWKMNtGvFNHGXnAJJEGZbiYCmVp/F8I= github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg= github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA= -github.com/redis/go-redis/v9 v9.8.0 h1:q3nRvjrlge/6UD7eTu/DSg2uYiU2mCL0G/uzBWqhicI= -github.com/redis/go-redis/v9 v9.8.0/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw= github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o= -github.com/scaleway/scaleway-sdk-go v1.0.0-beta.33 h1:KhF0WejiUTDbL5X55nXowP7zNopwpowa6qaMAWyIE+0= -github.com/scaleway/scaleway-sdk-go v1.0.0-beta.33/go.mod h1:792k1RTU+5JeMXm35/e2Wgp71qPH/DmDoZrRc+EFZDk= -github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= -github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35 h1:8xfn1RzeI9yoCUuEwDy08F+No6PcKZGEDOQ6hrRyLts= +github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35/go.mod h1:47B1d/YXmSAxlJxUJxClzHR6b3T4M1WyCvwENPQNBWc= +github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o= +github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKkMo8ZTx3f+BZEkzsRUY10Xsm2mwU0= -github.com/stackitcloud/stackit-sdk-go/core v0.17.2 h1:jPyn+i8rkp2hM80+hOg0B/1EVRbMt778Tr5RWyK1m2E= -github.com/stackitcloud/stackit-sdk-go/core v0.17.2/go.mod h1:8KIw3czdNJ9sdil9QQimxjR6vHjeINFrRv0iZ67wfn0= +github.com/stackitcloud/stackit-sdk-go/core v0.17.3 h1:GsZGmRRc/3GJLmCUnsZswirr5wfLRrwavbnL/renOqg= +github.com/stackitcloud/stackit-sdk-go/core v0.17.3/go.mod h1:HBCXJGPgdRulplDzhrmwC+Dak9B/x0nzNtmOpu+1Ahg= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= @@ -332,14 +329,14 @@ go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJyS go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= go.opentelemetry.io/collector/component v1.45.0 h1:gGFfVdbQ+1YuyUkJjWo85I7euu3H/CiupuzCHv8OgHA= go.opentelemetry.io/collector/component v1.45.0/go.mod h1:xoNFnRKE8Iv6gmlqAKgjayWraRnDcYLLgrPt9VgyO2g= -go.opentelemetry.io/collector/component/componentstatus v0.129.0 h1:ejpBAt7hXAAZiQKcSxLvcy8sj8SjY4HOLdoXIlW6ybw= -go.opentelemetry.io/collector/component/componentstatus v0.129.0/go.mod h1:/dLPIxn/tRMWmGi+DPtuFoBsffOLqPpSZ2IpEQzYtwI= -go.opentelemetry.io/collector/component/componenttest v0.129.0 h1:gpKkZGCRPu3Yn0U2co09bMvhs17yLFb59oV8Gl9mmRI= -go.opentelemetry.io/collector/component/componenttest v0.129.0/go.mod h1:JR9k34Qvd/pap6sYkPr5QqdHpTn66A5lYeYwhenKBAM= -go.opentelemetry.io/collector/confmap v1.35.0 h1:U4JDATAl4PrKWe9bGHbZkoQXmJXefWgR2DIkFvw8ULQ= -go.opentelemetry.io/collector/confmap v1.35.0/go.mod h1:qX37ExVBa+WU4jWWJCZc7IJ+uBjb58/9oL+/ctF1Bt0= -go.opentelemetry.io/collector/confmap/xconfmap v0.129.0 h1:Q/+pJKrkCaMPSoSAH2BpC3UZCh+5hTiFkh/bdy5yChk= -go.opentelemetry.io/collector/confmap/xconfmap v0.129.0/go.mod h1:RNMnlay2meJDXcKjxiLbST9/YAhKLJlj0kZCrJrLGgw= +go.opentelemetry.io/collector/component/componentstatus v0.139.0 h1:bQmkv1t7xW7uIDireE0a2Am4IMOprXm6zQr/qDtGCIA= +go.opentelemetry.io/collector/component/componentstatus v0.139.0/go.mod h1:ibZOohpG0u081/NaT/jMCTsKwRbbwwxWrjZml+owpyM= +go.opentelemetry.io/collector/component/componenttest v0.139.0 h1:x9Yu2eYhrHxdZ7sFXWtAWVjQ3UIraje557LgNurDC2I= +go.opentelemetry.io/collector/component/componenttest v0.139.0/go.mod h1:S9cj+qkf9FgHMzjvlYsLwQKd9BiS7B7oLZvxvlENM/c= +go.opentelemetry.io/collector/confmap v1.45.0 h1:7M7TTlpzX4r+mIzP/ARdxZBAvI4N+1V96phDane+akU= +go.opentelemetry.io/collector/confmap v1.45.0/go.mod h1:AE1dnkjv0T9gptsh5+mTX0XFGdXx0n7JS4b7CcPfJ6Q= +go.opentelemetry.io/collector/confmap/xconfmap v0.139.0 h1:uQGpFuWnTCXqdMbI3gDSvkwU66/kF/aoC0kVMrit1EM= +go.opentelemetry.io/collector/confmap/xconfmap v0.139.0/go.mod h1:d0ucaeNq2rojFRSQsCHF/gkT3cgBx5H2bVkPQMj57ck= go.opentelemetry.io/collector/consumer v1.45.0 h1:TtqXxgW+1GSCwdoohq0fzqnfqrZBKbfo++1XRj8mrEA= go.opentelemetry.io/collector/consumer v1.45.0/go.mod h1:pJzqTWBubwLt8mVou+G4/Hs23b3m425rVmld3LqOYpY= go.opentelemetry.io/collector/consumer/consumertest v0.139.0 h1:06mu43mMO7l49ASJ/GEbKgTWcV3py5zE/pKhNBZ1b3k= @@ -352,16 +349,16 @@ go.opentelemetry.io/collector/pdata v1.45.0 h1:q4XaISpeX640BcwXwb2mKOVw/gb67r22H go.opentelemetry.io/collector/pdata v1.45.0/go.mod h1:5q2f001YhwMQO8QvpFhCOa4Cq/vtwX9W4HRMsXkU/nE= go.opentelemetry.io/collector/pdata/pprofile v0.139.0 h1:UA5TgFzYmRuJN3Wz0GR1efLUfjbs5rH0HTaxfASpTR8= go.opentelemetry.io/collector/pdata/pprofile v0.139.0/go.mod h1:sI5qHt+zzE2fhOWFdJIaiDBR0yGGjD4A4ZvDFU0tiHk= -go.opentelemetry.io/collector/pdata/testdata v0.129.0 h1:n1QLnLOtrcAR57oMSVzmtPsQEpCc/nE5Avk1xfuAkjY= -go.opentelemetry.io/collector/pdata/testdata v0.129.0/go.mod h1:RfY5IKpmcvkS2IGVjl9jG9fcT7xpQEBWpg9sQOn/7mY= +go.opentelemetry.io/collector/pdata/testdata v0.139.0 h1:n7O5bmLLhc3T6PePV4447fFcI/6QWcMhBsLtfCaD0do= +go.opentelemetry.io/collector/pdata/testdata v0.139.0/go.mod h1:fxZ2VrhYLYBLHYBHC1XQRKZ6IJXwy0I2rPaaRlebYaY= go.opentelemetry.io/collector/pipeline v1.45.0 h1:sn9JJAEBe3XABTkWechMk0eH60QMBjjNe5V+ccBl+Uo= go.opentelemetry.io/collector/pipeline v1.45.0/go.mod h1:xUrAqiebzYbrgxyoXSkk6/Y3oi5Sy3im2iCA51LwUAI= go.opentelemetry.io/collector/processor v1.45.0 h1:GH5km9BkDQOoz7MR0jzTnzB1Kb5vtKzPwa/wDmRg2dQ= go.opentelemetry.io/collector/processor v1.45.0/go.mod h1:wdlaTTC3wqlZIJP9R9/SLc2q7h+MFGARsxfjgPtwbes= -go.opentelemetry.io/collector/processor/processortest v0.129.0 h1:r5iJHdS7Ffdb2zmMVYx4ahe92PLrce5cas/AJEXivkY= -go.opentelemetry.io/collector/processor/processortest v0.129.0/go.mod h1:gdf8GzyzjGoDTA11+CPwC4jfXphtC+B7MWbWn+LIWXc= -go.opentelemetry.io/collector/processor/xprocessor v0.129.0 h1:V3Zgd+YIeu3Ij3DPlGtzdcTwpqOQIqQVcL5jdHHS7sc= -go.opentelemetry.io/collector/processor/xprocessor v0.129.0/go.mod h1:78T+AP5NO137W/E+SibQhaqOyS67fR+IN697b4JFh00= +go.opentelemetry.io/collector/processor/processortest v0.139.0 h1:30akUdruFNG7EDpayuBhXoX2lV+hcfxW9Gl3Z6MYHb0= +go.opentelemetry.io/collector/processor/processortest v0.139.0/go.mod h1:RTll3UKHrqj/VS6RGjTHtuGIJzyLEwFhbw8KuCL3pjo= +go.opentelemetry.io/collector/processor/xprocessor v0.139.0 h1:O9x9RF/OG8gZ+HrOcB4f6F1fjniby484xf2D8GBxgqU= +go.opentelemetry.io/collector/processor/xprocessor v0.139.0/go.mod h1:hqGhEZ1/PftD/QHaYna0o1xAqZUsb7GhqpOiaTTDJnQ= go.opentelemetry.io/collector/semconv v0.128.0 h1:MzYOz7Vgb3Kf5D7b49pqqgeUhEmOCuT10bIXb/Cc+k4= go.opentelemetry.io/collector/semconv v0.128.0/go.mod h1:OPXer4l43X23cnjLXIZnRj/qQOjSuq4TgBLI76P9hns= go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 h1:2pn7OzMewmYRiNtv1doZnLo3gONcnMHlFnmOR8Vgt+8= @@ -392,65 +389,67 @@ go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= -go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= -go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= +go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= +go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= +go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4= -golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc= -golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 h1:yqrTHse8TCMW1M1ZCP+VAR/l0kKxwaAIqN/il7x4voA= -golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8/go.mod h1:tujkw807nyEEAamNbDrEGzRav+ilXA7PCRAd6xsmwiU= +golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04= +golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0= +golang.org/x/exp v0.0.0-20250808145144-a408d31f581a h1:Y+7uR/b1Mw2iSXZ3G//1haIiSElDQZ8KWh0h+sZPG90= +golang.org/x/exp v0.0.0-20250808145144-a408d31f581a/go.mod h1:rT6SFzZ7oxADUDx58pcaKFTcZ+inxAa9fTrYx/uVYwg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.26.0 h1:EGMPT//Ezu+ylkCijjPc+f4Aih7sZvaAr+O3EHBxvZg= -golang.org/x/mod v0.26.0/go.mod h1:/j6NAhSk8iQ723BGAUyoAcn7SlD7s15Dp9Nd/SfeaFQ= +golang.org/x/mod v0.28.0 h1:gQBtGhjxykdjY9YhZpSlZIsbnaE2+PgjfLWUQTnoZ1U= +golang.org/x/mod v0.28.0/go.mod h1:yfB/L0NOf/kmEbXjzCPOx1iK1fRutOydrCMsqRhEBxI= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE= -golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg= -golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI= -golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU= +golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4= +golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210= +golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= +golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw= -golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= +golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= +golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= -golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= -golang.org/x/term v0.34.0 h1:O/2T7POpk0ZZ7MAzMeWFSg6S5IpWd/RXDlM9hgM3DR4= -golang.org/x/term v0.34.0/go.mod h1:5jC53AEywhIVebHgPVeg0mj8OD3VO9OzclacVrqpaAw= +golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ= +golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/term v0.36.0 h1:zMPR+aF8gfksFprF/Nc/rd1wRS1EI6nDBGyWAvDzx2Q= +golang.org/x/term v0.36.0/go.mod h1:Qu394IJq6V6dCBRgwqshf3mPF85AqzYEzofzRdZkWss= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng= -golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU= -golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE= -golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg= +golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k= +golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM= +golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= +golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.35.0 h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0= -golang.org/x/tools v0.35.0/go.mod h1:NKdj5HkL/73byiZSJjqJgKn3ep7KjFkBOkR/Hps3VPw= +golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE= +golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk= gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E= -google.golang.org/api v0.239.0 h1:2hZKUnFZEy81eugPs4e2XzIJ5SOwQg0G82bpXD65Puo= -google.golang.org/api v0.239.0/go.mod h1:cOVEm2TpdAGHL2z+UwyS+kmlGr3bVWQQ6sYEqkKje50= -google.golang.org/genproto v0.0.0-20250505200425-f936aa4a68b2 h1:1tXaIXCracvtsRxSBsYDiSBN0cuJvM7QYW+MrpIRY78= -google.golang.org/genproto/googleapis/api v0.0.0-20250804133106-a7a43d27e69b h1:ULiyYQ0FdsJhwwZUwbaXpZF5yUE3h+RA+gxvBu37ucc= -google.golang.org/genproto/googleapis/api v0.0.0-20250804133106-a7a43d27e69b/go.mod h1:oDOGiMSXHL4sDTJvFvIB9nRQCGdLP1o/iVaqQK8zB+M= -google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b h1:zPKJod4w6F1+nRGDI9ubnXYhU9NSWoFAijkHkUXeTK8= -google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A= +google.golang.org/api v0.252.0 h1:xfKJeAJaMwb8OC9fesr369rjciQ704AjU/psjkKURSI= +google.golang.org/api v0.252.0/go.mod h1:dnHOv81x5RAmumZ7BWLShB/u7JZNeyalImxHmtTHxqw= +google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4= +google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 h1:8XJ4pajGwOlasW+L13MnEGA8W4115jJySQtVfS2/IBU= +google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4/go.mod h1:NnuHhy+bxcg30o7FnVAZbXsPHUDQ9qKWAQKCD7VxFtk= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797 h1:CirRxTOwnRWVLKzDNrs0CXAaVozJoR4G9xvdRecrdpk= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797/go.mod h1:HSkG/KdJWusxU1F6CNrwNDjBMgisKxGnc5dAZfT0mjQ= google.golang.org/grpc v1.76.0 h1:UnVkv1+uMLYXoIz6o7chp59WfQUYA2ex/BXQ9rHZu7A= google.golang.org/grpc v1.76.0/go.mod h1:Ju12QI8M6iQJtbcsV+awF5a4hfJMLi4X0JLo94ULZ6c= google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= @@ -469,23 +468,23 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -k8s.io/api v0.33.5 h1:YR+uhYj05jdRpcksv8kjSliW+v9hwXxn6Cv10aR8Juw= -k8s.io/api v0.33.5/go.mod h1:2gzShdwXKT5yPGiqrTrn/U/nLZ7ZyT4WuAj3XGDVgVs= -k8s.io/apimachinery v0.33.5 h1:NiT64hln4TQXeYR18/ES39OrNsjGz8NguxsBgp+6QIo= -k8s.io/apimachinery v0.33.5/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM= -k8s.io/client-go v0.33.5 h1:I8BdmQGxInpkMEnJvV6iG7dqzP3JRlpZZlib3OMFc3o= -k8s.io/client-go v0.33.5/go.mod h1:W8PQP4MxbM4ypgagVE65mUUqK1/ByQkSALF9tzuQ6u0= +k8s.io/api v0.34.1 h1:jC+153630BMdlFukegoEL8E/yT7aLyQkIVuwhmwDgJM= +k8s.io/api v0.34.1/go.mod h1:SB80FxFtXn5/gwzCoN6QCtPD7Vbu5w2n1S0J5gFfTYk= +k8s.io/apimachinery v0.34.1 h1:dTlxFls/eikpJxmAC7MVE8oOeP1zryV7iRyIjB0gky4= +k8s.io/apimachinery v0.34.1/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw= +k8s.io/client-go v0.34.1 h1:ZUPJKgXsnKwVwmKKdPfw4tB58+7/Ik3CrjOEhsiZ7mY= +k8s.io/client-go v0.34.1/go.mod h1:kA8v0FP+tk6sZA0yKLRG67LWjqufAoSHA2xVGKw9Of8= k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk= k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE= -k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff h1:/usPimJzUKKu+m+TE36gUyGcf03XZEP0ZIKgKj35LS4= -k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff/go.mod h1:5jIi+8yX4RIb8wk3XwBo5Pq2ccx4FP10ohkbSKCZoK8= -k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 h1:M3sRQVHv7vB20Xc2ybTt7ODCeFj6JSWYFzOFnYeS6Ro= -k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= -sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8= -sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo= +k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b h1:MloQ9/bdJyIu9lb1PzujOPolHyvO06MXG5TUIj2mNAA= +k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b/go.mod h1:UZ2yyWbFTpuhSbFhv24aGNOdoRdJZgsIObGBUaYVsts= +k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 h1:hwvWFiBzdWw1FhfY1FooPn3kzWuJ8tmbZBHi4zVsl1Y= +k8s.io/utils v0.0.0-20250604170112-4c0f3b243397/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 h1:gBQPwqORJ8d8/YNZWEjoZs7npUVDpVXUUOFfW6CgAqE= +sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg= sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU= sigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY= -sigs.k8s.io/structured-merge-diff/v4 v4.6.0 h1:IUA9nvMmnKWcj5jl84xn+T5MnlZKThmUW1TdblaLVAc= -sigs.k8s.io/structured-merge-diff/v4 v4.6.0/go.mod h1:dDy58f92j70zLsuZVuUX5Wp9vtxXpaZnkPGWeqDfCps= -sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E= -sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY= +sigs.k8s.io/structured-merge-diff/v6 v6.3.0 h1:jTijUJbW353oVOd9oTlifJqOGEkUw2jB/fXCbTiQEco= +sigs.k8s.io/structured-merge-diff/v6 v6.3.0/go.mod h1:M3W8sfWvn2HhQDIbGWj3S099YozAsymCo/wrT5ohRUE= +sigs.k8s.io/yaml v1.6.0 h1:G8fkbMSAFqgEFgh4b1wmtzDnioxFCUgTZhlbj5P9QYs= +sigs.k8s.io/yaml v1.6.0/go.mod h1:796bPqUfzR/0jLAl6XjHl3Ck7MiyVv8dbTdyT3/pMf4= diff --git a/go.mod b/go.mod index 7d830e86a2..6ebb6c46fe 100644 --- a/go.mod +++ b/go.mod @@ -1,62 +1,62 @@ module github.com/prometheus/prometheus -go 1.24.0 +go 1.24.9 require ( - github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1 - github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0 + github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0 + github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.7.0 github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v4 v4.3.0 github.com/Code-Hex/go-generics-cache v1.5.1 - github.com/KimMachineGun/automemlimit v0.7.4 + github.com/KimMachineGun/automemlimit v0.7.5 github.com/alecthomas/kingpin/v2 v2.4.0 github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b - github.com/aws/aws-sdk-go-v2 v1.39.6 - github.com/aws/aws-sdk-go-v2/config v1.31.17 - github.com/aws/aws-sdk-go-v2/credentials v1.18.21 - github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0 - github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2 - github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4 - github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 - github.com/aws/smithy-go v1.23.2 + github.com/aws/aws-sdk-go-v2 v1.41.0 + github.com/aws/aws-sdk-go-v2/config v1.32.6 + github.com/aws/aws-sdk-go-v2/credentials v1.19.6 + github.com/aws/aws-sdk-go-v2/service/ec2 v1.277.0 + github.com/aws/aws-sdk-go-v2/service/ecs v1.69.5 + github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.10 + github.com/aws/aws-sdk-go-v2/service/sts v1.41.5 + github.com/aws/smithy-go v1.24.0 github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 github.com/cespare/xxhash/v2 v2.3.0 github.com/dennwc/varint v1.0.0 - github.com/digitalocean/godo v1.168.0 + github.com/digitalocean/godo v1.171.0 github.com/docker/docker v28.5.2+incompatible github.com/edsrzf/mmap-go v1.2.0 - github.com/envoyproxy/go-control-plane/envoy v1.35.0 - github.com/envoyproxy/protoc-gen-validate v1.2.1 + github.com/envoyproxy/go-control-plane/envoy v1.36.0 + github.com/envoyproxy/protoc-gen-validate v1.3.0 github.com/facette/natsort v0.0.0-20181210072756-2cd4dd1e2dcb github.com/fsnotify/fsnotify v1.9.0 - github.com/go-openapi/strfmt v0.24.0 + github.com/go-openapi/strfmt v0.25.0 github.com/go-zookeeper/zk v1.0.4 github.com/gogo/protobuf v1.3.2 github.com/golang/snappy v1.0.0 github.com/google/go-cmp v0.7.0 - github.com/google/pprof v0.0.0-20250923004556-9e5a51aed1e8 + github.com/google/pprof v0.0.0-20251213031049-b05bdaca462f github.com/google/uuid v1.6.0 - github.com/gophercloud/gophercloud/v2 v2.8.0 + github.com/gophercloud/gophercloud/v2 v2.9.0 github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 - github.com/hashicorp/consul/api v1.32.0 - github.com/hashicorp/nomad/api v0.0.0-20250930071859-eaa0fe0e27af - github.com/hetznercloud/hcloud-go/v2 v2.29.0 - github.com/ionos-cloud/sdk-go/v6 v6.3.4 + github.com/hashicorp/consul/api v1.32.1 + github.com/hashicorp/nomad/api v0.0.0-20251216171439-1dee0671280e + github.com/hetznercloud/hcloud-go/v2 v2.32.0 + github.com/ionos-cloud/sdk-go/v6 v6.3.5 github.com/json-iterator/go v1.1.12 - github.com/klauspost/compress v1.18.1 + github.com/klauspost/compress v1.18.2 github.com/kolo/xmlrpc v0.0.0-20220921171641-a4b6fa1dd06b - github.com/linode/linodego v1.60.0 - github.com/miekg/dns v1.1.68 + github.com/linode/linodego v1.63.0 + github.com/miekg/dns v1.1.69 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f github.com/nsf/jsondiff v0.0.0-20230430225905-43f6cf3098c1 github.com/oklog/run v1.2.0 github.com/oklog/ulid/v2 v2.1.1 - github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.139.0 + github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.142.0 github.com/ovh/go-ovh v1.9.0 - github.com/prometheus/alertmanager v0.28.1 + github.com/prometheus/alertmanager v0.30.0 github.com/prometheus/client_golang v1.23.2 - github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a + github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca github.com/prometheus/client_model v0.6.2 github.com/prometheus/common v0.67.4 github.com/prometheus/common/assets v0.2.0 @@ -64,47 +64,58 @@ require ( github.com/prometheus/sigv4 v0.3.0 github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35 github.com/shurcooL/httpfs v0.0.0-20230704072500-f1e31cf0ba5c - github.com/stackitcloud/stackit-sdk-go/core v0.17.3 + github.com/stackitcloud/stackit-sdk-go/core v0.20.1 github.com/stretchr/testify v1.11.1 github.com/vultr/govultr/v2 v2.17.2 - go.opentelemetry.io/collector/component v1.45.0 - go.opentelemetry.io/collector/consumer v1.45.0 - go.opentelemetry.io/collector/pdata v1.45.0 - go.opentelemetry.io/collector/processor v1.45.0 + go.opentelemetry.io/collector/component v1.48.0 + go.opentelemetry.io/collector/consumer v1.48.0 + go.opentelemetry.io/collector/pdata v1.48.0 + go.opentelemetry.io/collector/processor v1.48.0 go.opentelemetry.io/collector/semconv v0.128.0 - go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 - go.opentelemetry.io/otel v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 - go.opentelemetry.io/otel/metric v1.38.0 - go.opentelemetry.io/otel/sdk v1.38.0 - go.opentelemetry.io/otel/trace v1.38.0 + go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.64.0 + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 + go.opentelemetry.io/otel v1.39.0 + go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.39.0 + go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.39.0 + go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.39.0 + go.opentelemetry.io/otel/metric v1.39.0 + go.opentelemetry.io/otel/sdk v1.39.0 + go.opentelemetry.io/otel/trace v1.39.0 go.uber.org/atomic v1.11.0 go.uber.org/automaxprocs v1.6.0 go.uber.org/goleak v1.3.0 go.uber.org/multierr v1.11.0 go.yaml.in/yaml/v2 v2.4.3 - golang.org/x/oauth2 v0.32.0 - golang.org/x/sync v0.18.0 - golang.org/x/sys v0.38.0 - golang.org/x/text v0.31.0 - google.golang.org/api v0.252.0 - google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 - google.golang.org/grpc v1.76.0 - google.golang.org/protobuf v1.36.10 + golang.org/x/oauth2 v0.34.0 + golang.org/x/sync v0.19.0 + golang.org/x/sys v0.39.0 + golang.org/x/text v0.32.0 + google.golang.org/api v0.257.0 + google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2 + google.golang.org/grpc v1.77.0 + google.golang.org/protobuf v1.36.11 gopkg.in/yaml.v3 v3.0.1 - k8s.io/api v0.34.1 - k8s.io/apimachinery v0.34.1 - k8s.io/client-go v0.34.1 + k8s.io/api v0.34.3 + k8s.io/apimachinery v0.34.3 + k8s.io/client-go v0.34.3 k8s.io/klog v1.0.0 k8s.io/klog/v2 v2.130.1 ) require ( + github.com/aws/aws-sdk-go-v2/service/signin v1.0.4 // indirect + github.com/go-openapi/swag/cmdutils v0.25.4 // indirect + github.com/go-openapi/swag/conv v0.25.4 // indirect + github.com/go-openapi/swag/fileutils v0.25.4 // indirect + github.com/go-openapi/swag/jsonname v0.25.4 // indirect + github.com/go-openapi/swag/jsonutils v0.25.4 // indirect + github.com/go-openapi/swag/loading v0.25.4 // indirect + github.com/go-openapi/swag/mangling v0.25.4 // indirect + github.com/go-openapi/swag/netutils v0.25.4 // indirect + github.com/go-openapi/swag/stringutils v0.25.4 // indirect + github.com/go-openapi/swag/typeutils v0.25.4 // indirect + github.com/go-openapi/swag/yamlutils v0.25.4 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect - golang.org/x/tools/godoc v0.1.0-deprecated // indirect sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect ) @@ -113,21 +124,20 @@ require ( cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect cloud.google.com/go/compute/metadata v0.9.0 // indirect github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect - github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 // indirect + github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect github.com/Microsoft/go-winio v0.6.1 // indirect github.com/armon/go-metrics v0.4.1 // indirect - github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect - github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 - github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 // indirect - github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 // indirect + github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 + github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 // indirect + github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 // indirect github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 // indirect - github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 // indirect - github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 // indirect - github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect + github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.30.8 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cenkalti/backoff/v5 v5.0.3 // indirect - github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect + github.com/cncf/xds/go v0.0.0-20251022180443-0feb69152e9f // indirect github.com/containerd/errdefs v1.0.0 // indirect github.com/containerd/errdefs/pkg v0.3.0 // indirect github.com/containerd/log v0.1.0 // indirect @@ -142,25 +152,25 @@ require ( github.com/fxamacker/cbor/v2 v2.9.0 // indirect github.com/go-logr/logr v1.4.3 // indirect github.com/go-logr/stdr v1.2.2 // indirect - github.com/go-openapi/analysis v0.23.0 // indirect - github.com/go-openapi/errors v0.22.3 // indirect - github.com/go-openapi/jsonpointer v0.21.0 // indirect - github.com/go-openapi/jsonreference v0.21.0 // indirect - github.com/go-openapi/loads v0.22.0 // indirect - github.com/go-openapi/spec v0.21.0 // indirect - github.com/go-openapi/swag v0.23.0 // indirect - github.com/go-openapi/validate v0.24.0 // indirect - github.com/go-resty/resty/v2 v2.16.5 // indirect + github.com/go-openapi/analysis v0.24.1 // indirect + github.com/go-openapi/errors v0.22.4 // indirect + github.com/go-openapi/jsonpointer v0.22.1 // indirect + github.com/go-openapi/jsonreference v0.21.3 // indirect + github.com/go-openapi/loads v0.23.2 // indirect + github.com/go-openapi/spec v0.22.1 // indirect + github.com/go-openapi/swag v0.25.4 // indirect + github.com/go-openapi/validate v0.25.1 // indirect + github.com/go-resty/resty/v2 v2.17.1 // indirect github.com/go-viper/mapstructure/v2 v2.4.0 // indirect github.com/gobwas/glob v0.2.3 // indirect github.com/golang-jwt/jwt/v5 v5.3.0 // indirect github.com/google/gnostic-models v0.7.0 // indirect github.com/google/go-querystring v1.1.0 // indirect github.com/google/s2a-go v0.1.9 // indirect - github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect + github.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect github.com/googleapis/gax-go/v2 v2.15.0 // indirect github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect - github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 // indirect + github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 // indirect github.com/hashicorp/cronexpr v1.1.3 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/go-cleanhttp v0.5.2 // indirect @@ -169,17 +179,15 @@ require ( github.com/hashicorp/go-multierror v1.1.1 // indirect github.com/hashicorp/go-retryablehttp v0.7.7 // indirect github.com/hashicorp/go-rootcerts v1.0.2 // indirect - github.com/hashicorp/go-version v1.7.0 // indirect + github.com/hashicorp/go-version v1.8.0 // indirect github.com/hashicorp/golang-lru v0.6.0 // indirect github.com/hashicorp/serf v0.10.1 // indirect - github.com/josharian/intern v1.0.0 // indirect github.com/jpillora/backoff v1.0.0 // indirect github.com/julienschmidt/httprouter v1.3.0 // indirect github.com/knadh/koanf/maps v0.1.2 // indirect github.com/knadh/koanf/providers/confmap v1.0.0 // indirect github.com/knadh/koanf/v2 v2.3.0 // indirect github.com/kylelemons/godebug v1.1.0 // indirect - github.com/mailru/easyjson v0.7.7 // indirect github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/mdlayher/socket v0.4.1 // indirect @@ -195,8 +203,8 @@ require ( github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect github.com/morikuni/aec v1.0.0 // indirect github.com/oklog/ulid v1.3.1 // indirect - github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.139.0 // indirect - github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.139.0 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.142.0 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.142.0 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.0.2 // indirect github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect @@ -211,22 +219,22 @@ require ( github.com/stretchr/objx v0.5.2 // indirect github.com/x448/float16 v0.8.4 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - go.mongodb.org/mongo-driver v1.17.4 // indirect - go.opentelemetry.io/auto/sdk v1.1.0 // indirect - go.opentelemetry.io/collector/confmap v1.45.0 // indirect - go.opentelemetry.io/collector/confmap/xconfmap v0.139.0 // indirect - go.opentelemetry.io/collector/featuregate v1.45.0 // indirect - go.opentelemetry.io/collector/pipeline v1.45.0 // indirect - go.opentelemetry.io/proto/otlp v1.7.1 // indirect - go.uber.org/zap v1.27.0 // indirect - golang.org/x/crypto v0.45.0 // indirect + go.mongodb.org/mongo-driver v1.17.6 // indirect + go.opentelemetry.io/auto/sdk v1.2.1 // indirect + go.opentelemetry.io/collector/confmap v1.48.0 // indirect + go.opentelemetry.io/collector/confmap/xconfmap v0.142.0 // indirect + go.opentelemetry.io/collector/featuregate v1.48.0 // indirect + go.opentelemetry.io/collector/pipeline v1.48.0 // indirect + go.opentelemetry.io/proto/otlp v1.9.0 // indirect + go.uber.org/zap v1.27.1 // indirect + golang.org/x/crypto v0.46.0 // indirect golang.org/x/exp v0.0.0-20250808145144-a408d31f581a // indirect - golang.org/x/mod v0.29.0 // indirect - golang.org/x/net v0.47.0 // indirect - golang.org/x/term v0.37.0 // indirect - golang.org/x/time v0.13.0 // indirect - golang.org/x/tools v0.38.0 // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797 // indirect + golang.org/x/mod v0.30.0 // indirect + golang.org/x/net v0.48.0 // indirect + golang.org/x/term v0.38.0 // indirect + golang.org/x/time v0.14.0 // indirect + golang.org/x/tools v0.39.0 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/ini.v1 v1.67.0 // indirect diff --git a/go.sum b/go.sum index 70720765e7..b28b0eb3ff 100644 --- a/go.sum +++ b/go.sum @@ -4,10 +4,10 @@ cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIi cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c= cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs= cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= -github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1 h1:5YTBM8QDVIBN3sxBil89WfdAAqDZbyJTgh688DSxX5w= -github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1/go.mod h1:YD5h/ldMsG0XiIw7PdyNhLxaM317eFh5yNLccNfGdyw= -github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0 h1:wL5IEG5zb7BVv1Kv0Xm92orq+5hB5Nipn3B5tn4Rqfk= -github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0/go.mod h1:J7MUC/wtRpfGVbQ5sIItY5/FuVWmvzlY21WAOfQnq/I= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0 h1:JXg2dwJUmPB9JmtVmdEB16APJ7jurfbY5jnfXpJoRMc= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0/go.mod h1:YD5h/ldMsG0XiIw7PdyNhLxaM317eFh5yNLccNfGdyw= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 h1:Hk5QBxZQC1jb2Fwj6mpzme37xbCDdNTxU7O9eb5+LB4= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1/go.mod h1:IYus9qsFobWIc2YVwe/WPjcnyCkPKtnHAqUYeebc8z0= github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2 h1:yz1bePFlP5Vws5+8ez6T3HWXPmwOK7Yvq8QxDBD3SKY= github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2/go.mod h1:Pa9ZNPuoNu/GztvBSKk9J1cDJW6vk/n0zLtV4mgd8N8= github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 h1:9iefClla7iYpfYWdzPCRDozdmndjTm8DXdpCzPajMgA= @@ -24,13 +24,13 @@ github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOEl github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM= github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE= -github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 h1:XkkQbfMyuH2jTSjQjSoihryI8GINRcs4xp8lNawg0FI= -github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk= +github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 h1:XRzhVemXdgvJqCH0sFfrBUTnUJSBrBf7++ypk+twtRs= +github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk= github.com/Code-Hex/go-generics-cache v1.5.1 h1:6vhZGc5M7Y/YD8cIUcY8kcuQLB4cHR7U+0KMqAA0KcU= github.com/Code-Hex/go-generics-cache v1.5.1/go.mod h1:qxcC9kRVrct9rHeiYpFWSoW1vxyillCVzX13KZG8dl4= github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ= -github.com/KimMachineGun/automemlimit v0.7.4 h1:UY7QYOIfrr3wjjOAqahFmC3IaQCLWvur9nmfIn6LnWk= -github.com/KimMachineGun/automemlimit v0.7.4/go.mod h1:QZxpHaGOQoYvFhv/r4u3U0JTC2ZcOwbSr11UZF46UBM= +github.com/KimMachineGun/automemlimit v0.7.5 h1:RkbaC0MwhjL1ZuBKunGDjE/ggwAX43DwZrJqVwyveTk= +github.com/KimMachineGun/automemlimit v0.7.5/go.mod h1:QZxpHaGOQoYvFhv/r4u3U0JTC2ZcOwbSr11UZF46UBM= github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow= github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM= github.com/alecthomas/kingpin/v2 v2.4.0 h1:f48lwail6p8zpO1bC4TxtqACaGqHYA22qkHjHpqDjYY= @@ -47,40 +47,40 @@ github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJ github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4= github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/armon/go-radix v1.0.0/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= -github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so= -github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw= -github.com/aws/aws-sdk-go-v2 v1.39.6 h1:2JrPCVgWJm7bm83BDwY5z8ietmeJUbh3O2ACnn+Xsqk= -github.com/aws/aws-sdk-go-v2 v1.39.6/go.mod h1:c9pm7VwuW0UPxAEYGyTmyurVcNrbF6Rt/wixFqDhcjE= -github.com/aws/aws-sdk-go-v2/config v1.31.17 h1:QFl8lL6RgakNK86vusim14P2k8BFSxjvUkcWLDjgz9Y= -github.com/aws/aws-sdk-go-v2/config v1.31.17/go.mod h1:V8P7ILjp/Uef/aX8TjGk6OHZN6IKPM5YW6S78QnRD5c= -github.com/aws/aws-sdk-go-v2/credentials v1.18.21 h1:56HGpsgnmD+2/KpG0ikvvR8+3v3COCwaF4r+oWwOeNA= -github.com/aws/aws-sdk-go-v2/credentials v1.18.21/go.mod h1:3YELwedmQbw7cXNaII2Wywd+YY58AmLPwX4LzARgmmA= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 h1:T1brd5dR3/fzNFAQch/iBKeX07/ffu/cLu+q+RuzEWk= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13/go.mod h1:Peg/GBAQ6JDt+RoBf4meB1wylmAipb7Kg2ZFakZTlwk= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 h1:a+8/MLcWlIxo1lF9xaGt3J/u3yOZx+CdSveSNwjhD40= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13/go.mod h1:oGnKwIYZ4XttyU2JWxFrwvhF6YKiK/9/wmE3v3Iu9K8= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 h1:HBSI2kDkMdWz4ZM7FjwE7e/pWDEZ+nR95x8Ztet1ooY= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13/go.mod h1:YE94ZoDArI7awZqJzBAZ3PDD2zSfuP7w6P2knOzIn8M= +github.com/aws/aws-sdk-go-v2 v1.41.0 h1:tNvqh1s+v0vFYdA1xq0aOJH+Y5cRyZ5upu6roPgPKd4= +github.com/aws/aws-sdk-go-v2 v1.41.0/go.mod h1:MayyLB8y+buD9hZqkCW3kX1AKq07Y5pXxtgB+rRFhz0= +github.com/aws/aws-sdk-go-v2/config v1.32.6 h1:hFLBGUKjmLAekvi1evLi5hVvFQtSo3GYwi+Bx4lpJf8= +github.com/aws/aws-sdk-go-v2/config v1.32.6/go.mod h1:lcUL/gcd8WyjCrMnxez5OXkO3/rwcNmvfno62tnXNcI= +github.com/aws/aws-sdk-go-v2/credentials v1.19.6 h1:F9vWao2TwjV2MyiyVS+duza0NIRtAslgLUM0vTA1ZaE= +github.com/aws/aws-sdk-go-v2/credentials v1.19.6/go.mod h1:SgHzKjEVsdQr6Opor0ihgWtkWdfRAIwxYzSJ8O85VHY= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 h1:80+uETIWS1BqjnN9uJ0dBUaETh+P1XwFy5vwHwK5r9k= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16/go.mod h1:wOOsYuxYuB/7FlnVtzeBYRcjSRtQpAW0hCP7tIULMwo= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 h1:rgGwPzb82iBYSvHMHXc8h9mRoOUBZIGFgKb9qniaZZc= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16/go.mod h1:L/UxsGeKpGoIj6DxfhOWHWQ/kGKcd4I1VncE4++IyKA= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 h1:1jtGzuV7c82xnqOVfx2F0xmJcOw5374L7N6juGW6x6U= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16/go.mod h1:M2E5OQf+XLe+SZGmmpaI2yy+J326aFf6/+54PoxSANc= github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk= github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc= -github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0 h1:5qBb1XV/D18qtCHd3bmmxoVglI+fZ4QWuS/EB8kIXYQ= -github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0/go.mod h1:NDdDLLW5PtLLXN661gKcvJvqAH5OBXsfhMlmKVu1/pY= -github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2 h1:oeICOX/+D0XXV1aMYJPXVe3CO37zYr7fB6HFgxchleU= -github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2/go.mod h1:rrhqfkXfa2DSNq0RyFhnnFEAyI+yJB4+2QlZKeJvMjs= -github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o= -github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13/go.mod h1:lmKuogqSU3HzQCwZ9ZtcqOc5XGMqtDK7OIc2+DxiUEg= -github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4 h1:/1o2AYwHJojUDeMvQNyJiKZwcWCc3e4kQuTXqRLuThc= -github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4/go.mod h1:Nn2xx6HojGuNMtUFxxz/nyNLSS+tHMRsMhe3+W3wB5k= -github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 h1:0JPwLz1J+5lEOfy/g0SURC9cxhbQ1lIMHMa+AHZSzz0= -github.com/aws/aws-sdk-go-v2/service/sso v1.30.1/go.mod h1:fKvyjJcz63iL/ftA6RaM8sRCtN4r4zl4tjL3qw5ec7k= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 h1:OWs0/j2UYR5LOGi88sD5/lhN6TDLG6SfA7CqsQO9zF0= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5/go.mod h1:klO+ejMvYsB4QATfEOIXk8WAEwN4N0aBfJpvC+5SZBo= -github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 h1:mLlUgHn02ue8whiR4BmxxGJLR2gwU6s6ZzJ5wDamBUs= -github.com/aws/aws-sdk-go-v2/service/sts v1.39.1/go.mod h1:E19xDjpzPZC7LS2knI9E6BaRFDK43Eul7vd6rSq2HWk= -github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM= -github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0= +github.com/aws/aws-sdk-go-v2/service/ec2 v1.277.0 h1:RHJSkRXDGkAKrV4CTEsZsZkOmSpxXKO4aKx4rXd94K4= +github.com/aws/aws-sdk-go-v2/service/ec2 v1.277.0/go.mod h1:Wg68QRgy2gEGGdmTPU/UbVpdv8sM14bUZmF64KFwAsY= +github.com/aws/aws-sdk-go-v2/service/ecs v1.69.5 h1:5nkhwt0d/gjuT3AQ2LUK0aFRNB3MGlzB2elqy/ZsKP4= +github.com/aws/aws-sdk-go-v2/service/ecs v1.69.5/go.mod h1:LQMlcWBoiFVD3vUVEz42ST0yTiaDujv2dRE6sXt1yPE= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 h1:0ryTNEdJbzUCEWkVXEXoqlXV72J5keC1GvILMOuD00E= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4/go.mod h1:HQ4qwNZh32C3CBeO6iJLQlgtMzqeG17ziAA/3KDJFow= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 h1:oHjJHeUy0ImIV0bsrX0X91GkV5nJAyv1l1CC9lnO0TI= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16/go.mod h1:iRSNGgOYmiYwSCXxXaKb9HfOEj40+oTKn8pTxMlYkRM= +github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.10 h1:MQuZZ6Tq1qQabPlkVxrCMdyVl70Ogl4AERZKo+y9Wzo= +github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.10/go.mod h1:U5C3JME1ibKESmpzBAqlRpTYZfVbTqrb5ICJm+sVVd8= +github.com/aws/aws-sdk-go-v2/service/signin v1.0.4 h1:HpI7aMmJ+mm1wkSHIA2t5EaFFv5EFYXePW30p1EIrbQ= +github.com/aws/aws-sdk-go-v2/service/signin v1.0.4/go.mod h1:C5RdGMYGlfM0gYq/tifqgn4EbyX99V15P2V3R+VHbQU= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.8 h1:aM/Q24rIlS3bRAhTyFurowU8A0SMyGDtEOY/l/s/1Uw= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.8/go.mod h1:+fWt2UHSb4kS7Pu8y+BMBvJF0EWx+4H0hzNwtDNRTrg= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 h1:AHDr0DaHIAo8c9t1emrzAlVDFp+iMMKnPdYy6XO4MCE= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12/go.mod h1:GQ73XawFFiWxyWXMHWfhiomvP3tXtdNar/fi8z18sx0= +github.com/aws/aws-sdk-go-v2/service/sts v1.41.5 h1:SciGFVNZ4mHdm7gpD1dgZYnCuVdX1s+lFTg4+4DOy70= +github.com/aws/aws-sdk-go-v2/service/sts v1.41.5/go.mod h1:iW40X4QBmUxdP+fZNOpfmkdMZqsovezbAeO+Ubiv2pk= +github.com/aws/smithy-go v1.24.0 h1:LpilSUItNPFr1eY85RYgTIg5eIEPtvFbskaFcmmIUnk= +github.com/aws/smithy-go v1.24.0/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0= github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 h1:6df1vn4bBlDDo4tARvBm7l6KA9iVMnE3NWizDeWSrps= github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3/go.mod h1:CIWtjkly68+yqLPbvwwR/fjNJA/idrtULjZWh2v1ys0= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= @@ -95,8 +95,8 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag= github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I= -github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 h1:aQ3y1lwWyqYPiWZThqv1aFbZMiM9vblcSArJRf2Irls= -github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443/go.mod h1:W+zGtBO5Y1IgJhy4+A9GOqVhqLpfZi+vwmdNXUehLA8= +github.com/cncf/xds/go v0.0.0-20251022180443-0feb69152e9f h1:Y8xYupdHxryycyPlc9Y+bSQAYZnetRJ70VMVKm5CKI0= +github.com/cncf/xds/go v0.0.0-20251022180443-0feb69152e9f/go.mod h1:HlzOvOjVBOfTGSRXRyY0OiCS/3J1akRGQQpRO/7zyF4= github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= @@ -112,8 +112,8 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/dennwc/varint v1.0.0 h1:kGNFFSSw8ToIy3obO/kKr8U9GZYUAxQEVuix4zfDWzE= github.com/dennwc/varint v1.0.0/go.mod h1:hnItb35rvZvJrbTALZtY/iQfDs48JKRG1RPpgziApxA= -github.com/digitalocean/godo v1.168.0 h1:mlORtUcPD91LQeJoznrH3XvfvgK3t8Wvrpph9giUT/Q= -github.com/digitalocean/godo v1.168.0/go.mod h1:xQsWpVCCbkDrWisHA72hPzPlnC+4W5w/McZY5ij9uvU= +github.com/digitalocean/godo v1.171.0 h1:QwpkwWKr3v7yxc8D4NQG973NoR9APCEWjYnLOQeXVpQ= +github.com/digitalocean/godo v1.171.0/go.mod h1:xQsWpVCCbkDrWisHA72hPzPlnC+4W5w/McZY5ij9uvU= github.com/distribution/reference v0.5.0 h1:/FUIFXtfc/x2gpa5/VGfiGLuOIdYa1t65IKK2OFGvA0= github.com/distribution/reference v0.5.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI= @@ -128,10 +128,10 @@ github.com/edsrzf/mmap-go v1.2.0 h1:hXLYlkbaPzt1SaQk+anYwKSRNhufIDCchSPkUD6dD84= github.com/edsrzf/mmap-go v1.2.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q= github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU= github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= -github.com/envoyproxy/go-control-plane/envoy v1.35.0 h1:ixjkELDE+ru6idPxcHLj8LBVc2bFP7iBytj353BoHUo= -github.com/envoyproxy/go-control-plane/envoy v1.35.0/go.mod h1:09qwbGVuSWWAyN5t/b3iyVfz5+z8QWGrzkoqm/8SbEs= -github.com/envoyproxy/protoc-gen-validate v1.2.1 h1:DEo3O99U8j4hBFwbJfrz9VtgcDfUKS7KJ7spH3d86P8= -github.com/envoyproxy/protoc-gen-validate v1.2.1/go.mod h1:d/C80l/jxXLdfEIhX1W2TmLfsJ31lvEjwamM4DxlWXU= +github.com/envoyproxy/go-control-plane/envoy v1.36.0 h1:yg/JjO5E7ubRyKX3m07GF3reDNEnfOboJ0QySbH736g= +github.com/envoyproxy/go-control-plane/envoy v1.36.0/go.mod h1:ty89S1YCCVruQAm9OtKeEkQLTb+Lkz0k8v9W0Oxsv98= +github.com/envoyproxy/protoc-gen-validate v1.3.0 h1:TvGH1wof4H33rezVKWSpqKz5NXWg5VPuZ0uONDT6eb4= +github.com/envoyproxy/protoc-gen-validate v1.3.0/go.mod h1:HvYl7zwPa5mffgyeTUHA9zHIH36nmrm7oCbo4YKoSWA= github.com/facette/natsort v0.0.0-20181210072756-2cd4dd1e2dcb h1:IT4JYU7k4ikYg1SCxNI1/Tieq/NFvh6dzLdgi7eu0tM= github.com/facette/natsort v0.0.0-20181210072756-2cd4dd1e2dcb/go.mod h1:bH6Xx7IW64qjjJq8M2u4dxNaBiDfKK+z/3eGDpXEQhc= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= @@ -155,26 +155,54 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= -github.com/go-openapi/analysis v0.23.0 h1:aGday7OWupfMs+LbmLZG4k0MYXIANxcuBTYUC03zFCU= -github.com/go-openapi/analysis v0.23.0/go.mod h1:9mz9ZWaSlV8TvjQHLl2mUW2PbZtemkE8yA5v22ohupo= -github.com/go-openapi/errors v0.22.3 h1:k6Hxa5Jg1TUyZnOwV2Lh81j8ayNw5VVYLvKrp4zFKFs= -github.com/go-openapi/errors v0.22.3/go.mod h1:+WvbaBBULWCOna//9B9TbLNGSFOfF8lY9dw4hGiEiKQ= -github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ= -github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY= -github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF2+tg1TRrwQ= -github.com/go-openapi/jsonreference v0.21.0/go.mod h1:LmZmgsrTkVg9LG4EaHeY8cBDslNPMo06cago5JNLkm4= -github.com/go-openapi/loads v0.22.0 h1:ECPGd4jX1U6NApCGG1We+uEozOAvXvJSF4nnwHZ8Aco= -github.com/go-openapi/loads v0.22.0/go.mod h1:yLsaTCS92mnSAZX5WWoxszLj0u+Ojl+Zs5Stn1oF+rs= -github.com/go-openapi/spec v0.21.0 h1:LTVzPc3p/RzRnkQqLRndbAzjY0d0BCL72A6j3CdL9ZY= -github.com/go-openapi/spec v0.21.0/go.mod h1:78u6VdPw81XU44qEWGhtr982gJ5BWg2c0I5XwVMotYk= -github.com/go-openapi/strfmt v0.24.0 h1:dDsopqbI3wrrlIzeXRbqMihRNnjzGC+ez4NQaAAJLuc= -github.com/go-openapi/strfmt v0.24.0/go.mod h1:Lnn1Bk9rZjXxU9VMADbEEOo7D7CDyKGLsSKekhFr7s4= -github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE= -github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ= -github.com/go-openapi/validate v0.24.0 h1:LdfDKwNbpB6Vn40xhTdNZAnfLECL81w+VX3BumrGD58= -github.com/go-openapi/validate v0.24.0/go.mod h1:iyeX1sEufmv3nPbBdX3ieNviWnOZaJ1+zquzJEf2BAQ= -github.com/go-resty/resty/v2 v2.16.5 h1:hBKqmWrr7uRc3euHVqmh1HTHcKn99Smr7o5spptdhTM= -github.com/go-resty/resty/v2 v2.16.5/go.mod h1:hkJtXbA2iKHzJheXYvQ8snQES5ZLGKMwQ07xAwp/fiA= +github.com/go-openapi/analysis v0.24.1 h1:Xp+7Yn/KOnVWYG8d+hPksOYnCYImE3TieBa7rBOesYM= +github.com/go-openapi/analysis v0.24.1/go.mod h1:dU+qxX7QGU1rl7IYhBC8bIfmWQdX4Buoea4TGtxXY84= +github.com/go-openapi/errors v0.22.4 h1:oi2K9mHTOb5DPW2Zjdzs/NIvwi2N3fARKaTJLdNabaM= +github.com/go-openapi/errors v0.22.4/go.mod h1:z9S8ASTUqx7+CP1Q8dD8ewGH/1JWFFLX/2PmAYNQLgk= +github.com/go-openapi/jsonpointer v0.22.1 h1:sHYI1He3b9NqJ4wXLoJDKmUmHkWy/L7rtEo92JUxBNk= +github.com/go-openapi/jsonpointer v0.22.1/go.mod h1:pQT9OsLkfz1yWoMgYFy4x3U5GY5nUlsOn1qSBH5MkCM= +github.com/go-openapi/jsonreference v0.21.3 h1:96Dn+MRPa0nYAR8DR1E03SblB5FJvh7W6krPI0Z7qMc= +github.com/go-openapi/jsonreference v0.21.3/go.mod h1:RqkUP0MrLf37HqxZxrIAtTWW4ZJIK1VzduhXYBEeGc4= +github.com/go-openapi/loads v0.23.2 h1:rJXAcP7g1+lWyBHC7iTY+WAF0rprtM+pm8Jxv1uQJp4= +github.com/go-openapi/loads v0.23.2/go.mod h1:IEVw1GfRt/P2Pplkelxzj9BYFajiWOtY2nHZNj4UnWY= +github.com/go-openapi/spec v0.22.1 h1:beZMa5AVQzRspNjvhe5aG1/XyBSMeX1eEOs7dMoXh/k= +github.com/go-openapi/spec v0.22.1/go.mod h1:c7aeIQT175dVowfp7FeCvXXnjN/MrpaONStibD2WtDA= +github.com/go-openapi/strfmt v0.25.0 h1:7R0RX7mbKLa9EYCTHRcCuIPcaqlyQiWNPTXwClK0saQ= +github.com/go-openapi/strfmt v0.25.0/go.mod h1:nNXct7OzbwrMY9+5tLX4I21pzcmE6ccMGXl3jFdPfn8= +github.com/go-openapi/swag v0.25.4 h1:OyUPUFYDPDBMkqyxOTkqDYFnrhuhi9NR6QVUvIochMU= +github.com/go-openapi/swag v0.25.4/go.mod h1:zNfJ9WZABGHCFg2RnY0S4IOkAcVTzJ6z2Bi+Q4i6qFQ= +github.com/go-openapi/swag/cmdutils v0.25.4 h1:8rYhB5n6WawR192/BfUu2iVlxqVR9aRgGJP6WaBoW+4= +github.com/go-openapi/swag/cmdutils v0.25.4/go.mod h1:pdae/AFo6WxLl5L0rq87eRzVPm/XRHM3MoYgRMvG4A0= +github.com/go-openapi/swag/conv v0.25.4 h1:/Dd7p0LZXczgUcC/Ikm1+YqVzkEeCc9LnOWjfkpkfe4= +github.com/go-openapi/swag/conv v0.25.4/go.mod h1:3LXfie/lwoAv0NHoEuY1hjoFAYkvlqI/Bn5EQDD3PPU= +github.com/go-openapi/swag/fileutils v0.25.4 h1:2oI0XNW5y6UWZTC7vAxC8hmsK/tOkWXHJQH4lKjqw+Y= +github.com/go-openapi/swag/fileutils v0.25.4/go.mod h1:cdOT/PKbwcysVQ9Tpr0q20lQKH7MGhOEb6EwmHOirUk= +github.com/go-openapi/swag/jsonname v0.25.4 h1:bZH0+MsS03MbnwBXYhuTttMOqk+5KcQ9869Vye1bNHI= +github.com/go-openapi/swag/jsonname v0.25.4/go.mod h1:GPVEk9CWVhNvWhZgrnvRA6utbAltopbKwDu8mXNUMag= +github.com/go-openapi/swag/jsonutils v0.25.4 h1:VSchfbGhD4UTf4vCdR2F4TLBdLwHyUDTd1/q4i+jGZA= +github.com/go-openapi/swag/jsonutils v0.25.4/go.mod h1:7OYGXpvVFPn4PpaSdPHJBtF0iGnbEaTk8AvBkoWnaAY= +github.com/go-openapi/swag/jsonutils/fixtures_test v0.25.4 h1:IACsSvBhiNJwlDix7wq39SS2Fh7lUOCJRmx/4SN4sVo= +github.com/go-openapi/swag/jsonutils/fixtures_test v0.25.4/go.mod h1:Mt0Ost9l3cUzVv4OEZG+WSeoHwjWLnarzMePNDAOBiM= +github.com/go-openapi/swag/loading v0.25.4 h1:jN4MvLj0X6yhCDduRsxDDw1aHe+ZWoLjW+9ZQWIKn2s= +github.com/go-openapi/swag/loading v0.25.4/go.mod h1:rpUM1ZiyEP9+mNLIQUdMiD7dCETXvkkC30z53i+ftTE= +github.com/go-openapi/swag/mangling v0.25.4 h1:2b9kBJk9JvPgxr36V23FxJLdwBrpijI26Bx5JH4Hp48= +github.com/go-openapi/swag/mangling v0.25.4/go.mod h1:6dxwu6QyORHpIIApsdZgb6wBk/DPU15MdyYj/ikn0Hg= +github.com/go-openapi/swag/netutils v0.25.4 h1:Gqe6K71bGRb3ZQLusdI8p/y1KLgV4M/k+/HzVSqT8H0= +github.com/go-openapi/swag/netutils v0.25.4/go.mod h1:m2W8dtdaoX7oj9rEttLyTeEFFEBvnAx9qHd5nJEBzYg= +github.com/go-openapi/swag/stringutils v0.25.4 h1:O6dU1Rd8bej4HPA3/CLPciNBBDwZj9HiEpdVsb8B5A8= +github.com/go-openapi/swag/stringutils v0.25.4/go.mod h1:GTsRvhJW5xM5gkgiFe0fV3PUlFm0dr8vki6/VSRaZK0= +github.com/go-openapi/swag/typeutils v0.25.4 h1:1/fbZOUN472NTc39zpa+YGHn3jzHWhv42wAJSN91wRw= +github.com/go-openapi/swag/typeutils v0.25.4/go.mod h1:Ou7g//Wx8tTLS9vG0UmzfCsjZjKhpjxayRKTHXf2pTE= +github.com/go-openapi/swag/yamlutils v0.25.4 h1:6jdaeSItEUb7ioS9lFoCZ65Cne1/RZtPBZ9A56h92Sw= +github.com/go-openapi/swag/yamlutils v0.25.4/go.mod h1:MNzq1ulQu+yd8Kl7wPOut/YHAAU/H6hL91fF+E2RFwc= +github.com/go-openapi/testify/enable/yaml/v2 v2.0.2 h1:0+Y41Pz1NkbTHz8NngxTuAXxEodtNSI1WG1c/m5Akw4= +github.com/go-openapi/testify/enable/yaml/v2 v2.0.2/go.mod h1:kme83333GCtJQHXQ8UKX3IBZu6z8T5Dvy5+CW3NLUUg= +github.com/go-openapi/testify/v2 v2.0.2 h1:X999g3jeLcoY8qctY/c/Z8iBHTbwLz7R2WXd6Ub6wls= +github.com/go-openapi/testify/v2 v2.0.2/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16pjwMkYkP4Ywn54= +github.com/go-openapi/validate v0.25.1 h1:sSACUI6Jcnbo5IWqbYHgjibrhhmt3vR6lCzKZnmAgBw= +github.com/go-openapi/validate v0.25.1/go.mod h1:RMVyVFYte0gbSTaZ0N4KmTn6u/kClvAFp+mAVfS/DQc= +github.com/go-resty/resty/v2 v2.17.1 h1:x3aMpHK1YM9e4va/TMDRlusDDoZiQ+ViDu/WpA6xTM4= +github.com/go-resty/resty/v2 v2.17.1/go.mod h1:kCKZ3wWmwJaNc7S29BRtUhJwy7iqmn+2mLtQrOyQlVA= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI= github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8= @@ -210,26 +238,26 @@ github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8= github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/pprof v0.0.0-20250923004556-9e5a51aed1e8 h1:ZI8gCoCjGzPsum4L21jHdQs8shFBIQih1TM9Rd/c+EQ= -github.com/google/pprof v0.0.0-20250923004556-9e5a51aed1e8/go.mod h1:I6V7YzU0XDpsHqbsyrghnFZLO1gwK6NPTNvmetQIk9U= +github.com/google/pprof v0.0.0-20251213031049-b05bdaca462f h1:HU1RgM6NALf/KW9HEY6zry3ADbDKcmpQ+hJedoNGQYQ= +github.com/google/pprof v0.0.0-20251213031049-b05bdaca462f/go.mod h1:67FPmZWbr+KDT/VlpWtw6sO9XSjpJmLuHpoLmWiTGgY= github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0= github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4= -github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA= +github.com/googleapis/enterprise-certificate-proxy v0.3.7 h1:zrn2Ee/nWmHulBx5sAVrGgAa0f2/R35S4DJwfFaUPFQ= +github.com/googleapis/enterprise-certificate-proxy v0.3.7/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA= github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo= github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc= -github.com/gophercloud/gophercloud/v2 v2.8.0 h1:of2+8tT6+FbEYHfYC8GBu8TXJNsXYSNm9KuvpX7Neqo= -github.com/gophercloud/gophercloud/v2 v2.8.0/go.mod h1:Ki/ILhYZr/5EPebrPL9Ej+tUg4lqx71/YH2JWVeU+Qk= +github.com/gophercloud/gophercloud/v2 v2.9.0 h1:Y9OMrwKF9EDERcHFSOTpf/6XGoAI0yOxmsLmQki4LPM= +github.com/gophercloud/gophercloud/v2 v2.9.0/go.mod h1:Ki/ILhYZr/5EPebrPL9Ej+tUg4lqx71/YH2JWVeU+Qk= github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo= github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA= github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 h1:cLN4IBkmkYZNnk7EAJ0BHIethd+J6LqxFNw5mSiI2bM= github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 h1:8Tjv8EJ+pM1xP8mK6egEbD1OgnVTyacbefKhmbLhIhU= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2/go.mod h1:pkJQ2tZHJ0aFOVEEot6oZmaVEZcRme73eIFmhiVuRWs= -github.com/hashicorp/consul/api v1.32.0 h1:5wp5u780Gri7c4OedGEPzmlUEzi0g2KyiPphSr6zjVg= -github.com/hashicorp/consul/api v1.32.0/go.mod h1:Z8YgY0eVPukT/17ejW+l+C7zJmKwgPHtjU1q16v/Y40= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 h1:NmZ1PKzSTQbuGHw9DGPFomqkkLWMC+vZCkfs+FHv1Vg= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3/go.mod h1:zQrxl1YP88HQlA6i9c63DSVPFklWpGX4OWAc9bFuaH4= +github.com/hashicorp/consul/api v1.32.1 h1:0+osr/3t/aZNAdJX558crU3PEjVrG4x6715aZHRgceE= +github.com/hashicorp/consul/api v1.32.1/go.mod h1:mXUWLnxftwTmDv4W3lzxYCPD199iNLLUyLfLGFJbtl4= github.com/hashicorp/consul/sdk v0.16.1 h1:V8TxTnImoPD5cj0U9Spl0TUxcytjcbbJeADFF07KdHg= github.com/hashicorp/consul/sdk v0.16.1/go.mod h1:fSXvwxB2hmh1FMZCNl6PwX0Q/1wdWtHJcZ7Ea5tns0s= github.com/hashicorp/cronexpr v1.1.3 h1:rl5IkxXN2m681EfivTlccqIryzYJSXRGRNa0xeG7NA4= @@ -245,6 +273,8 @@ github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVH github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= github.com/hashicorp/go-immutable-radix v1.3.1 h1:DKHmCUm2hRBK510BaiZlwvpD40f8bJFeZnpfm2KLowc= github.com/hashicorp/go-immutable-radix v1.3.1/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-metrics v0.5.4 h1:8mmPiIJkTPPEbAiV97IxdAGNdRdaWwVap1BU6elejKY= +github.com/hashicorp/go-metrics v0.5.4/go.mod h1:CG5yz4NZ/AI/aQt9Ucm/vdBnbh7fvmv4lxZ350i+QQI= github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= github.com/hashicorp/go-msgpack v0.5.5 h1:i9R9JSrqIz0QVLz3sz+i3YJdT7TTSLcfLLzJi9aZTuI= github.com/hashicorp/go-msgpack v0.5.5/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= @@ -267,28 +297,26 @@ github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/b github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8= github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= -github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY= -github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= +github.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4= +github.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.6.0 h1:uL2shRDx7RTrOrTCUZEGP/wJUFiUI8QT6E7z5o8jga4= github.com/hashicorp/golang-lru v0.6.0/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/YAJqrc= github.com/hashicorp/memberlist v0.5.0/go.mod h1:yvyXLpo0QaGE59Y7hDTsTzDD25JYBZ4mHgHUZ8lrOI0= -github.com/hashicorp/memberlist v0.5.1 h1:mk5dRuzeDNis2bi6LLoQIXfMH7JQvAzt3mQD0vNZZUo= -github.com/hashicorp/memberlist v0.5.1/go.mod h1:zGDXV6AqbDTKTM6yxW0I4+JtFzZAJVoIPvss4hV8F24= -github.com/hashicorp/nomad/api v0.0.0-20250930071859-eaa0fe0e27af h1:ScAYf8O+9xTqTJPZH8MIlUfO+ak8cb31rW1aYJgS+jE= -github.com/hashicorp/nomad/api v0.0.0-20250930071859-eaa0fe0e27af/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE= +github.com/hashicorp/memberlist v0.5.3 h1:tQ1jOCypD0WvMemw/ZhhtH+PWpzcftQvgCorLu0hndk= +github.com/hashicorp/memberlist v0.5.3/go.mod h1:h60o12SZn/ua/j0B6iKAZezA4eDaGsIuPO70eOaJ6WE= +github.com/hashicorp/nomad/api v0.0.0-20251216171439-1dee0671280e h1:wGl06iy/H90NSbWjfXWeRwk9SJOks0u4voIryeJFlSA= +github.com/hashicorp/nomad/api v0.0.0-20251216171439-1dee0671280e/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE= github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY= github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4= -github.com/hetznercloud/hcloud-go/v2 v2.29.0 h1:LzNFw5XLBfftyu3WM1sdSLjOZBlWORtz2hgGydHaYV8= -github.com/hetznercloud/hcloud-go/v2 v2.29.0/go.mod h1:XBU4+EDH2KVqu2KU7Ws0+ciZcX4ygukQl/J0L5GS8P8= -github.com/ionos-cloud/sdk-go/v6 v6.3.4 h1:jTvGl4LOF8v8OYoEIBNVwbFoqSGAFqn6vGE7sp7/BqQ= -github.com/ionos-cloud/sdk-go/v6 v6.3.4/go.mod h1:wCVwNJ/21W29FWFUv+fNawOTMlFoP1dS3L+ZuztFW48= +github.com/hetznercloud/hcloud-go/v2 v2.32.0 h1:BRe+k7ESdYv3xQLBGdKUfk+XBFRJNGKzq70nJI24ciM= +github.com/hetznercloud/hcloud-go/v2 v2.32.0/go.mod h1:hAanyyfn9M0cMmZ68CXzPCF54KRb9EXd8eiE2FHKGIE= +github.com/ionos-cloud/sdk-go/v6 v6.3.5 h1:6fHArdV1lf50iRhCkCP7wkvGwWzVwi+l9w1t5mwkOa8= +github.com/ionos-cloud/sdk-go/v6 v6.3.5/go.mod h1:nUGHP4kZHAZngCVr4v6C8nuargFrtvt7GrzH/hqn7c4= github.com/jarcoal/httpmock v1.4.1 h1:0Ju+VCFuARfFlhVXFc2HxlcQkfB+Xq12/EotHko+x2A= github.com/jarcoal/httpmock v1.4.1/go.mod h1:ftW1xULwo+j0R0JJkJIIi7UKigZUXCLLanykgjwBXL0= -github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= -github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= @@ -302,8 +330,8 @@ github.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRt github.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.18.1 h1:bcSGx7UbpBqMChDtsF28Lw6v/G94LPrrbMbdC3JH2co= -github.com/klauspost/compress v1.18.1/go.mod h1:ZQFFVG+MdnR0P+l6wpXgIL4NTtwiKIdBnrBd8Nrxr+0= +github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= +github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo= github.com/knadh/koanf/maps v0.1.2/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI= github.com/knadh/koanf/providers/confmap v1.0.0 h1:mHKLJTE7iXEys6deO5p6olAiZdG5zwp8Aebir+/EaRE= @@ -323,10 +351,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= -github.com/linode/linodego v1.60.0 h1:SgsebJFRCi+lSmYy+C40wmKZeJllGGm+W12Qw4+yVdI= -github.com/linode/linodego v1.60.0/go.mod h1:1+Bt0oTz5rBnDOJbGhccxn7LYVytXTIIfAy7QYmijDs= -github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= -github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/linode/linodego v1.63.0 h1:MdjizfXNJDVJU6ggoJmMO5O9h4KGPGivNX0fzrAnstk= +github.com/linode/linodego v1.63.0/go.mod h1:GoiwLVuLdBQcAebxAVKVL3mMYUgJZR/puOUSla04xBE= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= @@ -351,8 +377,8 @@ github.com/mdlayher/vsock v1.2.1 h1:pC1mTJTvjo1r9n9fbm7S1j04rCgCzhCOS5DY0zqHlnQ= github.com/mdlayher/vsock v1.2.1/go.mod h1:NRfCibel++DgeMD8z/hP+PPTjlNJsdPOmxcnENvE+SE= github.com/miekg/dns v1.1.26/go.mod h1:bPDLeHnStXmXAq1m/Ch/hvfNHr14JKNPMBo3VZKjuso= github.com/miekg/dns v1.1.41/go.mod h1:p6aan82bvRIyn+zDIv9xYNUpwa73JcSh9BKwknJysuI= -github.com/miekg/dns v1.1.68 h1:jsSRkNozw7G/mnmXULynzMNIsgY2dHC8LO6U6Ij2JEA= -github.com/miekg/dns v1.1.68/go.mod h1:fujopn7TB3Pu3JM69XaawiU0wqjpL9/8xGop5UrTPps= +github.com/miekg/dns v1.1.69 h1:Kb7Y/1Jo+SG+a2GtfoFUfDkG//csdRPwRLkCsxDG9Sc= +github.com/miekg/dns v1.1.69/go.mod h1:7OyjD9nEba5OkqQ/hB4fy3PIoxafSZJtducccIelz3g= github.com/mitchellh/cli v1.1.0/go.mod h1:xcISNoH86gajksDmfB23e/pu+B+GeFRMYmoHXxx3xhI= github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw= github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s= @@ -398,12 +424,12 @@ github.com/onsi/ginkgo/v2 v2.21.0 h1:7rg/4f3rB88pb5obDgNZrNHrQ4e6WpjonchcpuBRnZM github.com/onsi/ginkgo/v2 v2.21.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo= github.com/onsi/gomega v1.35.1 h1:Cwbd75ZBPxFSuZ6T+rN/WCb/gOc6YgFBXLlZLhC7Ds4= github.com/onsi/gomega v1.35.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog= -github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.139.0 h1:D5aGQCErSCb4sKIHoZhgR4El6AzgviTRYlHUpbSFqDo= -github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.139.0/go.mod h1:ZjeRsA5oaVk89fg5D+iXStx2QncmhAvtGbdSumT07H4= -github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.139.0 h1:6/j0Ta8ZJnmAFVEoC3aZ1Hs19RB4fHzlN6kOZhsBJqM= -github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.139.0/go.mod h1:VfA8xHz4xg7Fyj5bBsCDbOO3iVYzDn9wP/QFsjcAE5c= -github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.139.0 h1:iRNX/ueuad1psOVgnNkxuQmXxvF3ze5ZZCP66xKFk/w= -github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.139.0/go.mod h1:bW09lo3WgHsPsZ1mgsJvby9wCefT5o13patM5phdfIU= +github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.142.0 h1:agYk41V3eIfV6aIMxIeRQ7SFhfaW5k2O96HEebpmPwM= +github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics v0.142.0/go.mod h1:ZmMdcBia20ih8NYia5b4dNhfNLT68xHgaqF+fNW+TLM= +github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.142.0 h1:bLp+Ii1UQ9cNr+Dm1jKzbcklhd0eBnPuIFQY6NPzkZ0= +github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.142.0/go.mod h1:6N36UrFd9Yiz2aYpXM5xiK7Eqp2RyAr3O8lUE+wK2Y8= +github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.142.0 h1:fL8LBVeje+nbts2VIInvRa4T5LlsC0BZCI60wNGoS+Y= +github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor v0.142.0/go.mod h1:fSnKuTN91I68Ou1Lgfwe3Mt6BGl9kcA8PYCpnGkPnsY= github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.0.2 h1:9yCKha/T5XdGtO0q9Q9a6T5NUCsTn/DrBg0D7ufOcFM= @@ -431,15 +457,15 @@ github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndr github.com/posener/complete v1.2.3/go.mod h1:WZIdtGGp+qx0sLrYKtIRAruyNpv6hFCicSgv7Sy7s/s= github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g= github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U= -github.com/prometheus/alertmanager v0.28.1 h1:BK5pCoAtaKg01BYRUJhEDV1tqJMEtYBGzPw8QdvnnvA= -github.com/prometheus/alertmanager v0.28.1/go.mod h1:0StpPUDDHi1VXeM7p2yYfeZgLVi/PPlt39vo9LQUHxM= +github.com/prometheus/alertmanager v0.30.0 h1:E4dnxSFXK8V2Bb8iqudlisTmaIrF3hRJSWnliG08tBM= +github.com/prometheus/alertmanager v0.30.0/go.mod h1:93PBumcTLr/gNtNtM0m7BcCffbvYP5bKuLBWiOnISaA= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU= github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= -github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a h1:RF1vfKM34/3DbGNis22BGd6sDDY3XBi0eM7pYqmOEO0= -github.com/prometheus/client_golang/exp v0.0.0-20250914183048-a974e0d45e0a/go.mod h1:FGJuwvfcPY0V5enm+w8zF1RNS062yugQtPPQp1c4Io4= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca h1:BOxmsLoL2ymn8lXJtorca7N/m+2vDQUDoEtPjf0iAxA= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca/go.mod h1:gndBHh3ZdjBozGcGrjUYjN3UJLRS3l2drALtu4lUt+k= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= @@ -464,8 +490,8 @@ github.com/prometheus/sigv4 v0.3.0 h1:QIG7nTbu0JTnNidGI1Uwl5AGVIChWUACxn2B/BQ1km github.com/prometheus/sigv4 v0.3.0/go.mod h1:fKtFYDus2M43CWKMNtGvFNHGXnAJJEGZbiYCmVp/F8I= github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg= github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA= -github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= -github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o= +github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= +github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35 h1:8xfn1RzeI9yoCUuEwDy08F+No6PcKZGEDOQ6hrRyLts= github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35/go.mod h1:47B1d/YXmSAxlJxUJxClzHR6b3T4M1WyCvwENPQNBWc= @@ -482,8 +508,8 @@ github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVs github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o= github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/stackitcloud/stackit-sdk-go/core v0.17.3 h1:GsZGmRRc/3GJLmCUnsZswirr5wfLRrwavbnL/renOqg= -github.com/stackitcloud/stackit-sdk-go/core v0.17.3/go.mod h1:HBCXJGPgdRulplDzhrmwC+Dak9B/x0nzNtmOpu+1Ahg= +github.com/stackitcloud/stackit-sdk-go/core v0.20.1 h1:odiuhhRXmxvEvnVTeZSN9u98edvw2Cd3DcnkepncP3M= +github.com/stackitcloud/stackit-sdk-go/core v0.20.1/go.mod h1:fqto7M82ynGhEnpZU6VkQKYWYoFG5goC076JWXTUPRQ= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= @@ -509,72 +535,74 @@ github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8 github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -go.mongodb.org/mongo-driver v1.17.4 h1:jUorfmVzljjr0FLzYQsGP8cgN/qzzxlY9Vh0C9KFXVw= -go.mongodb.org/mongo-driver v1.17.4/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ= -go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= -go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= -go.opentelemetry.io/collector/component v1.45.0 h1:gGFfVdbQ+1YuyUkJjWo85I7euu3H/CiupuzCHv8OgHA= -go.opentelemetry.io/collector/component v1.45.0/go.mod h1:xoNFnRKE8Iv6gmlqAKgjayWraRnDcYLLgrPt9VgyO2g= -go.opentelemetry.io/collector/component/componentstatus v0.139.0 h1:bQmkv1t7xW7uIDireE0a2Am4IMOprXm6zQr/qDtGCIA= -go.opentelemetry.io/collector/component/componentstatus v0.139.0/go.mod h1:ibZOohpG0u081/NaT/jMCTsKwRbbwwxWrjZml+owpyM= -go.opentelemetry.io/collector/component/componenttest v0.139.0 h1:x9Yu2eYhrHxdZ7sFXWtAWVjQ3UIraje557LgNurDC2I= -go.opentelemetry.io/collector/component/componenttest v0.139.0/go.mod h1:S9cj+qkf9FgHMzjvlYsLwQKd9BiS7B7oLZvxvlENM/c= -go.opentelemetry.io/collector/confmap v1.45.0 h1:7M7TTlpzX4r+mIzP/ARdxZBAvI4N+1V96phDane+akU= -go.opentelemetry.io/collector/confmap v1.45.0/go.mod h1:AE1dnkjv0T9gptsh5+mTX0XFGdXx0n7JS4b7CcPfJ6Q= -go.opentelemetry.io/collector/confmap/xconfmap v0.139.0 h1:uQGpFuWnTCXqdMbI3gDSvkwU66/kF/aoC0kVMrit1EM= -go.opentelemetry.io/collector/confmap/xconfmap v0.139.0/go.mod h1:d0ucaeNq2rojFRSQsCHF/gkT3cgBx5H2bVkPQMj57ck= -go.opentelemetry.io/collector/consumer v1.45.0 h1:TtqXxgW+1GSCwdoohq0fzqnfqrZBKbfo++1XRj8mrEA= -go.opentelemetry.io/collector/consumer v1.45.0/go.mod h1:pJzqTWBubwLt8mVou+G4/Hs23b3m425rVmld3LqOYpY= -go.opentelemetry.io/collector/consumer/consumertest v0.139.0 h1:06mu43mMO7l49ASJ/GEbKgTWcV3py5zE/pKhNBZ1b3k= -go.opentelemetry.io/collector/consumer/consumertest v0.139.0/go.mod h1:gaeCpRQGbCFYTeLzi+Z2cTDt40GiIa3hgIEgLEmiC78= -go.opentelemetry.io/collector/consumer/xconsumer v0.139.0 h1:FhzDv+idglnrfjqPvnUw3YAEOkXSNv/FuNsuMiXQwcY= -go.opentelemetry.io/collector/consumer/xconsumer v0.139.0/go.mod h1:yWrg/6FE/A4Q7eo/Mg++CzkBoSILHdeMnTlxV3serI0= -go.opentelemetry.io/collector/featuregate v1.45.0 h1:D06hpf1F2KzKC+qXLmVv5e8IZpgCyZVeVVC8iOQxVmw= -go.opentelemetry.io/collector/featuregate v1.45.0/go.mod h1:d0tiRzVYrytB6LkcYgz2ESFTv7OktRPQe0QEQcPt1L4= -go.opentelemetry.io/collector/pdata v1.45.0 h1:q4XaISpeX640BcwXwb2mKOVw/gb67r22HjGWl8sbWsk= -go.opentelemetry.io/collector/pdata v1.45.0/go.mod h1:5q2f001YhwMQO8QvpFhCOa4Cq/vtwX9W4HRMsXkU/nE= -go.opentelemetry.io/collector/pdata/pprofile v0.139.0 h1:UA5TgFzYmRuJN3Wz0GR1efLUfjbs5rH0HTaxfASpTR8= -go.opentelemetry.io/collector/pdata/pprofile v0.139.0/go.mod h1:sI5qHt+zzE2fhOWFdJIaiDBR0yGGjD4A4ZvDFU0tiHk= -go.opentelemetry.io/collector/pdata/testdata v0.139.0 h1:n7O5bmLLhc3T6PePV4447fFcI/6QWcMhBsLtfCaD0do= -go.opentelemetry.io/collector/pdata/testdata v0.139.0/go.mod h1:fxZ2VrhYLYBLHYBHC1XQRKZ6IJXwy0I2rPaaRlebYaY= -go.opentelemetry.io/collector/pipeline v1.45.0 h1:sn9JJAEBe3XABTkWechMk0eH60QMBjjNe5V+ccBl+Uo= -go.opentelemetry.io/collector/pipeline v1.45.0/go.mod h1:xUrAqiebzYbrgxyoXSkk6/Y3oi5Sy3im2iCA51LwUAI= -go.opentelemetry.io/collector/processor v1.45.0 h1:GH5km9BkDQOoz7MR0jzTnzB1Kb5vtKzPwa/wDmRg2dQ= -go.opentelemetry.io/collector/processor v1.45.0/go.mod h1:wdlaTTC3wqlZIJP9R9/SLc2q7h+MFGARsxfjgPtwbes= -go.opentelemetry.io/collector/processor/processortest v0.139.0 h1:30akUdruFNG7EDpayuBhXoX2lV+hcfxW9Gl3Z6MYHb0= -go.opentelemetry.io/collector/processor/processortest v0.139.0/go.mod h1:RTll3UKHrqj/VS6RGjTHtuGIJzyLEwFhbw8KuCL3pjo= -go.opentelemetry.io/collector/processor/xprocessor v0.139.0 h1:O9x9RF/OG8gZ+HrOcB4f6F1fjniby484xf2D8GBxgqU= -go.opentelemetry.io/collector/processor/xprocessor v0.139.0/go.mod h1:hqGhEZ1/PftD/QHaYna0o1xAqZUsb7GhqpOiaTTDJnQ= +go.mongodb.org/mongo-driver v1.17.6 h1:87JUG1wZfWsr6rIz3ZmpH90rL5tea7O3IHuSwHUpsss= +go.mongodb.org/mongo-driver v1.17.6/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ= +go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= +go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= +go.opentelemetry.io/collector/component v1.48.0 h1:0hZKOvT6fIlXoE+6t40UXbXOH7r/h9jyE3eIt0W19Qg= +go.opentelemetry.io/collector/component v1.48.0/go.mod h1:Kmc9Z2CT53M2oRRf+WXHUHHgjCC+ADbiqfPO5mgZe3g= +go.opentelemetry.io/collector/component/componentstatus v0.142.0 h1:a1KkLCtShI5SfhO2ga75VqWjjBRGgrerelt/2JXWLBI= +go.opentelemetry.io/collector/component/componentstatus v0.142.0/go.mod h1:IRWKvFcUrFrkz1gJEV+cKAdE2ZBT128gk1sHt0OzKI4= +go.opentelemetry.io/collector/component/componenttest v0.142.0 h1:a8XclEutO5dv4AnzThHK8dfqR4lDWjJKLtRNM2aVUFM= +go.opentelemetry.io/collector/component/componenttest v0.142.0/go.mod h1:JhX/zKaEbjhFcsiV2ha2spzo24A6RL/jqNBS0svURD0= +go.opentelemetry.io/collector/confmap v1.48.0 h1:vGhg25NEUX5DiYziJEw2siwdzsvtXBRZVuYyLVinFR8= +go.opentelemetry.io/collector/confmap v1.48.0/go.mod h1:8tJHJowmvUkJ8AHzZ6SaH61dcWbdfRE9Sd/hwsKLgRE= +go.opentelemetry.io/collector/confmap/xconfmap v0.142.0 h1:SNfuFP8TA0PmUkx6ryY63uNjLN2HMh5VeGO++IYdPgA= +go.opentelemetry.io/collector/confmap/xconfmap v0.142.0/go.mod h1:FXuX6B8b7Ub7qkLqloWKanmPhADL18EEkaFptcd4eDQ= +go.opentelemetry.io/collector/consumer v1.48.0 h1:g1uroz2AA0cqnEsjqFTSZG+y8uH1gQBqqyzk8kd3QiM= +go.opentelemetry.io/collector/consumer v1.48.0/go.mod h1:lC6PnVXBwI456SV5WtvJqE7vjCNN6DAUc8xjFQ9wUV4= +go.opentelemetry.io/collector/consumer/consumertest v0.142.0 h1:TRt8zR57Vk1PTjtqjHOwOAMbIl+IeloHxWAuF8sWdRw= +go.opentelemetry.io/collector/consumer/consumertest v0.142.0/go.mod h1:yq2dhMxFUlCFkRN7LES3fzsTmUDw9VaunyRAka2TEaY= +go.opentelemetry.io/collector/consumer/xconsumer v0.142.0 h1:qOoQnLZXQ9sRLexTkkmBx3qfaOmEgco9VBPmryg5UhA= +go.opentelemetry.io/collector/consumer/xconsumer v0.142.0/go.mod h1:oPN0yJzEpovwlWvmSaiYgtDqGuOmMMLmmg352sqZdsE= +go.opentelemetry.io/collector/featuregate v1.48.0 h1:jiGRcl93yzUFgZVDuskMAftFraE21jANdxXTQfSQScc= +go.opentelemetry.io/collector/featuregate v1.48.0/go.mod h1:/1bclXgP91pISaEeNulRxzzmzMTm4I5Xih2SnI4HRSo= +go.opentelemetry.io/collector/internal/testutil v0.142.0 h1:MHnAVRimQdsfYqYHC3YuJRkIUap4VmSpJkkIT2N7jJA= +go.opentelemetry.io/collector/internal/testutil v0.142.0/go.mod h1:YAD9EAkwh/l5asZNbEBEUCqEjoL1OKMjAMoPjPqH76c= +go.opentelemetry.io/collector/pdata v1.48.0 h1:CKZ+9v/lGTX/cTGx2XVp8kp0E8R//60kHFCBdZudrTg= +go.opentelemetry.io/collector/pdata v1.48.0/go.mod h1:jaf2JQGpfUreD1TOtGBPsq00ecOqM66NG15wALmdxKA= +go.opentelemetry.io/collector/pdata/pprofile v0.142.0 h1:Ivyw7WY8SIIWqzXsnNmjEgz3ysVs/OkIf0KIpJUnuuo= +go.opentelemetry.io/collector/pdata/pprofile v0.142.0/go.mod h1:94GAph54K4WDpYz9xirhroHB3ptNLuPiY02k8fyoNUI= +go.opentelemetry.io/collector/pdata/testdata v0.142.0 h1:+jf9RyLWl8WyhIVjpg7yuH+bRdQH4mW20cPtCMlY1cI= +go.opentelemetry.io/collector/pdata/testdata v0.142.0/go.mod h1:kgAu5ZLEcVuPH3RFiHDg23RGitgm1M0cUAVwiGX4SB8= +go.opentelemetry.io/collector/pipeline v1.48.0 h1:E4zyQ7+4FTGvdGS4pruUnItuyRTGhN0Qqk1CN71lfW0= +go.opentelemetry.io/collector/pipeline v1.48.0/go.mod h1:xUrAqiebzYbrgxyoXSkk6/Y3oi5Sy3im2iCA51LwUAI= +go.opentelemetry.io/collector/processor v1.48.0 h1:3Kttw79mnrf463QKJGoGZzFfiNzQuMWK0p2nHuvOhaQ= +go.opentelemetry.io/collector/processor v1.48.0/go.mod h1:A3OsW6ga+a48J1mrnVNH5L5kB0v+n9nVFlmOQB5/Jwk= +go.opentelemetry.io/collector/processor/processortest v0.142.0 h1:wQnJeXDejBL6r8ov66AYAGf8Q0/JspjuqAjPVBdCUoI= +go.opentelemetry.io/collector/processor/processortest v0.142.0/go.mod h1:QU5SWj0L+92MSvQxZDjwWCsKssNDm+nD6SHn7IvviUE= +go.opentelemetry.io/collector/processor/xprocessor v0.142.0 h1:7a1Crxrd5iBMVnebTxkcqxVkRHAlOBUUmNTUVUTnlCU= +go.opentelemetry.io/collector/processor/xprocessor v0.142.0/go.mod h1:LY/GS2DiJILJKS3ynU3eOLLWSP8CmN1FtdpAMsVV8AU= go.opentelemetry.io/collector/semconv v0.128.0 h1:MzYOz7Vgb3Kf5D7b49pqqgeUhEmOCuT10bIXb/Cc+k4= go.opentelemetry.io/collector/semconv v0.128.0/go.mod h1:OPXer4l43X23cnjLXIZnRj/qQOjSuq4TgBLI76P9hns= -go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 h1:2pn7OzMewmYRiNtv1doZnLo3gONcnMHlFnmOR8Vgt+8= -go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0/go.mod h1:rjbQTDEPQymPE0YnRQp9/NuPwwtL0sesz/fnqRW/v84= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg= -go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8= -go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 h1:GqRJVj7UmLjCVyVJ3ZFLdPRmhDUp2zFmQe3RHIOsw24= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0/go.mod h1:ri3aaHSmCTVYu2AWv44YMauwAQc0aqI9gHKIcSbI1pU= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 h1:lwI4Dc5leUqENgGuQImwLo4WnuXFPetmPpkLi2IrX54= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0/go.mod h1:Kz/oCE7z5wuyhPxsXDuaPteSWqjSBD5YaSdbxZYGbGk= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 h1:aTL7F04bJHUlztTsNGJ2l+6he8c+y/b//eR0jjjemT4= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0/go.mod h1:kldtb7jDTeol0l3ewcmd8SDvx3EmIE7lyvqbasU3QC4= -go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA= -go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI= -go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E= -go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg= -go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM= -go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA= -go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE= -go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs= -go.opentelemetry.io/proto/otlp v1.7.1 h1:gTOMpGDb0WTBOP8JaO72iL3auEZhVmAQg4ipjOVAtj4= -go.opentelemetry.io/proto/otlp v1.7.1/go.mod h1:b2rVh6rfI/s2pHWNlB7ILJcRALpcNDzKhACevjI+ZnE= -go.opentelemetry.io/proto/slim/otlp v1.8.0 h1:afcLwp2XOeCbGrjufT1qWyruFt+6C9g5SOuymrSPUXQ= -go.opentelemetry.io/proto/slim/otlp v1.8.0/go.mod h1:Yaa5fjYm1SMCq0hG0x/87wV1MP9H5xDuG/1+AhvBcsI= -go.opentelemetry.io/proto/slim/otlp/collector/profiles/v1development v0.1.0 h1:Uc+elixz922LHx5colXGi1ORbsW8DTIGM+gg+D9V7HE= -go.opentelemetry.io/proto/slim/otlp/collector/profiles/v1development v0.1.0/go.mod h1:VyU6dTWBWv6h9w/+DYgSZAPMabWbPTFTuxp25sM8+s0= -go.opentelemetry.io/proto/slim/otlp/profiles/v1development v0.1.0 h1:i8YpvWGm/Uq1koL//bnbJ/26eV3OrKWm09+rDYo7keU= -go.opentelemetry.io/proto/slim/otlp/profiles/v1development v0.1.0/go.mod h1:pQ70xHY/ZVxNUBPn+qUWPl8nwai87eWdqL3M37lNi9A= +go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.64.0 h1:OXSUzgmIFkcC4An+mv+lqqZSndTffXpjAyoR+1f8k/A= +go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.64.0/go.mod h1:1A4GVLFIm54HFqVdOpWmukap7rgb0frrE3zWXohLPdM= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 h1:ssfIgGNANqpVFCndZvcuyKbl0g+UAVcbBcqGkG28H0Y= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0/go.mod h1:GQ/474YrbE4Jx8gZ4q5I4hrhUzM6UPzyrqJYV2AqPoQ= +go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48= +go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8= +go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.39.0 h1:f0cb2XPmrqn4XMy9PNliTgRKJgS5WcL/u0/WRYGz4t0= +go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.39.0/go.mod h1:vnakAaFckOMiMtOIhFI2MNH4FYrZzXCYxmb1LlhoGz8= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.39.0 h1:in9O8ESIOlwJAEGTkkf34DesGRAc/Pn8qJ7k3r/42LM= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.39.0/go.mod h1:Rp0EXBm5tfnv0WL+ARyO/PHBEaEAT8UUHQ6AGJcSq6c= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.39.0 h1:Ckwye2FpXkYgiHX7fyVrN1uA/UYd9ounqqTuSNAv0k4= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.39.0/go.mod h1:teIFJh5pW2y+AN7riv6IBPX2DuesS3HgP39mwOspKwU= +go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0= +go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs= +go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18= +go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE= +go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8= +go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew= +go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI= +go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA= +go.opentelemetry.io/proto/otlp v1.9.0 h1:l706jCMITVouPOqEnii2fIAuO3IVGBRPV5ICjceRb/A= +go.opentelemetry.io/proto/otlp v1.9.0/go.mod h1:xE+Cx5E/eEHw+ISFkwPLwCZefwVjY+pqKg1qcK03+/4= +go.opentelemetry.io/proto/slim/otlp v1.9.0 h1:fPVMv8tP3TrsqlkH1HWYUpbCY9cAIemx184VGkS6vlE= +go.opentelemetry.io/proto/slim/otlp v1.9.0/go.mod h1:xXdeJJ90Gqyll+orzUkY4bOd2HECo5JofeoLpymVqdI= +go.opentelemetry.io/proto/slim/otlp/collector/profiles/v1development v0.2.0 h1:o13nadWDNkH/quoDomDUClnQBpdQQ2Qqv0lQBjIXjE8= +go.opentelemetry.io/proto/slim/otlp/collector/profiles/v1development v0.2.0/go.mod h1:Gyb6Xe7FTi/6xBHwMmngGoHqL0w29Y4eW8TGFzpefGA= +go.opentelemetry.io/proto/slim/otlp/profiles/v1development v0.2.0 h1:EiUYvtwu6PMrMHVjcPfnsG3v+ajPkbUeH+IL93+QYyk= +go.opentelemetry.io/proto/slim/otlp/profiles/v1development v0.2.0/go.mod h1:mUUHKFiN2SST3AhJ8XhJxEoeVW12oqfXog0Bo8W3Ec4= go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs= @@ -583,8 +611,8 @@ go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= -go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= -go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= +go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc= +go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= @@ -594,14 +622,14 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= -golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= +golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU= +golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0= golang.org/x/exp v0.0.0-20250808145144-a408d31f581a h1:Y+7uR/b1Mw2iSXZ3G//1haIiSElDQZ8KWh0h+sZPG90= golang.org/x/exp v0.0.0-20250808145144-a408d31f581a/go.mod h1:rT6SFzZ7oxADUDx58pcaKFTcZ+inxAa9fTrYx/uVYwg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA= -golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w= +golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk= +golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= @@ -612,18 +640,18 @@ golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLL golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy7fQ90B1CfIiPueXVOjqfkSzI8= -golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= -golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= -golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= -golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= +golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU= +golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY= +golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw= +golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I= -golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= +golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -649,27 +677,27 @@ golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= -golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= +golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU= -golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254= +golang.org/x/term v0.38.0 h1:PQ5pkm/rLO6HnxFR7N2lJHOZX6Kez5Y1gDSJla6jo7Q= +golang.org/x/term v0.38.0/go.mod h1:bSEAKrOT1W+VSu9TSCMtoGEOUcKxOKgl3LE5QEF/xVg= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= -golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= -golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= -golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= +golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY= +golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= +golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ= -golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs= +golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ= +golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ= golang.org/x/tools/godoc v0.1.0-deprecated h1:o+aZ1BOj6Hsx/GBdJO/s815sqftjSnrZZwyYTHODvtk= golang.org/x/tools/godoc v0.1.0-deprecated/go.mod h1:qM63CriJ961IHWmnWa9CjZnBndniPt4a3CK0PVB9bIg= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -678,18 +706,18 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk= gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E= -google.golang.org/api v0.252.0 h1:xfKJeAJaMwb8OC9fesr369rjciQ704AjU/psjkKURSI= -google.golang.org/api v0.252.0/go.mod h1:dnHOv81x5RAmumZ7BWLShB/u7JZNeyalImxHmtTHxqw= +google.golang.org/api v0.257.0 h1:8Y0lzvHlZps53PEaw+G29SsQIkuKrumGWs9puiexNAA= +google.golang.org/api v0.257.0/go.mod h1:4eJrr+vbVaZSqs7vovFd1Jb/A6ml6iw2e6FBYf3GAO4= google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4= google.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s= -google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 h1:8XJ4pajGwOlasW+L13MnEGA8W4115jJySQtVfS2/IBU= -google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4/go.mod h1:NnuHhy+bxcg30o7FnVAZbXsPHUDQ9qKWAQKCD7VxFtk= -google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797 h1:CirRxTOwnRWVLKzDNrs0CXAaVozJoR4G9xvdRecrdpk= -google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797/go.mod h1:HSkG/KdJWusxU1F6CNrwNDjBMgisKxGnc5dAZfT0mjQ= -google.golang.org/grpc v1.76.0 h1:UnVkv1+uMLYXoIz6o7chp59WfQUYA2ex/BXQ9rHZu7A= -google.golang.org/grpc v1.76.0/go.mod h1:Ju12QI8M6iQJtbcsV+awF5a4hfJMLi4X0JLo94ULZ6c= -google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= -google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2 h1:7LRqPCEdE4TP4/9psdaB7F2nhZFfBiGJomA5sojLWdU= +google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2/go.mod h1:+rXWjjaukWZun3mLfjmVnQi18E1AsFbDN9QdJ5YXLto= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk= +google.golang.org/grpc v1.77.0 h1:wVVY6/8cGA6vvffn+wWK5ToddbgdU3d8MNENr4evgXM= +google.golang.org/grpc v1.77.0/go.mod h1:z0BY1iVj0q8E1uSQCjL9cppRj+gnZjzDnzV0dHhrNig= +google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= +google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= @@ -713,12 +741,12 @@ gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= gotest.tools/v3 v3.0.3 h1:4AuOwCGf4lLR9u3YOe2awrHygurzhO/HeQ6laiA6Sx0= gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8= -k8s.io/api v0.34.1 h1:jC+153630BMdlFukegoEL8E/yT7aLyQkIVuwhmwDgJM= -k8s.io/api v0.34.1/go.mod h1:SB80FxFtXn5/gwzCoN6QCtPD7Vbu5w2n1S0J5gFfTYk= -k8s.io/apimachinery v0.34.1 h1:dTlxFls/eikpJxmAC7MVE8oOeP1zryV7iRyIjB0gky4= -k8s.io/apimachinery v0.34.1/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw= -k8s.io/client-go v0.34.1 h1:ZUPJKgXsnKwVwmKKdPfw4tB58+7/Ik3CrjOEhsiZ7mY= -k8s.io/client-go v0.34.1/go.mod h1:kA8v0FP+tk6sZA0yKLRG67LWjqufAoSHA2xVGKw9Of8= +k8s.io/api v0.34.3 h1:D12sTP257/jSH2vHV2EDYrb16bS7ULlHpdNdNhEw2S4= +k8s.io/api v0.34.3/go.mod h1:PyVQBF886Q5RSQZOim7DybQjAbVs8g7gwJNhGtY5MBk= +k8s.io/apimachinery v0.34.3 h1:/TB+SFEiQvN9HPldtlWOTp0hWbJ+fjU+wkxysf/aQnE= +k8s.io/apimachinery v0.34.3/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw= +k8s.io/client-go v0.34.3 h1:wtYtpzy/OPNYf7WyNBTj3iUA0XaBHVqhv4Iv3tbrF5A= +k8s.io/client-go v0.34.3/go.mod h1:OxxeYagaP9Kdf78UrKLa3YZixMCfP6bgPwPwNBQBzpM= k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8= k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I= k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk= diff --git a/internal/tools/go.mod b/internal/tools/go.mod index e4817a35cd..2334ae7bd0 100644 --- a/internal/tools/go.mod +++ b/internal/tools/go.mod @@ -1,58 +1,61 @@ module github.com/prometheus/prometheus/internal/tools -go 1.24.0 +go 1.24.9 require ( - github.com/bufbuild/buf v1.57.2 + github.com/bufbuild/buf v1.61.0 github.com/daixiang0/gci v0.13.7 github.com/gogo/protobuf v1.3.2 github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 ) require ( - buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.9-20250718181942-e35f9b667443.1 // indirect - buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.9-20250912141014-52f32327d4b0.1 // indirect - buf.build/gen/go/bufbuild/registry/connectrpc/go v1.18.1-20250903170917-c4be0f57e197.1 // indirect - buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.9-20250903170917-c4be0f57e197.1 // indirect - buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.8-20241007202033-cf42259fcbfc.1 // indirect - buf.build/go/app v0.1.0 // indirect + buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.10-20250718181942-e35f9b667443.1 // indirect + buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.10-20250109164928-1da0de137947.1 // indirect + buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.10-20250912141014-52f32327d4b0.1 // indirect + buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251027152159-f1066ce064ca.2 // indirect + buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.10-20251027152159-f1066ce064ca.1 // indirect + buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.10-20241007202033-cf42259fcbfc.1 // indirect + buf.build/go/app v0.2.0 // indirect buf.build/go/bufplugin v0.9.0 // indirect + buf.build/go/bufprivateusage v0.1.0 // indirect buf.build/go/interrupt v1.1.0 // indirect - buf.build/go/protovalidate v1.0.0 // indirect + buf.build/go/protovalidate v1.0.1 // indirect buf.build/go/protoyaml v0.6.0 // indirect buf.build/go/spdx v0.2.0 // indirect buf.build/go/standard v0.1.0 // indirect cel.dev/expr v0.24.0 // indirect - connectrpc.com/connect v1.18.1 // indirect + connectrpc.com/connect v1.19.1 // indirect connectrpc.com/otelconnect v0.8.0 // indirect github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect github.com/Microsoft/go-winio v0.6.2 // indirect github.com/antlr4-go/antlr/v4 v4.13.1 // indirect - github.com/bufbuild/protocompile v0.14.1 // indirect + github.com/bufbuild/protocompile v0.14.2-0.20251120233202-3f9009bcd6c8 // indirect github.com/bufbuild/protoplugin v0.0.0-20250218205857-750e09ce93e1 // indirect + github.com/cli/browser v1.3.0 // indirect github.com/containerd/errdefs v1.0.0 // indirect github.com/containerd/errdefs/pkg v0.3.0 // indirect github.com/containerd/stargz-snapshotter/estargz v0.17.0 // indirect github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect github.com/distribution/reference v0.6.0 // indirect - github.com/docker/cli v28.4.0+incompatible // indirect + github.com/docker/cli v28.5.1+incompatible // indirect github.com/docker/distribution v2.8.3+incompatible // indirect - github.com/docker/docker v28.4.0+incompatible // indirect - github.com/docker/docker-credential-helpers v0.9.3 // indirect + github.com/docker/docker v28.5.2+incompatible // indirect + github.com/docker/docker-credential-helpers v0.9.4 // indirect github.com/docker/go-connections v0.6.0 // indirect github.com/docker/go-units v0.5.0 // indirect github.com/felixge/httpsnoop v1.0.4 // indirect github.com/go-chi/chi/v5 v5.2.3 // indirect github.com/go-logr/logr v1.4.3 // indirect github.com/go-logr/stdr v1.2.2 // indirect - github.com/gofrs/flock v0.12.1 // indirect + github.com/gofrs/flock v0.13.0 // indirect github.com/google/cel-go v0.26.1 // indirect github.com/google/go-containerregistry v0.20.6 // indirect github.com/google/uuid v1.6.0 // indirect github.com/hexops/gotextdiff v1.0.3 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/jdx/go-netrc v1.0.0 // indirect - github.com/klauspost/compress v1.18.0 // indirect + github.com/klauspost/compress v1.18.1 // indirect github.com/klauspost/pgzip v1.2.6 // indirect github.com/mattn/go-colorable v0.1.14 // indirect github.com/mattn/go-isatty v0.0.20 // indirect @@ -62,19 +65,21 @@ require ( github.com/morikuni/aec v1.0.0 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.1.1 // indirect - github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect + github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 // indirect github.com/pkg/errors v0.9.1 // indirect github.com/quic-go/qpack v0.5.1 // indirect - github.com/quic-go/quic-go v0.54.0 // indirect + github.com/quic-go/quic-go v0.56.0 // indirect + github.com/rivo/uniseg v0.4.7 // indirect github.com/rs/cors v1.11.1 // indirect github.com/russross/blackfriday/v2 v2.1.0 // indirect - github.com/segmentio/asm v1.2.0 // indirect + github.com/segmentio/asm v1.2.1 // indirect github.com/segmentio/encoding v0.5.3 // indirect github.com/sirupsen/logrus v1.9.3 // indirect github.com/spf13/cobra v1.10.1 // indirect github.com/spf13/pflag v1.0.10 // indirect github.com/stoewer/go-strcase v1.3.1 // indirect github.com/tetratelabs/wazero v1.9.0 // indirect + github.com/tidwall/btree v1.8.1 // indirect github.com/vbatts/tar-split v0.12.1 // indirect go.lsp.dev/jsonrpc2 v0.10.0 // indirect go.lsp.dev/pkg v0.0.0-20210717090340-384b27a52fb2 // indirect @@ -87,19 +92,18 @@ require ( go.opentelemetry.io/otel/trace v1.38.0 // indirect go.uber.org/mock v0.6.0 // indirect go.uber.org/multierr v1.11.0 // indirect - go.uber.org/zap v1.27.0 // indirect + go.uber.org/zap v1.27.1 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect golang.org/x/crypto v0.45.0 // indirect - golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b // indirect - golang.org/x/mod v0.29.0 // indirect + golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b // indirect + golang.org/x/mod v0.30.0 // indirect golang.org/x/net v0.47.0 // indirect golang.org/x/sync v0.18.0 // indirect golang.org/x/sys v0.38.0 // indirect golang.org/x/term v0.37.0 // indirect golang.org/x/text v0.31.0 // indirect - golang.org/x/tools v0.38.0 // indirect - google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4 // indirect + google.golang.org/genproto/googleapis/api v0.0.0-20251007200510-49b9836ed3ff // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff // indirect google.golang.org/grpc v1.75.1 // indirect google.golang.org/protobuf v1.36.10 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect diff --git a/internal/tools/go.sum b/internal/tools/go.sum index 26df5c98a2..fb63670b60 100644 --- a/internal/tools/go.sum +++ b/internal/tools/go.sum @@ -1,21 +1,25 @@ -buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.9-20250718181942-e35f9b667443.1 h1:HiLfreYRsqycF5QDlsnvSQOnl4tvhBoROl8+DkbaphI= -buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.9-20250718181942-e35f9b667443.1/go.mod h1:WSxC6zKCpqVRcGZCpOgVwkATp9XBIleoAdSAnkq7dhw= -buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.9-20250912141014-52f32327d4b0.1 h1:DQLS/rRxLHuugVzjJU5AvOwD57pdFl9he/0O7e5P294= -buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.9-20250912141014-52f32327d4b0.1/go.mod h1:aY3zbkNan5F+cGm9lITDP6oxJIwu0dn9KjJuJjWaHkg= -buf.build/gen/go/bufbuild/registry/connectrpc/go v1.18.1-20250903170917-c4be0f57e197.1 h1:isqFuFhL6JRd7+KF/vivWqZGJMCaTuAccZIWwneCcqE= -buf.build/gen/go/bufbuild/registry/connectrpc/go v1.18.1-20250903170917-c4be0f57e197.1/go.mod h1:eGjb9P6sl1irS46NKyXnxkyozT2aWs3BF4tbYWQuCsw= -buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.9-20250903170917-c4be0f57e197.1 h1:q+tABqEH2Cpcp8fO9TBZlvKok7zorHGy+/UyywXaAKo= -buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.9-20250903170917-c4be0f57e197.1/go.mod h1:Y3m+VD8IH6JTgnFYggPHvFul/ry6dL3QDliy8xH7610= -buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.8-20241007202033-cf42259fcbfc.1 h1:KuP+b+in6LGh2ukof5KgDCD8hPXotEq6EVOo13Wg1pE= -buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.8-20241007202033-cf42259fcbfc.1/go.mod h1:dV1Kz6zdmyXt7QWm5OXby44OFpyLemllUDBUG5HMLio= -buf.build/go/app v0.1.0 h1:nlqD/h0rhIN73ZoiDElprrPiO2N6JV+RmNK34K29Ihg= -buf.build/go/app v0.1.0/go.mod h1:0XVOYemubVbxNXVY0DnsVgWeGkcbbAvjDa1fmhBC+Wo= +buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.10-20250718181942-e35f9b667443.1 h1:FzJGrb8r7vir+P3zJ5Ebey8p54LYTYtQsrM/U35YO9Q= +buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.10-20250718181942-e35f9b667443.1/go.mod h1:E6HwqUm4Ag7bXtg/tX7jHWO7CgpknbmeACgDax0icV0= +buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.10-20250109164928-1da0de137947.1 h1:9hkMnVoImDlY7rTlAWIWXdkGUKOjf3YlyZeSbYT29uA= +buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.10-20250109164928-1da0de137947.1/go.mod h1:/AouMCAeQ+kB7+RRFpdUlZe3503p18VoUNcU2AFqZXM= +buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.10-20250912141014-52f32327d4b0.1 h1:31on4W/yPcV4nZHL4+UCiCvLPsMqe/vJcNg8Rci0scc= +buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.10-20250912141014-52f32327d4b0.1/go.mod h1:fUl8CEN/6ZAMk6bP8ahBJPUJw7rbp+j4x+wCcYi2IG4= +buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251027152159-f1066ce064ca.2 h1:Dbh4Edwy5qHlz1/boPAQ7T5Q7ZDMgEuQlEbXa94+JEo= +buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251027152159-f1066ce064ca.2/go.mod h1:SqqTA3aiYVDkpDINxgbxDT6QBjkVjdqUXtbiz6DiWIg= +buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.10-20251027152159-f1066ce064ca.1 h1:5tUFlRgcC+N2JJtjwlwyb2J4bBk/bJYLXk50zlewtzk= +buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.10-20251027152159-f1066ce064ca.1/go.mod h1:AaYXXeRvnOc151wEuupAmn58Mh9bccKce2kk3QKMIrQ= +buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.10-20241007202033-cf42259fcbfc.1 h1:CzM0kZcoaIr8+R4i8QVorUNRM/CqMr87i3j+w2pdpCc= +buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.10-20241007202033-cf42259fcbfc.1/go.mod h1:bG+Fa7tcA+4pW0JdOh4h7iKjleyZIKhfVzVS10qfrnk= +buf.build/go/app v0.2.0 h1:NYaH13A+RzPb7M5vO8uZYZ2maBZI5+MS9A9tQm66fy8= +buf.build/go/app v0.2.0/go.mod h1:0XVOYemubVbxNXVY0DnsVgWeGkcbbAvjDa1fmhBC+Wo= buf.build/go/bufplugin v0.9.0 h1:ktZJNP3If7ldcWVqh46XKeiYJVPxHQxCfjzVQDzZ/lo= buf.build/go/bufplugin v0.9.0/go.mod h1:Z0CxA3sKQ6EPz/Os4kJJneeRO6CjPeidtP1ABh5jPPY= +buf.build/go/bufprivateusage v0.1.0 h1:SzCoCcmzS3zyXHEXHeSQhGI7OTkgtljoknLzsUz9Gg4= +buf.build/go/bufprivateusage v0.1.0/go.mod h1:GlCCJ3VVF7EqqU0CoRmo1FzAwwaKymEWSr+ty69xU5w= buf.build/go/interrupt v1.1.0 h1:olBuhgv9Sav4/9pkSLoxgiOsZDgM5VhRhvRpn3DL0lE= buf.build/go/interrupt v1.1.0/go.mod h1:ql56nXPG1oHlvZa6efNC7SKAQ/tUjS6z0mhJl0gyeRM= -buf.build/go/protovalidate v1.0.0 h1:IAG1etULddAy93fiBsFVhpj7es5zL53AfB/79CVGtyY= -buf.build/go/protovalidate v1.0.0/go.mod h1:KQmEUrcQuC99hAw+juzOEAmILScQiKBP1Oc36vvCLW8= +buf.build/go/protovalidate v1.0.1 h1:Fwmf08OOUuKVeMvEnDmcKxQam4PJc/zFgvVX64BhTms= +buf.build/go/protovalidate v1.0.1/go.mod h1:SoZmvk/3ZzOVg9YSkTdm4grMAByjf8zgZq4ZNaLZXoQ= buf.build/go/protoyaml v0.6.0 h1:Nzz1lvcXF8YgNZXk+voPPwdU8FjDPTUV4ndNTXN0n2w= buf.build/go/protoyaml v0.6.0/go.mod h1:RgUOsBu/GYKLDSIRgQXniXbNgFlGEZnQpRAUdLAFV2Q= buf.build/go/spdx v0.2.0 h1:IItqM0/cMxvFJJumcBuP8NrsIzMs/UYjp/6WSpq8LTw= @@ -24,8 +28,8 @@ buf.build/go/standard v0.1.0 h1:g98T9IyvAl0vS3Pq8iVk6Cvj2ZiFvoUJRtfyGa0120U= buf.build/go/standard v0.1.0/go.mod h1:PiqpHz/7ZFq+kqvYhc/SK3lxFIB9N/aiH2CFC2JHIQg= cel.dev/expr v0.24.0 h1:56OvJKSH3hDGL0ml5uSxZmz3/3Pq4tJ+fb1unVLAFcY= cel.dev/expr v0.24.0/go.mod h1:hLPLo1W4QUmuYdA72RBX06QTs6MXw941piREPl3Yfiw= -connectrpc.com/connect v1.18.1 h1:PAg7CjSAGvscaf6YZKUefjoih5Z/qYkyaTrBW8xvYPw= -connectrpc.com/connect v1.18.1/go.mod h1:0292hj1rnx8oFrStN7cB4jjVBeqs+Yx5yDIC2prWDO8= +connectrpc.com/connect v1.19.1 h1:R5M57z05+90EfEvCY1b7hBxDVOUl45PrtXtAV2fOC14= +connectrpc.com/connect v1.19.1/go.mod h1:tN20fjdGlewnSFeZxLKb0xwIZ6ozc3OQs2hTXy4du9w= connectrpc.com/otelconnect v0.8.0 h1:a4qrN4H8aEE2jAoCxheZYYfEjXMgVPyL9OzPQLBEFXU= connectrpc.com/otelconnect v0.8.0/go.mod h1:AEkVLjCPXra+ObGFCOClcJkNjS7zPaQSqvO0lCyjfZc= github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg= @@ -34,14 +38,20 @@ github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERo github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ= github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw= -github.com/bufbuild/buf v1.57.2 h1:2vxP0giB8DVo0Lkem9T8WDUYIEC3zqY98+NHqAlP4ig= -github.com/bufbuild/buf v1.57.2/go.mod h1:8cygE3L/J84dtgQAaquZKpXLo9MjAn+dSdFuXvbUNYg= -github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/FBatYVw= -github.com/bufbuild/protocompile v0.14.1/go.mod h1:ppVdAIhbr2H8asPk6k4pY7t9zB1OU5DoEw9xY/FUi1c= +github.com/bmatcuk/doublestar/v4 v4.9.1 h1:X8jg9rRZmJd4yRy7ZeNDRnM+T3ZfHv15JiBJ/avrEXE= +github.com/bmatcuk/doublestar/v4 v4.9.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= +github.com/brianvoe/gofakeit/v6 v6.28.0 h1:Xib46XXuQfmlLS2EXRuJpqcw8St6qSZz75OUo0tgAW4= +github.com/brianvoe/gofakeit/v6 v6.28.0/go.mod h1:Xj58BMSnFqcn/fAQeSK+/PLtC5kSb7FJIq4JyGa8vEs= +github.com/bufbuild/buf v1.61.0 h1:JPaK/RM2eoheyzznW+1LxaFgN6xjBCi8s25q2kUbH9A= +github.com/bufbuild/buf v1.61.0/go.mod h1:Xs3leBmxjL5tTnSVYfNwNXHXD1k5et3fR/tJyIyQl4s= +github.com/bufbuild/protocompile v0.14.2-0.20251120233202-3f9009bcd6c8 h1:l4PKzJ7Usff8j5/e+YaWZPaM+rJHIghgDxRn8vDNxNo= +github.com/bufbuild/protocompile v0.14.2-0.20251120233202-3f9009bcd6c8/go.mod h1:HKN246DRQwavs64sr2xYmSL+RFOFxmLti+WGCZ2jh9U= github.com/bufbuild/protoplugin v0.0.0-20250218205857-750e09ce93e1 h1:V1xulAoqLqVg44rY97xOR+mQpD2N+GzhMHVwJ030WEU= github.com/bufbuild/protoplugin v0.0.0-20250218205857-750e09ce93e1/go.mod h1:c5D8gWRIZ2HLWO3gXYTtUfw/hbJyD8xikv2ooPxnklQ= github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8= github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE= +github.com/cli/browser v1.3.0 h1:LejqCrpWr+1pRqmEPDGnTZOjsMe7sehifLynZJuqJpo= +github.com/cli/browser v1.3.0/go.mod h1:HH8s+fOAxjhQoBUAsKuPCbqUuxZDhQ2/aD+SzsEfBTk= github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= @@ -62,14 +72,14 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= -github.com/docker/cli v28.4.0+incompatible h1:RBcf3Kjw2pMtwui5V0DIMdyeab8glEw5QY0UUU4C9kY= -github.com/docker/cli v28.4.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= +github.com/docker/cli v28.5.1+incompatible h1:ESutzBALAD6qyCLqbQSEf1a/U8Ybms5agw59yGVc+yY= +github.com/docker/cli v28.5.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk= github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/docker v28.4.0+incompatible h1:KVC7bz5zJY/4AZe/78BIvCnPsLaC9T/zh72xnlrTTOk= -github.com/docker/docker v28.4.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8= -github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo= +github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM= +github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/docker/docker-credential-helpers v0.9.4 h1:76ItO69/AP/V4yT9V4uuuItG0B1N8hvt0T0c0NN/DzI= +github.com/docker/docker-credential-helpers v0.9.4/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c= github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94= github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= @@ -83,8 +93,8 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= -github.com/gofrs/flock v0.12.1 h1:MTLVXXHf8ekldpJk3AKicLij9MdwOWkZ+a/jHHZby9E= -github.com/gofrs/flock v0.12.1/go.mod h1:9zxTsyu5xtJ9DK+1tFZyibEV7y3uwDxPPfbxeeHCoD0= +github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw= +github.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/google/cel-go v0.26.1 h1:iPbVVEdkhTX++hpe3lzSk7D3G3QSYqLGoHOcEio+UXQ= @@ -108,8 +118,8 @@ github.com/jhump/protoreflect/v2 v2.0.0-beta.2 h1:qZU+rEZUOYTz1Bnhi3xbwn+VxdXkLV github.com/jhump/protoreflect/v2 v2.0.0-beta.2/go.mod h1:4tnOYkB/mq7QTyS3YKtVtNrJv4Psqout8HA1U+hZtgM= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= -github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/klauspost/compress v1.18.1 h1:bcSGx7UbpBqMChDtsF28Lw6v/G94LPrrbMbdC3JH2co= +github.com/klauspost/compress v1.18.1/go.mod h1:ZQFFVG+MdnR0P+l6wpXgIL4NTtwiKIdBnrBd8Nrxr+0= github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU= github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= @@ -136,24 +146,30 @@ github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8 github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= -github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ= -github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU= +github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 h1:QTvNkZ5ylY0PGgA+Lih+GdboMLY/G9SEGLMEGVjTVA4= +github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/protocolbuffers/protoscope v0.0.0-20221109213918-8e7a6aafa2c9 h1:arwj11zP0yJIxIRiDn22E0H8PxfF7TsTrc2wIPFIsf4= +github.com/protocolbuffers/protoscope v0.0.0-20221109213918-8e7a6aafa2c9/go.mod h1:SKZx6stCn03JN3BOWTwvVIO2ajMkb/zQdTceXYhKw/4= github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI= github.com/quic-go/qpack v0.5.1/go.mod h1:+PC4XFrEskIVkcLzpEkbLqq1uCoxPhQuvK5rH1ZgaEg= -github.com/quic-go/quic-go v0.54.0 h1:6s1YB9QotYI6Ospeiguknbp2Znb/jZYjZLRXn9kMQBg= -github.com/quic-go/quic-go v0.54.0/go.mod h1:e68ZEaCdyviluZmy44P6Iey98v/Wfz6HCjQEm+l8zTY= +github.com/quic-go/quic-go v0.56.0 h1:q/TW+OLismmXAehgFLczhCDTYB3bFmua4D9lsNBWxvY= +github.com/quic-go/quic-go v0.56.0/go.mod h1:9gx5KsFQtw2oZ6GZTyh+7YEvOxWCL9WZAepnHxgAo6c= +github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= +github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= +github.com/rodaine/protogofakeit v0.1.1 h1:ZKouljuRM3A+TArppfBqnH8tGZHOwM/pjvtXe9DaXH8= +github.com/rodaine/protogofakeit v0.1.1/go.mod h1:pXn/AstBYMaSfc1/RqH3N82pBuxtWgejz1AlYpY1mI0= github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/rs/cors v1.11.1 h1:eU3gRzXLRK57F5rKMGMZURNdIG4EoAmX8k94r9wXWHA= github.com/rs/cors v1.11.1/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU= github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/segmentio/asm v1.2.0 h1:9BQrFxC+YOHJlTlHGkTrFWf59nbL3XnCoFLTwDCI7ys= -github.com/segmentio/asm v1.2.0/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs= +github.com/segmentio/asm v1.2.1 h1:DTNbBqs57ioxAD4PrArqftgypG4/qNpXoJx8TVXxPR0= +github.com/segmentio/asm v1.2.1/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs= github.com/segmentio/encoding v0.5.3 h1:OjMgICtcSFuNvQCdwqMCv9Tg7lEOXGwm1J5RPQccx6w= github.com/segmentio/encoding v0.5.3/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0= github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= @@ -176,6 +192,8 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I= github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM= +github.com/tidwall/btree v1.8.1 h1:27ehoXvm5AG/g+1VxLS1SD3vRhp/H7LuEfwNvddEdmA= +github.com/tidwall/btree v1.8.1/go.mod h1:jBbTdUWhSZClZWoDg54VnvV7/54modSOzDN7VXftj1A= github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo= github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= @@ -214,8 +232,8 @@ go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y= go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= -go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= -go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= +go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc= +go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= @@ -223,12 +241,12 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= -golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b h1:DXr+pvt3nC887026GRP39Ej11UATqWDmWuS99x26cD0= -golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b/go.mod h1:4QTo5u+SEIbbKW1RacMZq1YEfOBqeXa19JeshGi+zc4= +golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b h1:18qgiDvlvH7kk8Ioa8Ov+K6xCi0GMvmGfGW0sgd/SYA= +golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA= -golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w= +golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk= +golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= @@ -245,7 +263,6 @@ golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= @@ -255,22 +272,22 @@ golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= -golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs= -golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE= +golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ= -golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs= +golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ= +golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 h1:8XJ4pajGwOlasW+L13MnEGA8W4115jJySQtVfS2/IBU= -google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4/go.mod h1:NnuHhy+bxcg30o7FnVAZbXsPHUDQ9qKWAQKCD7VxFtk= -google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4 h1:i8QOKZfYg6AbGVZzUAY3LrNWCKF8O6zFisU9Wl9RER4= -google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4/go.mod h1:HSkG/KdJWusxU1F6CNrwNDjBMgisKxGnc5dAZfT0mjQ= +google.golang.org/genproto/googleapis/api v0.0.0-20251007200510-49b9836ed3ff h1:8Zg5TdmcbU8A7CXGjGXF1Slqu/nIFCRaR3S5gT2plIA= +google.golang.org/genproto/googleapis/api v0.0.0-20251007200510-49b9836ed3ff/go.mod h1:dbWfpVPvW/RqafStmRWBUpMN14puDezDMHxNYiRfQu0= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff h1:A90eA31Wq6HOMIQlLfzFwzqGKBTuaVztYu/g8sn+8Zc= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk= google.golang.org/grpc v1.75.1 h1:/ODCNEuf9VghjgO3rqLcfg8fiOP0nSluljWFlDxELLI= google.golang.org/grpc v1.75.1/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ= google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= diff --git a/web/ui/mantine-ui/src/promql/tools/go.mod b/web/ui/mantine-ui/src/promql/tools/go.mod index 6983cf4fe6..32b64019e9 100644 --- a/web/ui/mantine-ui/src/promql/tools/go.mod +++ b/web/ui/mantine-ui/src/promql/tools/go.mod @@ -1,10 +1,10 @@ module github.com/prometheus/prometheus/web/ui/mantine-ui/src/promql/tools -go 1.24.0 +go 1.24.9 require ( - github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc - github.com/prometheus/prometheus v0.54.1 + github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 + github.com/prometheus/prometheus v0.308.1 github.com/russross/blackfriday/v2 v2.1.0 ) @@ -12,15 +12,15 @@ require ( github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/dennwc/varint v1.0.0 // indirect - github.com/go-kit/log v0.2.1 // indirect - github.com/go-logfmt/logfmt v0.6.0 // indirect + github.com/kr/text v0.2.0 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect - github.com/prometheus/client_golang v1.19.1 // indirect - github.com/prometheus/client_model v0.6.1 // indirect - github.com/prometheus/common v0.55.0 // indirect - github.com/prometheus/procfs v0.15.1 // indirect + github.com/prometheus/client_golang v1.23.2 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.67.4 // indirect + github.com/prometheus/procfs v0.16.1 // indirect go.uber.org/atomic v1.11.0 // indirect - golang.org/x/sys v0.22.0 // indirect - golang.org/x/text v0.16.0 // indirect - google.golang.org/protobuf v1.34.2 // indirect + go.yaml.in/yaml/v2 v2.4.3 // indirect + golang.org/x/sys v0.37.0 // indirect + golang.org/x/text v0.30.0 // indirect + google.golang.org/protobuf v1.36.10 // indirect ) diff --git a/web/ui/mantine-ui/src/promql/tools/go.sum b/web/ui/mantine-ui/src/promql/tools/go.sum index e7ed7cec79..40c792d93d 100644 --- a/web/ui/mantine-ui/src/promql/tools/go.sum +++ b/web/ui/mantine-ui/src/promql/tools/go.sum @@ -1,47 +1,86 @@ -github.com/Azure/azure-sdk-for-go/sdk/azcore v1.13.0 h1:GJHeeA2N7xrG3q30L2UXDyuWRzDM900/65j70wcM4Ww= -github.com/Azure/azure-sdk-for-go/sdk/azcore v1.13.0/go.mod h1:l38EPgmsp71HHLq9j7De57JcKOWPyhrsW1Awm1JS6K0= -github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0 h1:tfLQ34V6F7tVSwoTf/4lH5sE0o6eCJuNDTmH09nDpbc= -github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0/go.mod h1:9kIvujWAA58nmPmWB1m23fyWic1kYZMxD9CxaWn4Qpg= -github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY= -github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY= -github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 h1:XHOnouVk1mxXfQidrMEnLlPk9UMeRtyBTnEFtxkV0kU= -github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI= -github.com/alecthomas/units v0.0.0-20240626203959-61d1e3462e30 h1:t3eaIm0rUkzbrIewtiFmMK5RXHej2XnoXNhxVsAYUfg= -github.com/alecthomas/units v0.0.0-20240626203959-61d1e3462e30/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs= -github.com/aws/aws-sdk-go v1.54.19 h1:tyWV+07jagrNiCcGRzRhdtVjQs7Vy41NwsuOcl0IbVI= -github.com/aws/aws-sdk-go v1.54.19/go.mod h1:eRwEWoyTWFMVYVQzKMNHWP5/RV4xIUGMQfXQHfHkpNU= +cloud.google.com/go/auth v0.17.0 h1:74yCm7hCj2rUyyAocqnFzsAYXgJhrG26XCFimrc/Kz4= +cloud.google.com/go/auth v0.17.0/go.mod h1:6wv/t5/6rOPAX4fJiRjKkJCvswLwdet7G8+UGXt7nCQ= +cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc= +cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c= +cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs= +cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1 h1:5YTBM8QDVIBN3sxBil89WfdAAqDZbyJTgh688DSxX5w= +github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1/go.mod h1:YD5h/ldMsG0XiIw7PdyNhLxaM317eFh5yNLccNfGdyw= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0 h1:wL5IEG5zb7BVv1Kv0Xm92orq+5hB5Nipn3B5tn4Rqfk= +github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0/go.mod h1:J7MUC/wtRpfGVbQ5sIItY5/FuVWmvzlY21WAOfQnq/I= +github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 h1:9iefClla7iYpfYWdzPCRDozdmndjTm8DXdpCzPajMgA= +github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2/go.mod h1:XtLgD3ZD34DAaVIIAyG3objl5DynM3CQ/vMcbBNJZGI= +github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 h1:XkkQbfMyuH2jTSjQjSoihryI8GINRcs4xp8lNawg0FI= +github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk= +github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0= +github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b/go.mod h1:fvzegU4vN3H1qMT+8wDmzjAcDONcgo2/SZ/TyfdUOFs= +github.com/aws/aws-sdk-go-v2 v1.39.6 h1:2JrPCVgWJm7bm83BDwY5z8ietmeJUbh3O2ACnn+Xsqk= +github.com/aws/aws-sdk-go-v2 v1.39.6/go.mod h1:c9pm7VwuW0UPxAEYGyTmyurVcNrbF6Rt/wixFqDhcjE= +github.com/aws/aws-sdk-go-v2/config v1.31.17 h1:QFl8lL6RgakNK86vusim14P2k8BFSxjvUkcWLDjgz9Y= +github.com/aws/aws-sdk-go-v2/config v1.31.17/go.mod h1:V8P7ILjp/Uef/aX8TjGk6OHZN6IKPM5YW6S78QnRD5c= +github.com/aws/aws-sdk-go-v2/credentials v1.18.21 h1:56HGpsgnmD+2/KpG0ikvvR8+3v3COCwaF4r+oWwOeNA= +github.com/aws/aws-sdk-go-v2/credentials v1.18.21/go.mod h1:3YELwedmQbw7cXNaII2Wywd+YY58AmLPwX4LzARgmmA= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 h1:T1brd5dR3/fzNFAQch/iBKeX07/ffu/cLu+q+RuzEWk= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13/go.mod h1:Peg/GBAQ6JDt+RoBf4meB1wylmAipb7Kg2ZFakZTlwk= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 h1:a+8/MLcWlIxo1lF9xaGt3J/u3yOZx+CdSveSNwjhD40= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13/go.mod h1:oGnKwIYZ4XttyU2JWxFrwvhF6YKiK/9/wmE3v3Iu9K8= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 h1:HBSI2kDkMdWz4ZM7FjwE7e/pWDEZ+nR95x8Ztet1ooY= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13/go.mod h1:YE94ZoDArI7awZqJzBAZ3PDD2zSfuP7w6P2knOzIn8M= +github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk= +github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o= +github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13/go.mod h1:lmKuogqSU3HzQCwZ9ZtcqOc5XGMqtDK7OIc2+DxiUEg= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 h1:0JPwLz1J+5lEOfy/g0SURC9cxhbQ1lIMHMa+AHZSzz0= +github.com/aws/aws-sdk-go-v2/service/sso v1.30.1/go.mod h1:fKvyjJcz63iL/ftA6RaM8sRCtN4r4zl4tjL3qw5ec7k= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 h1:OWs0/j2UYR5LOGi88sD5/lhN6TDLG6SfA7CqsQO9zF0= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5/go.mod h1:klO+ejMvYsB4QATfEOIXk8WAEwN4N0aBfJpvC+5SZBo= +github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 h1:mLlUgHn02ue8whiR4BmxxGJLR2gwU6s6ZzJ5wDamBUs= +github.com/aws/aws-sdk-go-v2/service/sts v1.39.1/go.mod h1:E19xDjpzPZC7LS2knI9E6BaRFDK43Eul7vd6rSq2HWk= +github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM= +github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0= github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 h1:6df1vn4bBlDDo4tARvBm7l6KA9iVMnE3NWizDeWSrps= github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3/go.mod h1:CIWtjkly68+yqLPbvwwR/fjNJA/idrtULjZWh2v1ys0= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/dennwc/varint v1.0.0 h1:kGNFFSSw8ToIy3obO/kKr8U9GZYUAxQEVuix4zfDWzE= github.com/dennwc/varint v1.0.0/go.mod h1:hnItb35rvZvJrbTALZtY/iQfDs48JKRG1RPpgziApxA= -github.com/go-kit/log v0.2.1 h1:MRVx0/zhvdseW+Gza6N9rVzU/IVzaeE1SFI4raAhmBU= -github.com/go-kit/log v0.2.1/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0= -github.com/go-logfmt/logfmt v0.6.0 h1:wGYYu3uicYdqXVgoYbvnkrPVXkuLM1p1ifugDMEdRi4= -github.com/go-logfmt/logfmt v0.6.0/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs= -github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= -github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= -github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk= -github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk= -github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM= -github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= -github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= -github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo= +github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= +github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs= +github.com/golang/snappy v1.0.0/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0= +github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc h1:GN2Lv3MGO7AS6PrRoT6yV5+wkrOpcszoIsO4+4ds248= -github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk= -github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg= -github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo= +github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4= +github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA= +github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo= +github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc= +github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 h1:cLN4IBkmkYZNnk7EAJ0BHIethd+J6LqxFNw5mSiI2bM= +github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk= github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= -github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA= -github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw= +github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= +github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= @@ -49,59 +88,85 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8m github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f h1:KUppIJq7/+SVif2QVs3tOP0zanoHgBEVAwHxUSIzRqU= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4= -github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= +github.com/oklog/ulid/v2 v2.1.1 h1:suPZ4ARWLOJLegGFiZZ1dFAkqzhMjL3J1TzI+5wHz8s= +github.com/oklog/ulid/v2 v2.1.1/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ= github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ= github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.19.1 h1:wZWJDwK+NameRJuPGDhlnFgx8e8HN3XHQeLaYJFJBOE= -github.com/prometheus/client_golang v1.19.1/go.mod h1:mP78NwGzrVks5S2H6ab8+ZZGJLZUq1hoULYBAYBw1Ho= -github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= -github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= -github.com/prometheus/common v0.55.0 h1:KEi6DK7lXW/m7Ig5i47x0vRzuBsHuvJdi5ee6Y3G1dc= -github.com/prometheus/common v0.55.0/go.mod h1:2SECS4xJG1kd8XF9IcM1gMX6510RAEL65zxzNImwdc8= -github.com/prometheus/common/sigv4 v0.1.0 h1:qoVebwtwwEhS85Czm2dSROY5fTo2PAPEVdDeppTwGX4= -github.com/prometheus/common/sigv4 v0.1.0/go.mod h1:2Jkxxk9yYvCkE5G1sQT7GuEXm57JrvHu9k5YwTjsNtI= -github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= -github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= -github.com/prometheus/prometheus v0.54.1 h1:vKuwQNjnYN2/mDoWfHXDhAsz/68q/dQDb+YbcEqU7MQ= -github.com/prometheus/prometheus v0.54.1/go.mod h1:xlLByHhk2g3ycakQGrMaU8K7OySZx98BzeCR99991NY= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca h1:BOxmsLoL2ymn8lXJtorca7N/m+2vDQUDoEtPjf0iAxA= +github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca/go.mod h1:gndBHh3ZdjBozGcGrjUYjN3UJLRS3l2drALtu4lUt+k= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.67.4 h1:yR3NqWO1/UyO1w2PhUvXlGQs/PtFmoveVO0KZ4+Lvsc= +github.com/prometheus/common v0.67.4/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI= +github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEoIwkU+A6qos= +github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/prometheus/prometheus v0.308.1 h1:ApMNI/3/es3Ze90Z7CMb+wwU2BsSYur0m5VKeqHj7h4= +github.com/prometheus/prometheus v0.308.1/go.mod h1:aHjYCDz9zKRyoUXvMWvu13K9XHOkBB12XrEqibs3e0A= +github.com/prometheus/sigv4 v0.3.0 h1:QIG7nTbu0JTnNidGI1Uwl5AGVIChWUACxn2B/BQ1kms= +github.com/prometheus/sigv4 v0.3.0/go.mod h1:fKtFYDus2M43CWKMNtGvFNHGXnAJJEGZbiYCmVp/F8I= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= -github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= +go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg= +go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8= +go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM= +go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA= +go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI= +go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE= +go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs= go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -golang.org/x/crypto v0.25.0 h1:ypSNr+bnYL2YhwoMt2zPxHFmbAN1KZs/njMG3hxUp30= -golang.org/x/crypto v0.25.0/go.mod h1:T+wALwcMOSE0kXgUAnPAHqTLW+XHgcELELW8VaDgm/M= -golang.org/x/exp v0.0.0-20240119083558-1b970713d09a h1:Q8/wZp0KX97QFTc2ywcOE0YRjZPVIx+MXInMzdvQqcA= -golang.org/x/exp v0.0.0-20240119083558-1b970713d09a/go.mod h1:idGWGoKP1toJGkd5/ig9ZLuPcZBC3ewk7SzmH0uou08= -golang.org/x/net v0.27.0 h1:5K3Njcw06/l2y9vpGCSdcxWOYHOUk3dVNGDXN+FvAys= -golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE= -golang.org/x/oauth2 v0.21.0 h1:tsimM75w1tF/uws5rbeHzIWxEqElMehnc+iW793zsZs= -golang.org/x/oauth2 v0.21.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= -golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M= -golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI= -golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4= -golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI= -golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk= -golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= -google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg= -google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw= -go.yaml.in/yaml/v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= -go.yaml.in/yaml/v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= +go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= +golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04= +golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0= +golang.org/x/exp v0.0.0-20250808145144-a408d31f581a h1:Y+7uR/b1Mw2iSXZ3G//1haIiSElDQZ8KWh0h+sZPG90= +golang.org/x/exp v0.0.0-20250808145144-a408d31f581a/go.mod h1:rT6SFzZ7oxADUDx58pcaKFTcZ+inxAa9fTrYx/uVYwg= +golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4= +golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210= +golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY= +golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= +golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= +golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ= +golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k= +golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM= +golang.org/x/time v0.13.0 h1:eUlYslOIt32DgYD6utsuUeHs4d7AsEYLuIAdg7FlYgI= +golang.org/x/time v0.13.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +google.golang.org/api v0.252.0 h1:xfKJeAJaMwb8OC9fesr369rjciQ704AjU/psjkKURSI= +google.golang.org/api v0.252.0/go.mod h1:dnHOv81x5RAmumZ7BWLShB/u7JZNeyalImxHmtTHxqw= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797 h1:CirRxTOwnRWVLKzDNrs0CXAaVozJoR4G9xvdRecrdpk= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251002232023-7c0ddcbb5797/go.mod h1:HSkG/KdJWusxU1F6CNrwNDjBMgisKxGnc5dAZfT0mjQ= +google.golang.org/grpc v1.76.0 h1:UnVkv1+uMLYXoIz6o7chp59WfQUYA2ex/BXQ9rHZu7A= +google.golang.org/grpc v1.76.0/go.mod h1:Ju12QI8M6iQJtbcsV+awF5a4hfJMLi4X0JLo94ULZ6c= +google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= +google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -k8s.io/apimachinery v0.29.3 h1:2tbx+5L7RNvqJjn7RIuIKu9XTsIZ9Z5wX2G22XAa5EU= -k8s.io/apimachinery v0.29.3/go.mod h1:hx/S4V2PNW4OMg3WizRrHutyB5la0iCUbZym+W0EQIU= -k8s.io/client-go v0.29.3 h1:R/zaZbEAxqComZ9FHeQwOh3Y1ZUs7FaHKZdQtIc2WZg= -k8s.io/client-go v0.29.3/go.mod h1:tkDisCvgPfiRpxGnOORfkljmS+UrW+WtXAy2fTvXJB0= +k8s.io/apimachinery v0.34.1 h1:dTlxFls/eikpJxmAC7MVE8oOeP1zryV7iRyIjB0gky4= +k8s.io/apimachinery v0.34.1/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw= +k8s.io/client-go v0.34.1 h1:ZUPJKgXsnKwVwmKKdPfw4tB58+7/Ik3CrjOEhsiZ7mY= +k8s.io/client-go v0.34.1/go.mod h1:kA8v0FP+tk6sZA0yKLRG67LWjqufAoSHA2xVGKw9Of8= k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8= k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk= k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE= -k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI= -k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 h1:hwvWFiBzdWw1FhfY1FooPn3kzWuJ8tmbZBHi4zVsl1Y= +k8s.io/utils v0.0.0-20250604170112-4c0f3b243397/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= From b8b55d0f432a29aaf7c4b32385e243d1e0c5b016 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Thu, 18 Dec 2025 12:22:15 +0100 Subject: [PATCH 161/439] Add Go workspace for multi-module development. This allows nested modules to reference the root prometheus/prometheus module from the local filesystem instead of downloading versioned releases. Improves development workflow and ensures CI tests against current code. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- .gitignore | 1 + Makefile | 2 ++ go.work | 8 ++++++++ 3 files changed, 11 insertions(+) create mode 100644 go.work diff --git a/.gitignore b/.gitignore index 0d99305f69..f64f775993 100644 --- a/.gitignore +++ b/.gitignore @@ -26,6 +26,7 @@ npm_licenses.tar.bz2 /vendor /.build +/go.work.sum /**/node_modules diff --git a/Makefile b/Makefile index 834f0e3ce2..1611dacd6f 100644 --- a/Makefile +++ b/Makefile @@ -194,6 +194,8 @@ GO_SUBMODULE_DIRS := documentation/examples/remote_storage internal/tools web/ui .PHONY: update-all-go-deps update-all-go-deps: update-go-deps $(foreach dir,$(GO_SUBMODULE_DIRS),$(MAKE) update-go-deps-in-dir DIR=$(dir);) + @echo ">> syncing Go workspace" + @$(GO) work sync .PHONY: update-go-deps-in-dir update-go-deps-in-dir: diff --git a/go.work b/go.work new file mode 100644 index 0000000000..5ec4aeab50 --- /dev/null +++ b/go.work @@ -0,0 +1,8 @@ +go 1.24.9 + +use ( + . + ./documentation/examples/remote_storage + ./internal/tools + ./web/ui/mantine-ui/src/promql/tools +) From 030cb5e4ee0a0342ea013af85b4ec201d7d5f454 Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Thu, 18 Dec 2025 12:48:18 +0100 Subject: [PATCH 162/439] web/ui: Add make targets for PromQL function generation. Add make targets to generate and check PromQL function signatures and documentation for the Mantine UI. The generate-promql-functions target runs the Go generators and automatically lints the output files. The check-generated-promql-functions target verifies that generated files are up to date, similar to check-generated-parser. Fix the gen_functions_list generator to output properly formatted TypeScript code with correct indentation and semicolons. Add check-generated-promql-functions to the UI tests CI job to ensure generated files stay in sync with upstream changes. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- .github/workflows/ci.yml | 18 +++++++++--------- Makefile | 14 ++++++++++++++ .../promql/tools/gen_functions_list/main.go | 4 ++-- 3 files changed, 25 insertions(+), 11 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e4c2fbce18..1e1f7804dd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -202,20 +202,20 @@ jobs: if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }} run: exit 1 check_generated_parser: - name: Check generated parser + name: Check generated parser and functions runs-on: ubuntu-latest + container: + image: quay.io/prometheus/golang-builder:1.25-base steps: - - name: Checkout repository - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: persist-credentials: false - - name: Install Go - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0 + - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3 + - uses: ./.github/promci/actions/setup_environment with: - cache: false - go-version: 1.25.x - - name: Run goyacc and check for diff - run: make install-goyacc check-generated-parser + enable_npm: true + - run: make install-goyacc check-generated-parser + - run: make check-generated-promql-functions golangci: name: golangci-lint runs-on: ubuntu-latest diff --git a/Makefile b/Makefile index 1611dacd6f..197fd17c19 100644 --- a/Makefile +++ b/Makefile @@ -79,6 +79,20 @@ ui-lint: # new Mantine-based UI is fully integrated and the old app can be removed. cd $(UI_PATH)/react-app && npm run lint +.PHONY: generate-promql-functions +generate-promql-functions: ui-install + @echo ">> generating PromQL function signatures" + @cd $(UI_PATH)/mantine-ui/src/promql/tools && $(GO) run ./gen_functions_list > ../functionSignatures.ts + @echo ">> generating PromQL function documentation" + @cd $(UI_PATH)/mantine-ui/src/promql/tools && $(GO) run ./gen_functions_docs $(CURDIR)/docs/querying/functions.md > ../functionDocs.tsx + @echo ">> formatting generated files" + @cd $(UI_PATH)/mantine-ui && npx prettier --write --print-width 120 src/promql/functionSignatures.ts src/promql/functionDocs.tsx + +.PHONY: check-generated-promql-functions +check-generated-promql-functions: generate-promql-functions + @echo ">> checking generated PromQL functions" + @git diff --exit-code -- $(UI_PATH)/mantine-ui/src/promql/functionSignatures.ts $(UI_PATH)/mantine-ui/src/promql/functionDocs.tsx || (echo "Generated PromQL function files are out of date. Please run 'make generate-promql-functions' and commit the changes." && false) + .PHONY: assets ifndef SKIP_UI_BUILD assets: check-node-version ui-install ui-build diff --git a/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go b/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go index f479b6d36a..8713772dfe 100644 --- a/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go +++ b/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go @@ -41,10 +41,10 @@ func main() { sort.Strings(fnNames) fmt.Println(`import { valueType, Func } from './ast'; - export const functionSignatures: Record = {`) +export const functionSignatures: Record = {`) for _, fnName := range fnNames { fn := parser.Functions[fnName] fmt.Printf(" %s: { name: '%s', argTypes: [%s], variadic: %d, returnType: %s },\n", fn.Name, fn.Name, formatValueTypes(fn.ArgTypes), fn.Variadic, formatValueType(fn.ReturnType)) } - fmt.Println("}") + fmt.Println("};") } From 6613c09ad743f68c5f936d631d8386e469dede6b Mon Sep 17 00:00:00 2001 From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Date: Thu, 18 Dec 2025 12:48:32 +0100 Subject: [PATCH 163/439] web/ui: Regenerate PromQL function files. Update generated files with latest functions from Prometheus, adding support for first_over_time, info, ts_of_first_over_time, ts_of_last_over_time, ts_of_max_over_time, and ts_of_min_over_time. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> --- .github/workflows/ci.yml | 3 +- web/ui/mantine-ui/src/promql/functionDocs.tsx | 2892 ++++++++++++----- .../src/promql/functionSignatures.ts | 214 +- .../promql/tools/gen_functions_docs/main.go | 44 +- 4 files changed, 2260 insertions(+), 893 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1e1f7804dd..f0195f02d5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -202,7 +202,8 @@ jobs: if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }} run: exit 1 check_generated_parser: - name: Check generated parser and functions + # Checks generated parser and UI functions list. Not renaming as it is a required check. + name: Check generated parser runs-on: ubuntu-latest container: image: quay.io/prometheus/golang-builder:1.25-base diff --git a/web/ui/mantine-ui/src/promql/functionDocs.tsx b/web/ui/mantine-ui/src/promql/functionDocs.tsx index 91221666d7..a9d9ca53a9 100644 --- a/web/ui/mantine-ui/src/promql/functionDocs.tsx +++ b/web/ui/mantine-ui/src/promql/functionDocs.tsx @@ -1,28 +1,31 @@ -import React from 'react'; +import React from "react"; const funcDocs: Record = { abs: ( <>

- abs(v instant-vector) returns the input vector with all sample values converted to their absolute value. + abs(v instant-vector) returns a vector containing all float samples in the input vector converted + to their absolute value. Histogram samples in the input vector are ignored silently.

), absent: ( <>

- absent(v instant-vector) returns an empty vector if the vector passed to it has any elements (floats or - native histograms) and a 1-element vector with the value 1 if the vector passed to it has no elements. + absent(v instant-vector) returns an empty vector if the vector passed to it has any elements (float + samples or histogram samples) and a 1-element vector with the value 1 if the vector passed to it has no + elements.

This is useful for alerting on when no time series exist for a given metric name and label combination.

         
-          absent(nonexistent{'{'}job="myjob"{'}'}) # => {'{'}job="myjob"{'}'}
-          absent(nonexistent{'{'}job="myjob",instance=~".*"{'}'}) # => {'{'}job="myjob"{'}'}
-          absent(sum(nonexistent{'{'}job="myjob"{'}'})) # => {'{'}
-          {'}'}
+          absent(nonexistent{"{"}job="myjob"{"}"}) # => {"{"}job="myjob"{"}"}
+          absent(nonexistent{"{"}job="myjob",instance=~".*"{"}"}) # => {"{"}job="myjob"
+          {"}"}
+          absent(sum(nonexistent{"{"}job="myjob"{"}"})) # => {"{"}
+          {"}"}
@@ -36,83 +39,83 @@ const funcDocs: Record = { <>

absent_over_time(v range-vector) returns an empty vector if the range vector passed to it has any - elements (floats or native histograms) and a 1-element vector with the value 1 if the range vector passed to it has - no elements. + elements (float samples or histogram samples) and a 1-element vector with the value 1 if the range vector passed + to it has no elements.

- This is useful for alerting on when no time series exist for a given metric name and label combination for a certain - amount of time. + This is useful for alerting on when no time series exist for a given metric name and label combination for a + certain amount of time. 

         
-          absent_over_time(nonexistent{'{'}job="myjob"{'}'}[1h]) # => {'{'}job="myjob"{'}'}
-          absent_over_time(nonexistent{'{'}job="myjob",instance=~".*"{'}'}[1h]) # => {'{'}
-          job="myjob"{'}'}
-          absent_over_time(sum(nonexistent{'{'}job="myjob"{'}'})[1h:]) # => {'{'}
-          {'}'}
+          absent_over_time(nonexistent{"{"}job="myjob"{"}"}[1h]) # => {"{"}job="myjob"{"}"}
+          absent_over_time(nonexistent{"{"}job="myjob",instance=~".*"{"}"}[1h]) # => {"{"}
+          job="myjob"{"}"}
+          absent_over_time(sum(nonexistent{"{"}job="myjob"{"}"})[1h:]) # => {"{"}
+          {"}"}

- In the first two examples, absent_over_time() tries to be smart about deriving labels of the 1-element - output vector from the input vector. + In the first two examples, absent_over_time() tries to be smart about deriving labels of the + 1-element output vector from the input vector.

), acos: ( <> -

The trigonometric functions work in radians:

+

The trigonometric functions work in radians. They ignore histogram samples in the input vector.

  • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
  • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
  • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
  • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
  • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
  • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
  • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
  • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
  • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
  • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
  • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
  • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
@@ -120,69 +123,69 @@ const funcDocs: Record = {
  • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
  • pi(): returns pi.
  • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
), acosh: ( <> -

The trigonometric functions work in radians:

+

The trigonometric functions work in radians. They ignore histogram samples in the input vector.

  • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
  • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
  • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
  • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
  • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
  • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
  • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
  • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
  • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
  • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
  • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
  • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
@@ -190,69 +193,69 @@ const funcDocs: Record = {
  • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
  • pi(): returns pi.
  • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
), asin: ( <> -

The trigonometric functions work in radians:

+

The trigonometric functions work in radians. They ignore histogram samples in the input vector.

  • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
  • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
  • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
  • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
  • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
  • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
  • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
  • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
  • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
  • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
  • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
  • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
@@ -260,69 +263,69 @@ const funcDocs: Record = {
  • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
  • pi(): returns pi.
  • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
), asinh: ( <> -

The trigonometric functions work in radians:

+

The trigonometric functions work in radians. They ignore histogram samples in the input vector.

  • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
  • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
  • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
  • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
  • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
  • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
  • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
  • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
  • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
  • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
  • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
  • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
@@ -330,69 +333,69 @@ const funcDocs: Record = {
  • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
  • pi(): returns pi.
  • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
), atan: ( <> -

The trigonometric functions work in radians:

+

The trigonometric functions work in radians. They ignore histogram samples in the input vector.

  • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
  • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
  • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
  • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
  • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
  • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
  • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
  • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
  • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
  • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
  • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
  • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
@@ -400,69 +403,69 @@ const funcDocs: Record = {
  • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
  • pi(): returns pi.
  • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
), atanh: ( <> -

The trigonometric functions work in radians:

+

The trigonometric functions work in radians. They ignore histogram samples in the input vector.

  • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
  • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
  • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
  • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
  • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
  • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
  • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
  • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
  • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
  • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
  • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
  • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
@@ -470,13 +473,13 @@ const funcDocs: Record = {
  • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
  • pi(): returns pi.
  • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
@@ -484,40 +487,42 @@ const funcDocs: Record = { avg_over_time: ( <>

- The following functions allow aggregating each series of a given range vector over time and return an instant vector - with per-series aggregation results: + The following functions allow aggregating each series of a given range vector over time and return an instant + vector with per-series aggregation results:

  • - avg_over_time(range-vector): the average value of all points in the specified interval. + avg_over_time(range-vector): the average value of all float or histogram samples in the specified + interval (see details below).
  • - min_over_time(range-vector): the minimum value of all points in the specified interval. + min_over_time(range-vector): the minimum value of all float samples in the specified interval.
  • - max_over_time(range-vector): the maximum value of all points in the specified interval. + max_over_time(range-vector): the maximum value of all float samples in the specified interval.
  • - sum_over_time(range-vector): the sum of all values in the specified interval. + sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval + (see details below).
  • - count_over_time(range-vector): the count of all values in the specified interval. + count_over_time(range-vector): the count of all samples in the specified interval.
  • - quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified - interval. + quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the + specified interval.
  • - stddev_over_time(range-vector): the population standard deviation of the values in the specified - interval. + stddev_over_time(range-vector): the population standard deviation of all float samples in the + specified interval.
  • - stdvar_over_time(range-vector): the population standard variance of the values in the specified - interval. + stdvar_over_time(range-vector): the population standard variance of all float samples in the + specified interval.
  • - last_over_time(range-vector): the most recent point value in the specified interval. + last_over_time(range-vector): the most recent sample in the specified interval.
  • present_over_time(range-vector): the value 1 for any series in the specified interval. @@ -526,32 +531,75 @@ const funcDocs: Record = {

    If the feature flag - --enable-feature=promql-experimental-functions is set, the following additional functions are available: + --enable-feature=promql-experimental-functions is set, the following additional functions are + available:

    • - mad_over_time(range-vector): the median absolute deviation of all points in the specified interval. + mad_over_time(range-vector): the median absolute deviation of all float samples in the specified + interval. +
      • +
    • + ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum + value of all float samples in the specified interval. +
      • +
    • + ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum + value of all float samples in the specified interval. +
      • +
    • + ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval. +
      • +
    • + first_over_time(range-vector): the oldest sample in the specified interval. +
      • +
    • + ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.

    - Note that all values in the specified interval have the same weight in the aggregation even if the values are not - equally spaced throughout the interval. + Note that all values in the specified interval have the same weight in the aggregation even if the values are + not equally spaced throughout the interval.

    +

    These functions act on histograms in the following way:

    + +
      +
    • + count_over_time, first_over_time, last_over_time, and + present_over_time() act on float and histogram samples in the same way. +
      • +
    • + avg_over_time() and sum_over_time() act on histogram samples in a way that + corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram + samples within the range, the corresponding result is removed entirely from the output vector. Such a removal + is flagged by a warn-level annotation. +
      • +
    • + All other functions ignore histogram samples in the following way: Input ranges containing only histogram + samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the + float samples are processed and the omission of the histogram samples is flagged by an info-level annotation. +
      • +
    +

    - avg_over_time, sum_over_time, count_over_time, last_over_time, - and - present_over_time handle native histograms as expected. All other functions ignore histogram samples. + first_over_time(m[1m]) differs from m offset 1m in that the former will select the + first sample of m within the 1m range, where m offset 1m will select the most + recent sample within the lookback interval outside and prior to the 1m offset. This is particularly + useful with first_over_time(m[step()]) + in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the + sample selected is within the range step.

    ), ceil: ( <>

    - ceil(v instant-vector) rounds the sample values of all elements in v up to the nearest - integer value greater than or equal to v. + ceil(v instant-vector) returns a vector containing all float samples in the input vector rounded up + to the nearest integer value greater than or equal to their original value. Histogram samples in the input + vector are ignored silently.

      @@ -573,17 +621,19 @@ const funcDocs: Record = { changes: ( <>

      - For each input time series, changes(v range-vector) returns the number of times its value has changed - within the provided time range as an instant vector. + For each input time series, changes(v range-vector) returns the number of times its value has + changed within the provided time range as an instant vector. A float sample followed by a histogram sample, or + vice versa, counts as a change. A counter histogram sample followed by a gauge histogram sample with otherwise + exactly the same values, or vice versa, does not count as a change.

      ), clamp: ( <>

      - clamp(v instant-vector, min scalar, max scalar) - clamps the sample values of all elements in v to have a lower limit of min and an upper - limit of max. + clamp(v instant-vector, min scalar, max scalar) clamps the values of all float samples in{" "} + v to have a lower limit of min and an upper limit of + max. Histogram samples in the input vector are ignored silently.

      Special cases:

      @@ -593,7 +643,7 @@ const funcDocs: Record = { Return an empty vector if min > max
    • - Return NaN if min or max is NaN + Float samples are clamped to NaN if min or max is NaN
    @@ -601,71 +651,71 @@ const funcDocs: Record = { clamp_max: ( <>

    - clamp_max(v instant-vector, max scalar) clamps the sample values of all elements in v to - have an upper limit of max. + clamp_max(v instant-vector, max scalar) clamps the values of all float samples in v to + have an upper limit of max. Histogram samples in the input vector are ignored silently.

    ), clamp_min: ( <>

    - clamp_min(v instant-vector, min scalar) clamps the sample values of all elements in v to - have a lower limit of min. + clamp_min(v instant-vector, min scalar) clamps the values of all float samples in v to + have a lower limit of min. Histogram samples in the input vector are ignored silently.

    ), cos: ( <> -

    The trigonometric functions work in radians:

    +

    The trigonometric functions work in radians. They ignore histogram samples in the input vector.

    • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
    • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
    • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
    • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
    • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
    • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
    • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
    • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
    • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
    • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
    • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
    • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
    @@ -673,69 +723,69 @@ const funcDocs: Record = {
    • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
    • pi(): returns pi.
    • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
    ), cosh: ( <> -

    The trigonometric functions work in radians:

    +

    The trigonometric functions work in radians. They ignore histogram samples in the input vector.

    • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
    • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
    • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
    • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
    • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
    • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
    • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
    • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
    • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
    • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
    • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
    • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
    @@ -743,13 +793,13 @@ const funcDocs: Record = {
    • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
    • pi(): returns pi.
    • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
    @@ -757,40 +807,42 @@ const funcDocs: Record = { count_over_time: ( <>

    - The following functions allow aggregating each series of a given range vector over time and return an instant vector - with per-series aggregation results: + The following functions allow aggregating each series of a given range vector over time and return an instant + vector with per-series aggregation results:

    • - avg_over_time(range-vector): the average value of all points in the specified interval. + avg_over_time(range-vector): the average value of all float or histogram samples in the specified + interval (see details below).
    • - min_over_time(range-vector): the minimum value of all points in the specified interval. + min_over_time(range-vector): the minimum value of all float samples in the specified interval.
    • - max_over_time(range-vector): the maximum value of all points in the specified interval. + max_over_time(range-vector): the maximum value of all float samples in the specified interval.
    • - sum_over_time(range-vector): the sum of all values in the specified interval. + sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval + (see details below).
    • - count_over_time(range-vector): the count of all values in the specified interval. + count_over_time(range-vector): the count of all samples in the specified interval.
    • - quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified - interval. + quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the + specified interval.
    • - stddev_over_time(range-vector): the population standard deviation of the values in the specified - interval. + stddev_over_time(range-vector): the population standard deviation of all float samples in the + specified interval.
    • - stdvar_over_time(range-vector): the population standard variance of the values in the specified - interval. + stdvar_over_time(range-vector): the population standard variance of all float samples in the + specified interval.
    • - last_over_time(range-vector): the most recent point value in the specified interval. + last_over_time(range-vector): the most recent sample in the specified interval.
    • present_over_time(range-vector): the value 1 for any series in the specified interval. @@ -799,111 +851,161 @@ const funcDocs: Record = {

      If the feature flag - --enable-feature=promql-experimental-functions is set, the following additional functions are available: + --enable-feature=promql-experimental-functions is set, the following additional functions are + available:

      • - mad_over_time(range-vector): the median absolute deviation of all points in the specified interval. + mad_over_time(range-vector): the median absolute deviation of all float samples in the specified + interval. +
        • +
      • + ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum + value of all float samples in the specified interval. +
        • +
      • + ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum + value of all float samples in the specified interval. +
        • +
      • + ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval. +
        • +
      • + first_over_time(range-vector): the oldest sample in the specified interval. +
        • +
      • + ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.

      - Note that all values in the specified interval have the same weight in the aggregation even if the values are not - equally spaced throughout the interval. + Note that all values in the specified interval have the same weight in the aggregation even if the values are + not equally spaced throughout the interval.

      +

      These functions act on histograms in the following way:

      + +
        +
      • + count_over_time, first_over_time, last_over_time, and + present_over_time() act on float and histogram samples in the same way. +
        • +
      • + avg_over_time() and sum_over_time() act on histogram samples in a way that + corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram + samples within the range, the corresponding result is removed entirely from the output vector. Such a removal + is flagged by a warn-level annotation. +
        • +
      • + All other functions ignore histogram samples in the following way: Input ranges containing only histogram + samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the + float samples are processed and the omission of the histogram samples is flagged by an info-level annotation. +
        • +
      +

      - avg_over_time, sum_over_time, count_over_time, last_over_time, - and - present_over_time handle native histograms as expected. All other functions ignore histogram samples. + first_over_time(m[1m]) differs from m offset 1m in that the former will select the + first sample of m within the 1m range, where m offset 1m will select the most + recent sample within the lookback interval outside and prior to the 1m offset. This is particularly + useful with first_over_time(m[step()]) + in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the + sample selected is within the range step.

      ), day_of_month: ( <>

      - day_of_month(v=vector(time()) instant-vector) returns the day of the month for each of the given times - in UTC. Returned values are from 1 to 31. + day_of_month(v=vector(time()) instant-vector) interprets float samples in + v as timestamps (number of seconds since January 1, 1970 UTC) and returns the day of the month (in + UTC) for each of those timestamps. Returned values are from 1 to 31. Histogram samples in the input vector are + ignored silently.

      ), day_of_week: ( <>

      - day_of_week(v=vector(time()) instant-vector) returns the day of the week for each of the given times in - UTC. Returned values are from 0 to 6, where 0 means Sunday etc. + day_of_week(v=vector(time()) instant-vector) interprets float samples in v + as timestamps (number of seconds since January 1, 1970 UTC) and returns the day of the week (in UTC) for each of + those timestamps. Returned values are from 0 to 6, where 0 means Sunday etc. Histogram samples in the input + vector are ignored silently.

      ), day_of_year: ( <>

      - day_of_year(v=vector(time()) instant-vector) returns the day of the year for each of the given times in - UTC. Returned values are from 1 to 365 for non-leap years, and 1 to 366 in leap years. + day_of_year(v=vector(time()) instant-vector) interprets float samples in v + as timestamps (number of seconds since January 1, 1970 UTC) and returns the day of the year (in UTC) for each of + those timestamps. Returned values are from 1 to 365 for non-leap years, and 1 to 366 in leap years. Histogram + samples in the input vector are ignored silently.

      ), days_in_month: ( <>

      - days_in_month(v=vector(time()) instant-vector) returns number of days in the month for each of the given - times in UTC. Returned values are from 28 to 31. + days_in_month(v=vector(time()) instant-vector) interprets float samples in + v as timestamps (number of seconds since January 1, 1970 UTC) and returns the number of days in the + month of each of those timestamps (in UTC). Returned values are from 28 to 31. Histogram samples in the input + vector are ignored silently.

      ), deg: ( <> -

      The trigonometric functions work in radians:

      +

      The trigonometric functions work in radians. They ignore histogram samples in the input vector.

      • - acos(v instant-vector): calculates the arccosine of all elements in v ( + acos(v instant-vector): calculates the arccosine of all float samples in v ( special cases).
      • - acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v ( - special cases). + acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "} + v (special cases).
      • - asin(v instant-vector): calculates the arcsine of all elements in v ( + asin(v instant-vector): calculates the arcsine of all float samples in v ( special cases).
      • - asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v ( - special cases). + asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "} + v (special cases).
      • - atan(v instant-vector): calculates the arctangent of all elements in v ( + atan(v instant-vector): calculates the arctangent of all float samples in v ( special cases).
      • - atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v ( - special cases). + atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "} + v (special cases).
      • - cos(v instant-vector): calculates the cosine of all elements in v ( + cos(v instant-vector): calculates the cosine of all float samples in v ( special cases).
      • - cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v ( + cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v ( special cases).
      • - sin(v instant-vector): calculates the sine of all elements in v ( + sin(v instant-vector): calculates the sine of all float samples in v ( special cases).
      • - sinh(v instant-vector): calculates the hyperbolic sine of all elements in v ( + sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v ( special cases).
      • - tan(v instant-vector): calculates the tangent of all elements in v ( + tan(v instant-vector): calculates the tangent of all float samples in v ( special cases).
      • - tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v ( - special cases). + tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "} + (special cases).
      @@ -911,13 +1013,13 @@ const funcDocs: Record = {
      • - deg(v instant-vector): converts radians to degrees for all elements in v. + deg(v instant-vector): converts radians to degrees for all float samples in v.
      • pi(): returns pi.
      • - rad(v instant-vector): converts degrees to radians for all elements in v. + rad(v instant-vector): converts degrees to radians for all float samples in v.
      @@ -925,52 +1027,86 @@ const funcDocs: Record = { delta: ( <>

      - delta(v range-vector) calculates the difference between the first and last value of each time series - element in a range vector v, returning an instant vector with the given deltas and equivalent labels. - The delta is extrapolated to cover the full time range as specified in the range vector selector, so that it is - possible to get a non-integer result even if the sample values are all integers. + delta(v range-vector) calculates the difference between the first and last value of each time + series element in a range vector v, returning an instant vector with the given deltas and + equivalent labels. The delta is extrapolated to cover the full time range as specified in the range vector + selector, so that it is possible to get a non-integer result even if the sample values are all integers.

      The following example expression returns the difference in CPU temperature between now and 2 hours ago:

               
-          delta(cpu_temp_celsius{'{'}host="zeus"{'}'}[2h])
+          delta(cpu_temp_celsius{"{"}host="zeus"{"}"}[2h])

      - delta acts on native histograms by calculating a new histogram where each component (sum and count of - observations, buckets) is the difference between the respective component in the first and last native histogram in - v. However, each element in v that contains a mix of float and native histogram samples - within the range, will be missing from the result vector. + delta acts on histogram samples by calculating a new histogram where each component (sum and count + of observations, buckets) is the difference between the respective component in the first and last native + histogram in v. However, each element in v that contains a mix of float samples and + histogram samples within the range will be omitted from the result vector, flagged by a warn-level annotation.

      - delta should only be used with gauges and native histograms where the components behave like gauges - (so-called gauge histograms). + delta should only be used with gauges (for both floats and histograms).

      ), deriv: ( <>

      - deriv(v range-vector) calculates the per-second derivative of the time series in a range vector{' '} - v, using simple linear regression. - The range vector must have at least two samples in order to perform the calculation. When +Inf or + deriv(v range-vector) calculates the per-second derivative of each float time series in the range + vector v, using{" "} + simple linear regression. The range vector + must have at least two float samples in order to perform the calculation. When +Inf or{" "} -Inf are found in the range vector, the slope and offset value calculated will be NaN.

      - deriv should only be used with gauges. + deriv should only be used with gauges and only works for float samples. Elements in the range + vector that contain only histogram samples are ignored entirely. For elements that contain a mix of float and + histogram samples, only the float samples are used as input, which is flagged by an info-level annotation. +

      + + ), + double_exponential_smoothing: ( + <> +

      + + This function has to be enabled via the{" "} + feature flag + --enable-feature=promql-experimental-functions. + +

      + +

      + double_exponential_smoothing(v range-vector, sf scalar, tf scalar) produces a smoothed value for + each float time series in the range in v. The lower the smoothing factor sf, the more + importance is given to old data. The higher the trend factor tf, the more trends in the data is + considered. Both sf and + tf must be between 0 and 1. For additional details, refer to{" "} + + NIST Engineering Statistics Handbook + + . In Prometheus V2 this function was called holt_winters. This caused confusion since the + Holt-Winters method usually refers to triple exponential smoothing. Double exponential smoothing as implemented + here is also referred to as “Holt Linear”. +

      + +

      + double_exponential_smoothing should only be used with gauges and only works for float samples. + Elements in the range vector that contain only histogram samples are ignored entirely. For elements that contain + a mix of float and histogram samples, only the float samples are used as input, which is flagged by an + info-level annotation.

      ), exp: ( <>

      - exp(v instant-vector) calculates the exponential function for all elements in v. Special - cases are: + exp(v instant-vector) calculates the exponential function for all float samples in v. + Histogram samples are ignored silently. Special cases are:

        @@ -983,11 +1119,122 @@ const funcDocs: Record = {
      ), + first_over_time: ( + <> +

      + The following functions allow aggregating each series of a given range vector over time and return an instant + vector with per-series aggregation results: +

      + +
        +
      • + avg_over_time(range-vector): the average value of all float or histogram samples in the specified + interval (see details below). +
        • +
      • + min_over_time(range-vector): the minimum value of all float samples in the specified interval. +
        • +
      • + max_over_time(range-vector): the maximum value of all float samples in the specified interval. +
        • +
      • + sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval + (see details below). +
        • +
      • + count_over_time(range-vector): the count of all samples in the specified interval. +
        • +
      • + quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the + specified interval. +
        • +
      • + stddev_over_time(range-vector): the population standard deviation of all float samples in the + specified interval. +
        • +
      • + stdvar_over_time(range-vector): the population standard variance of all float samples in the + specified interval. +
        • +
      • + last_over_time(range-vector): the most recent sample in the specified interval. +
        • +
      • + present_over_time(range-vector): the value 1 for any series in the specified interval. +
        • +
      + +

      + If the feature flag + --enable-feature=promql-experimental-functions is set, the following additional functions are + available: +

      + +
        +
      • + mad_over_time(range-vector): the median absolute deviation of all float samples in the specified + interval. +
        • +
      • + ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum + value of all float samples in the specified interval. +
        • +
      • + ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum + value of all float samples in the specified interval. +
        • +
      • + ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval. +
        • +
      • + first_over_time(range-vector): the oldest sample in the specified interval. +
        • +
      • + ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval. +
        • +
      + +

      + Note that all values in the specified interval have the same weight in the aggregation even if the values are + not equally spaced throughout the interval. +

      + +

      These functions act on histograms in the following way:

      + +
        +
      • + count_over_time, first_over_time, last_over_time, and + present_over_time() act on float and histogram samples in the same way. +
        • +
      • + avg_over_time() and sum_over_time() act on histogram samples in a way that + corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram + samples within the range, the corresponding result is removed entirely from the output vector. Such a removal + is flagged by a warn-level annotation. +
        • +
      • + All other functions ignore histogram samples in the following way: Input ranges containing only histogram + samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the + float samples are processed and the omission of the histogram samples is flagged by an info-level annotation. +
        • +
      + +

      + first_over_time(m[1m]) differs from m offset 1m in that the former will select the + first sample of m within the 1m range, where m offset 1m will select the most + recent sample within the lookback interval outside and prior to the 1m offset. This is particularly + useful with first_over_time(m[step()]) + in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the + sample selected is within the range step. +

      + + ), floor: ( <>

      - floor(v instant-vector) rounds the sample values of all elements in v down to the nearest - integer value smaller than or equal to v. + floor(v instant-vector) returns a vector containing all float samples in the input vector rounded + down to the nearest integer value smaller than or equal to their original value. Histogram samples in the input + vector are ignored silently.

        @@ -1009,19 +1256,13 @@ const funcDocs: Record = { histogram_avg: ( <>

        - - This function only acts on native histograms. - + histogram_avg(v instant-vector) returns the arithmetic average of observed values stored in each + histogram sample in v. Float samples are ignored and do not show up in the returned vector.

        - histogram_avg(v instant-vector) returns the arithmetic average of observed values stored in a native - histogram. Samples that are not native histograms are ignored and do not show up in the returned vector. -

        - -

        - Use histogram_avg as demonstrated below to compute the average request duration over a 5-minute window - from a native histogram: + Use histogram_avg as demonstrated below to compute the average request duration over a 5-minute + window from a native histogram: 

        @@ -1032,32 +1273,28 @@ const funcDocs: Record = {
 
       
                 
-          {' '}
-          histogram_sum(rate(http_request_duration_seconds[5m])) / histogram_count(rate(http_request_duration_seconds[5m]))
+          {" "}
+          histogram_sum(rate(http_request_duration_seconds[5m])) /
+          histogram_count(rate(http_request_duration_seconds[5m]))
         
       
        
     
   ),
-  'histogram_count()` and `histogram_sum': (
+  histogram_count: (
     <>
       
        
-        
-          Both functions only act on native histograms.
-        
+        histogram_count(v instant-vector) returns the count of observations stored in each histogram sample
+        in v. Float samples are ignored and do not show up in the returned vector.
       
        
 
       
        
-        histogram_count(v instant-vector) returns the count of observations stored in a native histogram.
-        Samples that are not native histograms are ignored and do not show up in the returned vector.
+        Similarly, histogram_sum(v instant-vector) returns the sum of observations stored in each histogram
+        sample.
       
        
 
       
        
-        Similarly, histogram_sum(v instant-vector) returns the sum of observations stored in a native histogram.
-      
        
-
-      
        
-        Use histogram_count in the following way to calculate a rate of observations (in this case corresponding
-        to “requests per second”) from a native histogram:
+        Use histogram_count in the following way to calculate a rate of observations (in this case
+        corresponding to “requests per second”) from a series of histogram samples:
       
        
 
       
        @@ -1068,20 +1305,28 @@ const funcDocs: Record = {
   histogram_fraction: (
     <>
       
        
-        
-          This function only acts on native histograms.
-        
+        histogram_fraction(lower scalar, upper scalar, b instant-vector) returns the estimated fraction of
+        observations between the provided lower and upper values for each classic or native histogram contained in{" "}
+        b. Float samples in b are considered the counts of observations in each bucket of one
+        or more classic histograms, while native histogram samples in b are treated each individually as a
+        separate histogram. This works in the same way as for histogram_quantile(). (See there for more
+        details.)
       
        
 
       
        
-        For a native histogram, histogram_fraction(lower scalar, upper scalar, v instant-vector) returns the
-        estimated fraction of observations between the provided lower and upper values. Samples that are not native
-        histograms are ignored and do not show up in the returned vector.
+        If the provided lower and upper values do not coincide with bucket boundaries, the calculated fraction is an
+        estimate, using the same interpolation method as for
+        histogram_quantile(). (See there for more details.) Especially with classic histograms, it is easy
+        to accidentally pick lower or upper values that are very far away from any bucket boundary, leading to large
+        margins of error. Rather than using histogram_fraction() with classic histograms, it is often a
+        more robust approach to directly act on the bucket series when calculating fractions. See the
+        calculation of the Apdex score
+        as a typical example.
       
        
 
       
        
-        For example, the following expression calculates the fraction of HTTP requests over the last hour that took 200ms or
-        less:
+        For example, the following expression calculates the fraction of HTTP requests over the last hour that took
+        200ms or less:
       
        
 
       
        @@ -1089,48 +1334,56 @@ const funcDocs: Record = {
       
        
 
       
        
-        The error of the estimation depends on the resolution of the underlying native histogram and how closely the provided
-        boundaries are aligned with the bucket boundaries in the histogram.
+        The error of the estimation depends on the resolution of the underlying native histogram and how closely the
+        provided boundaries are aligned with the bucket boundaries in the histogram.
       
        
 
       
        
-        +Inf and -Inf are valid boundary values. For example, if the histogram in the expression
-        above included negative observations (which shouldn’t be the case for request durations), the appropriate lower
-        boundary to include all observations less than or equal 0.2 would be -Inf rather than 0.
+        +Inf and -Inf are valid boundary values. For example, if the histogram in the
+        expression above included negative observations (which shouldn’t be the case for request durations), the
+        appropriate lower boundary to include all observations less than or equal 0.2 would be -Inf rather
+        than 0.
       
        
 
       
        
-        Whether the provided boundaries are inclusive or exclusive is only relevant if the provided boundaries are precisely
-        aligned with bucket boundaries in the underlying native histogram. In this case, the behavior depends on the schema
-        definition of the histogram. The currently supported schemas all feature inclusive upper boundaries and exclusive
-        lower boundaries for positive values (and vice versa for negative values). Without a precise alignment of boundaries,
-        the function uses linear interpolation to estimate the fraction. With the resulting uncertainty, it becomes
-        irrelevant if the boundaries are inclusive or exclusive.
+        Whether the provided boundaries are inclusive or exclusive is only relevant if the provided boundaries are
+        precisely aligned with bucket boundaries in the underlying native histogram. In this case, the behavior depends
+        on the schema definition of the histogram. (The usual standard exponential schemas all feature inclusive upper
+        boundaries and exclusive lower boundaries for positive values, and vice versa for negative values.) Without a
+        precise alignment of boundaries, the function uses interpolation to estimate the fraction. With the resulting
+        uncertainty, it becomes irrelevant if the boundaries are inclusive or exclusive.
+      
        
+
+      
        
+        Special case for native histograms with standard exponential buckets:
+        NaN observations are considered outside of any buckets in this case.
+        histogram_fraction(-Inf, +Inf, b) effectively returns the fraction of non-NaN{" "}
+        observations and may therefore be less than 1.
       
        
     
   ),
   histogram_quantile: (
     <>
       
        
-        histogram_quantile(φ scalar, b instant-vector) calculates the φ-quantile (0 ≤ φ ≤ 1) from a{' '}
+        histogram_quantile(φ scalar, b instant-vector) calculates the φ-quantile (0 ≤ φ ≤ 1) from a{" "}
         classic histogram or from a native
-        histogram. (See histograms and summaries for a detailed
-        explanation of φ-quantiles and the usage of the (classic) histogram metric type in general.)
+        histogram. (See histograms and summaries for a
+        detailed explanation of φ-quantiles and the usage of the (classic) histogram metric type in general.)
       
        
 
       
        
-        The float samples in b are considered the counts of observations in each bucket of one or more classic
-        histograms. Each float sample must have a label
-        le where the label value denotes the inclusive upper bound of the bucket. (Float samples without such a
-        label are silently ignored.) The other labels and the metric name are used to identify the buckets belonging to each
-        classic histogram. The{' '}
+        The float samples in b are considered the counts of observations in each bucket of one or more
+        classic histograms. Each float sample must have a label
+        le where the label value denotes the inclusive upper bound of the bucket. (Float samples without
+        such a label are silently ignored.) The other labels and the metric name are used to identify the buckets
+        belonging to each classic histogram. The{" "}
         histogram metric type
         automatically provides time series with the _bucket suffix and the appropriate labels.
       
        
 
       
        
-        The native histogram samples in b are treated each individually as a separate histogram to calculate the
-        quantile from.
+        The (native) histogram samples in b are treated each individually as a separate histogram to
+        calculate the quantile from.
       
        
 
       
        
@@ -1142,10 +1395,10 @@ const funcDocs: Record = {
       
        
 
       
        
-        Example: A histogram metric is called http_request_duration_seconds (and therefore the metric name for
-        the buckets of a classic histogram is
-        http_request_duration_seconds_bucket). To calculate the 90th percentile of request durations over the
-        last 10m, use the following expression in case
+        Example: A histogram metric is called http_request_duration_seconds (and therefore the metric name
+        for the buckets of a classic histogram is
+        http_request_duration_seconds_bucket). To calculate the 90th percentile of request durations over
+        the last 10m, use the following expression in case
         http_request_duration_seconds is a classic histogram:
       
        
 
@@ -1161,9 +1414,9 @@ const funcDocs: Record = {
 
       
        
         The quantile is calculated for each label combination in
-        http_request_duration_seconds. To aggregate, use the sum() aggregator around the{' '}
+        http_request_duration_seconds. To aggregate, use the sum() aggregator around the{" "}
         rate() function. Since the le label is required by
-        histogram_quantile() to deal with classic histograms, it has to be included in the by{' '}
+        histogram_quantile() to deal with classic histograms, it has to be included in the by{" "}
         clause. The following expression aggregates the 90th percentile by job for classic histograms:
       
        
 
@@ -1194,23 +1447,67 @@ const funcDocs: Record = {
       
        
 
       
        
-        The histogram_quantile() function interpolates quantile values by assuming a linear distribution within
-        a bucket.
+        In the (common) case that a quantile value does not coincide with a bucket boundary, the{" "}
+        histogram_quantile() function interpolates the quantile value within the bucket the quantile value
+        falls into. For classic histograms, for native histograms with custom bucket boundaries, and for the zero bucket
+        of other native histograms, it assumes a uniform distribution of observations within the bucket (also called{" "}
+        linear interpolation). For the non-zero-buckets of native histograms with a standard exponential
+        bucketing schema, the interpolation is done under the assumption that the samples within the bucket are
+        distributed in a way that they would uniformly populate the buckets in a hypothetical histogram with higher
+        resolution. (This is also called exponential interpolation. See the{" "}
+        
+          native histogram specification
+        
+        for more details.)
       
        
 
       
        
-        If b has 0 observations, NaN is returned. For φ < 0, -Inf is returned. For
-        φ > 1, +Inf is returned. For φ = NaN, NaN is returned.
+        If b has 0 observations, NaN is returned. For φ < 0, -Inf is returned.
+        For φ > 1, +Inf is returned. For φ = NaN, NaN is returned.
       
        
 
-      
        
-        The following is only relevant for classic histograms: If b contains fewer than two buckets,{' '}
-        NaN is returned. The highest bucket must have an upper bound of +Inf. (Otherwise,{' '}
-        NaN is returned.) If a quantile is located in the highest bucket, the upper bound of the second highest
-        bucket is returned. A lower limit of the lowest bucket is assumed to be 0 if the upper bound of that bucket is
-        greater than 0. In that case, the usual linear interpolation is applied within that bucket. Otherwise, the upper
-        bound of the lowest bucket is returned for quantiles located in the lowest bucket.
-      
        
+      
        Special cases for classic histograms:
        
+
+      
          
+        
        • 
+          If b contains fewer than two buckets, NaN is returned.
+        
          • 
+        
        • 
+          The highest bucket must have an upper bound of +Inf. (Otherwise, NaN is returned.)
+        
          • 
+        
        • 
+          If a quantile is located in the highest bucket, the upper bound of the second highest bucket is returned.
+        
          • 
+        
        • 
+          The lower limit of the lowest bucket is assumed to be 0 if the upper bound of that bucket is greater than 0.
+          In that case, the usual linear interpolation is applied within that bucket. Otherwise, the upper bound of the
+          lowest bucket is returned for quantiles located in the lowest bucket.
+        
          • 
+      
        
+
+      
        Special cases for native histograms:
        
+
+      
          
+        
        • 
+          If a native histogram with standard exponential buckets has NaN
+          observations and the quantile falls into one of the existing exponential buckets, the result is skewed towards
+          higher values due to NaN
+          observations treated as +Inf. This is flagged with an info level annotation.
+        
          • 
+        
        • 
+          If a native histogram with standard exponential buckets has NaN
+          observations and the quantile falls above all of the existing exponential buckets, NaN is
+          returned. This is flagged with an info level annotation.
+        
          • 
+        
        • 
+          A zero bucket with finite width is assumed to contain no negative observations if the histogram has
+          observations in positive buckets, but none in negative buckets.
+        
          • 
+        
        • 
+          A zero bucket with finite width is assumed to contain no positive observations if the histogram has
+          observations in negative buckets, but none in positive buckets.
+        
          • 
+      
        
 
       
        
         You can use histogram_quantile(0, v instant-vector) to get the estimated minimum value stored in a
@@ -1227,78 +1524,100 @@ const funcDocs: Record = {
       
          
         
        • The counts in the buckets are monotonically increasing (strictly non-decreasing).
          • 
         
        • 
-          A lack of observations between the upper limits of two consecutive buckets results in equal counts in those two
-          buckets.
+          A lack of observations between the upper limits of two consecutive buckets results in equal counts in those
+          two buckets.
         
          • 
       
        
 
       
        
-        However, floating point precision issues (e.g. small discrepancies introduced by computing of buckets with{' '}
-        sum(rate(...))) or invalid data might violate these assumptions. In that case,
-        histogram_quantile would be unable to return meaningful results. To mitigate the issue,
+        However, floating point precision issues (e.g. small discrepancies introduced by computing of buckets with{" "}
+        sum(rate(...))) or invalid data might violate these assumptions. In that case,{" "}
+        histogram_quantile would be unable to return meaningful results. To mitigate the issue,{" "}
         histogram_quantile assumes that tiny relative differences between consecutive buckets are happening
         because of floating point precision errors and ignores them. (The threshold to ignore a difference between two
-        buckets is a trillionth (1e-12) of the sum of both buckets.) Furthermore, if there are non-monotonic bucket counts
-        even after this adjustment, they are increased to the value of the previous buckets to enforce monotonicity. The
-        latter is evidence for an actual issue with the input data and is therefore flagged with an informational annotation
-        reading input to histogram_quantile needed to be fixed for monotonicity. If you encounter this
-        annotation, you should find and remove the source of the invalid data.
+        buckets is a trillionth (1e-12) of the sum of both buckets.) Furthermore, if there are non-monotonic bucket
+        counts even after this adjustment, they are increased to the value of the previous buckets to enforce
+        monotonicity. The latter is evidence for an actual issue with the input data and is therefore flagged by an
+        info-level annotation reading input to histogram_quantile needed to be fixed for monotonicity. If
+        you encounter this annotation, you should find and remove the source of the invalid data.
       
        
     
   ),
-  'histogram_stddev()` and `histogram_stdvar': (
+  histogram_stddev: (
     <>
       
        
-        
-          Both functions only act on native histograms.
-        
+        histogram_stddev(v instant-vector) returns the estimated standard deviation of observations for
+        each histogram sample in v. For this estimation, all observations in a bucket are assumed to have
+        the value of the mean of the bucket boundaries. For the zero bucket and for buckets with custom boundaries, the
+        arithmetic mean is used. For the usual exponential buckets, the geometric mean is used. Float samples are
+        ignored and do not show up in the returned vector.
       
        
 
       
        
-        
-        histogram_stddev(v instant-vector) returns the estimated standard deviation of observations in a native
-        histogram. For this estimation, all observations in a bucket are assumed to have the value of the mean of the bucket boundaries.
-        For the zero bucket and for buckets with custom boundaries, the arithmetic mean is used. For the usual exponential buckets,
-        the geometric mean is used. Samples that are not native histograms are ignored and do not show up in the returned vector.
-      
        
-
-      
        
-        Similarly, histogram_stdvar(v instant-vector) returns the estimated standard variance of observations in
-        a native histogram.
+        Similarly, histogram_stdvar(v instant-vector) returns the estimated standard variance of
+        observations for each histogram sample in v.
       
        
     
   ),
-  double_exponential_smoothing: (
+  histogram_stdvar: (
     <>
       
        
-        double_exponential_smoothing(v range-vector, sf scalar, tf scalar) produces a smoothed value for time series based on
-        the range in v. The lower the smoothing factor sf, the more importance is given to old
-        data. The higher the trend factor tf, the more trends in the data is considered. Both sf{' '}
-        and tf must be between 0 and 1.
+        histogram_stddev(v instant-vector) returns the estimated standard deviation of observations for
+        each histogram sample in v. For this estimation, all observations in a bucket are assumed to have
+        the value of the mean of the bucket boundaries. For the zero bucket and for buckets with custom boundaries, the
+        arithmetic mean is used. For the usual exponential buckets, the geometric mean is used. Float samples are
+        ignored and do not show up in the returned vector.
       
        
 
       
        
-        double_exponential_smoothing should only be used with gauges.
+        Similarly, histogram_stdvar(v instant-vector) returns the estimated standard variance of
+        observations for each histogram sample in v.
       
        
     
   ),
+  histogram_sum: (
+    <>
+      
        
+        histogram_count(v instant-vector) returns the count of observations stored in each histogram sample
+        in v. Float samples are ignored and do not show up in the returned vector.
+      
        
+
+      
        
+        Similarly, histogram_sum(v instant-vector) returns the sum of observations stored in each histogram
+        sample.
+      
        
+
+      
        
+        Use histogram_count in the following way to calculate a rate of observations (in this case
+        corresponding to “requests per second”) from a series of histogram samples:
+      
        
+
+      
        +        histogram_count(rate(http_request_duration_seconds[10m]))
+      
        
+    
+  ),
   hour: (
     <>
       
        
-        hour(v=vector(time()) instant-vector) returns the hour of the day for each of the given times in UTC.
-        Returned values are from 0 to 23.
+        hour(v=vector(time()) instant-vector) interprets float samples in v as timestamps
+        (number of seconds since January 1, 1970 UTC) and returns the hour of the day (in UTC) for each of those
+        timestamps. Returned values are from 0 to 23. Histogram samples in the input vector are ignored silently.
       
        
     
   ),
   idelta: (
     <>
       
        
-        idelta(v range-vector) calculates the difference between the last two samples in the range vector{' '}
-        v, returning an instant vector with the given deltas and equivalent labels.
+        idelta(v range-vector) calculates the difference between the last two samples in the range vector{" "}
+        v, returning an instant vector with the given deltas and equivalent labels. Both samples must be
+        either float samples or histogram samples. Elements in v where one of the last two samples is a
+        float sample and the other is a histogram sample will be omitted from the result vector, flagged by a warn-level
+        annotation.
       
        
 
       
        
-        idelta should only be used with gauges.
+        idelta should only be used with gauges (for both floats and histograms).
       
        
     
   ),
@@ -1307,79 +1626,208 @@ const funcDocs: Record = {
       
        
         increase(v range-vector) calculates the increase in the time series in the range vector. Breaks in
         monotonicity (such as counter resets due to target restarts) are automatically adjusted for. The increase is
-        extrapolated to cover the full time range as specified in the range vector selector, so that it is possible to get a
-        non-integer result even if a counter increases only by integer increments.
+        extrapolated to cover the full time range as specified in the range vector selector, so that it is possible to
+        get a non-integer result even if a counter increases only by integer increments.
       
        
 
       
        
-        The following example expression returns the number of HTTP requests as measured over the last 5 minutes, per time
-        series in the range vector:
+        The following example expression returns the number of HTTP requests as measured over the last 5 minutes, per
+        time series in the range vector:
       
        
 
       
                 
-          increase(http_requests_total{'{'}job="api-server"{'}'}[5m])
+          increase(http_requests_total{"{"}job="api-server"{"}"}[5m])
         
       
        
 
       
        
-        increase acts on native histograms by calculating a new histogram where each component (sum and count of
-        observations, buckets) is the increase between the respective component in the first and last native histogram in
-        v. However, each element in v that contains a mix of float and native histogram samples
-        within the range, will be missing from the result vector.
+        increase acts on histogram samples by calculating a new histogram where each component (sum and
+        count of observations, buckets) is the increase between the respective component in the first and last native
+        histogram in v. However, each element in v that contains a mix of float samples and
+        histogram samples within the range, will be omitted from the result vector, flagged by a warn-level annotation.
       
        
 
       
        
-        increase should only be used with counters and native histograms where the components behave like
-        counters. It is syntactic sugar for rate(v) multiplied by the number of seconds under the specified time
-        range window, and should be used primarily for human readability. Use rate in recording rules so that
-        increases are tracked consistently on a per-second basis.
+        increase should only be used with counters (for both floats and histograms). It is syntactic sugar
+        for rate(v) multiplied by the number of seconds under the specified time range window, and should
+        be used primarily for human readability. Use rate in recording rules so that increases are tracked
+        consistently on a per-second basis.
+      
        
+    
+  ),
+  info: (
+    <>
+      
        
+        _The info function is an experiment to improve UX around including labels from{" "}
+        
+          info metrics
+        
+        . The behavior of this function may change in future versions of Prometheus, including its removal from PromQL.{" "}
+        info has to be enabled via the
+        feature flag{" "}
+        --enable-feature=promql-experimental-functions._
+      
        
+
+      
        
+        info(v instant-vector, [data-label-selector instant-vector]) finds, for each time series in{" "}
+        v, all info series with matching identifying labels (more on this later), and adds the
+        union of their data (i.e., non-identifying) labels to the time series. The second argument{" "}
+        data-label-selector is optional. It is not a real instant vector, but uses a subset of its syntax.
+        It must start and end with curly braces (
+        
+          {"{"} ... {"}"}
+        
+        ) and may only contain label matchers. The label matchers are used to constrain which info series to consider
+        and which data labels to add to v.
+      
        
+
+      
        
+        Identifying labels of an info series are the subset of labels that uniquely identify the info series. The
+        remaining labels are considered
+        data labels (also called non-identifying). (Note that Prometheus’s concept of time series
+        identity always includes all the labels. For the sake of the info
+        function, we “logically” define info series identity in a different way than in the conventional Prometheus
+        view.) The identifying labels of an info series are used to join it to regular (non-info) series, i.e. those
+        series that have the same labels as the identifying labels of the info series. The data labels, which are the
+        ones added to the regular series by the info function, effectively encode metadata key value pairs.
+        (This implies that a change in the data labels in the conventional Prometheus view constitutes the end of one
+        info series and the beginning of a new info series, while the “logical” view of the info function
+        is that the same info series continues to exist, just with different “data”.)
+      
        
+
+      
        
+        The conventional approach of adding data labels is sometimes called a “join query”, as illustrated by the
+        following example:
+      
        
+
+      
        +        
+          {" "}
+          rate(http_server_request_duration_seconds_count[2m]) * on (job, instance) group_left (k8s_cluster_name)
+          target_info
+        
+      
        
+
+      
        
+        The core of the query is the expression rate(http_server_request_duration_seconds_count[2m]). But
+        to add data labels from an info metric, the user has to use elaborate (and not very obvious) syntax to specify
+        which info metric to use (target_info), what the identifying labels are (
+        on (job, instance)), and which data labels to add (group_left (k8s_cluster_name)).
+      
        
+
+      
        
+        This query is not only verbose and hard to write, it might also run into an “identity crisis”: If any of the
+        data labels of target_info changes, Prometheus sees that as a change of series (as alluded to
+        above, Prometheus just has no native concept of non-identifying labels). If the old target_info{" "}
+        series is not properly marked as stale (which can happen with certain ingestion paths), the query above will
+        fail for up to 5m (the lookback delta) because it will find a conflicting match with both the old and the new
+        version of target_info.
+      
        
+
+      
        
+        The info function not only resolves this conflict in favor of the newer series, it also simplifies
+        the syntax because it knows about the available info series and what their identifying labels are. The example
+        query looks like this with the info function:
+      
        
+
+      
        +        
+          info( rate(http_server_request_duration_seconds_count[2m]),
+          {"{"}k8s_cluster_name=~".+"{"}"})
+        
+      
        
+
+      
        
+        The common case of adding all data labels can be achieved by omitting the 2nd argument of the{" "}
+        info function entirely, simplifying the example even more:
+      
        
+
+      
        +        info(rate(http_server_request_duration_seconds_count[2m]))
+      
        
+
+      
        
+        While info normally automatically finds all matching info series, it’s possible to restrict
+        them by providing a __name__ label matcher, e.g.
+        
+          {"{"}__name__="target_info"{"}"}
+        
+        .
+      
        
+
+      
        Limitations
        
+
+      
        
+        In its current iteration, info defaults to considering only info series with the name{" "}
+        target_info. It also assumes that the identifying info series labels are
+        instance and job. info does support other info series names however,
+        through
+        __name__ label matchers. E.g., one can explicitly say to consider both
+        target_info and build_info as follows:
+        
+          {"{"}__name__=~"(target|build)_info"{"}"}
+        
+        . However, the identifying labels always have to be instance and job.
+      
        
+
+      
        
+        These limitations are partially defeating the purpose of the info function. At the current stage,
+        this is an experiment to find out how useful the approach turns out to be in practice. A final version of the{" "}
+        info function will indeed consider all matching info series and with their appropriate identifying
+        labels.
       
        
     
   ),
   irate: (
     <>
       
        
-        irate(v range-vector) calculates the per-second instant rate of increase of the time series in the range
-        vector. This is based on the last two data points. Breaks in monotonicity (such as counter resets due to target
-        restarts) are automatically adjusted for.
+        irate(v range-vector) calculates the per-second instant rate of increase of the time series in the
+        range vector. This is based on the last two data points. Breaks in monotonicity (such as counter resets due to
+        target restarts) are automatically adjusted for. Both samples must be either float samples or histogram samples.
+        Elements in v where one of the last two samples is a float sample and the other is a histogram
+        sample will be omitted from the result vector, flagged by a warn-level annotation.
       
        
 
       
        
-        The following example expression returns the per-second rate of HTTP requests looking up to 5 minutes back for the
-        two most recent data points, per time series in the range vector:
+        irate should only be used with counters (for both floats and histograms).
+      
        
+
+      
        
+        The following example expression returns the per-second rate of HTTP requests looking up to 5 minutes back for
+        the two most recent data points, per time series in the range vector:
       
        
 
       
                 
-          irate(http_requests_total{'{'}job="api-server"{'}'}[5m])
+          irate(http_requests_total{"{"}job="api-server"{"}"}[5m])
         
       
        
 
       
        
-        irate should only be used when graphing volatile, fast-moving counters. Use rate for alerts
-        and slow-moving counters, as brief changes in the rate can reset the FOR clause and graphs consisting
-        entirely of rare spikes are hard to read.
+        irate should only be used when graphing volatile, fast-moving counters. Use rate for
+        alerts and slow-moving counters, as brief changes in the rate can reset the FOR clause and graphs
+        consisting entirely of rare spikes are hard to read.
       
        
 
       
        
         Note that when combining irate() with an
         aggregation operator (e.g. sum()) or a function
-        aggregating over time (any function ending in _over_time), always take a irate() first,
-        then aggregate. Otherwise irate() cannot detect counter resets when your target restarts.
+        aggregating over time (any function ending in _over_time), always take an irate(){" "}
+        first, then aggregate. Otherwise irate() cannot detect counter resets when your target restarts.
       
        
     
   ),
   label_join: (
     <>
       
        
-        For each timeseries in v,{' '}
+        For each timeseries in v,{" "}
         
           label_join(v instant-vector, dst_label string, separator string, src_label_1 string, src_label_2 string, ...)
-        {' '}
+        {" "}
         joins all the values of all the src_labels
-        using separator and returns the timeseries with the label dst_label containing the joined
-        value. There can be any number of src_labels in this function.
+        using separator and returns the timeseries with the label dst_label containing the
+        joined value. There can be any number of src_labels in this function.
       
        
 
       
        
@@ -1387,13 +1835,13 @@ const funcDocs: Record = {
       
        
 
       
        
-        This example will return a vector with each time series having a foo label with the value{' '}
+        This example will return a vector with each time series having a foo label with the value{" "}
         a,b,c added to it:
       
        
 
       
                 
-          label_join(up{'{'}job="api-server",src1="a",src2="b",src3="c"{'}'},
+          label_join(up{"{"}job="api-server",src1="a",src2="b",src3="c"{"}"},
           "foo", ",", "src1", "src2", "src3")
         
       
        
@@ -1402,14 +1850,17 @@ const funcDocs: Record = {
   label_replace: (
     <>
       
        
-        For each timeseries in v,{' '}
-        label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string)
-        matches the regular expression regex against the
+        For each timeseries in v,{" "}
+        
+          label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string)
+        
+        matches the regular expression regex against the
         value of the label src_label. If it matches, the value of the label dst_label in the
         returned timeseries will be the expansion of replacement, together with the original labels in the
-        input. Capturing groups in the regular expression can be referenced with $1, $2, etc. Named
-        capturing groups in the regular expression can be referenced with $name (where name is the
-        capturing group name). If the regular expression doesn’t match then the timeseries is returned unchanged.
+        input. Capturing groups in the regular expression can be referenced with $1, $2, etc.
+        Named capturing groups in the regular expression can be referenced with $name (where{" "}
+        name is the capturing group name). If the regular expression doesn’t match then the
+        timeseries is returned unchanged.
       
        
 
       
        
@@ -1417,23 +1868,25 @@ const funcDocs: Record = {
       
        
 
       
        
-        This example will return timeseries with the values a:c at label service and a{' '}
-        at label foo:
+        This example will return timeseries with the values a:c at label service and{" "}
+        a at label foo:
       
        
 
       
                 
-          label_replace(up{'{'}job="api-server",service="a:c"{'}'}, "foo", "$1",
+          label_replace(up{"{"}job="api-server",service="a:c"{"}"}, "foo", "$1",
           "service", "(.*):.*")
         
       
        
 
-      
        This second example has the same effect than the first example, and illustrates use of named capturing groups:
        
+      
        
+        This second example has the same effect than the first example, and illustrates use of named capturing groups:
+      
        
 
       
                 
-          label_replace(up{'{'}job="api-server",service="a:c"{'}'}, "foo", "$name",
-          "service", "(?P<name>.*):(?P<version>.*)")
+          label_replace(up{"{"}job="api-server",service="a:c"{"}"}, "foo",
+          "$name", "service", "(?P<name>.*):(?P<version>.*)")
         
       
        
     
@@ -1441,40 +1894,42 @@ const funcDocs: Record = {
   last_over_time: (
     <>
       
        
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
        
 
       
          
         
        • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
          • 
         
        • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
          • 
         
        • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
          • 
         
        • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
          • 
         
        • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
          • 
         
        • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
          • 
         
        • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
          • 
         
        • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
          • 
         
        • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
          • 
         
        • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -1483,32 +1938,74 @@ const funcDocs: Record = {
 
       
          
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
          
 
       
            
         
          • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
            • 
+        
          • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
            • 
+        
          • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
            • 
+        
          • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
            • 
+        
          • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
            • 
+        
          • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
            • 
       
          
 
       
          
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
          
 
+      
          These functions act on histograms in the following way:
          
+
+      
            
+        
          • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
            • 
+        
          • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
            • 
+        
          • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
            • 
+      
          
+
       
          
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
          
     
   ),
   ln: (
     <>
       
          
-        ln(v instant-vector) calculates the natural logarithm for all elements in v. Special cases
-        are:
+        ln(v instant-vector) calculates the natural logarithm for all float samples in v.
+        Histogram samples in the input vector are ignored silently. Special cases are:
       
          
 
       
            
@@ -1530,56 +2027,60 @@ const funcDocs: Record = {
   log10: (
     <>
       
            
-        log10(v instant-vector) calculates the decimal logarithm for all elements in v. The special
-        cases are equivalent to those in ln.
+        log10(v instant-vector) calculates the decimal logarithm for all float samples in v.
+        Histogram samples in the input vector are ignored silently. The special cases are equivalent to those in{" "}
+        ln.
       
            
     
   ),
   log2: (
     <>
       
            
-        log2(v instant-vector) calculates the binary logarithm for all elements in v. The special
-        cases are equivalent to those in ln.
+        log2(v instant-vector) calculates the binary logarithm for all float samples in v.
+        Histogram samples in the input vector are ignored silently. The special cases are equivalent to those in{" "}
+        ln.
       
            
     
   ),
   mad_over_time: (
     <>
       
            
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
            
 
       
              
         
            • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
              • 
         
            • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
              • 
         
            • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
              • 
         
            • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
              • 
         
            • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
              • 
         
            • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
              • 
         
            • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
              • 
         
            • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
              • 
         
            • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
              • 
         
            • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -1588,64 +2089,108 @@ const funcDocs: Record = {
 
       
              
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
              
 
       
                
         
              • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                • 
+        
              • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                • 
+        
              • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                • 
+        
              • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                • 
+        
              • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                • 
+        
              • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                • 
       
              
 
       
              
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
              
 
+      
              These functions act on histograms in the following way:
              
+
+      
                
+        
              • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                • 
+        
              • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                • 
+        
              • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                • 
+      
              
+
       
              
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
              
     
   ),
   max_over_time: (
     <>
       
              
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
              
 
       
                
         
              • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                • 
         
              • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                • 
         
              • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                • 
         
              • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                • 
         
              • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                • 
         
              • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                • 
         
              • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                • 
         
              • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                • 
         
              • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                • 
         
              • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -1654,64 +2199,108 @@ const funcDocs: Record = {
 
       
                
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                
 
       
                  
         
                • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                  • 
+        
                • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                  • 
+        
                • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                  • 
+        
                • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                  • 
+        
                • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                  • 
+        
                • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                  • 
       
                
 
       
                
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                
 
+      
                These functions act on histograms in the following way:
                
+
+      
                  
+        
                • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                  • 
+        
                • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                  • 
+        
                • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                  • 
+      
                
+
       
                
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                
     
   ),
   min_over_time: (
     <>
       
                
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
                
 
       
                  
         
                • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                  • 
         
                • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                  • 
         
                • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                  • 
         
                • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                  • 
         
                • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                  • 
         
                • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                  • 
         
                • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                  • 
         
                • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                  • 
         
                • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                  • 
         
                • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -1720,95 +2309,140 @@ const funcDocs: Record = {
 
       
                  
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                  
 
       
                    
         
                  • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                    • 
+        
                  • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                    • 
+        
                  • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                    • 
+        
                  • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                    • 
+        
                  • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                    • 
+        
                  • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                    • 
       
                  
 
       
                  
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                  
 
+      
                  These functions act on histograms in the following way:
                  
+
+      
                    
+        
                  • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                    • 
+        
                  • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                    • 
+        
                  • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                    • 
+      
                  
+
       
                  
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                  
     
   ),
   minute: (
     <>
       
                  
-        minute(v=vector(time()) instant-vector) returns the minute of the hour for each of the given times in
-        UTC. Returned values are from 0 to 59.
+        minute(v=vector(time()) instant-vector) interprets float samples in v as timestamps
+        (number of seconds since January 1, 1970 UTC) and returns the minute of the hour (in UTC) for each of those
+        timestamps. Returned values are from 0 to 59. Histogram samples in the input vector are ignored silently.
       
                  
     
   ),
   month: (
     <>
       
                  
-        month(v=vector(time()) instant-vector) returns the month of the year for each of the given times in UTC.
-        Returned values are from 1 to 12, where 1 means January etc.
+        month(v=vector(time()) instant-vector) interprets float samples in v as timestamps
+        (number of seconds since January 1, 1970 UTC) and returns the month of the year (in UTC) for each of those
+        timestamps. Returned values are from 1 to 12, where 1 means January etc. Histogram samples in the input vector
+        are ignored silently.
       
                  
     
   ),
   pi: (
     <>
-      
                  The trigonometric functions work in radians:
                  
+      
                  The trigonometric functions work in radians. They ignore histogram samples in the input vector.
                  
 
       
                    
         
                  • 
-          acos(v instant-vector): calculates the arccosine of all elements in v (
+          acos(v instant-vector): calculates the arccosine of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v (
-          special cases).
+          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "}
+          v (special cases).
         
                    • 
         
                  • 
-          asin(v instant-vector): calculates the arcsine of all elements in v (
+          asin(v instant-vector): calculates the arcsine of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v (
-          special cases).
+          asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "}
+          v (special cases).
         
                    • 
         
                  • 
-          atan(v instant-vector): calculates the arctangent of all elements in v (
+          atan(v instant-vector): calculates the arctangent of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v (
-          special cases).
+          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "}
+          v (special cases).
         
                    • 
         
                  • 
-          cos(v instant-vector): calculates the cosine of all elements in v (
+          cos(v instant-vector): calculates the cosine of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v (
+          cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          sin(v instant-vector): calculates the sine of all elements in v (
+          sin(v instant-vector): calculates the sine of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          sinh(v instant-vector): calculates the hyperbolic sine of all elements in v (
+          sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          tan(v instant-vector): calculates the tangent of all elements in v (
+          tan(v instant-vector): calculates the tangent of all float samples in v (
           special cases).
         
                    • 
         
                  • 
-          tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v (
-          special cases).
+          tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "}
+          (special cases).
         
                    • 
       
                  
 
@@ -1816,13 +2450,13 @@ const funcDocs: Record = {
 
       
                    
         
                  • 
-          deg(v instant-vector): converts radians to degrees for all elements in v.
+          deg(v instant-vector): converts radians to degrees for all float samples in v.
         
                    • 
         
                  • 
           pi(): returns pi.
         
                    • 
         
                  • 
-          rad(v instant-vector): converts degrees to radians for all elements in v.
+          rad(v instant-vector): converts degrees to radians for all float samples in v.
         
                    • 
       
                  
     
@@ -1831,54 +2465,58 @@ const funcDocs: Record = {
     <>
       
                  
         predict_linear(v range-vector, t scalar) predicts the value of time series
-        t seconds from now, based on the range vector v, using{' '}
-        simple linear regression. The range vector must
-        have at least two samples in order to perform the calculation. When +Inf or -Inf are found
-        in the range vector, the slope and offset value calculated will be NaN.
+        t seconds from now, based on the range vector v, using{" "}
+        simple linear regression. The range vector
+        must have at least two float samples in order to perform the calculation. When +Inf or{" "}
+        -Inf are found in the range vector, the predicted value will be NaN.
       
                  
 
       
                  
-        predict_linear should only be used with gauges.
+        predict_linear should only be used with gauges and only works for float samples. Elements in the
+        range vector that contain only histogram samples are ignored entirely. For elements that contain a mix of float
+        and histogram samples, only the float samples are used as input, which is flagged by an info-level annotation.
       
                  
     
   ),
   present_over_time: (
     <>
       
                  
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
                  
 
       
                    
         
                  • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                    • 
         
                  • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                    • 
         
                  • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                    • 
         
                  • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                    • 
         
                  • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                    • 
         
                  • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                    • 
         
                  • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                    • 
         
                  • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                    • 
         
                  • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                    • 
         
                  • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -1887,64 +2525,108 @@ const funcDocs: Record = {
 
       
                    
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                    
 
       
                      
         
                    • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                      • 
+        
                    • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                      • 
+        
                    • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                      • 
+        
                    • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                      • 
+        
                    • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                      • 
+        
                    • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                      • 
       
                    
 
       
                    
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                    
 
+      
                    These functions act on histograms in the following way:
                    
+
+      
                      
+        
                    • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                      • 
+        
                    • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                      • 
+        
                    • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                      • 
+      
                    
+
       
                    
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                    
     
   ),
   quantile_over_time: (
     <>
       
                    
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
                    
 
       
                      
         
                    • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                      • 
         
                    • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                      • 
         
                    • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                      • 
         
                    • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                      • 
         
                    • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                      • 
         
                    • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                      • 
         
                    • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                      • 
         
                    • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                      • 
         
                    • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                      • 
         
                    • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -1953,79 +2635,121 @@ const funcDocs: Record = {
 
       
                      
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                      
 
       
                        
         
                      • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                        • 
+        
                      • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                        • 
+        
                      • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                        • 
+        
                      • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                        • 
+        
                      • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                        • 
+        
                      • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                        • 
       
                      
 
       
                      
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                      
 
+      
                      These functions act on histograms in the following way:
                      
+
+      
                        
+        
                      • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                        • 
+        
                      • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                        • 
+        
                      • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                        • 
+      
                      
+
       
                      
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                      
     
   ),
   rad: (
     <>
-      
                      The trigonometric functions work in radians:
                      
+      
                      The trigonometric functions work in radians. They ignore histogram samples in the input vector.
                      
 
       
                        
         
                      • 
-          acos(v instant-vector): calculates the arccosine of all elements in v (
+          acos(v instant-vector): calculates the arccosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v (
-          special cases).
+          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          asin(v instant-vector): calculates the arcsine of all elements in v (
+          asin(v instant-vector): calculates the arcsine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v (
-          special cases).
+          asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          atan(v instant-vector): calculates the arctangent of all elements in v (
+          atan(v instant-vector): calculates the arctangent of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v (
-          special cases).
+          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          cos(v instant-vector): calculates the cosine of all elements in v (
+          cos(v instant-vector): calculates the cosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v (
+          cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          sin(v instant-vector): calculates the sine of all elements in v (
+          sin(v instant-vector): calculates the sine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          sinh(v instant-vector): calculates the hyperbolic sine of all elements in v (
+          sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          tan(v instant-vector): calculates the tangent of all elements in v (
+          tan(v instant-vector): calculates the tangent of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v (
-          special cases).
+          tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "}
+          (special cases).
         
                        • 
       
                      
 
@@ -2033,13 +2757,13 @@ const funcDocs: Record = {
 
       
                        
         
                      • 
-          deg(v instant-vector): converts radians to degrees for all elements in v.
+          deg(v instant-vector): converts radians to degrees for all float samples in v.
         
                        • 
         
                      • 
           pi(): returns pi.
         
                        • 
         
                      • 
-          rad(v instant-vector): converts degrees to radians for all elements in v.
+          rad(v instant-vector): converts degrees to radians for all float samples in v.
         
                        • 
       
                      
     
@@ -2047,40 +2771,40 @@ const funcDocs: Record = {
   rate: (
     <>
       
                      
-        rate(v range-vector) calculates the per-second average rate of increase of the time series in the range
-        vector. Breaks in monotonicity (such as counter resets due to target restarts) are automatically adjusted for. Also,
-        the calculation extrapolates to the ends of the time range, allowing for missed scrapes or imperfect alignment of
-        scrape cycles with the range’s time period.
+        rate(v range-vector) calculates the per-second average rate of increase of the time series in the
+        range vector. Breaks in monotonicity (such as counter resets due to target restarts) are automatically adjusted
+        for. Also, the calculation extrapolates to the ends of the time range, allowing for missed scrapes or imperfect
+        alignment of scrape cycles with the range’s time period.
       
                      
 
       
                      
-        The following example expression returns the per-second rate of HTTP requests as measured over the last 5 minutes,
+        The following example expression returns the per-second average rate of HTTP requests over the last 5 minutes,
         per time series in the range vector:
       
                      
 
       
                               
-          rate(http_requests_total{'{'}job="api-server"{'}'}[5m])
+          rate(http_requests_total{"{"}job="api-server"{"}"}[5m])
         
       
                      
 
       
                      
-        rate acts on native histograms by calculating a new histogram where each component (sum and count of
-        observations, buckets) is the rate of increase between the respective component in the first and last native
-        histogram in
-        v. However, each element in v that contains a mix of float and native histogram samples
-        within the range, will be missing from the result vector.
+        rate acts on native histograms by calculating a new histogram where each component (sum and count
+        of observations, buckets) is the rate of increase between the respective component in the first and last native
+        histogram in v. However, each element in v that contains a mix of float and native
+        histogram samples within the range, will be omitted from the result vector, flagged by a warn-level annotation.
       
                      
 
       
                      
-        rate should only be used with counters and native histograms where the components behave like counters.
-        It is best suited for alerting, and for graphing of slow-moving counters.
+        rate should only be used with counters (for both floats and histograms). It is best suited for
+        alerting, and for graphing of slow-moving counters.
       
                      
 
       
                      
-        Note that when combining rate() with an aggregation operator (e.g. sum()) or a function
-        aggregating over time (any function ending in _over_time), always take a rate() first, then
-        aggregate. Otherwise rate() cannot detect counter resets when your target restarts.
+        Note that when combining rate() with an aggregation operator (e.g. sum()) or a
+        function aggregating over time (any function ending in _over_time), always take a{" "}
+        rate() first, then aggregate. Otherwise rate() cannot detect counter resets when your
+        target restarts.
       
                      
     
   ),
@@ -2089,102 +2813,104 @@ const funcDocs: Record = {
       
                      
         For each input time series, resets(v range-vector) returns the number of counter resets within the
         provided time range as an instant vector. Any decrease in the value between two consecutive float samples is
-        interpreted as a counter reset. A reset in a native histogram is detected in a more complex way: Any decrease in any
-        bucket, including the zero bucket, or in the count of observation constitutes a counter reset, but also the
-        disappearance of any previously populated bucket, an increase in bucket resolution, or a decrease of the zero-bucket
-        width.
+        interpreted as a counter reset. A reset in a native histogram is detected in a more complex way: Any decrease in
+        any bucket, including the zero bucket, or in the count of observation constitutes a counter reset, but also the
+        disappearance of any previously populated bucket, a decrease of the zero-bucket width, or any schema change that
+        is not a compatible decrease of resolution.
       
                      
 
       
                      
-        resets should only be used with counters and counter-like native histograms.
+        resets should only be used with counters (for both floats and histograms).
       
                      
 
       
                      
-        If the range vector contains a mix of float and histogram samples for the same series, counter resets are detected
-        separately and their numbers added up. The change from a float to a histogram sample is not considered a
-        counter reset. Each float sample is compared to the next float sample, and each histogram is comprared to the next
-        histogram.
+        A float sample followed by a histogram sample, or vice versa, counts as a reset. A counter histogram sample
+        followed by a gauge histogram sample, or vice versa, also counts as a reset (but note that resets{" "}
+        should not be used on gauges in the first place, see above).
       
                      
     
   ),
   round: (
     <>
       
                      
-        round(v instant-vector, to_nearest=1 scalar) rounds the sample values of all elements in v{' '}
-        to the nearest integer. Ties are resolved by rounding up. The optional to_nearest argument allows
-        specifying the nearest multiple to which the sample values should be rounded. This multiple may also be a fraction.
+        round(v instant-vector, to_nearest=1 scalar) rounds the sample values of all elements in{" "}
+        v to the nearest integer. Ties are resolved by rounding up. The optional to_nearest{" "}
+        argument allows specifying the nearest multiple to which the sample values should be rounded. This multiple may
+        also be a fraction. Histogram samples in the input vector are ignored silently.
       
                      
     
   ),
   scalar: (
     <>
       
                      
-        Given a single-element input vector, scalar(v instant-vector) returns the sample value of that single
-        element as a scalar. If the input vector does not have exactly one element, scalar will return{' '}
-        NaN.
+        Given an input vector that contains only one element with a float sample,
+        scalar(v instant-vector) returns the sample value of that float sample as a scalar. If the input
+        vector does not have exactly one element with a float sample, scalar will return NaN.
+        Histogram samples in the input vector are ignored silently.
       
                      
     
   ),
   sgn: (
     <>
       
                      
-        sgn(v instant-vector) returns a vector with all sample values converted to their sign, defined as this:
-        1 if v is positive, -1 if v is negative and 0 if v is equal to zero.
+        sgn(v instant-vector) returns a vector with all float sample values converted to their sign,
+        defined as this: 1 if v is positive, -1 if v is negative and 0 if v is equal to zero. Histogram samples in the
+        input vector are ignored silently.
       
                      
     
   ),
   sin: (
     <>
-      
                      The trigonometric functions work in radians:
                      
+      
                      The trigonometric functions work in radians. They ignore histogram samples in the input vector.
                      
 
       
                        
         
                      • 
-          acos(v instant-vector): calculates the arccosine of all elements in v (
+          acos(v instant-vector): calculates the arccosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v (
-          special cases).
+          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          asin(v instant-vector): calculates the arcsine of all elements in v (
+          asin(v instant-vector): calculates the arcsine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v (
-          special cases).
+          asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          atan(v instant-vector): calculates the arctangent of all elements in v (
+          atan(v instant-vector): calculates the arctangent of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v (
-          special cases).
+          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          cos(v instant-vector): calculates the cosine of all elements in v (
+          cos(v instant-vector): calculates the cosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v (
+          cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          sin(v instant-vector): calculates the sine of all elements in v (
+          sin(v instant-vector): calculates the sine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          sinh(v instant-vector): calculates the hyperbolic sine of all elements in v (
+          sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          tan(v instant-vector): calculates the tangent of all elements in v (
+          tan(v instant-vector): calculates the tangent of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v (
-          special cases).
+          tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "}
+          (special cases).
         
                        • 
       
                      
 
@@ -2192,69 +2918,69 @@ const funcDocs: Record = {
 
       
                        
         
                      • 
-          deg(v instant-vector): converts radians to degrees for all elements in v.
+          deg(v instant-vector): converts radians to degrees for all float samples in v.
         
                        • 
         
                      • 
           pi(): returns pi.
         
                        • 
         
                      • 
-          rad(v instant-vector): converts degrees to radians for all elements in v.
+          rad(v instant-vector): converts degrees to radians for all float samples in v.
         
                        • 
       
                      
     
   ),
   sinh: (
     <>
-      
                      The trigonometric functions work in radians:
                      
+      
                      The trigonometric functions work in radians. They ignore histogram samples in the input vector.
                      
 
       
                        
         
                      • 
-          acos(v instant-vector): calculates the arccosine of all elements in v (
+          acos(v instant-vector): calculates the arccosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v (
-          special cases).
+          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          asin(v instant-vector): calculates the arcsine of all elements in v (
+          asin(v instant-vector): calculates the arcsine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v (
-          special cases).
+          asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          atan(v instant-vector): calculates the arctangent of all elements in v (
+          atan(v instant-vector): calculates the arctangent of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v (
-          special cases).
+          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "}
+          v (special cases).
         
                        • 
         
                      • 
-          cos(v instant-vector): calculates the cosine of all elements in v (
+          cos(v instant-vector): calculates the cosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v (
+          cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          sin(v instant-vector): calculates the sine of all elements in v (
+          sin(v instant-vector): calculates the sine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          sinh(v instant-vector): calculates the hyperbolic sine of all elements in v (
+          sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          tan(v instant-vector): calculates the tangent of all elements in v (
+          tan(v instant-vector): calculates the tangent of all float samples in v (
           special cases).
         
                        • 
         
                      • 
-          tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v (
-          special cases).
+          tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "}
+          (special cases).
         
                        • 
       
                      
 
@@ -2262,13 +2988,13 @@ const funcDocs: Record = {
 
       
                        
         
                      • 
-          deg(v instant-vector): converts radians to degrees for all elements in v.
+          deg(v instant-vector): converts radians to degrees for all float samples in v.
         
                        • 
         
                      • 
           pi(): returns pi.
         
                        • 
         
                      • 
-          rad(v instant-vector): converts degrees to radians for all elements in v.
+          rad(v instant-vector): converts degrees to radians for all float samples in v.
         
                        • 
       
                      
     
@@ -2276,13 +3002,13 @@ const funcDocs: Record = {
   sort: (
     <>
       
                      
-        sort(v instant-vector) returns vector elements sorted by their sample values, in ascending order. Native
-        histograms are sorted by their sum of observations.
+        sort(v instant-vector) returns vector elements sorted by their float sample values, in ascending
+        order. Histogram samples in the input vector are ignored silently.
       
                      
 
       
                      
-        Please note that sort only affects the results of instant queries, as range query results always have a
-        fixed output ordering.
+        Please note that sort only affects the results of instant queries, as range query results always
+        have a fixed output ordering.
       
                      
     
   ),
@@ -2290,24 +3016,27 @@ const funcDocs: Record = {
     <>
       
                      
         
-          This function has to be enabled via the{' '}
-          feature flag{' '}
+          This function has to be enabled via the{" "}
+          feature flag
           --enable-feature=promql-experimental-functions.
         
       
                      
 
       
                      
-        sort_by_label(v instant-vector, label string, ...) returns vector elements sorted by the values of the
-        given labels in ascending order. In case these label values are equal, elements are sorted by their full label sets.
+        sort_by_label(v instant-vector, label string, ...) returns vector elements sorted by the values of
+        the given labels in ascending order. In case these label values are equal, elements are sorted by their full
+        label sets.
+        sort_by_label acts on float and histogram samples in the same way.
       
                      
 
       
                      
-        Please note that the sort by label functions only affect the results of instant queries, as range query results
+        Please note that sort_by_label only affects the results of instant queries, as range query results
         always have a fixed output ordering.
       
                      
 
       
                      
-        This function uses natural sort order.
+        sort_by_label uses{" "}
+        natural sort order.
       
                      
     
   ),
@@ -2315,8 +3044,8 @@ const funcDocs: Record = {
     <>
       
                      
         
-          This function has to be enabled via the{' '}
-          feature flag{' '}
+          This function has to be enabled via the{" "}
+          feature flag
           --enable-feature=promql-experimental-functions.
         
       
                      
@@ -2324,15 +3053,6 @@ const funcDocs: Record = {
       
                      
         Same as sort_by_label, but sorts in descending order.
       
                      
-
-      
                      
-        Please note that the sort by label functions only affect the results of instant queries, as range query results
-        always have a fixed output ordering.
-      
                      
-
-      
                      
-        This function uses natural sort order.
-      
                      
     
   ),
   sort_desc: (
@@ -2340,57 +3060,55 @@ const funcDocs: Record = {
       
                      
         Same as sort, but sorts in descending order.
       
                      
-
-      
                      
-        Like sort, sort_desc only affects the results of instant queries, as range query results
-        always have a fixed output ordering.
-      
                      
     
   ),
   sqrt: (
     <>
       
                      
-        sqrt(v instant-vector) calculates the square root of all elements in v.
+        sqrt(v instant-vector) calculates the square root of all float samples in
+        v. Histogram samples in the input vector are ignored silently.
       
                      
     
   ),
   stddev_over_time: (
     <>
       
                      
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
                      
 
       
                        
         
                      • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                        • 
         
                      • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                        • 
         
                      • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                        • 
         
                      • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                        • 
         
                      • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                        • 
         
                      • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                        • 
         
                      • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                        • 
         
                      • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                        • 
         
                      • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                        • 
         
                      • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -2399,64 +3117,108 @@ const funcDocs: Record = {
 
       
                        
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                        
 
       
                          
         
                        • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                          • 
+        
                        • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                          • 
+        
                        • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                          • 
+        
                        • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                          • 
+        
                        • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                          • 
+        
                        • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                          • 
       
                        
 
       
                        
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                        
 
+      
                        These functions act on histograms in the following way:
                        
+
+      
                          
+        
                        • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                          • 
+        
                        • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                          • 
+        
                        • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                          • 
+      
                        
+
       
                        
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                        
     
   ),
   stdvar_over_time: (
     <>
       
                        
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
                        
 
       
                          
         
                        • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                          • 
         
                        • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                          • 
         
                        • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                          • 
         
                        • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                          • 
         
                        • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                          • 
         
                        • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                          • 
         
                        • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                          • 
         
                        • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                          • 
         
                        • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                          • 
         
                        • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -2465,64 +3227,108 @@ const funcDocs: Record = {
 
       
                          
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                          
 
       
                            
         
                          • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                            • 
+        
                          • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                            • 
+        
                          • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                            • 
+        
                          • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                            • 
+        
                          • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                            • 
+        
                          • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                            • 
       
                          
 
       
                          
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                          
 
+      
                          These functions act on histograms in the following way:
                          
+
+      
                            
+        
                          • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                            • 
+        
                          • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                            • 
+        
                          • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                            • 
+      
                          
+
       
                          
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                          
     
   ),
   sum_over_time: (
     <>
       
                          
-        The following functions allow aggregating each series of a given range vector over time and return an instant vector
-        with per-series aggregation results:
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
       
                          
 
       
                            
         
                          • 
-          avg_over_time(range-vector): the average value of all points in the specified interval.
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
         
                            • 
         
                          • 
-          min_over_time(range-vector): the minimum value of all points in the specified interval.
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
         
                            • 
         
                          • 
-          max_over_time(range-vector): the maximum value of all points in the specified interval.
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
         
                            • 
         
                          • 
-          sum_over_time(range-vector): the sum of all values in the specified interval.
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
         
                            • 
         
                          • 
-          count_over_time(range-vector): the count of all values in the specified interval.
+          count_over_time(range-vector): the count of all samples in the specified interval.
         
                            • 
         
                          • 
-          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of the values in the specified
-          interval.
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
         
                            • 
         
                          • 
-          stddev_over_time(range-vector): the population standard deviation of the values in the specified
-          interval.
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
         
                            • 
         
                          • 
-          stdvar_over_time(range-vector): the population standard variance of the values in the specified
-          interval.
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
         
                            • 
         
                          • 
-          last_over_time(range-vector): the most recent point value in the specified interval.
+          last_over_time(range-vector): the most recent sample in the specified interval.
         
                            • 
         
                          • 
           present_over_time(range-vector): the value 1 for any series in the specified interval.
@@ -2531,79 +3337,121 @@ const funcDocs: Record = {
 
       
                            
         If the feature flag
-        --enable-feature=promql-experimental-functions is set, the following additional functions are available:
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
       
                            
 
       
                              
         
                            • 
-          mad_over_time(range-vector): the median absolute deviation of all points in the specified interval.
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                              • 
+        
                            • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                              • 
+        
                            • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
         
                              • 
       
                            
 
       
                            
-        Note that all values in the specified interval have the same weight in the aggregation even if the values are not
-        equally spaced throughout the interval.
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
       
                            
 
+      
                            These functions act on histograms in the following way:
                            
+
+      
                              
+        
                            • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                              • 
+        
                            • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                              • 
+        
                            • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                              • 
+      
                            
+
       
                            
-        avg_over_time, sum_over_time, count_over_time, last_over_time,
-        and
-        present_over_time handle native histograms as expected. All other functions ignore histogram samples.
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                            
     
   ),
   tan: (
     <>
-      
                            The trigonometric functions work in radians:
                            
+      
                            The trigonometric functions work in radians. They ignore histogram samples in the input vector.
                            
 
       
                              
         
                            • 
-          acos(v instant-vector): calculates the arccosine of all elements in v (
+          acos(v instant-vector): calculates the arccosine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v (
-          special cases).
+          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "}
+          v (special cases).
         
                              • 
         
                            • 
-          asin(v instant-vector): calculates the arcsine of all elements in v (
+          asin(v instant-vector): calculates the arcsine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v (
-          special cases).
+          asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "}
+          v (special cases).
         
                              • 
         
                            • 
-          atan(v instant-vector): calculates the arctangent of all elements in v (
+          atan(v instant-vector): calculates the arctangent of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v (
-          special cases).
+          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "}
+          v (special cases).
         
                              • 
         
                            • 
-          cos(v instant-vector): calculates the cosine of all elements in v (
+          cos(v instant-vector): calculates the cosine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v (
+          cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          sin(v instant-vector): calculates the sine of all elements in v (
+          sin(v instant-vector): calculates the sine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          sinh(v instant-vector): calculates the hyperbolic sine of all elements in v (
+          sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          tan(v instant-vector): calculates the tangent of all elements in v (
+          tan(v instant-vector): calculates the tangent of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v (
-          special cases).
+          tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "}
+          (special cases).
         
                              • 
       
                            
 
@@ -2611,69 +3459,69 @@ const funcDocs: Record = {
 
       
                              
         
                            • 
-          deg(v instant-vector): converts radians to degrees for all elements in v.
+          deg(v instant-vector): converts radians to degrees for all float samples in v.
         
                              • 
         
                            • 
           pi(): returns pi.
         
                              • 
         
                            • 
-          rad(v instant-vector): converts degrees to radians for all elements in v.
+          rad(v instant-vector): converts degrees to radians for all float samples in v.
         
                              • 
       
                            
     
   ),
   tanh: (
     <>
-      
                            The trigonometric functions work in radians:
                            
+      
                            The trigonometric functions work in radians. They ignore histogram samples in the input vector.
                            
 
       
                              
         
                            • 
-          acos(v instant-vector): calculates the arccosine of all elements in v (
+          acos(v instant-vector): calculates the arccosine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all elements in v (
-          special cases).
+          acosh(v instant-vector): calculates the inverse hyperbolic cosine of all float samples in{" "}
+          v (special cases).
         
                              • 
         
                            • 
-          asin(v instant-vector): calculates the arcsine of all elements in v (
+          asin(v instant-vector): calculates the arcsine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          asinh(v instant-vector): calculates the inverse hyperbolic sine of all elements in v (
-          special cases).
+          asinh(v instant-vector): calculates the inverse hyperbolic sine of all float samples in{" "}
+          v (special cases).
         
                              • 
         
                            • 
-          atan(v instant-vector): calculates the arctangent of all elements in v (
+          atan(v instant-vector): calculates the arctangent of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all elements in v (
-          special cases).
+          atanh(v instant-vector): calculates the inverse hyperbolic tangent of all float samples in{" "}
+          v (special cases).
         
                              • 
         
                            • 
-          cos(v instant-vector): calculates the cosine of all elements in v (
+          cos(v instant-vector): calculates the cosine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          cosh(v instant-vector): calculates the hyperbolic cosine of all elements in v (
+          cosh(v instant-vector): calculates the hyperbolic cosine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          sin(v instant-vector): calculates the sine of all elements in v (
+          sin(v instant-vector): calculates the sine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          sinh(v instant-vector): calculates the hyperbolic sine of all elements in v (
+          sinh(v instant-vector): calculates the hyperbolic sine of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          tan(v instant-vector): calculates the tangent of all elements in v (
+          tan(v instant-vector): calculates the tangent of all float samples in v (
           special cases).
         
                              • 
         
                            • 
-          tanh(v instant-vector): calculates the hyperbolic tangent of all elements in v (
-          special cases).
+          tanh(v instant-vector): calculates the hyperbolic tangent of all float samples in v{" "}
+          (special cases).
         
                              • 
       
                            
 
@@ -2681,13 +3529,13 @@ const funcDocs: Record = {
 
       
                              
         
                            • 
-          deg(v instant-vector): converts radians to degrees for all elements in v.
+          deg(v instant-vector): converts radians to degrees for all float samples in v.
         
                              • 
         
                            • 
           pi(): returns pi.
         
                              • 
         
                            • 
-          rad(v instant-vector): converts degrees to radians for all elements in v.
+          rad(v instant-vector): converts degrees to radians for all float samples in v.
         
                              • 
       
                            
     
@@ -2695,8 +3543,8 @@ const funcDocs: Record = {
   time: (
     <>
       
                            
-        time() returns the number of seconds since January 1, 1970 UTC. Note that this does not actually return
-        the current time, but the time at which the expression is to be evaluated.
+        time() returns the number of seconds since January 1, 1970 UTC. Note that this does not actually
+        return the current time, but the time at which the expression is to be evaluated.
       
                            
     
   ),
@@ -2704,14 +3552,455 @@ const funcDocs: Record = {
     <>
       
                            
         timestamp(v instant-vector) returns the timestamp of each of the samples of the given vector as the
-        number of seconds since January 1, 1970 UTC. It also works with histogram samples.
+        number of seconds since January 1, 1970 UTC. It acts on float and histogram samples in the same way.
+      
                            
+    
+  ),
+  ts_of_first_over_time: (
+    <>
+      
                            
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
+      
                            
+
+      
                              
+        
                            • 
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
+        
                              • 
+        
                            • 
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
+        
                              • 
+        
                            • 
+          count_over_time(range-vector): the count of all samples in the specified interval.
+        
                              • 
+        
                            • 
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          last_over_time(range-vector): the most recent sample in the specified interval.
+        
                              • 
+        
                            • 
+          present_over_time(range-vector): the value 1 for any series in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        If the feature flag
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
+      
                            
+
+      
                              
+        
                            • 
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                              • 
+        
                            • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                              • 
+        
                            • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
+      
                            
+
+      
                            These functions act on histograms in the following way:
                            
+
+      
                              
+        
                            • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                              • 
+        
                            • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                              • 
+        
                            • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                              • 
+      
                            
+
+      
                            
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
+      
                            
+    
+  ),
+  ts_of_last_over_time: (
+    <>
+      
                            
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
+      
                            
+
+      
                              
+        
                            • 
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
+        
                              • 
+        
                            • 
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
+        
                              • 
+        
                            • 
+          count_over_time(range-vector): the count of all samples in the specified interval.
+        
                              • 
+        
                            • 
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          last_over_time(range-vector): the most recent sample in the specified interval.
+        
                              • 
+        
                            • 
+          present_over_time(range-vector): the value 1 for any series in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        If the feature flag
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
+      
                            
+
+      
                              
+        
                            • 
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                              • 
+        
                            • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                              • 
+        
                            • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
+      
                            
+
+      
                            These functions act on histograms in the following way:
                            
+
+      
                              
+        
                            • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                              • 
+        
                            • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                              • 
+        
                            • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                              • 
+      
                            
+
+      
                            
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
+      
                            
+    
+  ),
+  ts_of_max_over_time: (
+    <>
+      
                            
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
+      
                            
+
+      
                              
+        
                            • 
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
+        
                              • 
+        
                            • 
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
+        
                              • 
+        
                            • 
+          count_over_time(range-vector): the count of all samples in the specified interval.
+        
                              • 
+        
                            • 
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          last_over_time(range-vector): the most recent sample in the specified interval.
+        
                              • 
+        
                            • 
+          present_over_time(range-vector): the value 1 for any series in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        If the feature flag
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
+      
                            
+
+      
                              
+        
                            • 
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                              • 
+        
                            • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                              • 
+        
                            • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
+      
                            
+
+      
                            These functions act on histograms in the following way:
                            
+
+      
                              
+        
                            • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                              • 
+        
                            • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                              • 
+        
                            • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                              • 
+      
                            
+
+      
                            
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
+      
                            
+    
+  ),
+  ts_of_min_over_time: (
+    <>
+      
                            
+        The following functions allow aggregating each series of a given range vector over time and return an instant
+        vector with per-series aggregation results:
+      
                            
+
+      
                              
+        
                            • 
+          avg_over_time(range-vector): the average value of all float or histogram samples in the specified
+          interval (see details below).
+        
                              • 
+        
                            • 
+          min_over_time(range-vector): the minimum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          max_over_time(range-vector): the maximum value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          sum_over_time(range-vector): the sum of all float or histogram samples in the specified interval
+          (see details below).
+        
                              • 
+        
                            • 
+          count_over_time(range-vector): the count of all samples in the specified interval.
+        
                              • 
+        
                            • 
+          quantile_over_time(scalar, range-vector): the φ-quantile (0 ≤ φ ≤ 1) of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stddev_over_time(range-vector): the population standard deviation of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          stdvar_over_time(range-vector): the population standard variance of all float samples in the
+          specified interval.
+        
                              • 
+        
                            • 
+          last_over_time(range-vector): the most recent sample in the specified interval.
+        
                              • 
+        
                            • 
+          present_over_time(range-vector): the value 1 for any series in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        If the feature flag
+        --enable-feature=promql-experimental-functions is set, the following additional functions are
+        available:
+      
                            
+
+      
                              
+        
                            • 
+          mad_over_time(range-vector): the median absolute deviation of all float samples in the specified
+          interval.
+        
                              • 
+        
                            • 
+          ts_of_min_over_time(range-vector): the timestamp of the last float sample that has the minimum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_max_over_time(range-vector): the timestamp of the last float sample that has the maximum
+          value of all float samples in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_last_over_time(range-vector): the timestamp of last sample in the specified interval.
+        
                              • 
+        
                            • 
+          first_over_time(range-vector): the oldest sample in the specified interval.
+        
                              • 
+        
                            • 
+          ts_of_first_over_time(range-vector): the timestamp of earliest sample in the specified interval.
+        
                              • 
+      
                            
+
+      
                            
+        Note that all values in the specified interval have the same weight in the aggregation even if the values are
+        not equally spaced throughout the interval.
+      
                            
+
+      
                            These functions act on histograms in the following way:
                            
+
+      
                              
+        
                            • 
+          count_over_time, first_over_time, last_over_time, and
+          present_over_time() act on float and histogram samples in the same way.
+        
                              • 
+        
                            • 
+          avg_over_time() and sum_over_time() act on histogram samples in a way that
+          corresponds to the respective aggregation operators. If a series contains a mix of float samples and histogram
+          samples within the range, the corresponding result is removed entirely from the output vector. Such a removal
+          is flagged by a warn-level annotation.
+        
                              • 
+        
                            • 
+          All other functions ignore histogram samples in the following way: Input ranges containing only histogram
+          samples are silently removed from the output. For ranges with a mix of histogram and float samples, only the
+          float samples are processed and the omission of the histogram samples is flagged by an info-level annotation.
+        
                              • 
+      
                            
+
+      
                            
+        first_over_time(m[1m]) differs from m offset 1m in that the former will select the
+        first sample of m within the 1m range, where m offset 1m will select the most
+        recent sample within the lookback interval outside and prior to the 1m offset. This is particularly
+        useful with first_over_time(m[step()])
+        in range queries (available when --enable-feature=promql-duration-expr is set) to ensure that the
+        sample selected is within the range step.
       
                            
     
   ),
   vector: (
     <>
       
                            
-        vector(s scalar) returns the scalar s as a vector with no labels.
+        vector(s scalar) converts the scalar s to a float sample and returns it as a
+        single-element instant vector with no labels.
       
                            
     
   ),
@@ -2719,6 +4008,7 @@ const funcDocs: Record = {
     <>
       
                            
         year(v=vector(time()) instant-vector) returns the year for each of the given times in UTC.
+        Histogram samples in the input vector are ignored silently.
       
                            
     
   ),
diff --git a/web/ui/mantine-ui/src/promql/functionSignatures.ts b/web/ui/mantine-ui/src/promql/functionSignatures.ts
index 472d54ac5a..da21a2d4aa 100644
--- a/web/ui/mantine-ui/src/promql/functionSignatures.ts
+++ b/web/ui/mantine-ui/src/promql/functionSignatures.ts
@@ -1,140 +1,196 @@
-import { valueType, Func } from './ast';
+import { valueType, Func } from "./ast";
 
 export const functionSignatures: Record = {
-  abs: { name: 'abs', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  absent: { name: 'absent', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  absent_over_time: { name: 'absent_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  acos: { name: 'acos', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  acosh: { name: 'acosh', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  asin: { name: 'asin', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  asinh: { name: 'asinh', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  atan: { name: 'atan', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  atanh: { name: 'atanh', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  avg_over_time: { name: 'avg_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  ceil: { name: 'ceil', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  changes: { name: 'changes', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  abs: { name: "abs", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  absent: { name: "absent", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  absent_over_time: {
+    name: "absent_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  acos: { name: "acos", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  acosh: { name: "acosh", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  asin: { name: "asin", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  asinh: { name: "asinh", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  atan: { name: "atan", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  atanh: { name: "atanh", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  avg_over_time: { name: "avg_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  ceil: { name: "ceil", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  changes: { name: "changes", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
   clamp: {
-    name: 'clamp',
+    name: "clamp",
     argTypes: [valueType.vector, valueType.scalar, valueType.scalar],
     variadic: 0,
     returnType: valueType.vector,
   },
   clamp_max: {
-    name: 'clamp_max',
+    name: "clamp_max",
     argTypes: [valueType.vector, valueType.scalar],
     variadic: 0,
     returnType: valueType.vector,
   },
   clamp_min: {
-    name: 'clamp_min',
+    name: "clamp_min",
     argTypes: [valueType.vector, valueType.scalar],
     variadic: 0,
     returnType: valueType.vector,
   },
-  cos: { name: 'cos', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  cosh: { name: 'cosh', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  count_over_time: { name: 'count_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  day_of_month: { name: 'day_of_month', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  day_of_week: { name: 'day_of_week', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  day_of_year: { name: 'day_of_year', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  days_in_month: { name: 'days_in_month', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  deg: { name: 'deg', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  delta: { name: 'delta', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  deriv: { name: 'deriv', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  exp: { name: 'exp', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  floor: { name: 'floor', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  histogram_avg: { name: 'histogram_avg', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  histogram_count: { name: 'histogram_count', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  cos: { name: "cos", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  cosh: { name: "cosh", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  count_over_time: { name: "count_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  day_of_month: { name: "day_of_month", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  day_of_week: { name: "day_of_week", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  day_of_year: { name: "day_of_year", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  days_in_month: { name: "days_in_month", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  deg: { name: "deg", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  delta: { name: "delta", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  deriv: { name: "deriv", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  double_exponential_smoothing: {
+    name: "double_exponential_smoothing",
+    argTypes: [valueType.matrix, valueType.scalar, valueType.scalar],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  exp: { name: "exp", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  first_over_time: { name: "first_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  floor: { name: "floor", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  histogram_avg: { name: "histogram_avg", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  histogram_count: { name: "histogram_count", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
   histogram_fraction: {
-    name: 'histogram_fraction',
+    name: "histogram_fraction",
     argTypes: [valueType.scalar, valueType.scalar, valueType.vector],
     variadic: 0,
     returnType: valueType.vector,
   },
   histogram_quantile: {
-    name: 'histogram_quantile',
+    name: "histogram_quantile",
     argTypes: [valueType.scalar, valueType.vector],
     variadic: 0,
     returnType: valueType.vector,
   },
-  histogram_stddev: { name: 'histogram_stddev', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  histogram_stdvar: { name: 'histogram_stdvar', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  histogram_sum: { name: 'histogram_sum', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  double_exponential_smoothing: {
-    name: 'double_exponential_smoothing',
-    argTypes: [valueType.matrix, valueType.scalar, valueType.scalar],
+  histogram_stddev: {
+    name: "histogram_stddev",
+    argTypes: [valueType.vector],
     variadic: 0,
     returnType: valueType.vector,
   },
-  hour: { name: 'hour', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  idelta: { name: 'idelta', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  increase: { name: 'increase', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  irate: { name: 'irate', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  histogram_stdvar: {
+    name: "histogram_stdvar",
+    argTypes: [valueType.vector],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  histogram_sum: { name: "histogram_sum", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  hour: { name: "hour", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  idelta: { name: "idelta", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  increase: { name: "increase", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  info: { name: "info", argTypes: [valueType.vector, valueType.vector], variadic: 1, returnType: valueType.vector },
+  irate: { name: "irate", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
   label_join: {
-    name: 'label_join',
+    name: "label_join",
     argTypes: [valueType.vector, valueType.string, valueType.string, valueType.string],
     variadic: -1,
     returnType: valueType.vector,
   },
   label_replace: {
-    name: 'label_replace',
+    name: "label_replace",
     argTypes: [valueType.vector, valueType.string, valueType.string, valueType.string, valueType.string],
     variadic: 0,
     returnType: valueType.vector,
   },
-  last_over_time: { name: 'last_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  ln: { name: 'ln', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  log10: { name: 'log10', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  log2: { name: 'log2', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  mad_over_time: { name: 'mad_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  max_over_time: { name: 'max_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  min_over_time: { name: 'min_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  minute: { name: 'minute', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  month: { name: 'month', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
-  pi: { name: 'pi', argTypes: [], variadic: 0, returnType: valueType.scalar },
+  last_over_time: { name: "last_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  ln: { name: "ln", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  log10: { name: "log10", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  log2: { name: "log2", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  mad_over_time: { name: "mad_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  max_over_time: { name: "max_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  min_over_time: { name: "min_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  minute: { name: "minute", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  month: { name: "month", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  pi: { name: "pi", argTypes: [], variadic: 0, returnType: valueType.scalar },
   predict_linear: {
-    name: 'predict_linear',
+    name: "predict_linear",
     argTypes: [valueType.matrix, valueType.scalar],
     variadic: 0,
     returnType: valueType.vector,
   },
-  present_over_time: { name: 'present_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  present_over_time: {
+    name: "present_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
   quantile_over_time: {
-    name: 'quantile_over_time',
+    name: "quantile_over_time",
     argTypes: [valueType.scalar, valueType.matrix],
     variadic: 0,
     returnType: valueType.vector,
   },
-  rad: { name: 'rad', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  rate: { name: 'rate', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  resets: { name: 'resets', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  round: { name: 'round', argTypes: [valueType.vector, valueType.scalar], variadic: 1, returnType: valueType.vector },
-  scalar: { name: 'scalar', argTypes: [valueType.vector], variadic: 0, returnType: valueType.scalar },
-  sgn: { name: 'sgn', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  sin: { name: 'sin', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  sinh: { name: 'sinh', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  sort: { name: 'sort', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  rad: { name: "rad", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  rate: { name: "rate", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  resets: { name: "resets", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  round: { name: "round", argTypes: [valueType.vector, valueType.scalar], variadic: 1, returnType: valueType.vector },
+  scalar: { name: "scalar", argTypes: [valueType.vector], variadic: 0, returnType: valueType.scalar },
+  sgn: { name: "sgn", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  sin: { name: "sin", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  sinh: { name: "sinh", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  sort: { name: "sort", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
   sort_by_label: {
-    name: 'sort_by_label',
+    name: "sort_by_label",
     argTypes: [valueType.vector, valueType.string],
     variadic: -1,
     returnType: valueType.vector,
   },
   sort_by_label_desc: {
-    name: 'sort_by_label_desc',
+    name: "sort_by_label_desc",
     argTypes: [valueType.vector, valueType.string],
     variadic: -1,
     returnType: valueType.vector,
   },
-  sort_desc: { name: 'sort_desc', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  sqrt: { name: 'sqrt', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  stddev_over_time: { name: 'stddev_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  stdvar_over_time: { name: 'stdvar_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  sum_over_time: { name: 'sum_over_time', argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
-  tan: { name: 'tan', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  tanh: { name: 'tanh', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  time: { name: 'time', argTypes: [], variadic: 0, returnType: valueType.scalar },
-  timestamp: { name: 'timestamp', argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
-  vector: { name: 'vector', argTypes: [valueType.scalar], variadic: 0, returnType: valueType.vector },
-  year: { name: 'year', argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
+  sort_desc: { name: "sort_desc", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  sqrt: { name: "sqrt", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  stddev_over_time: {
+    name: "stddev_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  stdvar_over_time: {
+    name: "stdvar_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  sum_over_time: { name: "sum_over_time", argTypes: [valueType.matrix], variadic: 0, returnType: valueType.vector },
+  tan: { name: "tan", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  tanh: { name: "tanh", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  time: { name: "time", argTypes: [], variadic: 0, returnType: valueType.scalar },
+  timestamp: { name: "timestamp", argTypes: [valueType.vector], variadic: 0, returnType: valueType.vector },
+  ts_of_first_over_time: {
+    name: "ts_of_first_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  ts_of_last_over_time: {
+    name: "ts_of_last_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  ts_of_max_over_time: {
+    name: "ts_of_max_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  ts_of_min_over_time: {
+    name: "ts_of_min_over_time",
+    argTypes: [valueType.matrix],
+    variadic: 0,
+    returnType: valueType.vector,
+  },
+  vector: { name: "vector", argTypes: [valueType.scalar], variadic: 0, returnType: valueType.vector },
+  year: { name: "year", argTypes: [valueType.vector], variadic: 1, returnType: valueType.vector },
 };
diff --git a/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go b/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go
index 89545c1e5e..1b58362393 100644
--- a/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go
+++ b/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go
@@ -18,7 +18,7 @@ import (
 	"fmt"
 	"io"
 	"log"
-	"net/http"
+	"os"
 	"sort"
 	"strings"
 
@@ -26,20 +26,23 @@ import (
 	"github.com/russross/blackfriday/v2"
 )
 
-var funcDocsRe = regexp.MustCompile("^## `(.+)\\(\\)`\n$|^## (Trigonometric Functions)\n$")
+var funcDocsRe = regexp.MustCompile("^## `([^)]+)\\(\\)` and `([^)]+)\\(\\)`\n$|^## `(.+)\\(\\)`\n$|^## (Trigonometric Functions)\n$")
 
 func main() {
-	resp, err := http.Get("https://raw.githubusercontent.com/prometheus/prometheus/master/docs/querying/functions.md")
+	// Read from local file instead of fetching from upstream.
+	if len(os.Args) < 2 {
+		log.Fatalln("Usage: gen_functions_docs ")
+	}
+	functionsPath := os.Args[1]
+	file, err := os.Open(functionsPath)
 	if err != nil {
-		log.Fatalln("Failed to fetch function docs:", err)
-	}
-	if resp.StatusCode != 200 {
-		log.Fatalln("Bad status code while fetching function docs:", resp.Status)
+		log.Fatalln("Failed to open function docs:", err)
 	}
+	defer file.Close()
 
 	funcDocs := map[string]string{}
 
-	r := bufio.NewReader(resp.Body)
+	r := bufio.NewReader(file)
 	currentFunc := ""
 	currentDocs := ""
 
@@ -58,6 +61,11 @@ func main() {
 				"last_over_time",
 				"present_over_time",
 				"mad_over_time",
+				"first_over_time",
+				"ts_of_first_over_time",
+				"ts_of_last_over_time",
+				"ts_of_max_over_time",
+				"ts_of_min_over_time",
 			} {
 				funcDocs[fn] = currentDocs
 			}
@@ -81,6 +89,12 @@ func main() {
 			} {
 				funcDocs[fn] = currentDocs
 			}
+		case "histogram_count_and_histogram_sum":
+			funcDocs["histogram_count"] = currentDocs
+			funcDocs["histogram_sum"] = currentDocs
+		case "histogram_stddev_and_histogram_stdvar":
+			funcDocs["histogram_stddev"] = currentDocs
+			funcDocs["histogram_stdvar"] = currentDocs
 		default:
 			funcDocs[currentFunc] = currentDocs
 		}
@@ -103,10 +117,16 @@ func main() {
 			}
 			currentDocs = ""
 
-			currentFunc = string(matches[1])
-			if matches[2] != "" {
-				// This is the case for "## Trigonometric Functions"
-				currentFunc = matches[2]
+			if matches[1] != "" && matches[2] != "" {
+				// Combined functions: "## `function1()` and `function2()`"
+				// Store as "function1_and_function2" and handle in saveCurrent.
+				currentFunc = matches[1] + "_and_" + matches[2]
+			} else if matches[3] != "" {
+				// Single function: "## `function_name()`"
+				currentFunc = string(matches[3])
+			} else if matches[4] != "" {
+				// Special section: "## Trigonometric Functions"
+				currentFunc = matches[4]
 			}
 		} else {
 			currentDocs += line

From a9f66529a71f38a2ad56cd3075c113d542c409a7 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Fri, 19 Dec 2025 16:04:41 +0100
Subject: [PATCH 164/439] chore(deps): update module github.com/quic-go/quic-go
 to v0.57.0 [security] (#17689)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 internal/tools/go.mod | 4 ++--
 internal/tools/go.sum | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/internal/tools/go.mod b/internal/tools/go.mod
index 2334ae7bd0..b31cd5bc3e 100644
--- a/internal/tools/go.mod
+++ b/internal/tools/go.mod
@@ -67,8 +67,8 @@ require (
 	github.com/opencontainers/image-spec v1.1.1 // indirect
 	github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
-	github.com/quic-go/qpack v0.5.1 // indirect
-	github.com/quic-go/quic-go v0.56.0 // indirect
+	github.com/quic-go/qpack v0.6.0 // indirect
+	github.com/quic-go/quic-go v0.57.0 // indirect
 	github.com/rivo/uniseg v0.4.7 // indirect
 	github.com/rs/cors v1.11.1 // indirect
 	github.com/russross/blackfriday/v2 v2.1.0 // indirect
diff --git a/internal/tools/go.sum b/internal/tools/go.sum
index fb63670b60..7f1161148b 100644
--- a/internal/tools/go.sum
+++ b/internal/tools/go.sum
@@ -154,10 +154,10 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/protocolbuffers/protoscope v0.0.0-20221109213918-8e7a6aafa2c9 h1:arwj11zP0yJIxIRiDn22E0H8PxfF7TsTrc2wIPFIsf4=
 github.com/protocolbuffers/protoscope v0.0.0-20221109213918-8e7a6aafa2c9/go.mod h1:SKZx6stCn03JN3BOWTwvVIO2ajMkb/zQdTceXYhKw/4=
-github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI=
-github.com/quic-go/qpack v0.5.1/go.mod h1:+PC4XFrEskIVkcLzpEkbLqq1uCoxPhQuvK5rH1ZgaEg=
-github.com/quic-go/quic-go v0.56.0 h1:q/TW+OLismmXAehgFLczhCDTYB3bFmua4D9lsNBWxvY=
-github.com/quic-go/quic-go v0.56.0/go.mod h1:9gx5KsFQtw2oZ6GZTyh+7YEvOxWCL9WZAepnHxgAo6c=
+github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
+github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
+github.com/quic-go/quic-go v0.57.0 h1:AsSSrrMs4qI/hLrKlTH/TGQeTMY0ib1pAOX7vA3AdqE=
+github.com/quic-go/quic-go v0.57.0/go.mod h1:ly4QBAjHA2VhdnxhojRsCUOeJwKYg+taDlos92xb1+s=
 github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
 github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
 github.com/rodaine/protogofakeit v0.1.1 h1:ZKouljuRM3A+TArppfBqnH8tGZHOwM/pjvtXe9DaXH8=

From e9283f99d30450e3fa3b49694236a37ab7dccb68 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Fri, 19 Dec 2025 16:04:57 +0100
Subject: [PATCH 165/439] chore(deps): bump github/codeql-action from 4.31.2 to
 4.31.4 (#17571)

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.31.2 to 4.31.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/0499de31b99561a6d14a36a5f662c2a54f91beee...e12f0178983d466f2f6028f5cc7a6d786fd97f4b)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.31.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/codeql-analysis.yml | 6 +++---
 .github/workflows/scorecards.yml      | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
index 2e2143f4c8..4c30e97f90 100644
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -29,12 +29,12 @@ jobs:
           persist-credentials: false
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
+        uses: github/codeql-action/init@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # v4.31.4
         with:
           languages: ${{ matrix.language }}
 
       - name: Autobuild
-        uses: github/codeql-action/autobuild@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
+        uses: github/codeql-action/autobuild@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # v4.31.4
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
+        uses: github/codeql-action/analyze@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # v4.31.4
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index 658e140f27..cb6ba9571d 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -45,6 +45,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@0499de31b99561a6d14a36a5f662c2a54f91beee # tag=v4.31.2
+        uses: github/codeql-action/upload-sarif@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # tag=v4.31.4
         with:
           sarif_file: results.sarif

From 119e75d78b8e2c98984b8bbec2eedf78a41533b9 Mon Sep 17 00:00:00 2001
From: aviralgarg05 
Date: Fri, 19 Dec 2025 23:32:08 +0530
Subject: [PATCH 166/439] promqltest: Properly distinguish explicit
 counter_reset_hint specification

This commit addresses the PR feedback for issue #17615. The previous
implementation could not distinguish between:
- No counter reset hint specified (meaning "don't care")
- counter_reset_hint:unknown explicitly specified (meaning "verify it's unknown")

Changes:
- Added CounterResetHintSet field to parser.SequenceValue to track
  whether counter_reset_hint was explicitly specified in the test file
- Modified buildHistogramFromMap to set this flag when the hint is
  present in the descriptor map
- Updated newHistogramSequenceValue helper and histogramsSeries
  functions to propagate the flag through histogram series creation
- Updated yacc grammar to use the new helper function
- Modified compareNativeHistogram to accept the flag and only compare
  hints when explicitly specified

This allows tests to:
1. Not specify a hint (no comparison, backward compatible)
2. Explicitly specify counter_reset_hint:unknown (verify it's unknown)
3. Explicitly specify counter_reset_hint:gauge/reset/not_reset (verify match)

Fixes #17615

Signed-off-by: aviralgarg05 
---
 promql/parser/generated_parser.y    |  5 ++--
 promql/parser/generated_parser.y.go |  5 ++--
 promql/parser/parse.go              | 39 +++++++++++++++++++++++++----
 promql/promqltest/test.go           | 25 ++++++++++++------
 4 files changed, 58 insertions(+), 16 deletions(-)

diff --git a/promql/parser/generated_parser.y b/promql/parser/generated_parser.y
index d9bbb10b28..0f196ef5af 100644
--- a/promql/parser/generated_parser.y
+++ b/promql/parser/generated_parser.y
@@ -790,14 +790,15 @@ series_item     : BLANK
                 // Histogram descriptions (part of unit testing).
                 | histogram_series_value
                         {
-                        $$ = []SequenceValue{{Histogram:$1}}
+                        $$ = []SequenceValue{yylex.(*parser).newHistogramSequenceValue($1)}
                         }
                 | histogram_series_value TIMES uint
                         {
                         $$ = []SequenceValue{}
                         // Add an additional value for time 0, which we ignore in tests.
+                        sv := yylex.(*parser).newHistogramSequenceValue($1)
                         for i:=uint64(0); i <= $3; i++{
-                                $$ = append($$, SequenceValue{Histogram:$1})
+                                $$ = append($$, sv)
                                 //$1 += $2
                         }
                         }
diff --git a/promql/parser/generated_parser.y.go b/promql/parser/generated_parser.y.go
index eb4b32129a..b649e86440 100644
--- a/promql/parser/generated_parser.y.go
+++ b/promql/parser/generated_parser.y.go
@@ -1835,15 +1835,16 @@ yydefault:
 	case 158:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
-			yyVAL.series = []SequenceValue{{Histogram: yyDollar[1].histogram}}
+			yyVAL.series = []SequenceValue{yylex.(*parser).newHistogramSequenceValue(yyDollar[1].histogram)}
 		}
 	case 159:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{}
 			// Add an additional value for time 0, which we ignore in tests.
+			sv := yylex.(*parser).newHistogramSequenceValue(yyDollar[1].histogram)
 			for i := uint64(0); i <= yyDollar[3].uint; i++ {
-				yyVAL.series = append(yyVAL.series, SequenceValue{Histogram: yyDollar[1].histogram})
+				yyVAL.series = append(yyVAL.series, sv)
 				//$1 += $2
 			}
 		}
diff --git a/promql/parser/parse.go b/promql/parser/parse.go
index bcd511f467..212a5758e7 100644
--- a/promql/parser/parse.go
+++ b/promql/parser/parse.go
@@ -67,6 +67,11 @@ type parser struct {
 
 	generatedParserResult any
 	parseErrors           ParseErrors
+
+	// lastHistogramCounterResetHintSet is set to true when the most recently
+	// built histogram had a counter_reset_hint explicitly specified.
+	// This is used to populate CounterResetHintSet in SequenceValue.
+	lastHistogramCounterResetHintSet bool
 }
 
 type Opt func(p *parser)
@@ -234,6 +239,11 @@ type SequenceValue struct {
 	Value     float64
 	Omitted   bool
 	Histogram *histogram.FloatHistogram
+	// CounterResetHintSet is true if the counter reset hint was explicitly
+	// specified in the test file using counter_reset_hint:... syntax.
+	// This allows distinguishing between "no hint specified" (don't care)
+	// vs "counter_reset_hint:unknown" (verify it's unknown).
+	CounterResetHintSet bool
 }
 
 func (v SequenceValue) String() string {
@@ -496,25 +506,30 @@ func (p *parser) mergeMaps(left, right *map[string]any) (ret *map[string]any) {
 }
 
 func (p *parser) histogramsIncreaseSeries(base, inc *histogram.FloatHistogram, times uint64) ([]SequenceValue, error) {
-	return p.histogramsSeries(base, inc, times, func(a, b *histogram.FloatHistogram) (*histogram.FloatHistogram, error) {
+	// Capture the hint set flag immediately after inc histogram is built.
+	// The base histogram's hint set flag was already captured.
+	hintSet := p.lastHistogramCounterResetHintSet
+	return p.histogramsSeries(base, inc, times, hintSet, func(a, b *histogram.FloatHistogram) (*histogram.FloatHistogram, error) {
 		res, _, _, err := a.Add(b)
 		return res, err
 	})
 }
 
 func (p *parser) histogramsDecreaseSeries(base, inc *histogram.FloatHistogram, times uint64) ([]SequenceValue, error) {
-	return p.histogramsSeries(base, inc, times, func(a, b *histogram.FloatHistogram) (*histogram.FloatHistogram, error) {
+	// Capture the hint set flag immediately after inc histogram is built.
+	hintSet := p.lastHistogramCounterResetHintSet
+	return p.histogramsSeries(base, inc, times, hintSet, func(a, b *histogram.FloatHistogram) (*histogram.FloatHistogram, error) {
 		res, _, _, err := a.Sub(b)
 		return res, err
 	})
 }
 
-func (*parser) histogramsSeries(base, inc *histogram.FloatHistogram, times uint64,
+func (*parser) histogramsSeries(base, inc *histogram.FloatHistogram, times uint64, counterResetHintSet bool,
 	combine func(*histogram.FloatHistogram, *histogram.FloatHistogram) (*histogram.FloatHistogram, error),
 ) ([]SequenceValue, error) {
 	ret := make([]SequenceValue, times+1)
 	// Add an additional value (the base) for time 0, which we ignore in tests.
-	ret[0] = SequenceValue{Histogram: base}
+	ret[0] = SequenceValue{Histogram: base, CounterResetHintSet: counterResetHintSet}
 	cur := base
 	for i := uint64(1); i <= times; i++ {
 		if cur.Schema > inc.Schema {
@@ -526,7 +541,7 @@ func (*parser) histogramsSeries(base, inc *histogram.FloatHistogram, times uint6
 		if err != nil {
 			return ret, err
 		}
-		ret[i] = SequenceValue{Histogram: cur}
+		ret[i] = SequenceValue{Histogram: cur, CounterResetHintSet: counterResetHintSet}
 	}
 
 	return ret, nil
@@ -535,6 +550,8 @@ func (*parser) histogramsSeries(base, inc *histogram.FloatHistogram, times uint6
 // buildHistogramFromMap is used in the grammar to take then individual parts of the histogram and complete it.
 func (p *parser) buildHistogramFromMap(desc *map[string]any) *histogram.FloatHistogram {
 	output := &histogram.FloatHistogram{}
+	// Reset the flag for each new histogram being built.
+	p.lastHistogramCounterResetHintSet = false
 
 	val, ok := (*desc)["schema"]
 	if ok {
@@ -595,6 +612,8 @@ func (p *parser) buildHistogramFromMap(desc *map[string]any) *histogram.FloatHis
 
 	val, ok = (*desc)["counter_reset_hint"]
 	if ok {
+		// Mark that the counter reset hint was explicitly specified.
+		p.lastHistogramCounterResetHintSet = true
 		resetHint, ok := val.(Item)
 
 		if ok {
@@ -626,6 +645,16 @@ func (p *parser) buildHistogramFromMap(desc *map[string]any) *histogram.FloatHis
 	return output
 }
 
+// newHistogramSequenceValue creates a SequenceValue for a histogram,
+// setting CounterResetHintSet based on whether counter_reset_hint was
+// explicitly specified in the histogram description.
+func (p *parser) newHistogramSequenceValue(h *histogram.FloatHistogram) SequenceValue {
+	return SequenceValue{
+		Histogram:           h,
+		CounterResetHintSet: p.lastHistogramCounterResetHintSet,
+	}
+}
+
 func (p *parser) buildHistogramBucketsAndSpans(desc *map[string]any, bucketsKey, offsetKey string,
 ) (buckets []float64, spans []histogram.Span) {
 	bucketCount := 0
diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go
index d1702ba61b..0170236587 100644
--- a/promql/promqltest/test.go
+++ b/promql/promqltest/test.go
@@ -1009,7 +1009,12 @@ func (ev *evalCmd) compareResult(result parser.Value) error {
 			exp := ev.expected[hash]
 
 			var expectedFloats []promql.FPoint
-			var expectedHistograms []promql.HPoint
+			// expectedHPoint wraps HPoint with CounterResetHintSet flag from SequenceValue.
+			type expectedHPoint struct {
+				promql.HPoint
+				CounterResetHintSet bool
+			}
+			var expectedHistograms []expectedHPoint
 
 			for i, e := range exp.vals {
 				ts := ev.start.Add(time.Duration(i) * ev.step)
@@ -1021,7 +1026,10 @@ func (ev *evalCmd) compareResult(result parser.Value) error {
 				t := ts.UnixNano() / int64(time.Millisecond/time.Nanosecond)
 
 				if e.Histogram != nil {
-					expectedHistograms = append(expectedHistograms, promql.HPoint{T: t, H: e.Histogram})
+					expectedHistograms = append(expectedHistograms, expectedHPoint{
+						HPoint:              promql.HPoint{T: t, H: e.Histogram},
+						CounterResetHintSet: e.CounterResetHintSet,
+					})
 				} else if !e.Omitted {
 					expectedFloats = append(expectedFloats, promql.FPoint{T: t, F: e.Value})
 				}
@@ -1050,7 +1058,7 @@ func (ev *evalCmd) compareResult(result parser.Value) error {
 					return fmt.Errorf("expected histogram value at index %v for %s to have timestamp %v, but it had timestamp %v (result has %s)", i, ev.metrics[hash], expected.T, actual.T, formatSeriesResult(s))
 				}
 
-				if !compareNativeHistogram(expected.H.Compact(0), actual.H.Compact(0)) {
+				if !compareNativeHistogram(expected.H.Compact(0), actual.H.Compact(0), expected.CounterResetHintSet) {
 					return fmt.Errorf("expected histogram value at index %v (t=%v) for %s to be %v, but got %v (result has %s)", i, actual.T, ev.metrics[hash], expected.H.TestExpression(), actual.H.TestExpression(), formatSeriesResult(s))
 				}
 			}
@@ -1089,7 +1097,7 @@ func (ev *evalCmd) compareResult(result parser.Value) error {
 			if expH != nil && v.H == nil {
 				return fmt.Errorf("expected histogram %s for %s but got float value %v", HistogramTestExpression(expH), v.Metric, v.F)
 			}
-			if expH != nil && !compareNativeHistogram(expH.Compact(0), v.H.Compact(0)) {
+			if expH != nil && !compareNativeHistogram(expH.Compact(0), v.H.Compact(0), exp0.CounterResetHintSet) {
 				return fmt.Errorf("expected %v for %s but got %s", HistogramTestExpression(expH), v.Metric, HistogramTestExpression(v.H))
 			}
 			if !almost.Equal(exp0.Value, v.F, defaultEpsilon) {
@@ -1127,7 +1135,9 @@ func (ev *evalCmd) compareResult(result parser.Value) error {
 
 // compareNativeHistogram is helper function to compare two native histograms
 // which can tolerate some differ in the field of float type, such as Count, Sum.
-func compareNativeHistogram(exp, cur *histogram.FloatHistogram) bool {
+// The counterResetHintSet parameter indicates whether the counter reset hint was
+// explicitly specified in the expected histogram (from the test file).
+func compareNativeHistogram(exp, cur *histogram.FloatHistogram, counterResetHintSet bool) bool {
 	if exp == nil || cur == nil {
 		return false
 	}
@@ -1164,8 +1174,9 @@ func compareNativeHistogram(exp, cur *histogram.FloatHistogram) bool {
 	}
 
 	// Compare CounterResetHint only if explicitly specified in expected histogram.
-	// UnknownCounterReset (the default) means "don't care about the hint".
-	if exp.CounterResetHint != histogram.UnknownCounterReset {
+	// When counterResetHintSet is false, no hint was specified, meaning "don't care".
+	// When counterResetHintSet is true, the hint was explicitly specified and must match.
+	if counterResetHintSet {
 		if exp.CounterResetHint != cur.CounterResetHint {
 			return false
 		}

From 3c92bb1c20d3d891ff9c054e231d15bbd8d3044d Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 21 Dec 2025 09:27:46 +0100
Subject: [PATCH 167/439] chore(deps): bump actions/checkout from 4.2.2 to
 5.0.1 (#17568)

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 5.0.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/11bd71901bbe5b1630ceea73d27597364c9af683...93cb6efe18208431cddfb8368fd83d5badbf9bfd)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 5.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/buf-lint.yml              |  2 +-
 .github/workflows/buf.yml                   |  2 +-
 .github/workflows/check_release_notes.yml   |  2 +-
 .github/workflows/ci.yml                    | 26 ++++++++++-----------
 .github/workflows/codeql-analysis.yml       |  2 +-
 .github/workflows/container_description.yml |  4 ++--
 .github/workflows/repo_sync.yml             |  2 +-
 .github/workflows/scorecards.yml            |  2 +-
 8 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/.github/workflows/buf-lint.yml b/.github/workflows/buf-lint.yml
index 4e942f1f3b..7b835b36f8 100644
--- a/.github/workflows/buf-lint.yml
+++ b/.github/workflows/buf-lint.yml
@@ -12,7 +12,7 @@ jobs:
     name: lint
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: bufbuild/buf-setup-action@a47c93e0b1648d5651a065437926377d060baa99 # v1.50.0
diff --git a/.github/workflows/buf.yml b/.github/workflows/buf.yml
index add72cc89c..da3cf4952a 100644
--- a/.github/workflows/buf.yml
+++ b/.github/workflows/buf.yml
@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     if: github.repository_owner == 'prometheus'
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: bufbuild/buf-setup-action@a47c93e0b1648d5651a065437926377d060baa99 # v1.50.0
diff --git a/.github/workflows/check_release_notes.yml b/.github/workflows/check_release_notes.yml
index b8381aff07..171af5f213 100644
--- a/.github/workflows/check_release_notes.yml
+++ b/.github/workflows/check_release_notes.yml
@@ -20,7 +20,7 @@ jobs:
     # Don't run it on dependabot PRs either as humans would take control in case a bump introduces a breaking change.
     if: (github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community') && github.event.pull_request.user.login != 'dependabot[bot]'
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
       - env:
           PR_DESCRIPTION: ${{ github.event.pull_request.body }}
         run: |
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index e4c2fbce18..0734a9de0d 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -16,7 +16,7 @@ jobs:
       # should also be updated.
       image: quay.io/prometheus/golang-builder:1.25-base
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -34,7 +34,7 @@ jobs:
     container:
       image: quay.io/prometheus/golang-builder:1.25-base
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -59,7 +59,7 @@ jobs:
       # The go version in this image should be N-1 wrt test_go.
       image: quay.io/prometheus/golang-builder:1.24-base
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - run: make build
@@ -78,7 +78,7 @@ jobs:
       image: quay.io/prometheus/golang-builder:1.25-base
 
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -97,7 +97,7 @@ jobs:
     name: Go tests on Windows
     runs-on: windows-latest
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
@@ -116,7 +116,7 @@ jobs:
     container:
       image: quay.io/prometheus/golang-builder:1.25-base
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - run: go install ./cmd/promtool/.
@@ -143,7 +143,7 @@ jobs:
       matrix:
         thread: [ 0, 1, 2 ]
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -170,7 +170,7 @@ jobs:
     # Whenever the Go version is updated here, .promu.yml
     # should also be updated.
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -206,7 +206,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - name: Install Go
@@ -221,7 +221,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - name: Install Go
@@ -265,7 +265,7 @@ jobs:
     needs: [test_ui, test_go, test_go_more, test_go_oldest, test_windows, golangci, codeql, build_all]
     if: github.event_name == 'push' && github.event.ref == 'refs/heads/main'
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -284,7 +284,7 @@ jobs:
       ||
       (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v3.'))
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
@@ -301,7 +301,7 @@ jobs:
     needs: [test_ui, codeql]
     steps:
       - name: Checkout
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
index 4c30e97f90..02f92b7e17 100644
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -24,7 +24,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
 
diff --git a/.github/workflows/container_description.yml b/.github/workflows/container_description.yml
index 7de8bb8da7..7b46e9532f 100644
--- a/.github/workflows/container_description.yml
+++ b/.github/workflows/container_description.yml
@@ -18,7 +18,7 @@ jobs:
     if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks.
     steps:
       - name: git checkout
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - name: Set docker hub repo name
@@ -42,7 +42,7 @@ jobs:
     if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks.
     steps:
       - name: git checkout
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - name: Set quay.io org name
diff --git a/.github/workflows/repo_sync.yml b/.github/workflows/repo_sync.yml
index fea1422fdc..afc589c6d7 100644
--- a/.github/workflows/repo_sync.yml
+++ b/.github/workflows/repo_sync.yml
@@ -14,7 +14,7 @@ jobs:
     container:
       image: quay.io/prometheus/golang-builder
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - run: ./scripts/sync_repo_files.sh
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index cb6ba9571d..c112b591dc 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -21,7 +21,7 @@ jobs:
 
     steps:
       - name: "Checkout code"
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # tag=v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # tag=v6.0.1
         with:
           persist-credentials: false
 

From a67e9ee37a0c943b14047f12092f09b808767ad8 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 21 Dec 2025 09:28:29 +0100
Subject: [PATCH 168/439] chore(deps): bump actions/setup-go from 6.0.0 to
 6.1.0 (#17569)

Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/44694675825211faa026b3c33043df3e48a5fa00...4dc6199c7b1a012772edbd06daecab0f50c9053c)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/ci.yml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0734a9de0d..3b8f4464fb 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -100,7 +100,7 @@ jobs:
       - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
-      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
+      - uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
         with:
           go-version: 1.25.x
       - run: |
@@ -210,7 +210,7 @@ jobs:
         with:
           persist-credentials: false
       - name: Install Go
-        uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
+        uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
         with:
           cache: false
           go-version: 1.25.x
@@ -225,7 +225,7 @@ jobs:
         with:
           persist-credentials: false
       - name: Install Go
-        uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
+        uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
         with:
           go-version: 1.25.x
       - name: Install snmp_exporter/generator dependencies

From bce607434331619caea21816b19a7eb7b1407307 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 21 Dec 2025 09:29:02 +0100
Subject: [PATCH 169/439] chore(deps): bump golangci/golangci-lint-action from
 8.0.0 to 9.0.0 (#17570)

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 8.0.0 to 9.0.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/4afd733a84b1f43292c63897423277bb7f4313a9...0a35821d5c230e903fcfe077583637dea1b27b47)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/ci.yml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 3b8f4464fb..1571558590 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -235,18 +235,18 @@ jobs:
         id: golangci-lint-version
         run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT
       - name: Lint
-        uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
+        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
         with:
           args: --verbose
           version: ${{ steps.golangci-lint-version.outputs.version }}
       - name: Lint with slicelabels
-        uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
+        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
         with:
           # goexperiment.synctest to ensure we don't miss files that depend on it.
           args: --verbose --build-tags=slicelabels,goexperiment.synctest
           version: ${{ steps.golangci-lint-version.outputs.version }}
       - name: Lint with dedupelabels
-        uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
+        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
         with:
           args: --verbose --build-tags=dedupelabels
           version: ${{ steps.golangci-lint-version.outputs.version }}

From a5811e2da9c901d9340d2464185fbf57f10d82b6 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 21 Dec 2025 09:29:37 +0100
Subject: [PATCH 170/439] chore(deps): bump actions/checkout from 4.2.2 to
 5.0.1 in /scripts (#17572)

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 5.0.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/11bd71901bbe5b1630ceea73d27597364c9af683...93cb6efe18208431cddfb8368fd83d5badbf9bfd)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 5.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 scripts/golangci-lint.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/golangci-lint.yml b/scripts/golangci-lint.yml
index 75f886d546..1b22eb6645 100644
--- a/scripts/golangci-lint.yml
+++ b/scripts/golangci-lint.yml
@@ -24,7 +24,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           persist-credentials: false
       - name: Install Go

From 3698cd044c33be89fcc0dfdcb1b279f9d30c8d53 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 21 Dec 2025 09:30:58 +0100
Subject: [PATCH 171/439] chore(deps): bump actions/setup-go from 6.0.0 to
 6.1.0 in /scripts (#17573)

Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/44694675825211faa026b3c33043df3e48a5fa00...4dc6199c7b1a012772edbd06daecab0f50c9053c)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 scripts/golangci-lint.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/golangci-lint.yml b/scripts/golangci-lint.yml
index 1b22eb6645..5fb28a738a 100644
--- a/scripts/golangci-lint.yml
+++ b/scripts/golangci-lint.yml
@@ -28,7 +28,7 @@ jobs:
         with:
           persist-credentials: false
       - name: Install Go
-        uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
+        uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
         with:
           go-version: 1.25.x
       - name: Install snmp_exporter/generator dependencies

From b34e3410f3103ed77d3e8e833603ffb210518e5a Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 21 Dec 2025 09:31:32 +0100
Subject: [PATCH 172/439] chore(deps): bump golangci/golangci-lint-action in
 /scripts (#17574)

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 8.0.0 to 9.0.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/4afd733a84b1f43292c63897423277bb7f4313a9...0a35821d5c230e903fcfe077583637dea1b27b47)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 scripts/golangci-lint.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/golangci-lint.yml b/scripts/golangci-lint.yml
index 5fb28a738a..2736e69b78 100644
--- a/scripts/golangci-lint.yml
+++ b/scripts/golangci-lint.yml
@@ -38,7 +38,7 @@ jobs:
         id: golangci-lint-version
         run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT
       - name: Lint
-        uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
+        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
         with:
           args: --verbose
           version: ${{ steps.golangci-lint-version.outputs.version }}

From a155ad55a35a527d4980d91e1619ccad659d3bc8 Mon Sep 17 00:00:00 2001
From: Joshua Rogers 
Date: Sun, 21 Dec 2025 17:15:35 +0800
Subject: [PATCH 173/439] httputil: add Vary: Accept-Encoding and fix
 compression headers (#17466)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

- Emit `Vary: Accept-Encoding` in newCompressedResponseWriter so shared caches
  key responses by content-coding. This prevents cache poisoning and
  undecodable bytes when a compressed variant is cached and later served to a
  client that didn't advertise support. (RFC 9110 §12.5.5 "Vary";
  RFC 9111 §4.1 cache key & Vary)

- When selecting gzip/deflate, set `Content-Encoding` and delete any existing
  `Content-Length` so Go's net/http can frame the message correctly
  (chunked for HTTP/1.1; implicit for HTTP/2+). This avoids stale length
  mismatches and related proxy/client issues.

Signed-off-by: Joshua Rogers 
---
 util/httputil/compression.go | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/util/httputil/compression.go b/util/httputil/compression.go
index d5bedb7fa9..e67f9ffd9f 100644
--- a/util/httputil/compression.go
+++ b/util/httputil/compression.go
@@ -56,6 +56,7 @@ func (c *compressedResponseWriter) Close() {
 
 // Constructs a new compressedResponseWriter based on client request headers.
 func newCompressedResponseWriter(writer http.ResponseWriter, req *http.Request) *compressedResponseWriter {
+	writer.Header().Add("Vary", acceptEncodingHeader)
 	raw := req.Header.Get(acceptEncodingHeader)
 	var (
 		encoding   string
@@ -65,13 +66,17 @@ func newCompressedResponseWriter(writer http.ResponseWriter, req *http.Request)
 		encoding, raw, commaFound = strings.Cut(raw, ",")
 		switch strings.TrimSpace(encoding) {
 		case gzipEncoding:
-			writer.Header().Set(contentEncodingHeader, gzipEncoding)
+			h := writer.Header()
+			h.Del("Content-Length") // avoid stale length after compression
+			h.Set(contentEncodingHeader, gzipEncoding)
 			return &compressedResponseWriter{
 				ResponseWriter: writer,
 				writer:         gzip.NewWriter(writer),
 			}
 		case deflateEncoding:
-			writer.Header().Set(contentEncodingHeader, deflateEncoding)
+			h := writer.Header()
+			h.Del("Content-Length")
+			h.Set(contentEncodingHeader, deflateEncoding)
 			return &compressedResponseWriter{
 				ResponseWriter: writer,
 				writer:         zlib.NewWriter(writer),

From e4b6d443fca2e612c6d9ef82c3721f07ff372528 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarkko=20P=C3=B6yry?= 
Date: Sun, 21 Dec 2025 21:55:02 +0200
Subject: [PATCH 174/439] tsdb: fix handle leak on mmap failure on MS Windows
 (#17310)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Jarkko Pöyry 
---
 tsdb/fileutil/mmap_windows.go | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tsdb/fileutil/mmap_windows.go b/tsdb/fileutil/mmap_windows.go
index b942264123..5704b3b96d 100644
--- a/tsdb/fileutil/mmap_windows.go
+++ b/tsdb/fileutil/mmap_windows.go
@@ -27,14 +27,15 @@ func mmap(f *os.File, size int) ([]byte, error) {
 	}
 
 	addr, errno := syscall.MapViewOfFile(h, syscall.FILE_MAP_READ, 0, 0, uintptr(size))
-	if addr == 0 {
-		return nil, os.NewSyscallError("MapViewOfFile", errno)
-	}
 
 	if err := syscall.CloseHandle(syscall.Handle(h)); err != nil {
 		return nil, os.NewSyscallError("CloseHandle", err)
 	}
 
+	if addr == 0 {
+		return nil, os.NewSyscallError("MapViewOfFile", errno)
+	}
+
 	return (*[maxMapSize]byte)(unsafe.Pointer(addr))[:size], nil
 }
 

From 17e06dbab54ed5da9d3fed1aa268a54bed16daae Mon Sep 17 00:00:00 2001
From: Bartlomiej Plotka 
Date: Mon, 22 Dec 2025 09:38:48 +0000
Subject: [PATCH 175/439] refactor(scrape)[PART2]: simplified scrapeLoop
 constructors & tests; add teststorage.Appendable mock (#17631)

* refactor(scrape): simplified scrapeLoop constructors & tests; add teststorage.Appender mock

Signed-off-by: bwplotka 

debug

* refactor(scrape): simplified newLoop even more

Signed-off-by: bwplotka 

* refactor(scrape): rename sl -> app, slApp -> app

Signed-off-by: bwplotka 

* fix TestScrapeLoopRun flakiness

Signed-off-by: bwplotka 

* fix lint

Signed-off-by: bwplotka 

* kill unused listSeriesSet code

Signed-off-by: bwplotka 

* fix closing to not panic

Signed-off-by: bwplotka 

* added extra benchmark for scrapeAndReport

Signed-off-by: bwplotka 

* added extra benchmark for restartLoops

Signed-off-by: bwplotka 

* addressed last comments

Signed-off-by: bwplotka 

* fix TestConcurrentAppender_ReturnsErrAppender naming

Signed-off-by: bwplotka 

* addressed small comments

Signed-off-by: bwplotka 

* refactor(scrape): ensure scrape config is reloaded; added test

Signed-off-by: bwplotka 

* addressed comments.

Signed-off-by: bwplotka 

---------

Signed-off-by: bwplotka 
---
 config/config.go                   |    2 +-
 model/histogram/float_histogram.go |    2 +-
 model/histogram/histogram.go       |    2 +-
 scrape/helpers_test.go             |  313 ++--
 scrape/manager.go                  |   22 +-
 scrape/manager_test.go             |  140 +-
 scrape/scrape.go                   |  669 ++++----
 scrape/scrape_test.go              | 2535 +++++++++++++---------------
 scrape/target.go                   |    2 +-
 scrape/target_test.go              |   24 +-
 util/teststorage/appender.go       |  399 +++++
 util/teststorage/appender_test.go  |  131 ++
 12 files changed, 2198 insertions(+), 2043 deletions(-)
 create mode 100644 util/teststorage/appender.go
 create mode 100644 util/teststorage/appender_test.go

diff --git a/config/config.go b/config/config.go
index 30c8a8ed21..113942b61a 100644
--- a/config/config.go
+++ b/config/config.go
@@ -1022,7 +1022,7 @@ func ToEscapingScheme(s string, v model.ValidationScheme) (model.EscapingScheme,
 		case model.LegacyValidation:
 			return model.UnderscoreEscaping, nil
 		case model.UnsetValidation:
-			return model.NoEscaping, fmt.Errorf("v is unset: %s", v)
+			return model.NoEscaping, fmt.Errorf("ValidationScheme is unset: %s", v)
 		default:
 			panic(fmt.Errorf("unhandled validation scheme: %s", v))
 		}
diff --git a/model/histogram/float_histogram.go b/model/histogram/float_histogram.go
index 91fcac1cfb..0acf9cb28f 100644
--- a/model/histogram/float_histogram.go
+++ b/model/histogram/float_histogram.go
@@ -484,7 +484,7 @@ func (h *FloatHistogram) Sub(other *FloatHistogram) (res *FloatHistogram, counte
 // supposed to be used according to the schema.
 func (h *FloatHistogram) Equals(h2 *FloatHistogram) bool {
 	if h2 == nil {
-		return false
+		return h == nil
 	}
 
 	if h.Schema != h2.Schema ||
diff --git a/model/histogram/histogram.go b/model/histogram/histogram.go
index 5fc68ef9d0..aa9f696be6 100644
--- a/model/histogram/histogram.go
+++ b/model/histogram/histogram.go
@@ -247,7 +247,7 @@ func (h *Histogram) CumulativeBucketIterator() BucketIterator[uint64] {
 // supposed to be used according to the schema.
 func (h *Histogram) Equals(h2 *Histogram) bool {
 	if h2 == nil {
-		return false
+		return h == nil
 	}
 
 	if h.Schema != h2.Schema || h.Count != h2.Count ||
diff --git a/scrape/helpers_test.go b/scrape/helpers_test.go
index ff7a7bf65a..dd5179b360 100644
--- a/scrape/helpers_test.go
+++ b/scrape/helpers_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -17,240 +17,127 @@ import (
 	"bytes"
 	"context"
 	"encoding/binary"
-	"fmt"
-	"math"
-	"strings"
-	"sync"
+	"net/http"
 	"testing"
+	"time"
 
 	"github.com/gogo/protobuf/proto"
 	dto "github.com/prometheus/client_model/go"
+	"github.com/prometheus/common/model"
+	"github.com/prometheus/common/promslog"
 	"github.com/stretchr/testify/require"
 
-	"github.com/prometheus/prometheus/model/exemplar"
+	"github.com/prometheus/prometheus/config"
 	"github.com/prometheus/prometheus/model/histogram"
 	"github.com/prometheus/prometheus/model/labels"
-	"github.com/prometheus/prometheus/model/metadata"
 	"github.com/prometheus/prometheus/storage"
+	"github.com/prometheus/prometheus/util/pool"
+	"github.com/prometheus/prometheus/util/teststorage"
 )
 
-type nopAppendable struct{}
+// For readability.
+type sample = teststorage.Sample
 
-func (nopAppendable) Appender(context.Context) storage.Appender {
-	return nopAppender{}
-}
-
-type nopAppender struct{}
-
-func (nopAppender) SetOptions(*storage.AppendOptions) {}
-
-func (nopAppender) Append(storage.SeriesRef, labels.Labels, int64, float64) (storage.SeriesRef, error) {
-	return 1, nil
-}
-
-func (nopAppender) AppendExemplar(storage.SeriesRef, labels.Labels, exemplar.Exemplar) (storage.SeriesRef, error) {
-	return 2, nil
-}
-
-func (nopAppender) AppendHistogram(storage.SeriesRef, labels.Labels, int64, *histogram.Histogram, *histogram.FloatHistogram) (storage.SeriesRef, error) {
-	return 3, nil
-}
-
-func (nopAppender) AppendHistogramSTZeroSample(storage.SeriesRef, labels.Labels, int64, int64, *histogram.Histogram, *histogram.FloatHistogram) (storage.SeriesRef, error) {
-	return 0, nil
-}
-
-func (nopAppender) UpdateMetadata(storage.SeriesRef, labels.Labels, metadata.Metadata) (storage.SeriesRef, error) {
-	return 4, nil
-}
-
-func (nopAppender) AppendSTZeroSample(storage.SeriesRef, labels.Labels, int64, int64) (storage.SeriesRef, error) {
-	return 5, nil
-}
-
-func (nopAppender) Commit() error   { return nil }
-func (nopAppender) Rollback() error { return nil }
-
-type floatSample struct {
-	metric labels.Labels
-	t      int64
-	f      float64
-}
-
-func equalFloatSamples(a, b floatSample) bool {
-	// Compare Float64bits so NaN values which are exactly the same will compare equal.
-	return labels.Equal(a.metric, b.metric) && a.t == b.t && math.Float64bits(a.f) == math.Float64bits(b.f)
-}
-
-type histogramSample struct {
-	metric labels.Labels
-	t      int64
-	h      *histogram.Histogram
-	fh     *histogram.FloatHistogram
-}
-
-type metadataEntry struct {
-	m      metadata.Metadata
-	metric labels.Labels
-}
-
-func metadataEntryEqual(a, b metadataEntry) bool {
-	if !labels.Equal(a.metric, b.metric) {
-		return false
+func withCtx(ctx context.Context) func(sl *scrapeLoop) {
+	return func(sl *scrapeLoop) {
+		sl.ctx = ctx
 	}
-	if a.m.Type != b.m.Type {
-		return false
-	}
-	if a.m.Unit != b.m.Unit {
-		return false
-	}
-	if a.m.Help != b.m.Help {
-		return false
-	}
-	return true
 }
 
-type collectResultAppendable struct {
-	*collectResultAppender
+func withAppendable(appendable storage.Appendable) func(sl *scrapeLoop) {
+	return func(sl *scrapeLoop) {
+		sl.appendable = appendable
+	}
 }
 
-func (a *collectResultAppendable) Appender(context.Context) storage.Appender {
-	return a
+// newTestScrapeLoop is the initial scrape loop for all tests.
+// It returns scrapeLoop and mock scraper you can customize.
+//
+// It's recommended to use withXYZ functions for simple option customizations, e.g:
+//
+//	appTest := teststorage.NewAppendable()
+//	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
+//
+// However, when changing more than one scrapeLoop options it's more readable to have one explicit opt function:
+//
+//	ctx, cancel := context.WithCancel(t.Context())
+//	appTest := teststorage.NewAppendable()
+//	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+//		sl.ctx = ctx
+//		sl.appendable = appTest
+//		// Since we're writing samples directly below we need to provide a protocol fallback.
+//		sl.fallbackScrapeProtocol = "text/plain"
+//	})
+//
+// NOTE: Try to NOT add more parameter to this function. Try to NOT add more
+// newTestScrapeLoop-like constructors. It should be flexible enough with scrapeLoop
+// used for initial options.
+func newTestScrapeLoop(t testing.TB, opts ...func(sl *scrapeLoop)) (_ *scrapeLoop, scraper *testScraper) {
+	metrics := newTestScrapeMetrics(t)
+	sl := &scrapeLoop{
+		stopped: make(chan struct{}),
+
+		l:     promslog.NewNopLogger(),
+		cache: newScrapeCache(metrics),
+
+		interval:            10 * time.Millisecond,
+		timeout:             1 * time.Hour,
+		sampleMutator:       nopMutator,
+		reportSampleMutator: nopMutator,
+
+		appendable:          teststorage.NewAppendable(),
+		buffers:             pool.New(1e3, 1e6, 3, func(sz int) any { return make([]byte, 0, sz) }),
+		metrics:             metrics,
+		maxSchema:           histogram.ExponentialSchemaMax,
+		honorTimestamps:     true,
+		enableCompression:   true,
+		validationScheme:    model.UTF8Validation,
+		symbolTable:         labels.NewSymbolTable(),
+		appendMetadataToWAL: true, // Tests assumes it's enabled, unless explicitly turned off.
+	}
+	for _, o := range opts {
+		o(sl)
+	}
+	// Validate user opts for convenience.
+	require.Nil(t, sl.parentCtx, "newTestScrapeLoop does not support injecting non-nil parent context")
+	require.Nil(t, sl.appenderCtx, "newTestScrapeLoop does not support injecting non-nil appender context")
+	require.Nil(t, sl.cancel, "newTestScrapeLoop does not support injecting custom cancel function")
+	require.Nil(t, sl.scraper, "newTestScrapeLoop does not support injecting scraper, it's mocked, use the returned scraper")
+
+	rootCtx := t.Context()
+	// Use sl.ctx for context injection.
+	// True contexts (sl.appenderCtx, sl.parentCtx, sl.ctx) are populated from it
+	if sl.ctx != nil {
+		rootCtx = sl.ctx
+	}
+	ctx, cancel := context.WithCancel(rootCtx)
+	sl.ctx = ctx
+	sl.cancel = cancel
+	sl.appenderCtx = rootCtx
+	sl.parentCtx = rootCtx
+
+	scraper = &testScraper{}
+	sl.scraper = scraper
+	return sl, scraper
 }
 
-// collectResultAppender records all samples that were added through the appender.
-// It can be used as its zero value or be backed by another appender it writes samples through.
-type collectResultAppender struct {
-	mtx sync.Mutex
+func newTestScrapePool(t *testing.T, injectNewLoop func(options scrapeLoopOptions) loop) *scrapePool {
+	return &scrapePool{
+		ctx:     t.Context(),
+		cancel:  func() {},
+		logger:  promslog.NewNopLogger(),
+		config:  &config.ScrapeConfig{},
+		options: &Options{},
+		client:  http.DefaultClient,
 
-	next                 storage.Appender
-	resultFloats         []floatSample
-	pendingFloats        []floatSample
-	rolledbackFloats     []floatSample
-	resultHistograms     []histogramSample
-	pendingHistograms    []histogramSample
-	rolledbackHistograms []histogramSample
-	resultExemplars      []exemplar.Exemplar
-	pendingExemplars     []exemplar.Exemplar
-	resultMetadata       []metadataEntry
-	pendingMetadata      []metadataEntry
-}
+		activeTargets:     map[uint64]*Target{},
+		loops:             map[uint64]loop{},
+		injectTestNewLoop: injectNewLoop,
 
-func (*collectResultAppender) SetOptions(*storage.AppendOptions) {}
-
-func (a *collectResultAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64, v float64) (storage.SeriesRef, error) {
-	a.mtx.Lock()
-	defer a.mtx.Unlock()
-	a.pendingFloats = append(a.pendingFloats, floatSample{
-		metric: lset,
-		t:      t,
-		f:      v,
-	})
-
-	if a.next == nil {
-		if ref == 0 {
-			// Use labels hash as a stand-in for unique series reference, to avoid having to track all series.
-			ref = storage.SeriesRef(lset.Hash())
-		}
-		return ref, nil
+		appendable:  teststorage.NewAppendable(),
+		symbolTable: labels.NewSymbolTable(),
+		metrics:     newTestScrapeMetrics(t),
 	}
-
-	ref, err := a.next.Append(ref, lset, t, v)
-	if err != nil {
-		return 0, err
-	}
-	return ref, nil
-}
-
-func (a *collectResultAppender) AppendExemplar(ref storage.SeriesRef, l labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) {
-	a.mtx.Lock()
-	defer a.mtx.Unlock()
-	a.pendingExemplars = append(a.pendingExemplars, e)
-	if a.next == nil {
-		return 0, nil
-	}
-
-	return a.next.AppendExemplar(ref, l, e)
-}
-
-func (a *collectResultAppender) AppendHistogram(ref storage.SeriesRef, l labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) {
-	a.mtx.Lock()
-	defer a.mtx.Unlock()
-	a.pendingHistograms = append(a.pendingHistograms, histogramSample{h: h, fh: fh, t: t, metric: l})
-	if a.next == nil {
-		return 0, nil
-	}
-
-	return a.next.AppendHistogram(ref, l, t, h, fh)
-}
-
-func (a *collectResultAppender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, _, st int64, h *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) {
-	if h != nil {
-		return a.AppendHistogram(ref, l, st, &histogram.Histogram{}, nil)
-	}
-	return a.AppendHistogram(ref, l, st, nil, &histogram.FloatHistogram{})
-}
-
-func (a *collectResultAppender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) {
-	a.mtx.Lock()
-	defer a.mtx.Unlock()
-	a.pendingMetadata = append(a.pendingMetadata, metadataEntry{metric: l, m: m})
-	if a.next == nil {
-		if ref == 0 {
-			ref = storage.SeriesRef(l.Hash())
-		}
-		return ref, nil
-	}
-
-	return a.next.UpdateMetadata(ref, l, m)
-}
-
-func (a *collectResultAppender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, _, st int64) (storage.SeriesRef, error) {
-	return a.Append(ref, l, st, 0.0)
-}
-
-func (a *collectResultAppender) Commit() error {
-	a.mtx.Lock()
-	defer a.mtx.Unlock()
-	a.resultFloats = append(a.resultFloats, a.pendingFloats...)
-	a.resultExemplars = append(a.resultExemplars, a.pendingExemplars...)
-	a.resultHistograms = append(a.resultHistograms, a.pendingHistograms...)
-	a.resultMetadata = append(a.resultMetadata, a.pendingMetadata...)
-	a.pendingFloats = nil
-	a.pendingExemplars = nil
-	a.pendingHistograms = nil
-	a.pendingMetadata = nil
-	if a.next == nil {
-		return nil
-	}
-	return a.next.Commit()
-}
-
-func (a *collectResultAppender) Rollback() error {
-	a.mtx.Lock()
-	defer a.mtx.Unlock()
-	a.rolledbackFloats = a.pendingFloats
-	a.rolledbackHistograms = a.pendingHistograms
-	a.pendingFloats = nil
-	a.pendingHistograms = nil
-	if a.next == nil {
-		return nil
-	}
-	return a.next.Rollback()
-}
-
-func (a *collectResultAppender) String() string {
-	var sb strings.Builder
-	for _, s := range a.resultFloats {
-		sb.WriteString(fmt.Sprintf("committed: %s %f %d\n", s.metric, s.f, s.t))
-	}
-	for _, s := range a.pendingFloats {
-		sb.WriteString(fmt.Sprintf("pending: %s %f %d\n", s.metric, s.f, s.t))
-	}
-	for _, s := range a.rolledbackFloats {
-		sb.WriteString(fmt.Sprintf("rolledback: %s %f %d\n", s.metric, s.f, s.t))
-	}
-	return sb.String()
 }
 
 // protoMarshalDelimited marshals a MetricFamily into a delimited
diff --git a/scrape/manager.go b/scrape/manager.go
index 9bb6988df9..bd68c186c0 100644
--- a/scrape/manager.go
+++ b/scrape/manager.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -39,8 +39,8 @@ import (
 	"github.com/prometheus/prometheus/util/pool"
 )
 
-// NewManager is the Manager constructor.
-func NewManager(o *Options, logger *slog.Logger, newScrapeFailureLogger func(string) (*logging.JSONFileLogger, error), app storage.Appendable, registerer prometheus.Registerer) (*Manager, error) {
+// NewManager is the Manager constructor using Appendable.
+func NewManager(o *Options, logger *slog.Logger, newScrapeFailureLogger func(string) (*logging.JSONFileLogger, error), appendable storage.Appendable, registerer prometheus.Registerer) (*Manager, error) {
 	if o == nil {
 		o = &Options{}
 	}
@@ -54,7 +54,7 @@ func NewManager(o *Options, logger *slog.Logger, newScrapeFailureLogger func(str
 	}
 
 	m := &Manager{
-		append:                 app,
+		appendable:             appendable,
 		opts:                   o,
 		logger:                 logger,
 		newScrapeFailureLogger: newScrapeFailureLogger,
@@ -87,15 +87,15 @@ type Options struct {
 	// Option to enable appending of scraped Metadata to the TSDB/other appenders. Individual appenders
 	// can decide what to do with metadata, but for practical purposes this flag exists so that metadata
 	// can be written to the WAL and thus read for remote write.
-	// TODO: implement some form of metadata storage
 	AppendMetadata bool
 	// Option to increase the interval used by scrape manager to throttle target groups updates.
 	DiscoveryReloadInterval model.Duration
+
 	// Option to enable the ingestion of the created timestamp as a synthetic zero sample.
 	// See: https://github.com/prometheus/proposals/blob/main/proposals/2023-06-13_created-timestamp.md
 	EnableStartTimestampZeroIngestion bool
 
-	// EnableTypeAndUnitLabels
+	// EnableTypeAndUnitLabels represents type-and-unit-labels feature flag.
 	EnableTypeAndUnitLabels bool
 
 	// Optional HTTP client options to use when scraping.
@@ -111,9 +111,11 @@ type Options struct {
 // Manager maintains a set of scrape pools and manages start/stop cycles
 // when receiving new target groups from the discovery manager.
 type Manager struct {
-	opts      *Options
-	logger    *slog.Logger
-	append    storage.Appendable
+	opts   *Options
+	logger *slog.Logger
+
+	appendable storage.Appendable
+
 	graceShut chan struct{}
 
 	offsetSeed             uint64     // Global offsetSeed seed is used to spread scrape workload across HA setup.
@@ -194,7 +196,7 @@ func (m *Manager) reload() {
 				continue
 			}
 			m.metrics.targetScrapePools.Inc()
-			sp, err := newScrapePool(scrapeConfig, m.append, m.offsetSeed, m.logger.With("scrape_pool", setName), m.buffers, m.opts, m.metrics)
+			sp, err := newScrapePool(scrapeConfig, m.appendable, m.offsetSeed, m.logger.With("scrape_pool", setName), m.buffers, m.opts, m.metrics)
 			if err != nil {
 				m.metrics.targetScrapePoolsFailed.Inc()
 				m.logger.Error("error creating new scrape pool", "err", err, "scrape_pool", setName)
diff --git a/scrape/manager_test.go b/scrape/manager_test.go
index 1ec4875d19..d4898eb996 100644
--- a/scrape/manager_test.go
+++ b/scrape/manager_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -51,6 +51,7 @@ import (
 	"github.com/prometheus/prometheus/storage"
 	"github.com/prometheus/prometheus/tsdb/tsdbutil"
 	"github.com/prometheus/prometheus/util/runutil"
+	"github.com/prometheus/prometheus/util/teststorage"
 	"github.com/prometheus/prometheus/util/testutil"
 )
 
@@ -527,21 +528,12 @@ scrape_configs:
 		ch <- struct{}{}
 		return noopLoop()
 	}
-	sp := &scrapePool{
-		appendable: &nopAppendable{},
-		activeTargets: map[uint64]*Target{
-			1: {},
-		},
-		loops: map[uint64]loop{
-			1: noopLoop(),
-		},
-		newLoop:     newLoop,
-		logger:      nil,
-		config:      cfg1.ScrapeConfigs[0],
-		client:      http.DefaultClient,
-		metrics:     scrapeManager.metrics,
-		symbolTable: labels.NewSymbolTable(),
-	}
+	sp := newTestScrapePool(t, newLoop)
+	sp.activeTargets[1] = &Target{}
+	sp.loops[1] = noopLoop()
+	sp.config = cfg1.ScrapeConfigs[0]
+	sp.metrics = scrapeManager.metrics
+
 	scrapeManager.scrapePools = map[string]*scrapePool{
 		"job1": sp,
 	}
@@ -691,18 +683,11 @@ scrape_configs:
 		for _, sc := range cfg.ScrapeConfigs {
 			_, cancel := context.WithCancel(context.Background())
 			defer cancel()
-			sp := &scrapePool{
-				appendable:    &nopAppendable{},
-				activeTargets: map[uint64]*Target{},
-				loops: map[uint64]loop{
-					1: noopLoop(),
-				},
-				newLoop: newLoop,
-				logger:  nil,
-				config:  sc,
-				client:  http.DefaultClient,
-				cancel:  cancel,
-			}
+
+			sp := newTestScrapePool(t, newLoop)
+			sp.loops[1] = noopLoop()
+			sp.config = cfg1.ScrapeConfigs[0]
+			sp.metrics = scrapeManager.metrics
 			for _, c := range sc.ServiceDiscoveryConfigs {
 				staticConfig := c.(discovery.StaticConfig)
 				for _, group := range staticConfig {
@@ -764,7 +749,7 @@ func TestManagerSTZeroIngestion(t *testing.T) {
 			for _, testWithST := range []bool{false, true} {
 				t.Run(fmt.Sprintf("withST=%v", testWithST), func(t *testing.T) {
 					for _, testSTZeroIngest := range []bool{false, true} {
-						t.Run(fmt.Sprintf("ctZeroIngest=%v", testSTZeroIngest), func(t *testing.T) {
+						t.Run(fmt.Sprintf("stZeroIngest=%v", testSTZeroIngest), func(t *testing.T) {
 							ctx, cancel := context.WithCancel(context.Background())
 							defer cancel()
 
@@ -777,11 +762,11 @@ func TestManagerSTZeroIngestion(t *testing.T) {
 							// TODO(bwplotka): Add more types than just counter?
 							encoded := prepareTestEncodedCounter(t, testFormat, expectedMetricName, expectedSampleValue, sampleTs, stTs)
 
-							app := &collectResultAppender{}
+							app := teststorage.NewAppendable()
 							discoveryManager, scrapeManager := runManagers(t, ctx, &Options{
 								EnableStartTimestampZeroIngestion: testSTZeroIngest,
 								skipOffsetting:                    true,
-							}, &collectResultAppendable{app})
+							}, app)
 							defer scrapeManager.Stop()
 
 							server := setupTestServer(t, config.ScrapeProtocolsHeaders[testFormat], encoded)
@@ -806,11 +791,8 @@ scrape_configs:
 							ctx, cancel = context.WithTimeout(ctx, 1*time.Minute)
 							defer cancel()
 							require.NoError(t, runutil.Retry(100*time.Millisecond, ctx.Done(), func() error {
-								app.mtx.Lock()
-								defer app.mtx.Unlock()
-
 								// Check if scrape happened and grab the relevant samples.
-								if len(app.resultFloats) > 0 {
+								if len(app.ResultSamples()) > 0 {
 									return nil
 								}
 								return errors.New("expected some float samples, got none")
@@ -818,32 +800,32 @@ scrape_configs:
 
 							// Verify results.
 							// Verify what we got vs expectations around ST injection.
-							samples := findSamplesForMetric(app.resultFloats, expectedMetricName)
+							got := findSamplesForMetric(app.ResultSamples(), expectedMetricName)
 							if testWithST && testSTZeroIngest {
-								require.Len(t, samples, 2)
-								require.Equal(t, 0.0, samples[0].f)
-								require.Equal(t, timestamp.FromTime(stTs), samples[0].t)
-								require.Equal(t, expectedSampleValue, samples[1].f)
-								require.Equal(t, timestamp.FromTime(sampleTs), samples[1].t)
+								require.Len(t, got, 2)
+								require.Equal(t, 0.0, got[0].V)
+								require.Equal(t, timestamp.FromTime(stTs), got[0].T)
+								require.Equal(t, expectedSampleValue, got[1].V)
+								require.Equal(t, timestamp.FromTime(sampleTs), got[1].T)
 							} else {
-								require.Len(t, samples, 1)
-								require.Equal(t, expectedSampleValue, samples[0].f)
-								require.Equal(t, timestamp.FromTime(sampleTs), samples[0].t)
+								require.Len(t, got, 1)
+								require.Equal(t, expectedSampleValue, got[0].V)
+								require.Equal(t, timestamp.FromTime(sampleTs), got[0].T)
 							}
 
 							// Verify what we got vs expectations around additional _created series for OM text.
 							// enableSTZeroInjection also kills that _created line.
-							createdSeriesSamples := findSamplesForMetric(app.resultFloats, expectedCreatedMetricName)
+							gotSTSeries := findSamplesForMetric(app.ResultSamples(), expectedCreatedMetricName)
 							if testFormat == config.OpenMetricsText1_0_0 && testWithST && !testSTZeroIngest {
 								// For OM Text, when counter has ST, and feature flag disabled we should see _created lines.
-								require.Len(t, createdSeriesSamples, 1)
+								require.Len(t, gotSTSeries, 1)
 								// Conversion taken from common/expfmt.writeOpenMetricsFloat.
 								// We don't check the st timestamp as explicit ts was not implemented in expfmt.Encoder,
 								// but exists in OM https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#:~:text=An%20example%20with%20a%20Metric%20with%20no%20labels%2C%20and%20a%20MetricPoint%20with%20a%20timestamp%20and%20a%20created
 								// We can implement this, but we want to potentially get rid of OM 1.0 ST lines
-								require.Equal(t, float64(timestamppb.New(stTs).AsTime().UnixNano())/1e9, createdSeriesSamples[0].f)
+								require.Equal(t, float64(timestamppb.New(stTs).AsTime().UnixNano())/1e9, gotSTSeries[0].V)
 							} else {
-								require.Empty(t, createdSeriesSamples)
+								require.Empty(t, gotSTSeries)
 							}
 						})
 					}
@@ -885,9 +867,9 @@ func prepareTestEncodedCounter(t *testing.T, format config.ScrapeProtocol, mName
 	}
 }
 
-func findSamplesForMetric(floats []floatSample, metricName string) (ret []floatSample) {
+func findSamplesForMetric(floats []sample, metricName string) (ret []sample) {
 	for _, f := range floats {
-		if f.metric.Get(model.MetricNameLabel) == metricName {
+		if f.L.Get(model.MetricNameLabel) == metricName {
 			ret = append(ret, f)
 		}
 	}
@@ -964,11 +946,11 @@ func TestManagerSTZeroIngestionHistogram(t *testing.T) {
 			ctx, cancel := context.WithCancel(context.Background())
 			defer cancel()
 
-			app := &collectResultAppender{}
+			app := teststorage.NewAppendable()
 			discoveryManager, scrapeManager := runManagers(t, ctx, &Options{
 				EnableStartTimestampZeroIngestion: tc.enableSTZeroIngestion,
 				skipOffsetting:                    true,
-			}, &collectResultAppendable{app})
+			}, app)
 			defer scrapeManager.Stop()
 
 			once := sync.Once{}
@@ -1012,43 +994,33 @@ scrape_configs:
 `, serverURL.Host)
 			applyConfig(t, testConfig, scrapeManager, discoveryManager)
 
-			var got []histogramSample
-
 			// Wait for one scrape.
 			ctx, cancel = context.WithTimeout(ctx, 1*time.Minute)
 			defer cancel()
 			require.NoError(t, runutil.Retry(100*time.Millisecond, ctx.Done(), func() error {
-				app.mtx.Lock()
-				defer app.mtx.Unlock()
-
-				// Check if scrape happened and grab the relevant histograms, they have to be there - or it's a bug
-				// and it's not worth waiting.
-				for _, h := range app.resultHistograms {
-					if h.metric.Get(model.MetricNameLabel) == mName {
-						got = append(got, h)
-					}
-				}
-				if len(app.resultHistograms) > 0 {
+				if len(app.ResultSamples()) > 0 {
 					return nil
 				}
 				return errors.New("expected some histogram samples, got none")
 			}), "after 1 minute")
 
+			got := findSamplesForMetric(app.ResultSamples(), mName)
+
 			// Check for zero samples, assuming we only injected always one histogram sample.
 			// Did it contain ST to inject? If yes, was ST zero enabled?
 			if tc.inputHistSample.CreatedTimestamp.IsValid() && tc.enableSTZeroIngestion {
 				require.Len(t, got, 2)
 				// Zero sample.
-				require.Equal(t, histogram.Histogram{}, *got[0].h)
+				require.Equal(t, histogram.Histogram{}, *got[0].H)
 				// Quick soft check to make sure it's the same sample or at least not zero.
-				require.Equal(t, tc.inputHistSample.GetSampleSum(), got[1].h.Sum)
+				require.Equal(t, tc.inputHistSample.GetSampleSum(), got[1].H.Sum)
 				return
 			}
 
 			// Expect only one, valid sample.
 			require.Len(t, got, 1)
 			// Quick soft check to make sure it's the same sample or at least not zero.
-			require.Equal(t, tc.inputHistSample.GetSampleSum(), got[0].h.Sum)
+			require.Equal(t, tc.inputHistSample.GetSampleSum(), got[0].H.Sum)
 		})
 	}
 }
@@ -1083,11 +1055,11 @@ func TestNHCBAndSTZeroIngestion(t *testing.T) {
 
 	ctx := t.Context()
 
-	app := &collectResultAppender{}
+	app := teststorage.NewAppendable()
 	discoveryManager, scrapeManager := runManagers(t, ctx, &Options{
 		EnableStartTimestampZeroIngestion: true,
 		skipOffsetting:                    true,
-	}, &collectResultAppendable{app})
+	}, app)
 	defer scrapeManager.Stop()
 
 	once := sync.Once{}
@@ -1146,33 +1118,19 @@ scrape_configs:
 		return exists
 	}, 5*time.Second, 100*time.Millisecond, "scrape pool should be created for job 'test'")
 
-	// Helper function to get matching histograms to avoid race conditions.
-	getMatchingHistograms := func() []histogramSample {
-		app.mtx.Lock()
-		defer app.mtx.Unlock()
-
-		var got []histogramSample
-		for _, h := range app.resultHistograms {
-			if h.metric.Get(model.MetricNameLabel) == mName {
-				got = append(got, h)
-			}
-		}
-		return got
-	}
-
 	require.Eventually(t, func() bool {
-		return len(getMatchingHistograms()) > 0
+		return len(app.ResultSamples()) > 0
 	}, 1*time.Minute, 100*time.Millisecond, "expected histogram samples, got none")
 
 	// Verify that samples were ingested (proving both features work together).
-	got := getMatchingHistograms()
+	got := findSamplesForMetric(app.ResultSamples(), mName)
 
 	// With ST zero ingestion enabled and a created timestamp present, we expect 2 samples:
 	// one zero sample and one actual sample.
 	require.Len(t, got, 2, "expected 2 histogram samples (zero sample + actual sample)")
-	require.Equal(t, histogram.Histogram{}, *got[0].h, "first sample should be zero sample")
-	require.InDelta(t, expectedHistogramSum, got[1].h.Sum, 1e-9, "second sample should retain the expected sum")
-	require.Len(t, app.resultExemplars, 2, "expected 2 exemplars from histogram buckets")
+	require.Equal(t, histogram.Histogram{}, *got[0].H, "first sample should be zero sample")
+	require.InDelta(t, expectedHistogramSum, got[1].H.Sum, 1e-9, "second sample should retain the expected sum")
+	require.Len(t, got[1].ES, 2, "expected 2 exemplars on second histogram")
 }
 
 func applyConfig(
@@ -1203,7 +1161,7 @@ func runManagers(t *testing.T, ctx context.Context, opts *Options, app storage.A
 	}
 	opts.DiscoveryReloadInterval = model.Duration(100 * time.Millisecond)
 	if app == nil {
-		app = nopAppendable{}
+		app = teststorage.NewAppendable()
 	}
 
 	reg := prometheus.NewRegistry()
@@ -1601,7 +1559,7 @@ scrape_configs:
 
 	cfg := loadConfiguration(t, cfgText)
 
-	m, err := NewManager(&Options{}, nil, nil, &nopAppendable{}, prometheus.NewRegistry())
+	m, err := NewManager(&Options{}, nil, nil, teststorage.NewAppendable(), prometheus.NewRegistry())
 	require.NoError(t, err)
 	defer m.Stop()
 	require.NoError(t, m.ApplyConfig(cfg))
diff --git a/scrape/scrape.go b/scrape/scrape.go
index b653873bad..6be2525fe0 100644
--- a/scrape/scrape.go
+++ b/scrape/scrape.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -59,6 +59,8 @@ import (
 	"github.com/prometheus/prometheus/util/pool"
 )
 
+var aOptionRejectEarlyOOO = storage.AppendOptions{DiscardOutOfOrder: true}
+
 // ScrapeTimestampTolerance is the tolerance for scrape appends timestamps
 // alignment, to enable better compression at the TSDB level.
 // See https://github.com/prometheus/prometheus/issues/7846
@@ -67,7 +69,7 @@ var ScrapeTimestampTolerance = 2 * time.Millisecond
 // AlignScrapeTimestamps enables the tolerance for scrape appends timestamps described above.
 var AlignScrapeTimestamps = true
 
-var errNameLabelMandatory = fmt.Errorf("missing metric name (%s label)", labels.MetricName)
+var errNameLabelMandatory = fmt.Errorf("missing metric name (%s label)", model.MetricNameLabel)
 
 var _ FailureLogger = (*logging.JSONFileLogger)(nil)
 
@@ -82,8 +84,9 @@ type FailureLogger interface {
 type scrapePool struct {
 	appendable storage.Appendable
 	logger     *slog.Logger
+	ctx        context.Context
 	cancel     context.CancelFunc
-	httpOpts   []config_util.HTTPClientOption
+	options    *Options
 
 	// mtx must not be taken after targetMtx.
 	mtx    sync.Mutex
@@ -102,16 +105,15 @@ type scrapePool struct {
 	droppedTargets      []*Target // Subject to KeepDroppedTargets limit.
 	droppedTargetsCount int       // Count of all dropped targets.
 
-	// Constructor for new scrape loops. This is settable for testing convenience.
-	newLoop func(scrapeLoopOptions) loop
+	// newLoop injection for testing purposes.
+	injectTestNewLoop func(scrapeLoopOptions) loop
 
-	metrics *scrapeMetrics
+	metrics    *scrapeMetrics
+	buffers    *pool.Pool
+	offsetSeed uint64
 
 	scrapeFailureLogger    FailureLogger
 	scrapeFailureLoggerMtx sync.RWMutex
-
-	validationScheme model.ValidationScheme
-	escapingScheme   model.EscapingScheme
 }
 
 type labelLimits struct {
@@ -120,118 +122,80 @@ type labelLimits struct {
 	labelValueLengthLimit int
 }
 
-type scrapeLoopOptions struct {
-	target                   *Target
-	scraper                  scraper
-	sampleLimit              int
-	bucketLimit              int
-	maxSchema                int32
-	labelLimits              *labelLimits
-	honorLabels              bool
-	honorTimestamps          bool
-	trackTimestampsStaleness bool
-	interval                 time.Duration
-	timeout                  time.Duration
-	scrapeNativeHist         bool
-	alwaysScrapeClassicHist  bool
-	convertClassicHistToNHCB bool
-	fallbackScrapeProtocol   string
-
-	mrc               []*relabel.Config
-	cache             *scrapeCache
-	enableCompression bool
-}
-
 const maxAheadTime = 10 * time.Minute
 
 // returning an empty label set is interpreted as "drop".
 type labelsMutator func(labels.Labels) labels.Labels
 
-func newScrapePool(cfg *config.ScrapeConfig, app storage.Appendable, offsetSeed uint64, logger *slog.Logger, buffers *pool.Pool, options *Options, metrics *scrapeMetrics) (*scrapePool, error) {
+// scrapeLoopAppendAdapter allows support for multiple storage.Appender versions.
+type scrapeLoopAppendAdapter interface {
+	Commit() error
+	Rollback() error
+
+	addReportSample(s reportSample, t int64, v float64, b *labels.Builder, rejectOOO bool) error
+	append(b []byte, contentType string, ts time.Time) (total, added, seriesAdded int, err error)
+}
+
+func newScrapePool(
+	cfg *config.ScrapeConfig,
+	appendable storage.Appendable,
+	offsetSeed uint64,
+	logger *slog.Logger,
+	buffers *pool.Pool,
+	options *Options,
+	metrics *scrapeMetrics,
+) (*scrapePool, error) {
 	if logger == nil {
 		logger = promslog.NewNopLogger()
 	}
+	if buffers == nil {
+		buffers = pool.New(1e3, 1e6, 3, func(sz int) any { return make([]byte, 0, sz) })
+	}
 
 	client, err := newScrapeClient(cfg.HTTPClientConfig, cfg.JobName, options.HTTPClientOptions...)
 	if err != nil {
 		return nil, err
 	}
 
+	// Validate scheme so we don't need to do it later.
+	// We also do it on scrapePool.reload(...)
+	// TODO(bwplotka): Can we move it to scrape config validation?
 	if err := namevalidationutil.CheckNameValidationScheme(cfg.MetricNameValidationScheme); err != nil {
 		return nil, errors.New("newScrapePool: MetricNameValidationScheme must be set in scrape configuration")
 	}
-	var escapingScheme model.EscapingScheme
-	escapingScheme, err = config.ToEscapingScheme(cfg.MetricNameEscapingScheme, cfg.MetricNameValidationScheme)
-	if err != nil {
+	if _, err = config.ToEscapingScheme(cfg.MetricNameEscapingScheme, cfg.MetricNameValidationScheme); err != nil {
 		return nil, fmt.Errorf("invalid metric name escaping scheme, %w", err)
 	}
 
+	symbols := labels.NewSymbolTable()
 	ctx, cancel := context.WithCancel(context.Background())
 	sp := &scrapePool{
+		appendable:           appendable,
+		logger:               logger,
+		ctx:                  ctx,
 		cancel:               cancel,
-		appendable:           app,
+		options:              options,
 		config:               cfg,
 		client:               client,
-		activeTargets:        map[uint64]*Target{},
 		loops:                map[uint64]loop{},
-		symbolTable:          labels.NewSymbolTable(),
+		symbolTable:          symbols,
 		lastSymbolTableCheck: time.Now(),
-		logger:               logger,
+		activeTargets:        map[uint64]*Target{},
 		metrics:              metrics,
-		httpOpts:             options.HTTPClientOptions,
-		validationScheme:     cfg.MetricNameValidationScheme,
-		escapingScheme:       escapingScheme,
-	}
-	sp.newLoop = func(opts scrapeLoopOptions) loop {
-		// Update the targets retrieval function for metadata to a new scrape cache.
-		cache := opts.cache
-		if cache == nil {
-			cache = newScrapeCache(metrics)
-		}
-		opts.target.SetMetadataStore(cache)
-
-		return newScrapeLoop(
-			ctx,
-			opts.scraper,
-			logger.With("target", opts.target),
-			buffers,
-			func(l labels.Labels) labels.Labels {
-				return mutateSampleLabels(l, opts.target, opts.honorLabels, opts.mrc)
-			},
-			func(l labels.Labels) labels.Labels { return mutateReportSampleLabels(l, opts.target) },
-			func(ctx context.Context) storage.Appender { return app.Appender(ctx) },
-			cache,
-			sp.symbolTable,
-			offsetSeed,
-			opts.honorTimestamps,
-			opts.trackTimestampsStaleness,
-			opts.enableCompression,
-			opts.sampleLimit,
-			opts.bucketLimit,
-			opts.maxSchema,
-			opts.labelLimits,
-			opts.interval,
-			opts.timeout,
-			opts.alwaysScrapeClassicHist,
-			opts.convertClassicHistToNHCB,
-			cfg.ScrapeNativeHistogramsEnabled(),
-			options.EnableStartTimestampZeroIngestion,
-			options.EnableTypeAndUnitLabels,
-			options.ExtraMetrics,
-			options.AppendMetadata,
-			opts.target,
-			options.PassMetadataInContext,
-			metrics,
-			options.skipOffsetting,
-			sp.validationScheme,
-			sp.escapingScheme,
-			opts.fallbackScrapeProtocol,
-		)
+		buffers:              buffers,
+		offsetSeed:           offsetSeed,
 	}
 	sp.metrics.targetScrapePoolTargetLimit.WithLabelValues(sp.config.JobName).Set(float64(sp.config.TargetLimit))
 	return sp, nil
 }
 
+func (sp *scrapePool) newLoop(opts scrapeLoopOptions) loop {
+	if sp.injectTestNewLoop != nil {
+		return sp.injectTestNewLoop(opts)
+	}
+	return newScrapeLoop(opts)
+}
+
 func (sp *scrapePool) ActiveTargets() []*Target {
 	sp.targetMtx.Lock()
 	defer sp.targetMtx.Unlock()
@@ -323,7 +287,7 @@ func (sp *scrapePool) reload(cfg *config.ScrapeConfig) error {
 	sp.metrics.targetScrapePoolReloads.Inc()
 	start := time.Now()
 
-	client, err := newScrapeClient(cfg.HTTPClientConfig, cfg.JobName, sp.httpOpts...)
+	client, err := newScrapeClient(cfg.HTTPClientConfig, cfg.JobName, sp.options.HTTPClientOptions...)
 	if err != nil {
 		sp.metrics.targetScrapePoolReloadsFailed.Inc()
 		return err
@@ -333,17 +297,14 @@ func (sp *scrapePool) reload(cfg *config.ScrapeConfig) error {
 	sp.config = cfg
 	oldClient := sp.client
 	sp.client = client
+
+	// Validate scheme so we don't need to do it later.
 	if err := namevalidationutil.CheckNameValidationScheme(cfg.MetricNameValidationScheme); err != nil {
 		return errors.New("scrapePool.reload: MetricNameValidationScheme must be set in scrape configuration")
 	}
-	sp.validationScheme = cfg.MetricNameValidationScheme
-	var escapingScheme model.EscapingScheme
-	escapingScheme, err = model.ToEscapingScheme(cfg.MetricNameEscapingScheme)
-	if err != nil {
-		return fmt.Errorf("invalid metric name escaping scheme, %w", err)
+	if _, err = config.ToEscapingScheme(cfg.MetricNameEscapingScheme, cfg.MetricNameValidationScheme); err != nil {
+		return fmt.Errorf("scrapePool.reload: invalid metric name escaping scheme, %w", err)
 	}
-	sp.escapingScheme = escapingScheme
-
 	sp.metrics.targetScrapePoolTargetLimit.WithLabelValues(sp.config.JobName).Set(float64(sp.config.TargetLimit))
 
 	sp.restartLoops(reuseCache)
@@ -355,30 +316,7 @@ func (sp *scrapePool) reload(cfg *config.ScrapeConfig) error {
 }
 
 func (sp *scrapePool) restartLoops(reuseCache bool) {
-	var (
-		wg            sync.WaitGroup
-		interval      = time.Duration(sp.config.ScrapeInterval)
-		timeout       = time.Duration(sp.config.ScrapeTimeout)
-		bodySizeLimit = int64(sp.config.BodySizeLimit)
-		sampleLimit   = int(sp.config.SampleLimit)
-		bucketLimit   = int(sp.config.NativeHistogramBucketLimit)
-		maxSchema     = pickSchema(sp.config.NativeHistogramMinBucketFactor)
-		labelLimits   = &labelLimits{
-			labelLimit:            int(sp.config.LabelLimit),
-			labelNameLengthLimit:  int(sp.config.LabelNameLengthLimit),
-			labelValueLengthLimit: int(sp.config.LabelValueLengthLimit),
-		}
-		honorLabels              = sp.config.HonorLabels
-		honorTimestamps          = sp.config.HonorTimestamps
-		enableCompression        = sp.config.EnableCompression
-		trackTimestampsStaleness = sp.config.TrackTimestampsStaleness
-		mrc                      = sp.config.MetricRelabelConfigs
-		fallbackScrapeProtocol   = sp.config.ScrapeFallbackProtocol.HeaderMediaType()
-		scrapeNativeHist         = sp.config.ScrapeNativeHistogramsEnabled()
-		alwaysScrapeClassicHist  = sp.config.AlwaysScrapeClassicHistogramsEnabled()
-		convertClassicHistToNHCB = sp.config.ConvertClassicHistogramsToNHCBEnabled()
-	)
-
+	var wg sync.WaitGroup
 	sp.targetMtx.Lock()
 
 	forcedErr := sp.refreshTargetLimitErr()
@@ -392,38 +330,27 @@ func (sp *scrapePool) restartLoops(reuseCache bool) {
 		}
 
 		t := sp.activeTargets[fp]
-		targetInterval, targetTimeout, err := t.intervalAndTimeout(interval, timeout)
-		var (
-			s = &targetScraper{
+		targetInterval, targetTimeout, err := t.intervalAndTimeout(
+			time.Duration(sp.config.ScrapeInterval),
+			time.Duration(sp.config.ScrapeTimeout),
+		)
+		escapingScheme, _ := config.ToEscapingScheme(sp.config.MetricNameEscapingScheme, sp.config.MetricNameValidationScheme)
+		newLoop := sp.newLoop(scrapeLoopOptions{
+			target: t,
+			scraper: &targetScraper{
 				Target:               t,
 				client:               sp.client,
 				timeout:              targetTimeout,
-				bodySizeLimit:        bodySizeLimit,
-				acceptHeader:         acceptHeader(sp.config.ScrapeProtocols, sp.escapingScheme),
-				acceptEncodingHeader: acceptEncodingHeader(enableCompression),
+				bodySizeLimit:        int64(sp.config.BodySizeLimit),
+				acceptHeader:         acceptHeader(sp.config.ScrapeProtocols, escapingScheme),
+				acceptEncodingHeader: acceptEncodingHeader(sp.config.EnableCompression),
 				metrics:              sp.metrics,
-			}
-			newLoop = sp.newLoop(scrapeLoopOptions{
-				target:                   t,
-				scraper:                  s,
-				sampleLimit:              sampleLimit,
-				bucketLimit:              bucketLimit,
-				maxSchema:                maxSchema,
-				labelLimits:              labelLimits,
-				honorLabels:              honorLabels,
-				honorTimestamps:          honorTimestamps,
-				enableCompression:        enableCompression,
-				trackTimestampsStaleness: trackTimestampsStaleness,
-				mrc:                      mrc,
-				cache:                    cache,
-				interval:                 targetInterval,
-				timeout:                  targetTimeout,
-				fallbackScrapeProtocol:   fallbackScrapeProtocol,
-				scrapeNativeHist:         scrapeNativeHist,
-				alwaysScrapeClassicHist:  alwaysScrapeClassicHist,
-				convertClassicHistToNHCB: convertClassicHistToNHCB,
-			})
-		)
+			},
+			cache:    cache,
+			interval: targetInterval,
+			timeout:  targetTimeout,
+			sp:       sp,
+		})
 		if err != nil {
 			newLoop.setForcedError(err)
 		}
@@ -516,31 +443,10 @@ func (sp *scrapePool) Sync(tgs []*targetgroup.Group) {
 // scrape loops for new targets, and stops scrape loops for disappeared targets.
 // It returns after all stopped scrape loops terminated.
 func (sp *scrapePool) sync(targets []*Target) {
-	var (
-		uniqueLoops   = make(map[uint64]loop)
-		interval      = time.Duration(sp.config.ScrapeInterval)
-		timeout       = time.Duration(sp.config.ScrapeTimeout)
-		bodySizeLimit = int64(sp.config.BodySizeLimit)
-		sampleLimit   = int(sp.config.SampleLimit)
-		bucketLimit   = int(sp.config.NativeHistogramBucketLimit)
-		maxSchema     = pickSchema(sp.config.NativeHistogramMinBucketFactor)
-		labelLimits   = &labelLimits{
-			labelLimit:            int(sp.config.LabelLimit),
-			labelNameLengthLimit:  int(sp.config.LabelNameLengthLimit),
-			labelValueLengthLimit: int(sp.config.LabelValueLengthLimit),
-		}
-		honorLabels              = sp.config.HonorLabels
-		honorTimestamps          = sp.config.HonorTimestamps
-		enableCompression        = sp.config.EnableCompression
-		trackTimestampsStaleness = sp.config.TrackTimestampsStaleness
-		mrc                      = sp.config.MetricRelabelConfigs
-		fallbackScrapeProtocol   = sp.config.ScrapeFallbackProtocol.HeaderMediaType()
-		scrapeNativeHist         = sp.config.ScrapeNativeHistogramsEnabled()
-		alwaysScrapeClassicHist  = sp.config.AlwaysScrapeClassicHistogramsEnabled()
-		convertClassicHistToNHCB = sp.config.ConvertClassicHistogramsToNHCBEnabled()
-	)
+	uniqueLoops := make(map[uint64]loop)
 
 	sp.targetMtx.Lock()
+	escapingScheme, _ := config.ToEscapingScheme(sp.config.MetricNameEscapingScheme, sp.config.MetricNameValidationScheme)
 	for _, t := range targets {
 		hash := t.hash()
 
@@ -549,34 +455,25 @@ func (sp *scrapePool) sync(targets []*Target) {
 			// so whether changed via relabeling or not, they'll exist and hold the correct values
 			// for every target.
 			var err error
-			interval, timeout, err = t.intervalAndTimeout(interval, timeout)
-			s := &targetScraper{
-				Target:               t,
-				client:               sp.client,
-				timeout:              timeout,
-				bodySizeLimit:        bodySizeLimit,
-				acceptHeader:         acceptHeader(sp.config.ScrapeProtocols, sp.escapingScheme),
-				acceptEncodingHeader: acceptEncodingHeader(enableCompression),
-				metrics:              sp.metrics,
-			}
+			targetInterval, targetTimeout, err := t.intervalAndTimeout(
+				time.Duration(sp.config.ScrapeInterval),
+				time.Duration(sp.config.ScrapeTimeout),
+			)
 			l := sp.newLoop(scrapeLoopOptions{
-				target:                   t,
-				scraper:                  s,
-				sampleLimit:              sampleLimit,
-				bucketLimit:              bucketLimit,
-				maxSchema:                maxSchema,
-				labelLimits:              labelLimits,
-				honorLabels:              honorLabels,
-				honorTimestamps:          honorTimestamps,
-				enableCompression:        enableCompression,
-				trackTimestampsStaleness: trackTimestampsStaleness,
-				mrc:                      mrc,
-				interval:                 interval,
-				timeout:                  timeout,
-				scrapeNativeHist:         scrapeNativeHist,
-				alwaysScrapeClassicHist:  alwaysScrapeClassicHist,
-				convertClassicHistToNHCB: convertClassicHistToNHCB,
-				fallbackScrapeProtocol:   fallbackScrapeProtocol,
+				target: t,
+				scraper: &targetScraper{
+					Target:               t,
+					client:               sp.client,
+					timeout:              targetTimeout,
+					bodySizeLimit:        int64(sp.config.BodySizeLimit),
+					acceptHeader:         acceptHeader(sp.config.ScrapeProtocols, escapingScheme),
+					acceptEncodingHeader: acceptEncodingHeader(sp.config.EnableCompression),
+					metrics:              sp.metrics,
+				},
+				cache:    newScrapeCache(sp.metrics),
+				interval: targetInterval,
+				timeout:  targetTimeout,
+				sp:       sp,
 			})
 			if err != nil {
 				l.setForcedError(err)
@@ -661,7 +558,7 @@ func verifyLabelLimits(lset labels.Labels, limits *labelLimits) error {
 		return nil
 	}
 
-	met := lset.Get(labels.MetricName)
+	met := lset.Get(model.MetricNameLabel)
 	if limits.labelLimit > 0 {
 		nbLabels := lset.Len()
 		if nbLabels > limits.labelLimit {
@@ -749,8 +646,8 @@ func mutateReportSampleLabels(lset labels.Labels, target *Target) labels.Labels
 	return lb.Labels()
 }
 
-// appender returns an appender for ingested samples from the target.
-func appender(app storage.Appender, sampleLimit, bucketLimit int, maxSchema int32) storage.Appender {
+// appenderWithLimits returns an appender with additional validation.
+func appenderWithLimits(app storage.Appender, sampleLimit, bucketLimit int, maxSchema int32) storage.Appender {
 	app = &timeLimitAppender{
 		Appender: app,
 		maxTime:  timestamp.FromTime(time.Now().Add(maxAheadTime)),
@@ -927,55 +824,63 @@ type cacheEntry struct {
 }
 
 type scrapeLoop struct {
-	scraper                  scraper
-	l                        *slog.Logger
-	scrapeFailureLogger      FailureLogger
-	scrapeFailureLoggerMtx   sync.RWMutex
-	cache                    *scrapeCache
-	lastScrapeSize           int
-	buffers                  *pool.Pool
-	offsetSeed               uint64
-	honorTimestamps          bool
-	trackTimestampsStaleness bool
-	enableCompression        bool
-	forcedErr                error
-	forcedErrMtx             sync.Mutex
-	sampleLimit              int
-	bucketLimit              int
-	maxSchema                int32
-	labelLimits              *labelLimits
-	interval                 time.Duration
-	timeout                  time.Duration
-	validationScheme         model.ValidationScheme
-	escapingScheme           model.EscapingScheme
-
-	alwaysScrapeClassicHist  bool
-	convertClassicHistToNHCB bool
-	enableSTZeroIngestion    bool
-	enableTypeAndUnitLabels  bool
-	fallbackScrapeProtocol   string
-
-	enableNativeHistogramScraping bool
-
-	appender            func(ctx context.Context) storage.Appender
-	symbolTable         *labels.SymbolTable
-	sampleMutator       labelsMutator
-	reportSampleMutator labelsMutator
-
-	parentCtx   context.Context
-	appenderCtx context.Context
+	// Parameters.
 	ctx         context.Context
 	cancel      func()
 	stopped     chan struct{}
+	parentCtx   context.Context
+	appenderCtx context.Context
+	l           *slog.Logger
+	cache       *scrapeCache
 
+	interval            time.Duration
+	timeout             time.Duration
+	sampleMutator       labelsMutator
+	reportSampleMutator labelsMutator
+	scraper             scraper
+
+	// Static params per scrapePool.
+	appendable  storage.Appendable
+	buffers     *pool.Pool
+	offsetSeed  uint64
+	symbolTable *labels.SymbolTable
+	metrics     *scrapeMetrics
+
+	// Options from config.ScrapeConfig.
+	sampleLimit                   int
+	bucketLimit                   int
+	maxSchema                     int32
+	labelLimits                   *labelLimits
+	honorLabels                   bool
+	honorTimestamps               bool
+	trackTimestampsStaleness      bool
+	enableNativeHistogramScraping bool
+	alwaysScrapeClassicHist       bool
+	convertClassicHistToNHCB      bool
+	fallbackScrapeProtocol        string
+	enableCompression             bool
+	mrc                           []*relabel.Config
+	validationScheme              model.ValidationScheme
+
+	// Options from scrape.Options.
+	enableSTZeroIngestion   bool
+	enableTypeAndUnitLabels bool
+	reportExtraMetrics      bool
+	appendMetadataToWAL     bool
+	passMetadataInContext   bool
+	skipOffsetting          bool // For testability.
+
+	// error injection through setForcedError.
+	forcedErr    error
+	forcedErrMtx sync.Mutex
+
+	// Special logger set on setScrapeFailureLogger
+	scrapeFailureLoggerMtx sync.RWMutex
+	scrapeFailureLogger    FailureLogger
+
+	// Locally cached data.
+	lastScrapeSize                   int
 	disabledEndOfRunStalenessMarkers atomic.Bool
-
-	reportExtraMetrics  bool
-	appendMetadataToWAL bool
-
-	metrics *scrapeMetrics
-
-	skipOffsetting bool // For testability.
 }
 
 // scrapeCache tracks mappings of exposed metric strings to label sets and
@@ -1000,8 +905,8 @@ type scrapeCache struct {
 	seriesCur  map[storage.SeriesRef]*cacheEntry
 	seriesPrev map[storage.SeriesRef]*cacheEntry
 
-	// TODO(bwplotka): Consider moving Metadata API to use WAL instead of scrape loop to
-	// avoid locking (using metadata API can block scraping).
+	// TODO(bwplotka): Consider moving metadata caching to head. See
+	// https://github.com/prometheus/prometheus/issues/17619.
 	metaMtx  sync.Mutex            // Mutex is needed due to api touching it when metadata is queried.
 	metadata map[string]*metaEntry // metadata by metric family name.
 
@@ -1236,99 +1141,87 @@ func (c *scrapeCache) LengthMetadata() int {
 	return len(c.metadata)
 }
 
-func newScrapeLoop(ctx context.Context,
-	sc scraper,
-	l *slog.Logger,
-	buffers *pool.Pool,
-	sampleMutator labelsMutator,
-	reportSampleMutator labelsMutator,
-	appender func(ctx context.Context) storage.Appender,
-	cache *scrapeCache,
-	symbolTable *labels.SymbolTable,
-	offsetSeed uint64,
-	honorTimestamps bool,
-	trackTimestampsStaleness bool,
-	enableCompression bool,
-	sampleLimit int,
-	bucketLimit int,
-	maxSchema int32,
-	labelLimits *labelLimits,
-	interval time.Duration,
-	timeout time.Duration,
-	alwaysScrapeClassicHist bool,
-	convertClassicHistToNHCB bool,
-	enableNativeHistogramScraping bool,
-	enableSTZeroIngestion bool,
-	enableTypeAndUnitLabels bool,
-	reportExtraMetrics bool,
-	appendMetadataToWAL bool,
-	target *Target,
-	passMetadataInContext bool,
-	metrics *scrapeMetrics,
-	skipOffsetting bool,
-	validationScheme model.ValidationScheme,
-	escapingScheme model.EscapingScheme,
-	fallbackScrapeProtocol string,
-) *scrapeLoop {
-	if l == nil {
-		l = promslog.NewNopLogger()
-	}
-	if buffers == nil {
-		buffers = pool.New(1e3, 1e6, 3, func(sz int) any { return make([]byte, 0, sz) })
-	}
-	if cache == nil {
-		cache = newScrapeCache(metrics)
-	}
+// scrapeLoopOptions contains static options that do not change per scrapePool lifecycle.
+type scrapeLoopOptions struct {
+	target            *Target
+	scraper           scraper
+	cache             *scrapeCache
+	interval, timeout time.Duration
 
-	appenderCtx := ctx
+	sp *scrapePool
+}
 
-	if passMetadataInContext {
+// newScrapeLoop constructs new scrapeLoop.
+// NOTE: Technically this could be a scrapePool method, but it's a standalone function to make it clear scrapeLoop
+// can be used outside scrapePool lifecycle (e.g. in tests).
+func newScrapeLoop(opts scrapeLoopOptions) *scrapeLoop {
+	// Update the targets retrieval function for metadata to a new target.
+	opts.target.SetMetadataStore(opts.cache)
+
+	appenderCtx := opts.sp.ctx
+	if opts.sp.options.PassMetadataInContext {
 		// Store the cache and target in the context. This is then used by downstream OTel Collector
 		// to lookup the metadata required to process the samples. Not used by Prometheus itself.
 		// TODO(gouthamve) We're using a dedicated context because using the parentCtx caused a memory
 		// leak. We should ideally fix the main leak. See: https://github.com/prometheus/prometheus/pull/10590
-		appenderCtx = ContextWithMetricMetadataStore(appenderCtx, cache)
-		appenderCtx = ContextWithTarget(appenderCtx, target)
+		// TODO(bwplotka): Remove once OpenTelemetry collector uses AppenderV2 (add issue)
+		appenderCtx = ContextWithMetricMetadataStore(appenderCtx, opts.cache)
+		appenderCtx = ContextWithTarget(appenderCtx, opts.target)
 	}
 
-	sl := &scrapeLoop{
-		scraper:                       sc,
-		buffers:                       buffers,
-		cache:                         cache,
-		appender:                      appender,
-		symbolTable:                   symbolTable,
-		sampleMutator:                 sampleMutator,
-		reportSampleMutator:           reportSampleMutator,
-		stopped:                       make(chan struct{}),
-		offsetSeed:                    offsetSeed,
-		l:                             l,
-		parentCtx:                     ctx,
-		appenderCtx:                   appenderCtx,
-		honorTimestamps:               honorTimestamps,
-		trackTimestampsStaleness:      trackTimestampsStaleness,
-		enableCompression:             enableCompression,
-		sampleLimit:                   sampleLimit,
-		bucketLimit:                   bucketLimit,
-		maxSchema:                     maxSchema,
-		labelLimits:                   labelLimits,
-		interval:                      interval,
-		timeout:                       timeout,
-		alwaysScrapeClassicHist:       alwaysScrapeClassicHist,
-		convertClassicHistToNHCB:      convertClassicHistToNHCB,
-		enableSTZeroIngestion:         enableSTZeroIngestion,
-		enableTypeAndUnitLabels:       enableTypeAndUnitLabels,
-		fallbackScrapeProtocol:        fallbackScrapeProtocol,
-		enableNativeHistogramScraping: enableNativeHistogramScraping,
-		reportExtraMetrics:            reportExtraMetrics,
-		appendMetadataToWAL:           appendMetadataToWAL,
-		metrics:                       metrics,
-		skipOffsetting:                skipOffsetting,
-		validationScheme:              validationScheme,
-		escapingScheme:                escapingScheme,
-	}
-	sl.ctx, sl.cancel = context.WithCancel(ctx)
+	ctx, cancel := context.WithCancel(opts.sp.ctx)
+	return &scrapeLoop{
+		ctx:         ctx,
+		cancel:      cancel,
+		stopped:     make(chan struct{}),
+		parentCtx:   opts.sp.ctx,
+		appenderCtx: appenderCtx,
+		l:           opts.sp.logger.With("target", opts.target),
+		cache:       opts.cache,
 
-	return sl
+		interval: opts.interval,
+		timeout:  opts.timeout,
+		sampleMutator: func(l labels.Labels) labels.Labels {
+			return mutateSampleLabels(l, opts.target, opts.sp.config.HonorTimestamps, opts.sp.config.MetricRelabelConfigs)
+		},
+		reportSampleMutator: func(l labels.Labels) labels.Labels { return mutateReportSampleLabels(l, opts.target) },
+		scraper:             opts.scraper,
+
+		// Static params per scrapePool.
+		appendable:  opts.sp.appendable,
+		buffers:     opts.sp.buffers,
+		offsetSeed:  opts.sp.offsetSeed,
+		symbolTable: opts.sp.symbolTable,
+		metrics:     opts.sp.metrics,
+
+		// config.ScrapeConfig.
+		sampleLimit: int(opts.sp.config.SampleLimit),
+		bucketLimit: int(opts.sp.config.NativeHistogramBucketLimit),
+		maxSchema:   pickSchema(opts.sp.config.NativeHistogramMinBucketFactor),
+		labelLimits: &labelLimits{
+			labelLimit:            int(opts.sp.config.LabelLimit),
+			labelNameLengthLimit:  int(opts.sp.config.LabelNameLengthLimit),
+			labelValueLengthLimit: int(opts.sp.config.LabelValueLengthLimit),
+		},
+		honorLabels:                   opts.sp.config.HonorLabels,
+		honorTimestamps:               opts.sp.config.HonorTimestamps,
+		trackTimestampsStaleness:      opts.sp.config.TrackTimestampsStaleness,
+		enableNativeHistogramScraping: opts.sp.config.ScrapeNativeHistogramsEnabled(),
+		alwaysScrapeClassicHist:       opts.sp.config.AlwaysScrapeClassicHistogramsEnabled(),
+		convertClassicHistToNHCB:      opts.sp.config.ConvertClassicHistogramsToNHCBEnabled(),
+		fallbackScrapeProtocol:        opts.sp.config.ScrapeFallbackProtocol.HeaderMediaType(),
+		enableCompression:             opts.sp.config.EnableCompression,
+		mrc:                           opts.sp.config.MetricRelabelConfigs,
+		validationScheme:              opts.sp.config.MetricNameValidationScheme,
+
+		// scrape.Options.
+		enableSTZeroIngestion:   opts.sp.options.EnableStartTimestampZeroIngestion,
+		enableTypeAndUnitLabels: opts.sp.options.EnableTypeAndUnitLabels,
+		reportExtraMetrics:      opts.sp.options.ExtraMetrics,
+		appendMetadataToWAL:     opts.sp.options.AppendMetadata,
+		passMetadataInContext:   opts.sp.options.PassMetadataInContext,
+		skipOffsetting:          opts.sp.options.skipOffsetting,
+	}
 }
 
 func (sl *scrapeLoop) setScrapeFailureLogger(l FailureLogger) {
@@ -1407,6 +1300,11 @@ mainLoop:
 	}
 }
 
+func (sl *scrapeLoop) appender() scrapeLoopAppendAdapter {
+	// NOTE(bwplotka): Add AppenderV2 implementation, see https://github.com/prometheus/prometheus/issues/17632.
+	return &scrapeLoopAppender{scrapeLoop: sl, Appender: sl.appendable.Appender(sl.appenderCtx)}
+}
+
 // scrapeAndReport performs a scrape and then appends the result to the storage
 // together with reporting metrics, by using as few appenders as possible.
 // In the happy scenario, a single appender is used.
@@ -1428,10 +1326,10 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er
 	var total, added, seriesAdded, bytesRead int
 	var err, appErr, scrapeErr error
 
-	app := sl.appender(sl.appenderCtx)
+	app := sl.appender()
 	defer func() {
 		if err != nil {
-			app.Rollback()
+			_ = app.Rollback()
 			return
 		}
 		err = app.Commit()
@@ -1449,9 +1347,9 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er
 	if forcedErr := sl.getForcedError(); forcedErr != nil {
 		scrapeErr = forcedErr
 		// Add stale markers.
-		if _, _, _, err := sl.append(app, []byte{}, "", appendTime); err != nil {
-			app.Rollback()
-			app = sl.appender(sl.appenderCtx)
+		if _, _, _, err := app.append([]byte{}, "", appendTime); err != nil {
+			_ = app.Rollback()
+			app = sl.appender()
 			sl.l.Warn("Append failed", "err", err)
 		}
 		if errc != nil {
@@ -1507,16 +1405,16 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er
 
 	// A failed scrape is the same as an empty scrape,
 	// we still call sl.append to trigger stale markers.
-	total, added, seriesAdded, appErr = sl.append(app, b, contentType, appendTime)
+	total, added, seriesAdded, appErr = app.append(b, contentType, appendTime)
 	if appErr != nil {
-		app.Rollback()
-		app = sl.appender(sl.appenderCtx)
+		_ = app.Rollback()
+		app = sl.appender()
 		sl.l.Debug("Append failed", "err", appErr)
 		// The append failed, probably due to a parse error or sample limit.
 		// Call sl.append again with an empty scrape to trigger stale markers.
-		if _, _, _, err := sl.append(app, []byte{}, "", appendTime); err != nil {
-			app.Rollback()
-			app = sl.appender(sl.appenderCtx)
+		if _, _, _, err := app.append([]byte{}, "", appendTime); err != nil {
+			_ = app.Rollback()
+			app = sl.appender()
 			sl.l.Warn("Append failed", "err", err)
 		}
 	}
@@ -1586,11 +1484,11 @@ func (sl *scrapeLoop) endOfRunStaleness(last time.Time, ticker *time.Ticker, int
 	// If the target has since been recreated and scraped, the
 	// stale markers will be out of order and ignored.
 	// sl.context would have been cancelled, hence using sl.appenderCtx.
-	app := sl.appender(sl.appenderCtx)
+	app := sl.appender()
 	var err error
 	defer func() {
 		if err != nil {
-			app.Rollback()
+			_ = app.Rollback()
 			return
 		}
 		err = app.Commit()
@@ -1598,9 +1496,9 @@ func (sl *scrapeLoop) endOfRunStaleness(last time.Time, ticker *time.Ticker, int
 			sl.l.Warn("Stale commit failed", "err", err)
 		}
 	}()
-	if _, _, _, err = sl.append(app, []byte{}, "", staleTime); err != nil {
-		app.Rollback()
-		app = sl.appender(sl.appenderCtx)
+	if _, _, _, err = app.append([]byte{}, "", staleTime); err != nil {
+		_ = app.Rollback()
+		app = sl.appender()
 		sl.l.Warn("Stale append failed", "err", err)
 	}
 	if err = sl.reportStale(app, staleTime); err != nil {
@@ -1634,7 +1532,7 @@ type appendErrors struct {
 func (sl *scrapeLoop) updateStaleMarkers(app storage.Appender, defTime int64) (err error) {
 	sl.cache.forEachStale(func(ref storage.SeriesRef, lset labels.Labels) bool {
 		// Series no longer exposed, mark it stale.
-		app.SetOptions(&storage.AppendOptions{DiscardOutOfOrder: true})
+		app.SetOptions(&aOptionRejectEarlyOOO)
 		_, err = app.Append(ref, lset, defTime, math.Float64frombits(value.StaleNaN))
 		app.SetOptions(nil)
 		switch {
@@ -1648,12 +1546,20 @@ func (sl *scrapeLoop) updateStaleMarkers(app storage.Appender, defTime int64) (e
 	return err
 }
 
-func (sl *scrapeLoop) append(app storage.Appender, b []byte, contentType string, ts time.Time) (total, added, seriesAdded int, err error) {
+type scrapeLoopAppender struct {
+	*scrapeLoop
+
+	storage.Appender
+}
+
+var _ scrapeLoopAppendAdapter = &scrapeLoopAppender{}
+
+func (sl *scrapeLoopAppender) append(b []byte, contentType string, ts time.Time) (total, added, seriesAdded int, err error) {
 	defTime := timestamp.FromTime(ts)
 
 	if len(b) == 0 {
 		// Empty scrape. Just update the stale makers and swap the cache (but don't flush it).
-		err = sl.updateStaleMarkers(app, defTime)
+		err = sl.updateStaleMarkers(sl.Appender, defTime)
 		sl.cache.iterDone(false)
 		return total, added, seriesAdded, err
 	}
@@ -1696,7 +1602,7 @@ func (sl *scrapeLoop) append(app storage.Appender, b []byte, contentType string,
 	exemplars := make([]exemplar.Exemplar, 0, 1)
 
 	// Take an appender with limits.
-	app = appender(app, sl.sampleLimit, sl.bucketLimit, sl.maxSchema)
+	app := appenderWithLimits(sl.Appender, sl.sampleLimit, sl.bucketLimit, sl.maxSchema)
 
 	defer func() {
 		if err != nil {
@@ -1785,7 +1691,7 @@ loop:
 				continue
 			}
 
-			if !lset.Has(labels.MetricName) {
+			if !lset.Has(model.MetricNameLabel) {
 				err = errNameLabelMandatory
 				break loop
 			}
@@ -1859,7 +1765,7 @@ loop:
 				// But make sure we only do this if we have a cache entry (ce) for our series.
 				sl.cache.trackStaleness(ref, ce)
 			}
-			if sampleAdded && sampleLimitErr == nil && bucketLimitErr == nil {
+			if sampleLimitErr == nil && bucketLimitErr == nil {
 				seriesAdded++
 			}
 		}
@@ -1917,7 +1823,7 @@ loop:
 				// In majority cases we can trust that the current series/histogram is matching the lastMeta and lastMFName.
 				// However, optional TYPE etc metadata and broken OM text can break this, detect those cases here.
 				// TODO(bwplotka): Consider moving this to parser as many parser users end up doing this (e.g. ST and NHCB parsing).
-				if isSeriesPartOfFamily(lset.Get(labels.MetricName), lastMFName, lastMeta.Type) {
+				if isSeriesPartOfFamily(lset.Get(model.MetricNameLabel), lastMFName, lastMeta.Type) {
 					if _, merr := app.UpdateMetadata(ref, lset, lastMeta.Metadata); merr != nil {
 						// No need to fail the scrape on errors appending metadata.
 						sl.l.Debug("Error when appending metadata in scrape loop", "ref", fmt.Sprintf("%d", ref), "metadata", fmt.Sprintf("%+v", lastMeta.Metadata), "err", merr)
@@ -2029,7 +1935,7 @@ func isSeriesPartOfFamily(mName string, mfName []byte, typ model.MetricType) boo
 // during normal operation (e.g., accidental cardinality explosion, sudden traffic spikes).
 // Current case ordering prevents exercising other cases when limits are exceeded.
 // Remaining error cases typically occur only a few times, often during initial setup.
-func (sl *scrapeLoop) checkAddError(met []byte, err error, sampleLimitErr, bucketLimitErr *error, appErrs *appendErrors) (bool, error) {
+func (sl *scrapeLoop) checkAddError(met []byte, err error, sampleLimitErr, bucketLimitErr *error, appErrs *appendErrors) (sampleAdded bool, _ error) {
 	switch {
 	case err == nil:
 		return true, nil
@@ -2141,7 +2047,7 @@ var (
 	}
 )
 
-func (sl *scrapeLoop) report(app storage.Appender, start time.Time, duration time.Duration, scraped, added, seriesAdded, bytes int, scrapeErr error) (err error) {
+func (sl *scrapeLoop) report(app scrapeLoopAppendAdapter, start time.Time, duration time.Duration, scraped, added, seriesAdded, bytes int, scrapeErr error) (err error) {
 	sl.scraper.Report(start, duration, scrapeErr)
 
 	ts := timestamp.FromTime(start)
@@ -2152,71 +2058,70 @@ func (sl *scrapeLoop) report(app storage.Appender, start time.Time, duration tim
 	}
 	b := labels.NewBuilderWithSymbolTable(sl.symbolTable)
 
-	if err = sl.addReportSample(app, scrapeHealthMetric, ts, health, b); err != nil {
+	if err = app.addReportSample(scrapeHealthMetric, ts, health, b, false); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, scrapeDurationMetric, ts, duration.Seconds(), b); err != nil {
+	if err = app.addReportSample(scrapeDurationMetric, ts, duration.Seconds(), b, false); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, scrapeSamplesMetric, ts, float64(scraped), b); err != nil {
+	if err = app.addReportSample(scrapeSamplesMetric, ts, float64(scraped), b, false); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, samplesPostRelabelMetric, ts, float64(added), b); err != nil {
+	if err = app.addReportSample(samplesPostRelabelMetric, ts, float64(added), b, false); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, scrapeSeriesAddedMetric, ts, float64(seriesAdded), b); err != nil {
+	if err = app.addReportSample(scrapeSeriesAddedMetric, ts, float64(seriesAdded), b, false); err != nil {
 		return err
 	}
 	if sl.reportExtraMetrics {
-		if err = sl.addReportSample(app, scrapeTimeoutMetric, ts, sl.timeout.Seconds(), b); err != nil {
+		if err = app.addReportSample(scrapeTimeoutMetric, ts, sl.timeout.Seconds(), b, false); err != nil {
 			return err
 		}
-		if err = sl.addReportSample(app, scrapeSampleLimitMetric, ts, float64(sl.sampleLimit), b); err != nil {
+		if err = app.addReportSample(scrapeSampleLimitMetric, ts, float64(sl.sampleLimit), b, false); err != nil {
 			return err
 		}
-		if err = sl.addReportSample(app, scrapeBodySizeBytesMetric, ts, float64(bytes), b); err != nil {
+		if err = app.addReportSample(scrapeBodySizeBytesMetric, ts, float64(bytes), b, false); err != nil {
 			return err
 		}
 	}
 	return err
 }
 
-func (sl *scrapeLoop) reportStale(app storage.Appender, start time.Time) (err error) {
+func (sl *scrapeLoop) reportStale(app scrapeLoopAppendAdapter, start time.Time) (err error) {
 	ts := timestamp.FromTime(start)
-	app.SetOptions(&storage.AppendOptions{DiscardOutOfOrder: true})
 	stale := math.Float64frombits(value.StaleNaN)
 	b := labels.NewBuilder(labels.EmptyLabels())
 
-	if err = sl.addReportSample(app, scrapeHealthMetric, ts, stale, b); err != nil {
+	if err = app.addReportSample(scrapeHealthMetric, ts, stale, b, true); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, scrapeDurationMetric, ts, stale, b); err != nil {
+	if err = app.addReportSample(scrapeDurationMetric, ts, stale, b, true); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, scrapeSamplesMetric, ts, stale, b); err != nil {
+	if err = app.addReportSample(scrapeSamplesMetric, ts, stale, b, true); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, samplesPostRelabelMetric, ts, stale, b); err != nil {
+	if err = app.addReportSample(samplesPostRelabelMetric, ts, stale, b, true); err != nil {
 		return err
 	}
-	if err = sl.addReportSample(app, scrapeSeriesAddedMetric, ts, stale, b); err != nil {
+	if err = app.addReportSample(scrapeSeriesAddedMetric, ts, stale, b, true); err != nil {
 		return err
 	}
 	if sl.reportExtraMetrics {
-		if err = sl.addReportSample(app, scrapeTimeoutMetric, ts, stale, b); err != nil {
+		if err = app.addReportSample(scrapeTimeoutMetric, ts, stale, b, true); err != nil {
 			return err
 		}
-		if err = sl.addReportSample(app, scrapeSampleLimitMetric, ts, stale, b); err != nil {
+		if err = app.addReportSample(scrapeSampleLimitMetric, ts, stale, b, true); err != nil {
 			return err
 		}
-		if err = sl.addReportSample(app, scrapeBodySizeBytesMetric, ts, stale, b); err != nil {
+		if err = app.addReportSample(scrapeBodySizeBytesMetric, ts, stale, b, true); err != nil {
 			return err
 		}
 	}
 	return err
 }
 
-func (sl *scrapeLoop) addReportSample(app storage.Appender, s reportSample, t int64, v float64, b *labels.Builder) error {
+func (sl *scrapeLoopAppender) addReportSample(s reportSample, t int64, v float64, b *labels.Builder, rejectOOO bool) (err error) {
 	ce, ok, _ := sl.cache.get(s.name)
 	var ref storage.SeriesRef
 	var lset labels.Labels
@@ -2228,18 +2133,26 @@ func (sl *scrapeLoop) addReportSample(app storage.Appender, s reportSample, t in
 		// with scraped metrics in the cache.
 		// We have to drop it when building the actual metric.
 		b.Reset(labels.EmptyLabels())
-		b.Set(labels.MetricName, string(s.name[:len(s.name)-1]))
+		b.Set(model.MetricNameLabel, string(s.name[:len(s.name)-1]))
 		lset = sl.reportSampleMutator(b.Labels())
 	}
 
-	ref, err := app.Append(ref, lset, t, v)
+	// This will be improved in AppenderV2.
+	if rejectOOO {
+		sl.SetOptions(&aOptionRejectEarlyOOO)
+		ref, err = sl.Append(ref, lset, t, v)
+		sl.SetOptions(nil)
+	} else {
+		ref, err = sl.Append(ref, lset, t, v)
+	}
+
 	switch {
 	case err == nil:
 		if !ok {
 			sl.cache.addRef(s.name, ref, lset, lset.Hash())
 			// We only need to add metadata once a scrape target appears.
 			if sl.appendMetadataToWAL {
-				if _, merr := app.UpdateMetadata(ref, lset, s.Metadata); merr != nil {
+				if _, merr := sl.UpdateMetadata(ref, lset, s.Metadata); merr != nil {
 					sl.l.Debug("Error when appending metadata in addReportSample", "ref", fmt.Sprintf("%d", ref), "metadata", fmt.Sprintf("%+v", s.Metadata), "err", merr)
 				}
 			}
diff --git a/scrape/scrape_test.go b/scrape/scrape_test.go
index eab1499158..ae004bbd56 100644
--- a/scrape/scrape_test.go
+++ b/scrape/scrape_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -89,7 +89,7 @@ func newTestScrapeMetrics(t testing.TB) *scrapeMetrics {
 
 func TestNewScrapePool(t *testing.T) {
 	var (
-		app = &nopAppendable{}
+		app = teststorage.NewAppendable()
 		cfg = &config.ScrapeConfig{
 			MetricNameValidationScheme: model.UTF8Validation,
 			MetricNameEscapingScheme:   model.AllowUTF8,
@@ -98,20 +98,17 @@ func TestNewScrapePool(t *testing.T) {
 	)
 	require.NoError(t, err)
 
-	a, ok := sp.appendable.(*nopAppendable)
+	a, ok := sp.appendable.(*teststorage.Appendable)
 	require.True(t, ok, "Failure to append.")
 	require.Equal(t, app, a, "Wrong sample appender.")
 	require.Equal(t, cfg, sp.config, "Wrong scrape config.")
-	require.NotNil(t, sp.newLoop, "newLoop function not initialized.")
 }
 
 func TestStorageHandlesOutOfOrderTimestamps(t *testing.T) {
 	// Test with default OutOfOrderTimeWindow (0)
 	t.Run("Out-Of-Order Sample Disabled", func(t *testing.T) {
 		s := teststorage.New(t)
-		t.Cleanup(func() {
-			_ = s.Close()
-		})
+		t.Cleanup(func() { _ = s.Close() })
 
 		runScrapeLoopTest(t, s, false)
 	})
@@ -119,19 +116,14 @@ func TestStorageHandlesOutOfOrderTimestamps(t *testing.T) {
 	// Test with specific OutOfOrderTimeWindow (600000)
 	t.Run("Out-Of-Order Sample Enabled", func(t *testing.T) {
 		s := teststorage.New(t, 600000)
-		t.Cleanup(func() {
-			_ = s.Close()
-		})
+		t.Cleanup(func() { _ = s.Close() })
 
 		runScrapeLoopTest(t, s, true)
 	})
 }
 
 func runScrapeLoopTest(t *testing.T, s *teststorage.TestStorage, expectOutOfOrder bool) {
-	// Create an appender for adding samples to the storage.
-	app := s.Appender(context.Background())
-	capp := &collectResultAppender{next: app}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return capp }, 0)
+	sl, _ := newTestScrapeLoop(t, withAppendable(s))
 
 	// Current time for generating timestamps.
 	now := time.Now()
@@ -142,37 +134,35 @@ func runScrapeLoopTest(t *testing.T, s *teststorage.TestStorage, expectOutOfOrde
 	timestampOutOfOrder := now.Add(-5 * time.Minute)
 	timestampInorder2 := now.Add(5 * time.Minute)
 
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte(`metric_total{a="1",b="1"} 1`), "text/plain", timestampInorder1)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(`metric_total{a="1",b="1"} 1`), "text/plain", timestampInorder1)
 	require.NoError(t, err)
 
-	_, _, _, err = sl.append(slApp, []byte(`metric_total{a="1",b="1"} 2`), "text/plain", timestampOutOfOrder)
+	_, _, _, err = app.append([]byte(`metric_total{a="1",b="1"} 2`), "text/plain", timestampOutOfOrder)
 	require.NoError(t, err)
 
-	_, _, _, err = sl.append(slApp, []byte(`metric_total{a="1",b="1"} 3`), "text/plain", timestampInorder2)
+	_, _, _, err = app.append([]byte(`metric_total{a="1",b="1"} 3`), "text/plain", timestampInorder2)
 	require.NoError(t, err)
 
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
 	// Query the samples back from the storage.
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
 	q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
 	require.NoError(t, err)
-	defer q.Close()
+	t.Cleanup(func() { _ = q.Close() })
 
 	// Use a matcher to filter the metric name.
-	series := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", "metric_total"))
+	series := q.Select(t.Context(), false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", "metric_total"))
 
-	var results []floatSample
+	var results []sample
 	for series.Next() {
 		it := series.At().Iterator(nil)
 		for it.Next() == chunkenc.ValFloat {
 			t, v := it.At()
-			results = append(results, floatSample{
-				metric: series.At().Labels(),
-				t:      t,
-				f:      v,
+			results = append(results, sample{
+				L: series.At().Labels(),
+				T: t,
+				V: v,
 			})
 		}
 		require.NoError(t, it.Err())
@@ -180,16 +170,16 @@ func runScrapeLoopTest(t *testing.T, s *teststorage.TestStorage, expectOutOfOrde
 	require.NoError(t, series.Err())
 
 	// Define the expected results
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings("__name__", "metric_total", "a", "1", "b", "1"),
-			t:      timestamp.FromTime(timestampInorder1),
-			f:      1,
+			L: labels.FromStrings("__name__", "metric_total", "a", "1", "b", "1"),
+			T: timestamp.FromTime(timestampInorder1),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings("__name__", "metric_total", "a", "1", "b", "1"),
-			t:      timestamp.FromTime(timestampInorder2),
-			f:      3,
+			L: labels.FromStrings("__name__", "metric_total", "a", "1", "b", "1"),
+			T: timestamp.FromTime(timestampInorder2),
+			V: 3,
 		},
 	}
 
@@ -201,7 +191,7 @@ func runScrapeLoopTest(t *testing.T, s *teststorage.TestStorage, expectOutOfOrde
 }
 
 // Regression test against https://github.com/prometheus/prometheus/issues/15831.
-func TestScrapeAppendMetadataUpdate(t *testing.T) {
+func TestScrapeAppend_MetadataUpdate(t *testing.T) {
 	const (
 		scrape1 = `# TYPE test_metric counter
 # HELP test_metric some help text
@@ -224,60 +214,54 @@ test_metric2{foo="bar"} 22
 # EOF`
 	)
 
-	// Create an appender for adding samples to the storage.
-	capp := &collectResultAppender{next: nopAppender{}}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return capp }, 0)
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte(scrape1), "application/openmetrics-text", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(scrape1), "application/openmetrics-text", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
-	testutil.RequireEqualWithOptions(t, []metadataEntry{
-		{metric: labels.FromStrings("__name__", "test_metric_total"), m: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
-		{metric: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), m: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}},
-	}, capp.resultMetadata, []cmp.Option{cmp.Comparer(metadataEntryEqual)})
-	capp.resultMetadata = nil
+	require.NoError(t, app.Commit())
+	testutil.RequireEqual(t, []sample{
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}},
+	}, appTest.ResultMetadata())
+	appTest.ResultReset()
 
-	// Next (the same) scrape should not add new metadata entries.
-	slApp = sl.appender(context.Background())
-	_, _, _, err = sl.append(slApp, []byte(scrape1), "application/openmetrics-text", now.Add(15*time.Second))
+	// Next (the same) scrape should not new metadata entries.
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(scrape1), "application/openmetrics-text", now.Add(15*time.Second))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
-	testutil.RequireEqualWithOptions(t, []metadataEntry(nil), capp.resultMetadata, []cmp.Option{cmp.Comparer(metadataEntryEqual)})
+	require.NoError(t, app.Commit())
+	require.Empty(t, appTest.ResultMetadata())
+	appTest.ResultReset()
 
-	slApp = sl.appender(context.Background())
-	_, _, _, err = sl.append(slApp, []byte(scrape2), "application/openmetrics-text", now.Add(15*time.Second))
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(scrape2), "application/openmetrics-text", now.Add(15*time.Second))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
-	testutil.RequireEqualWithOptions(t, []metadataEntry{
-		{metric: labels.FromStrings("__name__", "test_metric_total"), m: metadata.Metadata{Type: "counter", Unit: "metric", Help: "different help text"}}, // Here, technically we should have no unit, but it's a known limitation of the current implementation.
-		{metric: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), m: metadata.Metadata{Type: "gauge", Unit: "metric2", Help: "other help text"}},
-	}, capp.resultMetadata, []cmp.Option{cmp.Comparer(metadataEntryEqual)})
+	require.NoError(t, app.Commit())
+	testutil.RequireEqual(t, []sample{
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "different help text"}}, // Here, technically we should have no unit, but it's a known limitation of the current implementation.
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "metric2", Help: "other help text"}},
+	}, appTest.ResultMetadata())
+	appTest.ResultReset()
 }
 
-type nopScraper struct {
-	scraper
-}
+func TestScrapeReportMetadata(t *testing.T) {
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
+	app := sl.appender()
 
-func (nopScraper) Report(time.Time, time.Duration, error) {}
-
-func TestScrapeReportMetadataUpdate(t *testing.T) {
-	// Create an appender for adding samples to the storage.
-	capp := &collectResultAppender{next: nopAppender{}}
-	sl := newBasicScrapeLoop(t, context.Background(), nopScraper{}, func(context.Context) storage.Appender { return capp }, 0)
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-
-	require.NoError(t, sl.report(slApp, now, 2*time.Second, 1, 1, 1, 512, nil))
-	require.NoError(t, slApp.Commit())
-	testutil.RequireEqualWithOptions(t, []metadataEntry{
-		{metric: labels.FromStrings("__name__", "up"), m: scrapeHealthMetric.Metadata},
-		{metric: labels.FromStrings("__name__", "scrape_duration_seconds"), m: scrapeDurationMetric.Metadata},
-		{metric: labels.FromStrings("__name__", "scrape_samples_scraped"), m: scrapeSamplesMetric.Metadata},
-		{metric: labels.FromStrings("__name__", "scrape_samples_post_metric_relabeling"), m: samplesPostRelabelMetric.Metadata},
-		{metric: labels.FromStrings("__name__", "scrape_series_added"), m: scrapeSeriesAddedMetric.Metadata},
-	}, capp.resultMetadata, []cmp.Option{cmp.Comparer(metadataEntryEqual)})
+	require.NoError(t, sl.report(app, now, 2*time.Second, 1, 1, 1, 512, nil))
+	require.NoError(t, app.Commit())
+	testutil.RequireEqual(t, []sample{
+		{L: labels.FromStrings("__name__", "up"), M: scrapeHealthMetric.Metadata},
+		{L: labels.FromStrings("__name__", "scrape_duration_seconds"), M: scrapeDurationMetric.Metadata},
+		{L: labels.FromStrings("__name__", "scrape_samples_scraped"), M: scrapeSamplesMetric.Metadata},
+		{L: labels.FromStrings("__name__", "scrape_samples_post_metric_relabeling"), M: samplesPostRelabelMetric.Metadata},
+		{L: labels.FromStrings("__name__", "scrape_series_added"), M: scrapeSeriesAddedMetric.Metadata},
+	}, appTest.ResultMetadata())
 }
 
 func TestIsSeriesPartOfFamily(t *testing.T) {
@@ -330,7 +314,7 @@ func TestIsSeriesPartOfFamily(t *testing.T) {
 
 func TestDroppedTargetsList(t *testing.T) {
 	var (
-		app = &nopAppendable{}
+		app = teststorage.NewAppendable()
 		cfg = &config.ScrapeConfig{
 			JobName:                    "dropMe",
 			ScrapeInterval:             model.Duration(1),
@@ -374,9 +358,7 @@ func TestDroppedTargetsList(t *testing.T) {
 // TestDiscoveredLabelsUpdate checks that DiscoveredLabels are updated
 // even when new labels don't affect the target `hash`.
 func TestDiscoveredLabelsUpdate(t *testing.T) {
-	sp := &scrapePool{
-		metrics: newTestScrapeMetrics(t),
-	}
+	sp := newTestScrapePool(t, nil)
 
 	// These are used when syncing so need this to avoid a panic.
 	sp.config = &config.ScrapeConfig{
@@ -448,13 +430,8 @@ func (*testLoop) getCache() *scrapeCache {
 
 func TestScrapePoolStop(t *testing.T) {
 	t.Parallel()
-	sp := &scrapePool{
-		activeTargets: map[uint64]*Target{},
-		loops:         map[uint64]loop{},
-		cancel:        func() {},
-		client:        http.DefaultClient,
-		metrics:       newTestScrapeMetrics(t),
-	}
+	sp := newTestScrapePool(t, nil)
+
 	var mtx sync.Mutex
 	stopped := map[uint64]bool{}
 	numTargets := 20
@@ -506,26 +483,42 @@ func TestScrapePoolStop(t *testing.T) {
 	require.Empty(t, sp.loops, "Loops were not cleared on stopping: %d left", len(sp.loops))
 }
 
+// TestScrapePoolReload tests reloading logic, so:
+// * all loops are reloaded, reusing cache if scrape config changed.
+// * reloaded loops are stopped before new ones are started.
+// * new scrapeLoops are configured with the updated scrape config.
 func TestScrapePoolReload(t *testing.T) {
 	t.Parallel()
-	var mtx sync.Mutex
-	numTargets := 20
 
-	stopped := map[uint64]bool{}
+	var (
+		mtx        sync.Mutex
+		numTargets = 20
+		stopped    = map[uint64]bool{}
+	)
 
-	reloadCfg := &config.ScrapeConfig{
+	cfg0 := &config.ScrapeConfig{}
+	cfg1 := &config.ScrapeConfig{
 		ScrapeInterval:             model.Duration(3 * time.Second),
 		ScrapeTimeout:              model.Duration(2 * time.Second),
 		MetricNameValidationScheme: model.UTF8Validation,
 		MetricNameEscapingScheme:   model.AllowUTF8,
+
+		// Test a few example options.
+		SampleLimit:            123,
+		ScrapeFallbackProtocol: "application/vnd.google.protobuf;proto=io.prometheus.client.MetricFamily;encoding=delimited",
 	}
-	// On starting to run, new loops created on reload check whether their preceding
-	// equivalents have been stopped.
-	newLoop := func(opts scrapeLoopOptions) loop {
-		l := &testLoop{interval: time.Duration(reloadCfg.ScrapeInterval), timeout: time.Duration(reloadCfg.ScrapeTimeout)}
+	newLoopCfg1 := func(opts scrapeLoopOptions) loop {
+		// Test cfg1 is being used.
+		require.Equal(t, cfg1, opts.sp.config)
+
+		// Inject out testLoop that allows mocking start and stop.
+		l := &testLoop{interval: opts.interval, timeout: opts.timeout}
+
+		// On start, expect previous loop instances for the same target to be stopped.
 		l.startFunc = func(interval, timeout time.Duration, _ chan<- error) {
-			require.Equal(t, 3*time.Second, interval, "Unexpected scrape interval")
-			require.Equal(t, 2*time.Second, timeout, "Unexpected scrape timeout")
+			// Ensure cfg1 interval and timeout are correctly configured.
+			require.Equal(t, time.Duration(cfg1.ScrapeInterval), interval, "Unexpected scrape interval")
+			require.Equal(t, time.Duration(cfg1.ScrapeTimeout), timeout, "Unexpected scrape timeout")
 
 			mtx.Lock()
 			targetScraper := opts.scraper.(*targetScraper)
@@ -535,32 +528,21 @@ func TestScrapePoolReload(t *testing.T) {
 		return l
 	}
 
+	// Create test pool.
 	reg, metrics := newTestRegistryAndScrapeMetrics(t)
-	sp := &scrapePool{
-		appendable:    &nopAppendable{},
-		activeTargets: map[uint64]*Target{},
-		loops:         map[uint64]loop{},
-		newLoop:       newLoop,
-		logger:        nil,
-		client:        http.DefaultClient,
-		metrics:       metrics,
-		symbolTable:   labels.NewSymbolTable(),
-	}
-
-	// Reloading a scrape pool with a new scrape configuration must stop all scrape
-	// loops and start new ones. A new loop must not be started before the preceding
-	// one terminated.
+	sp := newTestScrapePool(t, newLoopCfg1)
+	sp.metrics = metrics
 
+	// Prefill pool with 20 loops, simulating 20 scrape targets.
 	for i := range numTargets {
-		labels := labels.FromStrings(model.AddressLabel, fmt.Sprintf("example.com:%d", i))
 		t := &Target{
-			labels:       labels,
-			scrapeConfig: &config.ScrapeConfig{},
+			labels:       labels.FromStrings(model.AddressLabel, fmt.Sprintf("example.com:%d", i)),
+			scrapeConfig: cfg0,
 		}
 		l := &testLoop{}
 		d := time.Duration((i+1)*20) * time.Millisecond
 		l.stopFunc = func() {
-			time.Sleep(d)
+			time.Sleep(d) // Sleep uneven time on stop.
 
 			mtx.Lock()
 			stopped[t.hash()] = true
@@ -570,36 +552,26 @@ func TestScrapePoolReload(t *testing.T) {
 		sp.activeTargets[t.hash()] = t
 		sp.loops[t.hash()] = l
 	}
-	done := make(chan struct{})
 
 	beforeTargets := map[uint64]*Target{}
 	maps.Copy(beforeTargets, sp.activeTargets)
 
-	reloadTime := time.Now()
-
-	go func() {
-		sp.reload(reloadCfg)
-		close(done)
-	}()
-
-	select {
-	case <-time.After(5 * time.Second):
-		require.FailNow(t, "scrapeLoop.reload() did not return as expected")
-	case <-done:
-		// This should have taken at least as long as the last target slept.
-		require.GreaterOrEqual(t, time.Since(reloadTime), time.Duration(numTargets*20)*time.Millisecond, "scrapeLoop.stop() exited before all targets stopped")
-	}
-
+	// Reloading a scrape pool with a new scrape configuration must stop all scrape
+	// loops and start new ones. A new loop must not be started before the preceding
+	// one terminated.
+	require.NoError(t, sp.reload(cfg1))
+	var stoppedCount int
 	mtx.Lock()
-	require.Len(t, stopped, numTargets, "Unexpected number of stopped loops")
+	stoppedCount = len(stopped)
 	mtx.Unlock()
-
+	require.Equal(t, numTargets, stoppedCount, "Unexpected number of stopped loops")
 	require.Equal(t, sp.activeTargets, beforeTargets, "Reloading affected target states unexpectedly")
-	require.Len(t, sp.loops, numTargets, "Unexpected number of stopped loops after reload")
+	require.Len(t, sp.loops, numTargets, "Unexpected number of loops after reload")
 
+	// Check if prometheus_target_reload_length_seconds points to cfg1.ScrapeInterval.
 	got, err := gatherLabels(reg, "prometheus_target_reload_length_seconds")
 	require.NoError(t, err)
-	expectedName, expectedValue := "interval", "3s"
+	expectedName, expectedValue := "interval", cfg1.ScrapeInterval.String()
 	require.Equal(t, [][]*dto.LabelPair{{{Name: &expectedName, Value: &expectedValue}}}, got)
 	require.Equal(t, 1.0, prom_testutil.ToFloat64(sp.metrics.targetScrapePoolReloads))
 }
@@ -620,22 +592,12 @@ func TestScrapePoolReloadPreserveRelabeledIntervalTimeout(t *testing.T) {
 		return l
 	}
 	reg, metrics := newTestRegistryAndScrapeMetrics(t)
-	sp := &scrapePool{
-		appendable: &nopAppendable{},
-		activeTargets: map[uint64]*Target{
-			1: {
-				labels: labels.FromStrings(model.ScrapeIntervalLabel, "5s", model.ScrapeTimeoutLabel, "3s"),
-			},
-		},
-		loops: map[uint64]loop{
-			1: noopLoop(),
-		},
-		newLoop:     newLoop,
-		logger:      nil,
-		client:      http.DefaultClient,
-		metrics:     metrics,
-		symbolTable: labels.NewSymbolTable(),
+	sp := newTestScrapePool(t, newLoop)
+	sp.activeTargets[1] = &Target{
+		labels: labels.FromStrings(model.ScrapeIntervalLabel, "5s", model.ScrapeTimeoutLabel, "3s"),
 	}
+	sp.metrics = metrics
+	sp.loops[1] = noopLoop()
 
 	err := sp.reload(reloadCfg)
 	if err != nil {
@@ -681,18 +643,10 @@ func TestScrapePoolTargetLimit(t *testing.T) {
 		}
 		return l
 	}
-	sp := &scrapePool{
-		appendable:    &nopAppendable{},
-		activeTargets: map[uint64]*Target{},
-		loops:         map[uint64]loop{},
-		newLoop:       newLoop,
-		logger:        promslog.NewNopLogger(),
-		client:        http.DefaultClient,
-		metrics:       newTestScrapeMetrics(t),
-		symbolTable:   labels.NewSymbolTable(),
-	}
 
-	tgs := []*targetgroup.Group{}
+	sp := newTestScrapePool(t, newLoop)
+
+	var tgs []*targetgroup.Group
 	for i := range 50 {
 		tgs = append(tgs,
 			&targetgroup.Group{
@@ -782,12 +736,12 @@ func TestScrapePoolTargetLimit(t *testing.T) {
 	tgs = append(tgs,
 		&targetgroup.Group{
 			Targets: []model.LabelSet{
-				{model.AddressLabel: model.LabelValue("127.0.0.1:1090")},
+				{model.AddressLabel: "127.0.0.1:1090"},
 			},
 		},
 		&targetgroup.Group{
 			Targets: []model.LabelSet{
-				{model.AddressLabel: model.LabelValue("127.0.0.1:1090")},
+				{model.AddressLabel: "127.0.0.1:1090"},
 			},
 		},
 	)
@@ -797,62 +751,48 @@ func TestScrapePoolTargetLimit(t *testing.T) {
 	validateErrorMessage(false)
 }
 
-func TestScrapePoolAppender(t *testing.T) {
-	cfg := &config.ScrapeConfig{
-		MetricNameValidationScheme: model.UTF8Validation,
-		MetricNameEscapingScheme:   model.AllowUTF8,
-	}
-	app := &nopAppendable{}
-	sp, _ := newScrapePool(cfg, app, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+func TestScrapePoolAppenderWithLimits(t *testing.T) {
+	// Create a unique value, to validate the correct chain of appenders.
+	baseAppender := struct{ storage.Appender }{}
+	appendable := appendableFunc(func(context.Context) storage.Appender { return baseAppender })
 
-	loop := sp.newLoop(scrapeLoopOptions{
-		target: &Target{},
-	})
-	appl, ok := loop.(*scrapeLoop)
-	require.True(t, ok, "Expected scrapeLoop but got %T", loop)
-
-	wrapped := appender(appl.appender(context.Background()), 0, 0, histogram.ExponentialSchemaMax)
+	sl, _ := newTestScrapeLoop(t, withAppendable(appendable))
+	wrapped := appenderWithLimits(sl.appendable.Appender(context.Background()), 0, 0, histogram.ExponentialSchemaMax)
 
 	tl, ok := wrapped.(*timeLimitAppender)
 	require.True(t, ok, "Expected timeLimitAppender but got %T", wrapped)
 
-	_, ok = tl.Appender.(nopAppender)
-	require.True(t, ok, "Expected base appender but got %T", tl.Appender)
+	require.Equal(t, baseAppender, tl.Appender, "Expected base appender but got %T", tl.Appender)
 
 	sampleLimit := 100
-	loop = sp.newLoop(scrapeLoopOptions{
-		target:      &Target{},
-		sampleLimit: sampleLimit,
+	sl, _ = newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appendable
+		sl.sampleLimit = sampleLimit
 	})
-	appl, ok = loop.(*scrapeLoop)
-	require.True(t, ok, "Expected scrapeLoop but got %T", loop)
+	wrapped = appenderWithLimits(sl.appendable.Appender(context.Background()), sampleLimit, 0, histogram.ExponentialSchemaMax)
 
-	wrapped = appender(appl.appender(context.Background()), sampleLimit, 0, histogram.ExponentialSchemaMax)
-
-	sl, ok := wrapped.(*limitAppender)
+	la, ok := wrapped.(*limitAppender)
 	require.True(t, ok, "Expected limitAppender but got %T", wrapped)
 
-	tl, ok = sl.Appender.(*timeLimitAppender)
-	require.True(t, ok, "Expected timeLimitAppender but got %T", sl.Appender)
+	tl, ok = la.Appender.(*timeLimitAppender)
+	require.True(t, ok, "Expected timeLimitAppender but got %T", la.Appender)
 
-	_, ok = tl.Appender.(nopAppender)
-	require.True(t, ok, "Expected base appender but got %T", tl.Appender)
+	require.Equal(t, baseAppender, tl.Appender, "Expected base appender but got %T", tl.Appender)
 
-	wrapped = appender(appl.appender(context.Background()), sampleLimit, 100, histogram.ExponentialSchemaMax)
+	wrapped = appenderWithLimits(sl.appendable.Appender(context.Background()), sampleLimit, 100, histogram.ExponentialSchemaMax)
 
 	bl, ok := wrapped.(*bucketLimitAppender)
 	require.True(t, ok, "Expected bucketLimitAppender but got %T", wrapped)
 
-	sl, ok = bl.Appender.(*limitAppender)
+	la, ok = bl.Appender.(*limitAppender)
 	require.True(t, ok, "Expected limitAppender but got %T", bl)
 
-	tl, ok = sl.Appender.(*timeLimitAppender)
-	require.True(t, ok, "Expected timeLimitAppender but got %T", sl.Appender)
+	tl, ok = la.Appender.(*timeLimitAppender)
+	require.True(t, ok, "Expected timeLimitAppender but got %T", la.Appender)
 
-	_, ok = tl.Appender.(nopAppender)
-	require.True(t, ok, "Expected base appender but got %T", tl.Appender)
+	require.Equal(t, baseAppender, tl.Appender, "Expected base appender but got %T", tl.Appender)
 
-	wrapped = appender(appl.appender(context.Background()), sampleLimit, 100, 0)
+	wrapped = appenderWithLimits(sl.appendable.Appender(context.Background()), sampleLimit, 100, 0)
 
 	ml, ok := wrapped.(*maxSchemaAppender)
 	require.True(t, ok, "Expected maxSchemaAppender but got %T", wrapped)
@@ -860,14 +800,13 @@ func TestScrapePoolAppender(t *testing.T) {
 	bl, ok = ml.Appender.(*bucketLimitAppender)
 	require.True(t, ok, "Expected bucketLimitAppender but got %T", wrapped)
 
-	sl, ok = bl.Appender.(*limitAppender)
+	la, ok = bl.Appender.(*limitAppender)
 	require.True(t, ok, "Expected limitAppender but got %T", bl)
 
-	tl, ok = sl.Appender.(*timeLimitAppender)
-	require.True(t, ok, "Expected timeLimitAppender but got %T", sl.Appender)
+	tl, ok = la.Appender.(*timeLimitAppender)
+	require.True(t, ok, "Expected timeLimitAppender but got %T", la.Appender)
 
-	_, ok = tl.Appender.(nopAppender)
-	require.True(t, ok, "Expected base appender but got %T", tl.Appender)
+	require.Equal(t, baseAppender, tl.Appender, "Expected base appender but got %T", tl.Appender)
 }
 
 func TestScrapePoolRaces(t *testing.T) {
@@ -882,7 +821,7 @@ func TestScrapePoolRaces(t *testing.T) {
 			MetricNameEscapingScheme:   model.AllowUTF8,
 		}
 	}
-	sp, _ := newScrapePool(newConfig(), &nopAppendable{}, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+	sp, _ := newScrapePool(newConfig(), teststorage.NewAppendable(), 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 	tgts := []*targetgroup.Group{
 		{
 			Targets: []model.LabelSet{
@@ -908,7 +847,7 @@ func TestScrapePoolRaces(t *testing.T) {
 
 	for range 20 {
 		time.Sleep(10 * time.Millisecond)
-		sp.reload(newConfig())
+		_ = sp.reload(newConfig())
 	}
 	sp.stop()
 }
@@ -925,16 +864,7 @@ func TestScrapePoolScrapeLoopsStarted(t *testing.T) {
 		}
 		return l
 	}
-	sp := &scrapePool{
-		appendable:    &nopAppendable{},
-		activeTargets: map[uint64]*Target{},
-		loops:         map[uint64]loop{},
-		newLoop:       newLoop,
-		logger:        nil,
-		client:        http.DefaultClient,
-		metrics:       newTestScrapeMetrics(t),
-		symbolTable:   labels.NewSymbolTable(),
-	}
+	sp := newTestScrapePool(t, newLoop)
 
 	tgs := []*targetgroup.Group{
 		{
@@ -965,51 +895,13 @@ func TestScrapePoolScrapeLoopsStarted(t *testing.T) {
 	}
 }
 
-func newBasicScrapeLoop(t testing.TB, ctx context.Context, scraper scraper, app func(ctx context.Context) storage.Appender, interval time.Duration) *scrapeLoop {
-	return newBasicScrapeLoopWithFallback(t, ctx, scraper, app, interval, "")
-}
-
-func newBasicScrapeLoopWithFallback(t testing.TB, ctx context.Context, scraper scraper, app func(ctx context.Context) storage.Appender, interval time.Duration, fallback string) *scrapeLoop {
-	return newScrapeLoop(ctx,
-		scraper,
-		nil, nil,
-		nopMutator,
-		nopMutator,
-		app,
-		nil,
-		labels.NewSymbolTable(),
-		0,
-		true,
-		false,
-		true,
-		0, 0, histogram.ExponentialSchemaMax,
-		nil,
-		interval,
-		time.Hour,
-		false,
-		false,
-		false,
-		false,
-		false,
-		false,
-		true,
-		nil,
-		false,
-		newTestScrapeMetrics(t),
-		false,
-		model.UTF8Validation,
-		model.NoEscaping,
-		fallback,
-	)
-}
-
 func TestScrapeLoopStopBeforeRun(t *testing.T) {
 	t.Parallel()
-	scraper := &testScraper{}
-	sl := newBasicScrapeLoop(t, context.Background(), scraper, nil, 1)
+
+	sl, scraper := newTestScrapeLoop(t)
 
 	// The scrape pool synchronizes on stopping scrape loops. However, new scrape
-	// loops are started asynchronously. Thus it's possible, that a loop is stopped
+	// loops are started asynchronously. Thus, it's possible, that a loop is stopped
 	// again before having started properly.
 	// Stopping not-yet-started loops must block until the run method was called and exited.
 	// The run method must exit immediately.
@@ -1054,26 +946,24 @@ func TestScrapeLoopStopBeforeRun(t *testing.T) {
 func nopMutator(l labels.Labels) labels.Labels { return l }
 
 func TestScrapeLoopStop(t *testing.T) {
-	var (
-		signal   = make(chan struct{}, 1)
-		appender = &collectResultAppender{}
-		scraper  = &testScraper{}
-		app      = func(context.Context) storage.Appender { return appender }
-	)
+	signal := make(chan struct{}, 1)
 
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, context.Background(), scraper, app, 10*time.Millisecond, "text/plain")
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		// Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+	})
 
 	// Terminate loop after 2 scrapes.
 	numScrapes := 0
-
 	scraper.scrapeFunc = func(ctx context.Context, w io.Writer) error {
 		numScrapes++
 		if numScrapes == 2 {
 			go sl.stop()
 			<-sl.ctx.Done()
 		}
-		w.Write([]byte("metric_a 42\n"))
+		_, _ = w.Write([]byte("metric_a 42\n"))
 		return ctx.Err()
 	}
 
@@ -1088,23 +978,24 @@ func TestScrapeLoopStop(t *testing.T) {
 		require.FailNow(t, "Scrape wasn't stopped.")
 	}
 
+	got := appTest.ResultSamples()
 	// We expected 1 actual sample for each scrape plus 5 for report samples.
 	// At least 2 scrapes were made, plus the final stale markers.
-	require.GreaterOrEqual(t, len(appender.resultFloats), 6*3, "Expected at least 3 scrapes with 6 samples each.")
-	require.Zero(t, len(appender.resultFloats)%6, "There is a scrape with missing samples.")
+	require.GreaterOrEqual(t, len(got), 6*3, "Expected at least 3 scrapes with 6 samples each.")
+	require.Zero(t, len(got)%6, "There is a scrape with missing samples.")
 	// All samples in a scrape must have the same timestamp.
 	var ts int64
-	for i, s := range appender.resultFloats {
+	for i, s := range got {
 		switch {
 		case i%6 == 0:
-			ts = s.t
-		case s.t != ts:
+			ts = s.T
+		case s.T != ts:
 			t.Fatalf("Unexpected multiple timestamps within single scrape")
 		}
 	}
 	// All samples from the last scrape must be stale markers.
-	for _, s := range appender.resultFloats[len(appender.resultFloats)-5:] {
-		require.True(t, value.IsStaleNaN(s.f), "Appended last sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(s.f))
+	for _, s := range got[len(got)-5:] {
+		require.True(t, value.IsStaleNaN(s.V), "Appended last sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(s.V))
 	}
 }
 
@@ -1113,45 +1004,10 @@ func TestScrapeLoopRun(t *testing.T) {
 	var (
 		signal = make(chan struct{}, 1)
 		errc   = make(chan error)
-
-		scraper       = &testScraper{}
-		app           = func(context.Context) storage.Appender { return &nopAppender{} }
-		scrapeMetrics = newTestScrapeMetrics(t)
-	)
-
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newScrapeLoop(ctx,
-		scraper,
-		nil, nil,
-		nopMutator,
-		nopMutator,
-		app,
-		nil,
-		nil,
-		0,
-		true,
-		false,
-		true,
-		0, 0, histogram.ExponentialSchemaMax,
-		nil,
-		time.Second,
-		time.Hour,
-		false,
-		false,
-		false,
-		false,
-		false,
-		false,
-		false,
-		nil,
-		false,
-		scrapeMetrics,
-		false,
-		model.UTF8Validation,
-		model.NoEscaping,
-		"",
 	)
 
+	ctx, cancel := context.WithCancel(t.Context())
+	sl, scraper := newTestScrapeLoop(t, withCtx(ctx))
 	// The loop must terminate during the initial offset if the context
 	// is canceled.
 	scraper.offsetDur = time.Hour
@@ -1173,24 +1029,26 @@ func TestScrapeLoopRun(t *testing.T) {
 		require.FailNow(t, "Unexpected error", "err: %s", err)
 	}
 
+	ctx, cancel = context.WithCancel(t.Context())
+	sl, scraper = newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.timeout = 100 * time.Millisecond
+	})
 	// The provided timeout must cause cancellation of the context passed down to the
 	// scraper. The scraper has to respect the context.
 	scraper.offsetDur = 0
 
-	block := make(chan struct{})
+	blockCtx, blockCancel := context.WithCancel(t.Context())
 	scraper.scrapeFunc = func(ctx context.Context, _ io.Writer) error {
 		select {
-		case <-block:
+		case <-blockCtx.Done():
+			cancel()
 		case <-ctx.Done():
 			return ctx.Err()
 		}
 		return nil
 	}
 
-	ctx, cancel = context.WithCancel(context.Background())
-	sl = newBasicScrapeLoop(t, ctx, scraper, app, time.Second)
-	sl.timeout = 100 * time.Millisecond
-
 	go func() {
 		sl.run(errc)
 		signal <- struct{}{}
@@ -1206,9 +1064,7 @@ func TestScrapeLoopRun(t *testing.T) {
 	// We already caught the timeout error and are certainly in the loop.
 	// Let the scrapes returns immediately to cause no further timeout errors
 	// and check whether canceling the parent context terminates the loop.
-	close(block)
-	cancel()
-
+	blockCancel()
 	select {
 	case <-signal:
 		// Loop terminated as expected.
@@ -1223,13 +1079,10 @@ func TestScrapeLoopForcedErr(t *testing.T) {
 	var (
 		signal = make(chan struct{}, 1)
 		errc   = make(chan error)
-
-		scraper = &testScraper{}
-		app     = func(context.Context) storage.Appender { return &nopAppender{} }
 	)
 
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, scraper, app, time.Second)
+	ctx, cancel := context.WithCancel(t.Context())
+	sl, scraper := newTestScrapeLoop(t, withCtx(ctx))
 
 	forcedErr := errors.New("forced err")
 	sl.setForcedError(forcedErr)
@@ -1264,15 +1117,12 @@ func TestScrapeLoopRun_ContextCancelTerminatesBlockedSend(t *testing.T) {
 	defer goleak.VerifyNone(t)
 
 	var (
-		signal  = make(chan struct{})
-		errc    = make(chan error)
-		scraper = &testScraper{}
-		app     = func(context.Context) storage.Appender { return &nopAppender{} }
+		signal = make(chan struct{})
+		errc   = make(chan error)
 	)
 
-	ctx, cancel := context.WithCancel(context.Background())
-
-	sl := newBasicScrapeLoop(t, ctx, scraper, app, 100*time.Millisecond)
+	ctx, cancel := context.WithCancel(t.Context())
+	sl, scraper := newTestScrapeLoop(t, withCtx(ctx))
 
 	forcedErr := errors.New("forced err")
 	sl.setForcedError(forcedErr)
@@ -1299,50 +1149,10 @@ func TestScrapeLoopRun_ContextCancelTerminatesBlockedSend(t *testing.T) {
 }
 
 func TestScrapeLoopMetadata(t *testing.T) {
-	var (
-		signal        = make(chan struct{})
-		scraper       = &testScraper{}
-		scrapeMetrics = newTestScrapeMetrics(t)
-		cache         = newScrapeCache(scrapeMetrics)
-	)
-	defer close(signal)
+	sl, _ := newTestScrapeLoop(t)
 
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newScrapeLoop(ctx,
-		scraper,
-		nil, nil,
-		nopMutator,
-		nopMutator,
-		func(context.Context) storage.Appender { return nopAppender{} },
-		cache,
-		labels.NewSymbolTable(),
-		0,
-		true,
-		false,
-		true,
-		0, 0, histogram.ExponentialSchemaMax,
-		nil,
-		0,
-		0,
-		false,
-		false,
-		false,
-		false,
-		false,
-		false,
-		false,
-		nil,
-		false,
-		scrapeMetrics,
-		false,
-		model.UTF8Validation,
-		model.NoEscaping,
-		"",
-	)
-	defer cancel()
-
-	slApp := sl.appender(ctx)
-	total, _, _, err := sl.append(slApp, []byte(`# TYPE test_metric counter
+	app := sl.appender()
+	total, _, _, err := app.append([]byte(`# TYPE test_metric counter
 # HELP test_metric some help text
 # UNIT test_metric metric
 test_metric_total 1
@@ -1350,54 +1160,42 @@ test_metric_total 1
 # HELP test_metric_no_type other help text
 # EOF`), "application/openmetrics-text", time.Now())
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 1, total)
 
-	md, ok := cache.GetMetadata("test_metric")
+	md, ok := sl.cache.GetMetadata("test_metric")
 	require.True(t, ok, "expected metadata to be present")
 	require.Equal(t, model.MetricTypeCounter, md.Type, "unexpected metric type")
 	require.Equal(t, "some help text", md.Help)
 	require.Equal(t, "metric", md.Unit)
 
-	md, ok = cache.GetMetadata("test_metric_no_help")
+	md, ok = sl.cache.GetMetadata("test_metric_no_help")
 	require.True(t, ok, "expected metadata to be present")
 	require.Equal(t, model.MetricTypeGauge, md.Type, "unexpected metric type")
 	require.Empty(t, md.Help)
 	require.Empty(t, md.Unit)
 
-	md, ok = cache.GetMetadata("test_metric_no_type")
+	md, ok = sl.cache.GetMetadata("test_metric_no_type")
 	require.True(t, ok, "expected metadata to be present")
 	require.Equal(t, model.MetricTypeUnknown, md.Type, "unexpected metric type")
 	require.Equal(t, "other help text", md.Help)
 	require.Empty(t, md.Unit)
 }
 
-func simpleTestScrapeLoop(t testing.TB) (context.Context, *scrapeLoop) {
-	// Need a full storage for correct Add/AddFast semantics.
-	s := teststorage.New(t)
-	t.Cleanup(func() { s.Close() })
-
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, &testScraper{}, s.Appender, 0)
-	t.Cleanup(func() { cancel() })
-
-	return ctx, sl
-}
-
 func TestScrapeLoopSeriesAdded(t *testing.T) {
-	ctx, sl := simpleTestScrapeLoop(t)
+	sl, _ := newTestScrapeLoop(t)
 
-	slApp := sl.appender(ctx)
-	total, added, seriesAdded, err := sl.append(slApp, []byte("test_metric 1\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("test_metric 1\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 1, total)
 	require.Equal(t, 1, added)
 	require.Equal(t, 1, seriesAdded)
 
-	slApp = sl.appender(ctx)
-	total, added, seriesAdded, err = sl.append(slApp, []byte("test_metric 1\n"), "text/plain", time.Time{})
-	require.NoError(t, slApp.Commit())
+	app = sl.appender()
+	total, added, seriesAdded, err = app.append([]byte("test_metric 1\n"), "text/plain", time.Time{})
+	require.NoError(t, app.Commit())
 	require.NoError(t, err)
 	require.Equal(t, 1, total)
 	require.Equal(t, 1, added)
@@ -1405,10 +1203,6 @@ func TestScrapeLoopSeriesAdded(t *testing.T) {
 }
 
 func TestScrapeLoopFailWithInvalidLabelsAfterRelabel(t *testing.T) {
-	s := teststorage.New(t)
-	defer s.Close()
-	ctx := t.Context()
-
 	target := &Target{
 		labels: labels.FromStrings("pod_label_invalid_012\xff", "test"),
 	}
@@ -1419,43 +1213,41 @@ func TestScrapeLoopFailWithInvalidLabelsAfterRelabel(t *testing.T) {
 		Replacement:          "$1",
 		NameValidationScheme: model.UTF8Validation,
 	}}
-	sl := newBasicScrapeLoop(t, ctx, &testScraper{}, s.Appender, 0)
-	sl.sampleMutator = func(l labels.Labels) labels.Labels {
-		return mutateSampleLabels(l, target, true, relabelConfig)
-	}
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.sampleMutator = func(l labels.Labels) labels.Labels {
+			return mutateSampleLabels(l, target, true, relabelConfig)
+		}
+	})
 
-	slApp := sl.appender(ctx)
-	total, added, seriesAdded, err := sl.append(slApp, []byte("test_metric 1\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("test_metric 1\n"), "text/plain", time.Time{})
 	require.ErrorContains(t, err, "invalid metric name or label names")
-	require.NoError(t, slApp.Rollback())
+	require.NoError(t, app.Rollback())
 	require.Equal(t, 1, total)
 	require.Equal(t, 0, added)
 	require.Equal(t, 0, seriesAdded)
 }
 
 func TestScrapeLoopFailLegacyUnderUTF8(t *testing.T) {
-	// Test that scrapes fail when default validation is utf8 but scrape config is
-	// legacy.
-	s := teststorage.New(t)
-	defer s.Close()
-	ctx := t.Context()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.validationScheme = model.LegacyValidation
+	})
 
-	sl := newBasicScrapeLoop(t, ctx, &testScraper{}, s.Appender, 0)
-	sl.validationScheme = model.LegacyValidation
-
-	slApp := sl.appender(ctx)
-	total, added, seriesAdded, err := sl.append(slApp, []byte("{\"test.metric\"} 1\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("{\"test.metric\"} 1\n"), "text/plain", time.Time{})
 	require.ErrorContains(t, err, "invalid metric name or label names")
-	require.NoError(t, slApp.Rollback())
+	require.NoError(t, app.Rollback())
 	require.Equal(t, 1, total)
 	require.Equal(t, 0, added)
 	require.Equal(t, 0, seriesAdded)
 
 	// When scrapeloop has validation set to UTF-8, the metric is allowed.
-	sl.validationScheme = model.UTF8Validation
+	sl, _ = newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.validationScheme = model.UTF8Validation
+	})
 
-	slApp = sl.appender(ctx)
-	total, added, seriesAdded, err = sl.append(slApp, []byte("{\"test.metric\"} 1\n"), "text/plain", time.Time{})
+	app = sl.appender()
+	total, added, seriesAdded, err = app.append([]byte("{\"test.metric\"} 1\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
 	require.Equal(t, 1, total)
 	require.Equal(t, 1, added)
@@ -1474,12 +1266,12 @@ func readTextParseTestMetrics(t testing.TB) []byte {
 
 func makeTestGauges(n int) []byte {
 	sb := bytes.Buffer{}
-	fmt.Fprintf(&sb, "# TYPE metric_a gauge\n")
-	fmt.Fprintf(&sb, "# HELP metric_a help text\n")
+	sb.WriteString("# TYPE metric_a gauge\n")
+	sb.WriteString("# HELP metric_a help text\n")
 	for i := range n {
-		fmt.Fprintf(&sb, "metric_a{foo=\"%d\",bar=\"%d\"} 1\n", i, i*100)
+		_, _ = fmt.Fprintf(&sb, "metric_a{foo=\"%d\",bar=\"%d\"} 1\n", i, i*100)
 	}
-	fmt.Fprintf(&sb, "# EOF\n")
+	sb.WriteString("# EOF\n")
 	return sb.Bytes()
 }
 
@@ -1550,7 +1342,7 @@ func TestPromTextToProto(t *testing.T) {
 //
 // Recommended CLI invocation:
 /*
-	export bench=append-v1 && go test ./scrape/... \
+	export bench=append && go test ./scrape/... \
 		-run '^$' -bench '^BenchmarkScrapeLoopAppend' \
 		-benchtime 5s -count 6 -cpu 2 -timeout 999m \
 		| tee ${bench}.txt
@@ -1576,16 +1368,19 @@ func BenchmarkScrapeLoopAppend(b *testing.B) {
 				{name: "PromProto", contentType: "application/vnd.google.protobuf", parsable: metricsProto},
 			} {
 				b.Run(fmt.Sprintf("fmt=%v", bcase.name), func(b *testing.B) {
-					ctx, sl := simpleTestScrapeLoop(b)
+					// Need a full storage for correct Add/AddFast semantics.
+					s := teststorage.New(b)
+					b.Cleanup(func() { _ = s.Close() })
 
-					slApp := sl.appender(ctx)
+					sl, _ := newTestScrapeLoop(b, withAppendable(s))
+					app := sl.appender()
 					ts := time.Time{}
 
 					b.ReportAllocs()
 					b.ResetTimer()
 					for b.Loop() {
 						ts = ts.Add(time.Second)
-						_, _, _, err := sl.append(slApp, bcase.parsable, bcase.contentType, ts)
+						_, _, _, err := app.append(bcase.parsable, bcase.contentType, ts)
 						if err != nil {
 							b.Fatal(err)
 						}
@@ -1596,30 +1391,85 @@ func BenchmarkScrapeLoopAppend(b *testing.B) {
 	}
 }
 
+func TestScrapeLoopScrapeAndReport(t *testing.T) {
+	parsableText := readTextParseTestMetrics(t)
+	// On windows \r is added when reading, but parsers do not support this. Kill it.
+	parsableText = bytes.ReplaceAll(parsableText, []byte("\r"), nil)
+
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.fallbackScrapeProtocol = "application/openmetrics-text"
+	})
+	scraper.scrapeFunc = func(_ context.Context, writer io.Writer) error {
+		_, err := writer.Write(parsableText)
+		return err
+	}
+
+	ts := time.Time{}
+
+	sl.scrapeAndReport(time.Time{}, ts, nil)
+	require.NoError(t, scraper.lastError)
+
+	require.Len(t, appTest.ResultSamples(), 1862)
+	require.Len(t, appTest.ResultMetadata(), 1862)
+}
+
+// Recommended CLI invocation:
+/*
+	export bench=scrapeAndReport && go test ./scrape/... \
+		-run '^$' -bench '^BenchmarkScrapeLoopScrapeAndReport' \
+		-benchtime 5s -count 6 -cpu 2 -timeout 999m \
+		| tee ${bench}.txt
+*/
+func BenchmarkScrapeLoopScrapeAndReport(b *testing.B) {
+	parsableText := readTextParseTestMetrics(b)
+
+	s := teststorage.New(b)
+	b.Cleanup(func() { _ = s.Close() })
+
+	sl, scraper := newTestScrapeLoop(b, func(sl *scrapeLoop) {
+		sl.appendable = s
+		sl.fallbackScrapeProtocol = "application/openmetrics-text"
+	})
+	scraper.scrapeFunc = func(_ context.Context, writer io.Writer) error {
+		_, err := writer.Write(parsableText)
+		return err
+	}
+
+	ts := time.Time{}
+
+	b.ReportAllocs()
+	b.ResetTimer()
+	for b.Loop() {
+		ts = ts.Add(time.Second)
+		sl.scrapeAndReport(time.Time{}, ts, nil)
+		require.NoError(b, scraper.lastError)
+	}
+}
+
 func TestSetOptionsHandlingStaleness(t *testing.T) {
 	s := teststorage.New(t, 600000)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
 	signal := make(chan struct{}, 1)
-	ctx, cancel := context.WithCancel(context.Background())
+	ctx, cancel := context.WithCancel(t.Context())
 	defer cancel()
 
 	// Function to run the scrape loop
 	runScrapeLoop := func(ctx context.Context, t *testing.T, cue int, action func(*scrapeLoop)) {
-		var (
-			scraper = &testScraper{}
-			app     = func(ctx context.Context) storage.Appender {
-				return s.Appender(ctx)
-			}
-		)
-		sl := newBasicScrapeLoop(t, ctx, scraper, app, 10*time.Millisecond)
+		sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+			sl.ctx = ctx
+			sl.appendable = s
+		})
+
 		numScrapes := 0
 		scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 			numScrapes++
 			if numScrapes == cue {
 				action(sl)
 			}
-			fmt.Fprintf(w, "metric_a{a=\"1\",b=\"1\"} %d\n", 42+numScrapes)
+			_, _ = fmt.Fprintf(w, "metric_a{a=\"1\",b=\"1\"} %d\n", 42+numScrapes)
 			return nil
 		}
 		sl.run(nil)
@@ -1644,25 +1494,25 @@ func TestSetOptionsHandlingStaleness(t *testing.T) {
 		t.Fatalf("Scrape wasn't stopped.")
 	}
 
-	ctx1, cancel := context.WithCancel(context.Background())
+	ctx1, cancel := context.WithCancel(t.Context())
 	defer cancel()
 
 	q, err := s.Querier(0, time.Now().UnixNano())
 
 	require.NoError(t, err)
-	defer q.Close()
+	t.Cleanup(func() { _ = q.Close() })
 
 	series := q.Select(ctx1, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", "metric_a"))
 
-	var results []floatSample
+	var results []sample
 	for series.Next() {
 		it := series.At().Iterator(nil)
 		for it.Next() == chunkenc.ValFloat {
 			t, v := it.At()
-			results = append(results, floatSample{
-				metric: series.At().Labels(),
-				t:      t,
-				f:      v,
+			results = append(results, sample{
+				L: series.At().Labels(),
+				T: t,
+				V: v,
 			})
 		}
 		require.NoError(t, it.Err())
@@ -1670,7 +1520,7 @@ func TestSetOptionsHandlingStaleness(t *testing.T) {
 	require.NoError(t, series.Err())
 	var c int
 	for _, s := range results {
-		if value.IsStaleNaN(s.f) {
+		if value.IsStaleNaN(s.V) {
 			c++
 		}
 	}
@@ -1678,25 +1528,25 @@ func TestSetOptionsHandlingStaleness(t *testing.T) {
 }
 
 func TestScrapeLoopRunCreatesStaleMarkersOnFailedScrape(t *testing.T) {
-	appender := &collectResultAppender{}
-	var (
-		signal  = make(chan struct{}, 1)
-		scraper = &testScraper{}
-		app     = func(context.Context) storage.Appender { return appender }
-	)
+	signal := make(chan struct{}, 1)
+
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = appTest
+		// Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+	})
 
-	ctx, cancel := context.WithCancel(context.Background())
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, ctx, scraper, app, 10*time.Millisecond, "text/plain")
 	// Succeed once, several failures, then stop.
 	numScrapes := 0
-
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		numScrapes++
 
 		switch numScrapes {
 		case 1:
-			w.Write([]byte("metric_a 42\n"))
+			_, _ = w.Write([]byte("metric_a 42\n"))
 			return nil
 		case 5:
 			cancel()
@@ -1715,36 +1565,39 @@ func TestScrapeLoopRunCreatesStaleMarkersOnFailedScrape(t *testing.T) {
 		require.FailNow(t, "Scrape wasn't stopped.")
 	}
 
-	// 1 successfully scraped sample, 1 stale marker after first fail, 5 report samples for
-	// each scrape successful or not.
-	require.Len(t, appender.resultFloats, 27, "Appended samples not as expected:\n%s", appender)
-	require.Equal(t, 42.0, appender.resultFloats[0].f, "Appended first sample not as expected")
-	require.True(t, value.IsStaleNaN(appender.resultFloats[6].f),
-		"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(appender.resultFloats[6].f))
+	got := appTest.ResultSamples()
+	// 1 successfully scraped sample
+	// 1 stale marker after first fail
+	// 5x 5 report samples for each scrape successful or not.
+	require.Len(t, got, 27, "Appended samples not as expected:\n%s", appTest)
+	require.Equal(t, 42.0, got[0].V, "Appended first sample not as expected")
+	require.True(t, value.IsStaleNaN(got[6].V),
+		"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(got[6].V))
 }
 
 func TestScrapeLoopRunCreatesStaleMarkersOnParseFailure(t *testing.T) {
-	appender := &collectResultAppender{}
-	var (
-		signal     = make(chan struct{}, 1)
-		scraper    = &testScraper{}
-		app        = func(context.Context) storage.Appender { return appender }
-		numScrapes = 0
-	)
+	signal := make(chan struct{}, 1)
 
-	ctx, cancel := context.WithCancel(context.Background())
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, ctx, scraper, app, 10*time.Millisecond, "text/plain")
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = appTest
+		// Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+	})
 
 	// Succeed once, several failures, then stop.
+	numScrapes := 0
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		numScrapes++
+
 		switch numScrapes {
 		case 1:
-			w.Write([]byte("metric_a 42\n"))
+			_, _ = w.Write([]byte("metric_a 42\n"))
 			return nil
 		case 2:
-			w.Write([]byte("7&-\n"))
+			_, _ = w.Write([]byte("7&-\n"))
 			return nil
 		case 3:
 			cancel()
@@ -1759,46 +1612,49 @@ func TestScrapeLoopRunCreatesStaleMarkersOnParseFailure(t *testing.T) {
 
 	select {
 	case <-signal:
+		// TODO(bwplotka): Prone to flakiness, depend on atomic numScrapes.
 	case <-time.After(5 * time.Second):
 		require.FailNow(t, "Scrape wasn't stopped.")
 	}
 
-	// 1 successfully scraped sample, 1 stale marker after first fail, 5 report samples for
-	// each scrape successful or not.
-	require.Len(t, appender.resultFloats, 17, "Appended samples not as expected:\n%s", appender)
-	require.Equal(t, 42.0, appender.resultFloats[0].f, "Appended first sample not as expected")
-	require.True(t, value.IsStaleNaN(appender.resultFloats[6].f),
-		"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(appender.resultFloats[6].f))
+	got := appTest.ResultSamples()
+	// 1 successfully scraped sample
+	// 1 stale marker after first fail
+	// 3x 5 report samples for each scrape successful or not.
+	require.Len(t, got, 17, "Appended samples not as expected:\n%s", appTest)
+	require.Equal(t, 42.0, got[0].V, "Appended first sample not as expected")
+	require.True(t, value.IsStaleNaN(got[6].V),
+		"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(got[6].V))
 }
 
-// If we have a target with sample_limit set and scrape initially works but then we hit the sample_limit error,
+// If we have a target with sample_limit set and scrape initially works, but then we hit the sample_limit error,
 // then we don't expect to see any StaleNaNs appended for the series that disappeared due to sample_limit error.
 func TestScrapeLoopRunCreatesStaleMarkersOnSampleLimit(t *testing.T) {
-	appender := &collectResultAppender{}
-	var (
-		signal     = make(chan struct{}, 1)
-		scraper    = &testScraper{}
-		app        = func(_ context.Context) storage.Appender { return appender }
-		numScrapes = 0
-	)
+	signal := make(chan struct{}, 1)
 
-	ctx, cancel := context.WithCancel(context.Background())
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, ctx, scraper, app, 10*time.Millisecond, "text/plain")
-	sl.sampleLimit = 4
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = appTest
+		// Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+		sl.sampleLimit = 4
+	})
 
 	// Succeed once, several failures, then stop.
+	numScrapes := 0
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		numScrapes++
 		switch numScrapes {
 		case 1:
-			w.Write([]byte("metric_a 10\nmetric_b 10\nmetric_c 10\nmetric_d 10\n"))
+			_, _ = w.Write([]byte("metric_a 10\nmetric_b 10\nmetric_c 10\nmetric_d 10\n"))
 			return nil
 		case 2:
-			w.Write([]byte("metric_a 20\nmetric_b 20\nmetric_c 20\nmetric_d 20\nmetric_e 999\n"))
+			_, _ = w.Write([]byte("metric_a 20\nmetric_b 20\nmetric_c 20\nmetric_d 20\nmetric_e 999\n"))
 			return nil
 		case 3:
-			w.Write([]byte("metric_a 30\nmetric_b 30\nmetric_c 30\nmetric_d 30\n"))
+			_, _ = w.Write([]byte("metric_a 30\nmetric_b 30\nmetric_c 30\nmetric_d 30\n"))
 			return nil
 		case 4:
 			cancel()
@@ -1817,49 +1673,52 @@ func TestScrapeLoopRunCreatesStaleMarkersOnSampleLimit(t *testing.T) {
 		require.FailNow(t, "Scrape wasn't stopped.")
 	}
 
+	got := appTest.ResultSamples()
+
 	// 4 scrapes in total:
 	// #1 - success - 4 samples appended + 5 report series
 	// #2 - sample_limit exceeded - no samples appended, only 5 report series
 	// #3 - success - 4 samples appended + 5 report series
 	// #4 - scrape canceled - 4 StaleNaNs appended because of scrape error + 5 report series
-	require.Len(t, appender.resultFloats, (4+5)+5+(4+5)+(4+5), "Appended samples not as expected:\n%s", appender)
+	require.Len(t, got, (4+5)+5+(4+5)+(4+5), "Appended samples not as expected:\n%s", appTest)
 	// Expect first 4 samples to be metric_X [0-3].
 	for i := range 4 {
-		require.Equal(t, 10.0, appender.resultFloats[i].f, "Appended %d sample not as expected", i)
+		require.Equal(t, 10.0, got[i].V, "Appended %d sample not as expected", i)
 	}
 	// Next 5 samples are report series [4-8].
 	// Next 5 samples are report series for the second scrape [9-13].
 	// Expect first 4 samples to be metric_X from the third scrape [14-17].
 	for i := 14; i <= 17; i++ {
-		require.Equal(t, 30.0, appender.resultFloats[i].f, "Appended %d sample not as expected", i)
+		require.Equal(t, 30.0, got[i].V, "Appended %d sample not as expected", i)
 	}
 	// Next 5 samples are report series [18-22].
 	// Next 5 samples are report series [23-26].
 	for i := 23; i <= 26; i++ {
-		require.True(t, value.IsStaleNaN(appender.resultFloats[i].f),
-			"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(appender.resultFloats[i].f))
+		require.True(t, value.IsStaleNaN(got[i].V),
+			"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(got[i].V))
 	}
 }
 
 func TestScrapeLoopCache(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	appender := &collectResultAppender{}
-	var (
-		signal  = make(chan struct{}, 1)
-		scraper = &testScraper{}
-		app     = func(ctx context.Context) storage.Appender { appender.next = s.Appender(ctx); return appender }
-	)
+	signal := make(chan struct{}, 1)
 
-	ctx, cancel := context.WithCancel(context.Background())
-	// Decreasing the scrape interval could make the test fail, as multiple scrapes might be initiated at identical millisecond timestamps.
-	// See https://github.com/prometheus/prometheus/issues/12727.
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, ctx, scraper, app, 100*time.Millisecond, "text/plain")
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable().Then(s)
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.l = promslog.New(&promslog.Config{})
+		sl.appendable = appTest
+		// Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+		// Decreasing the scrape interval could make the test fail, as multiple scrapes might be initiated at identical millisecond timestamps.
+		// See https://github.com/prometheus/prometheus/issues/12727.
+		sl.interval = 100 * time.Millisecond
+	})
 
 	numScrapes := 0
-
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		switch numScrapes {
 		case 1, 2:
@@ -1877,10 +1736,10 @@ func TestScrapeLoopCache(t *testing.T) {
 		numScrapes++
 		switch numScrapes {
 		case 1:
-			w.Write([]byte("metric_a 42\nmetric_b 43\n"))
+			_, _ = w.Write([]byte("metric_a 42\nmetric_b 43\n"))
 			return nil
 		case 3:
-			w.Write([]byte("metric_a 44\n"))
+			_, _ = w.Write([]byte("metric_a 44\n"))
 			return nil
 		case 4:
 			cancel()
@@ -1899,29 +1758,23 @@ func TestScrapeLoopCache(t *testing.T) {
 		require.FailNow(t, "Scrape wasn't stopped.")
 	}
 
-	// 1 successfully scraped sample, 1 stale marker after first fail, 5 report samples for
-	// each scrape successful or not.
-	require.Len(t, appender.resultFloats, 26, "Appended samples not as expected:\n%s", appender)
+	// 3 successfully scraped samples
+	// 3 stale marker after samples were missing.
+	// 4x 5 report samples for each scrape successful or not.
+	require.Len(t, appTest.ResultSamples(), 26, "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoopCacheMemoryExhaustionProtection(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	sapp := s.Appender(context.Background())
-
-	appender := &collectResultAppender{next: sapp}
-	var (
-		signal  = make(chan struct{}, 1)
-		scraper = &testScraper{}
-		app     = func(context.Context) storage.Appender { return appender }
-	)
-
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, scraper, app, 10*time.Millisecond)
+	signal := make(chan struct{}, 1)
 
+	ctx, cancel := context.WithCancel(t.Context())
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+	})
 	numScrapes := 0
-
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		numScrapes++
 		if numScrapes < 5 {
@@ -1929,7 +1782,7 @@ func TestScrapeLoopCacheMemoryExhaustionProtection(t *testing.T) {
 			for i := range 500 {
 				s = fmt.Sprintf("%smetric_%d_%d 42\n", s, i, numScrapes)
 			}
-			w.Write([]byte(s + "&"))
+			_, _ = w.Write([]byte(s + "&"))
 		} else {
 			cancel()
 		}
@@ -2004,37 +1857,38 @@ func TestScrapeLoopAppend(t *testing.T) {
 	}
 
 	for _, test := range tests {
-		app := &collectResultAppender{}
-
 		discoveryLabels := &Target{
 			labels: labels.FromStrings(test.discoveryLabels...),
 		}
 
-		sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-		sl.sampleMutator = func(l labels.Labels) labels.Labels {
-			return mutateSampleLabels(l, discoveryLabels, test.honorLabels, nil)
-		}
-		sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
-			return mutateReportSampleLabels(l, discoveryLabels)
-		}
+		appTest := teststorage.NewAppendable()
+		sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+			sl.appendable = appTest
+			sl.sampleMutator = func(l labels.Labels) labels.Labels {
+				return mutateSampleLabels(l, discoveryLabels, test.honorLabels, nil)
+			}
+			sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
+				return mutateReportSampleLabels(l, discoveryLabels)
+			}
+		})
 
 		now := time.Now()
 
-		slApp := sl.appender(context.Background())
-		_, _, _, err := sl.append(slApp, []byte(test.scrapeLabels), "text/plain", now)
+		app := sl.appender()
+		_, _, _, err := app.append([]byte(test.scrapeLabels), "text/plain", now)
 		require.NoError(t, err)
-		require.NoError(t, slApp.Commit())
+		require.NoError(t, app.Commit())
 
-		expected := []floatSample{
+		expected := []sample{
 			{
-				metric: test.expLset,
-				t:      timestamp.FromTime(now),
-				f:      test.expValue,
+				L: test.expLset,
+				T: timestamp.FromTime(now),
+				V: test.expValue,
 			},
 		}
 
 		t.Logf("Test:%s", test.title)
-		requireEqual(t, expected, app.resultFloats)
+		requireEqual(t, expected, appTest.ResultSamples())
 	}
 }
 
@@ -2042,13 +1896,12 @@ func requireEqual(t *testing.T, expected, actual any, msgAndArgs ...any) {
 	t.Helper()
 	testutil.RequireEqualWithOptions(t, expected, actual,
 		[]cmp.Option{
-			cmp.Comparer(equalFloatSamples),
-			cmp.AllowUnexported(histogramSample{}),
+			cmp.Comparer(func(a, b sample) bool { return a.Equals(b) }),
 			// StaleNaN samples are generated by iterating over a map, which means that the order
 			// of samples might be different on every test run. Sort series by label to avoid
 			// test failures because of that.
-			cmpopts.SortSlices(func(a, b floatSample) int {
-				return labels.Compare(a.metric, b.metric)
+			cmpopts.SortSlices(func(a, b sample) int {
+				return labels.Compare(a.L, b.L)
 			}),
 		},
 		msgAndArgs...)
@@ -2106,32 +1959,34 @@ func TestScrapeLoopAppendForConflictingPrefixedLabels(t *testing.T) {
 
 	for name, tc := range testcases {
 		t.Run(name, func(t *testing.T) {
-			app := &collectResultAppender{}
-			sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-			sl.sampleMutator = func(l labels.Labels) labels.Labels {
-				return mutateSampleLabels(l, &Target{labels: labels.FromStrings(tc.targetLabels...)}, false, nil)
-			}
-			slApp := sl.appender(context.Background())
-			_, _, _, err := sl.append(slApp, []byte(tc.exposedLabels), "text/plain", time.Date(2000, 1, 1, 1, 0, 0, 0, time.UTC))
+			appTest := teststorage.NewAppendable()
+			sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+				sl.appendable = appTest
+				sl.sampleMutator = func(l labels.Labels) labels.Labels {
+					return mutateSampleLabels(l, &Target{labels: labels.FromStrings(tc.targetLabels...)}, false, nil)
+				}
+			})
+
+			app := sl.appender()
+			_, _, _, err := app.append([]byte(tc.exposedLabels), "text/plain", time.Date(2000, 1, 1, 1, 0, 0, 0, time.UTC))
 			require.NoError(t, err)
 
-			require.NoError(t, slApp.Commit())
+			require.NoError(t, app.Commit())
 
-			requireEqual(t, []floatSample{
+			requireEqual(t, []sample{
 				{
-					metric: labels.FromStrings(tc.expected...),
-					t:      timestamp.FromTime(time.Date(2000, 1, 1, 1, 0, 0, 0, time.UTC)),
-					f:      0,
+					L: labels.FromStrings(tc.expected...),
+					T: timestamp.FromTime(time.Date(2000, 1, 1, 1, 0, 0, 0, time.UTC)),
+					V: 0,
 				},
-			}, app.resultFloats)
+			}, appTest.ResultSamples())
 		})
 	}
 }
 
 func TestScrapeLoopAppendCacheEntryButErrNotFound(t *testing.T) {
-	// collectResultAppender's AddFast always returns ErrNotFound if we don't give it a next.
-	app := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	fakeRef := storage.SeriesRef(1)
 	expValue := float64(1)
@@ -2141,7 +1996,8 @@ func TestScrapeLoopAppendCacheEntryButErrNotFound(t *testing.T) {
 	require.NoError(t, warning)
 
 	var lset labels.Labels
-	p.Next()
+	_, err := p.Next()
+	require.NoError(t, err)
 	p.Labels(&lset)
 	hash := lset.Hash()
 
@@ -2149,36 +2005,43 @@ func TestScrapeLoopAppendCacheEntryButErrNotFound(t *testing.T) {
 	sl.cache.addRef(metric, fakeRef, lset, hash)
 	now := time.Now()
 
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, metric, "text/plain", now)
+	app := sl.appender()
+	_, _, _, err = app.append(metric, "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	expected := []floatSample{
+	expected := []sample{
 		{
-			metric: lset,
-			t:      timestamp.FromTime(now),
-			f:      expValue,
+			L: lset,
+			T: timestamp.FromTime(now),
+			V: expValue,
 		},
 	}
 
-	require.Equal(t, expected, app.resultFloats)
+	require.Equal(t, expected, appTest.ResultSamples())
 }
 
+type appendableFunc func(ctx context.Context) storage.Appender
+
+func (a appendableFunc) Appender(ctx context.Context) storage.Appender { return a(ctx) }
+
 func TestScrapeLoopAppendSampleLimit(t *testing.T) {
-	resApp := &collectResultAppender{}
-	app := &limitAppender{Appender: resApp, limit: 1}
-
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-	sl.sampleMutator = func(l labels.Labels) labels.Labels {
-		if l.Has("deleteme") {
-			return labels.EmptyLabels()
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appendableFunc(func(ctx context.Context) storage.Appender {
+			// Chain appTest to verify what samples passed through.
+			return &limitAppender{Appender: appTest.Appender(ctx), limit: 1}
+		})
+		sl.sampleMutator = func(l labels.Labels) labels.Labels {
+			if l.Has("deleteme") {
+				return labels.EmptyLabels()
+			}
+			return l
 		}
-		return l
-	}
-	sl.sampleLimit = app.limit
+		sl.sampleLimit = 1 // Same as limitAppender.limit
+	})
 
-	// Get the value of the Counter before performing the append.
+	// Get the value of the Counter before performing append.
 	beforeMetric := dto.Metric{}
 	err := sl.metrics.targetScrapeSampleLimit.Write(&beforeMetric)
 	require.NoError(t, err)
@@ -2186,10 +2049,10 @@ func TestScrapeLoopAppendSampleLimit(t *testing.T) {
 	beforeMetricValue := beforeMetric.GetCounter().GetValue()
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	total, added, seriesAdded, err := sl.append(app, []byte("metric_a 1\nmetric_b 1\nmetric_c 1\n"), "text/plain", now)
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("metric_a 1\nmetric_b 1\nmetric_c 1\n"), "text/plain", now)
 	require.ErrorIs(t, err, errSampleLimit)
-	require.NoError(t, slApp.Rollback())
+	require.NoError(t, app.Rollback())
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
 	require.Equal(t, 1, seriesAdded)
@@ -2200,42 +2063,44 @@ func TestScrapeLoopAppendSampleLimit(t *testing.T) {
 	err = sl.metrics.targetScrapeSampleLimit.Write(&metric)
 	require.NoError(t, err)
 
-	value := metric.GetCounter().GetValue()
-	change := value - beforeMetricValue
+	v := metric.GetCounter().GetValue()
+	change := v - beforeMetricValue
 	require.Equal(t, 1.0, change, "Unexpected change of sample limit metric: %f", change)
 
 	// And verify that we got the samples that fit under the limit.
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 	}
-	requireEqual(t, want, resApp.rolledbackFloats, "Appended samples not as expected:\n%s", appender)
+	requireEqual(t, want, appTest.RolledbackSamples(), "Appended samples not as expected:\n%s", appTest)
 
 	now = time.Now()
-	slApp = sl.appender(context.Background())
-	total, added, seriesAdded, err = sl.append(slApp, []byte("metric_a 1\nmetric_b 1\nmetric_c{deleteme=\"yes\"} 1\nmetric_d 1\nmetric_e 1\nmetric_f 1\nmetric_g 1\nmetric_h{deleteme=\"yes\"} 1\nmetric_i{deleteme=\"yes\"} 1\n"), "text/plain", now)
+	app = sl.appender()
+	total, added, seriesAdded, err = app.append([]byte("metric_a 1\nmetric_b 1\nmetric_c{deleteme=\"yes\"} 1\nmetric_d 1\nmetric_e 1\nmetric_f 1\nmetric_g 1\nmetric_h{deleteme=\"yes\"} 1\nmetric_i{deleteme=\"yes\"} 1\n"), "text/plain", now)
 	require.ErrorIs(t, err, errSampleLimit)
-	require.NoError(t, slApp.Rollback())
+	require.NoError(t, app.Rollback())
 	require.Equal(t, 9, total)
 	require.Equal(t, 6, added)
-	require.Equal(t, 0, seriesAdded)
+	require.Equal(t, 1, seriesAdded)
 }
 
 func TestScrapeLoop_HistogramBucketLimit(t *testing.T) {
-	resApp := &collectResultAppender{}
-	app := &bucketLimitAppender{Appender: resApp, limit: 2}
-
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-	sl.enableNativeHistogramScraping = true
-	sl.sampleMutator = func(l labels.Labels) labels.Labels {
-		if l.Has("deleteme") {
-			return labels.EmptyLabels()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appendableFunc(func(ctx context.Context) storage.Appender {
+			return &bucketLimitAppender{Appender: teststorage.NewAppendable().Appender(ctx), limit: 2}
+		})
+		sl.enableNativeHistogramScraping = true
+		sl.sampleMutator = func(l labels.Labels) labels.Labels {
+			if l.Has("deleteme") {
+				return labels.EmptyLabels()
+			}
+			return l
 		}
-		return l
-	}
+	})
+	app := sl.appender()
 
 	metric := dto.Metric{}
 	err := sl.metrics.targetScrapeNativeHistogramBucketLimit.Write(&metric)
@@ -2254,7 +2119,7 @@ func TestScrapeLoop_HistogramBucketLimit(t *testing.T) {
 		[]string{"size"},
 	)
 	registry := prometheus.NewRegistry()
-	registry.Register(nativeHistogram)
+	require.NoError(t, registry.Register(nativeHistogram))
 	nativeHistogram.WithLabelValues("S").Observe(1.0)
 	nativeHistogram.WithLabelValues("M").Observe(1.0)
 	nativeHistogram.WithLabelValues("L").Observe(1.0)
@@ -2270,7 +2135,7 @@ func TestScrapeLoop_HistogramBucketLimit(t *testing.T) {
 	require.NoError(t, err)
 
 	now := time.Now()
-	total, added, seriesAdded, err := sl.append(app, msg, "application/vnd.google.protobuf", now)
+	total, added, seriesAdded, err := app.append(msg, "application/vnd.google.protobuf", now)
 	require.NoError(t, err)
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
@@ -2293,11 +2158,11 @@ func TestScrapeLoop_HistogramBucketLimit(t *testing.T) {
 	require.NoError(t, err)
 
 	now = time.Now()
-	total, added, seriesAdded, err = sl.append(app, msg, "application/vnd.google.protobuf", now)
+	total, added, seriesAdded, err = app.append(msg, "application/vnd.google.protobuf", now)
 	require.NoError(t, err)
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
-	require.Equal(t, 3, seriesAdded)
+	require.Equal(t, 0, seriesAdded) // Series are cached.
 
 	err = sl.metrics.targetScrapeNativeHistogramBucketLimit.Write(&metric)
 	require.NoError(t, err)
@@ -2316,14 +2181,14 @@ func TestScrapeLoop_HistogramBucketLimit(t *testing.T) {
 	require.NoError(t, err)
 
 	now = time.Now()
-	total, added, seriesAdded, err = sl.append(app, msg, "application/vnd.google.protobuf", now)
+	total, added, seriesAdded, err = app.append(msg, "application/vnd.google.protobuf", now)
 	if !errors.Is(err, errBucketLimit) {
 		t.Fatalf("Did not see expected histogram bucket limit error: %s", err)
 	}
 	require.NoError(t, app.Rollback())
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
-	require.Equal(t, 0, seriesAdded)
+	require.Equal(t, 0, seriesAdded) // Series are cached.
 
 	err = sl.metrics.targetScrapeNativeHistogramBucketLimit.Write(&metric)
 	require.NoError(t, err)
@@ -2333,151 +2198,149 @@ func TestScrapeLoop_HistogramBucketLimit(t *testing.T) {
 
 func TestScrapeLoop_ChangingMetricString(t *testing.T) {
 	// This is a regression test for the scrape loop cache not properly maintaining
-	// IDs when the string representation of a metric changes across a scrape. Thus
+	// IDs when the string representation of a metric changes across a scrape. Thus,
 	// we use a real storage appender here.
-	s := teststorage.New(t)
-	defer s.Close()
-
-	capp := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return capp }, 0)
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte(`metric_a{a="1",b="1"} 1`), "text/plain", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(`metric_a{a="1",b="1"} 1`), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	slApp = sl.appender(context.Background())
-	_, _, _, err = sl.append(slApp, []byte(`metric_a{b="1",a="1"} 2`), "text/plain", now.Add(time.Minute))
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(`metric_a{b="1",a="1"} 2`), "text/plain", now.Add(time.Minute))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
-			t:      timestamp.FromTime(now.Add(time.Minute)),
-			f:      2,
+			L: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
+			T: timestamp.FromTime(now.Add(time.Minute)),
+			V: 2,
 		},
 	}
-	require.Equal(t, want, capp.resultFloats, "Appended samples not as expected:\n%s", appender)
+	require.Equal(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoopAppendFailsWithNoContentType(t *testing.T) {
-	app := &collectResultAppender{}
-
-	// Explicitly setting the lack of fallback protocol here to make it obvious.
-	sl := newBasicScrapeLoopWithFallback(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0, "")
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		// Explicitly setting the lack of fallback protocol here to make it obvious.
+		sl.fallbackScrapeProtocol = ""
+	})
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte("metric_a 1\n"), "", now)
-	// We expect the appropriate error.
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("metric_a 1\n"), "", now)
+	// We expected the appropriate error.
 	require.ErrorContains(t, err, "non-compliant scrape target sending blank Content-Type and no fallback_scrape_protocol specified for target", "Expected \"non-compliant scrape\" error but got: %s", err)
 }
 
+// TestScrapeLoopAppendEmptyWithNoContentType ensures we there are no errors when we get a blank scrape or just want to append a stale marker.
 func TestScrapeLoopAppendEmptyWithNoContentType(t *testing.T) {
-	// This test ensures we there are no errors when we get a blank scrape or just want to append a stale marker.
-	app := &collectResultAppender{}
-
-	// Explicitly setting the lack of fallback protocol here to make it obvious.
-	sl := newBasicScrapeLoopWithFallback(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0, "")
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		// Explicitly setting the lack of fallback protocol here to make it obvious.
+		sl.fallbackScrapeProtocol = ""
+	})
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte(""), "", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(""), "", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 }
 
 func TestScrapeLoopAppendStaleness(t *testing.T) {
-	app := &collectResultAppender{}
-
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte("metric_a 1\n"), "text/plain", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("metric_a 1\n"), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	slApp = sl.appender(context.Background())
-	_, _, _, err = sl.append(slApp, []byte(""), "", now.Add(time.Second))
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(""), "", now.Add(time.Second))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now.Add(time.Second)),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now.Add(time.Second)),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 	}
-	requireEqual(t, want, app.resultFloats, "Appended samples not as expected:\n%s", appender)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoopAppendNoStalenessIfTimestamp(t *testing.T) {
-	app := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte("metric_a 1 1000\n"), "text/plain", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("metric_a 1 1000\n"), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	slApp = sl.appender(context.Background())
-	_, _, _, err = sl.append(slApp, []byte(""), "", now.Add(time.Second))
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(""), "", now.Add(time.Second))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      1000,
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: 1000,
+			V: 1,
 		},
 	}
-	require.Equal(t, want, app.resultFloats, "Appended samples not as expected:\n%s", appender)
+	require.Equal(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoopAppendStalenessIfTrackTimestampStaleness(t *testing.T) {
-	app := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-	sl.trackTimestampsStaleness = true
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.trackTimestampsStaleness = true
+	})
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte("metric_a 1 1000\n"), "text/plain", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("metric_a 1 1000\n"), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	slApp = sl.appender(context.Background())
-	_, _, _, err = sl.append(slApp, []byte(""), "", now.Add(time.Second))
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(""), "", now.Add(time.Second))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      1000,
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: 1000,
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now.Add(time.Second)),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now.Add(time.Second)),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 	}
-	requireEqual(t, want, app.resultFloats, "Appended samples not as expected:\n%s", appender)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoopAppendExemplar(t *testing.T) {
@@ -2488,18 +2351,16 @@ func TestScrapeLoopAppendExemplar(t *testing.T) {
 		scrapeText                      string
 		contentType                     string
 		discoveryLabels                 []string
-		floats                          []floatSample
-		histograms                      []histogramSample
-		exemplars                       []exemplar.Exemplar
+		samples                         []sample
 	}{
 		{
 			title:           "Metric without exemplars",
 			scrapeText:      "metric_total{n=\"1\"} 0\n# EOF",
 			contentType:     "application/openmetrics-text",
 			discoveryLabels: []string{"n", "2"},
-			floats: []floatSample{{
-				metric: labels.FromStrings("__name__", "metric_total", "exported_n", "1", "n", "2"),
-				f:      0,
+			samples: []sample{{
+				L: labels.FromStrings("__name__", "metric_total", "exported_n", "1", "n", "2"),
+				V: 0,
 			}},
 		},
 		{
@@ -2507,26 +2368,24 @@ func TestScrapeLoopAppendExemplar(t *testing.T) {
 			scrapeText:      "metric_total{n=\"1\"} 0 # {a=\"abc\"} 1.0\n# EOF",
 			contentType:     "application/openmetrics-text",
 			discoveryLabels: []string{"n", "2"},
-			floats: []floatSample{{
-				metric: labels.FromStrings("__name__", "metric_total", "exported_n", "1", "n", "2"),
-				f:      0,
+			samples: []sample{{
+				L: labels.FromStrings("__name__", "metric_total", "exported_n", "1", "n", "2"),
+				V: 0,
+				ES: []exemplar.Exemplar{
+					{Labels: labels.FromStrings("a", "abc"), Value: 1},
+				},
 			}},
-			exemplars: []exemplar.Exemplar{
-				{Labels: labels.FromStrings("a", "abc"), Value: 1},
-			},
 		},
 		{
 			title:           "Metric with exemplars and TS",
 			scrapeText:      "metric_total{n=\"1\"} 0 # {a=\"abc\"} 1.0 10000\n# EOF",
 			contentType:     "application/openmetrics-text",
 			discoveryLabels: []string{"n", "2"},
-			floats: []floatSample{{
-				metric: labels.FromStrings("__name__", "metric_total", "exported_n", "1", "n", "2"),
-				f:      0,
+			samples: []sample{{
+				L:  labels.FromStrings("__name__", "metric_total", "exported_n", "1", "n", "2"),
+				V:  0,
+				ES: []exemplar.Exemplar{{Labels: labels.FromStrings("a", "abc"), Value: 1, Ts: 10000000, HasTs: true}},
 			}},
-			exemplars: []exemplar.Exemplar{
-				{Labels: labels.FromStrings("a", "abc"), Value: 1, Ts: 10000000, HasTs: true},
-			},
 		},
 		{
 			title: "Two metrics and exemplars",
@@ -2534,17 +2393,15 @@ func TestScrapeLoopAppendExemplar(t *testing.T) {
 metric_total{n="2"} 2 # {t="2"} 2.0 20000
 # EOF`,
 			contentType: "application/openmetrics-text",
-			floats: []floatSample{{
-				metric: labels.FromStrings("__name__", "metric_total", "n", "1"),
-				f:      1,
+			samples: []sample{{
+				L:  labels.FromStrings("__name__", "metric_total", "n", "1"),
+				V:  1,
+				ES: []exemplar.Exemplar{{Labels: labels.FromStrings("t", "1"), Value: 1, Ts: 10000000, HasTs: true}},
 			}, {
-				metric: labels.FromStrings("__name__", "metric_total", "n", "2"),
-				f:      2,
+				L:  labels.FromStrings("__name__", "metric_total", "n", "2"),
+				V:  2,
+				ES: []exemplar.Exemplar{{Labels: labels.FromStrings("t", "2"), Value: 2, Ts: 20000000, HasTs: true}},
 			}},
-			exemplars: []exemplar.Exemplar{
-				{Labels: labels.FromStrings("t", "1"), Value: 1, Ts: 10000000, HasTs: true},
-				{Labels: labels.FromStrings("t", "2"), Value: 2, Ts: 20000000, HasTs: true},
-			},
 		},
 		{
 			title: "Native histogram with three exemplars from classic buckets",
@@ -2636,10 +2493,10 @@ metric: <
 
 `,
 			contentType: "application/vnd.google.protobuf",
-			histograms: []histogramSample{{
-				t:      1234568,
-				metric: labels.FromStrings("__name__", "test_histogram"),
-				h: &histogram.Histogram{
+			samples: []sample{{
+				T: 1234568,
+				L: labels.FromStrings("__name__", "test_histogram"),
+				H: &histogram.Histogram{
 					Count:         175,
 					ZeroCount:     2,
 					Sum:           0.0008280461746287094,
@@ -2656,12 +2513,12 @@ metric: <
 					PositiveBuckets: []int64{1, 2, -1, -1},
 					NegativeBuckets: []int64{1, 3, -2, -1, 1},
 				},
+				ES: []exemplar.Exemplar{
+					// Native histogram exemplars are arranged by timestamp, and those with missing timestamps are dropped.
+					{Labels: labels.FromStrings("dummyID", "58215"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
+					{Labels: labels.FromStrings("dummyID", "59727"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
+				},
 			}},
-			exemplars: []exemplar.Exemplar{
-				// Native histogram exemplars are arranged by timestamp, and those with missing timestamps are dropped.
-				{Labels: labels.FromStrings("dummyID", "58215"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
-				{Labels: labels.FromStrings("dummyID", "59727"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
-			},
 		},
 		{
 			title: "Native histogram with three exemplars scraped as classic histogram",
@@ -2754,46 +2611,50 @@ metric: <
 `,
 			alwaysScrapeClassicHist: true,
 			contentType:             "application/vnd.google.protobuf",
-			floats: []floatSample{
-				{metric: labels.FromStrings("__name__", "test_histogram_count"), t: 1234568, f: 175},
-				{metric: labels.FromStrings("__name__", "test_histogram_sum"), t: 1234568, f: 0.0008280461746287094},
-				{metric: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0004899999999999998"), t: 1234568, f: 2},
-				{metric: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0003899999999999998"), t: 1234568, f: 4},
-				{metric: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0002899999999999998"), t: 1234568, f: 16},
-				{metric: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0001899999999999998"), t: 1234568, f: 32},
-				{metric: labels.FromStrings("__name__", "test_histogram_bucket", "le", "+Inf"), t: 1234568, f: 175},
-			},
-			histograms: []histogramSample{{
-				t:      1234568,
-				metric: labels.FromStrings("__name__", "test_histogram"),
-				h: &histogram.Histogram{
-					Count:         175,
-					ZeroCount:     2,
-					Sum:           0.0008280461746287094,
-					ZeroThreshold: 2.938735877055719e-39,
-					Schema:        3,
-					PositiveSpans: []histogram.Span{
-						{Offset: -161, Length: 1},
-						{Offset: 8, Length: 3},
-					},
-					NegativeSpans: []histogram.Span{
-						{Offset: -162, Length: 1},
-						{Offset: 23, Length: 4},
-					},
-					PositiveBuckets: []int64{1, 2, -1, -1},
-					NegativeBuckets: []int64{1, 3, -2, -1, 1},
+			samples: []sample{
+				{L: labels.FromStrings("__name__", "test_histogram_count"), T: 1234568, V: 175},
+				{L: labels.FromStrings("__name__", "test_histogram_sum"), T: 1234568, V: 0.0008280461746287094},
+				{L: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0004899999999999998"), T: 1234568, V: 2},
+				{
+					L: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0003899999999999998"), T: 1234568, V: 4,
+					ES: []exemplar.Exemplar{{Labels: labels.FromStrings("dummyID", "59727"), Value: -0.00039, Ts: 1625851155146, HasTs: true}},
+				},
+				{
+					L: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0002899999999999998"), T: 1234568, V: 16,
+					ES: []exemplar.Exemplar{{Labels: labels.FromStrings("dummyID", "5617"), Value: -0.00029, Ts: 1234568, HasTs: false}},
+				},
+				{
+					L: labels.FromStrings("__name__", "test_histogram_bucket", "le", "-0.0001899999999999998"), T: 1234568, V: 32,
+					ES: []exemplar.Exemplar{{Labels: labels.FromStrings("dummyID", "58215"), Value: -0.00019, Ts: 1625851055146, HasTs: true}},
+				},
+				{L: labels.FromStrings("__name__", "test_histogram_bucket", "le", "+Inf"), T: 1234568, V: 175},
+				{
+					T: 1234568,
+					L: labels.FromStrings("__name__", "test_histogram"),
+					H: &histogram.Histogram{
+						Count:         175,
+						ZeroCount:     2,
+						Sum:           0.0008280461746287094,
+						ZeroThreshold: 2.938735877055719e-39,
+						Schema:        3,
+						PositiveSpans: []histogram.Span{
+							{Offset: -161, Length: 1},
+							{Offset: 8, Length: 3},
+						},
+						NegativeSpans: []histogram.Span{
+							{Offset: -162, Length: 1},
+							{Offset: 23, Length: 4},
+						},
+						PositiveBuckets: []int64{1, 2, -1, -1},
+						NegativeBuckets: []int64{1, 3, -2, -1, 1},
+					},
+					ES: []exemplar.Exemplar{
+						// Native histogram one is arranged by timestamp.
+						// Exemplars with missing timestamps are dropped for native histograms.
+						{Labels: labels.FromStrings("dummyID", "58215"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
+						{Labels: labels.FromStrings("dummyID", "59727"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
+					},
 				},
-			}},
-			exemplars: []exemplar.Exemplar{
-				// Native histogram one is arranged by timestamp.
-				// Exemplars with missing timestamps are dropped for native histograms.
-				{Labels: labels.FromStrings("dummyID", "58215"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
-				{Labels: labels.FromStrings("dummyID", "59727"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
-				// Classic histogram one is in order of appearance.
-				// Exemplars with missing timestamps are supported for classic histograms.
-				{Labels: labels.FromStrings("dummyID", "59727"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
-				{Labels: labels.FromStrings("dummyID", "5617"), Value: -0.00029, Ts: 1234568, HasTs: false},
-				{Labels: labels.FromStrings("dummyID", "58215"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
 			},
 		},
 		{
@@ -2869,10 +2730,10 @@ metric: <
 >
 
 `,
-			histograms: []histogramSample{{
-				t:      1234568,
-				metric: labels.FromStrings("__name__", "test_histogram"),
-				h: &histogram.Histogram{
+			samples: []sample{{
+				T: 1234568,
+				L: labels.FromStrings("__name__", "test_histogram"),
+				H: &histogram.Histogram{
 					Count:         175,
 					ZeroCount:     2,
 					Sum:           0.0008280461746287094,
@@ -2889,12 +2750,12 @@ metric: <
 					PositiveBuckets: []int64{1, 2, -1, -1},
 					NegativeBuckets: []int64{1, 3, -2, -1, 1},
 				},
+				ES: []exemplar.Exemplar{
+					// Exemplars with missing timestamps are dropped for native histograms.
+					{Labels: labels.FromStrings("dummyID", "58242"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
+					{Labels: labels.FromStrings("dummyID", "59732"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
+				},
 			}},
-			exemplars: []exemplar.Exemplar{
-				// Exemplars with missing timestamps are dropped for native histograms.
-				{Labels: labels.FromStrings("dummyID", "58242"), Value: -0.00019, Ts: 1625851055146, HasTs: true},
-				{Labels: labels.FromStrings("dummyID", "59732"), Value: -0.00039, Ts: 1625851155146, HasTs: true},
-			},
 		},
 		{
 			title:                           "Native histogram with exemplars but ingestion disabled",
@@ -2969,45 +2830,50 @@ metric: <
 >
 
 `,
-			floats: []floatSample{
-				{metric: labels.FromStrings("__name__", "test_histogram_count"), t: 1234568, f: 175},
-				{metric: labels.FromStrings("__name__", "test_histogram_sum"), t: 1234568, f: 0.0008280461746287094},
-				{metric: labels.FromStrings("__name__", "test_histogram_bucket", "le", "+Inf"), t: 1234568, f: 175},
+			samples: []sample{
+				{L: labels.FromStrings("__name__", "test_histogram_count"), T: 1234568, V: 175},
+				{L: labels.FromStrings("__name__", "test_histogram_sum"), T: 1234568, V: 0.0008280461746287094},
+				{L: labels.FromStrings("__name__", "test_histogram_bucket", "le", "+Inf"), T: 1234568, V: 175},
 			},
 		},
 	}
 
 	for _, test := range tests {
 		t.Run(test.title, func(t *testing.T) {
-			app := &collectResultAppender{}
-
 			discoveryLabels := &Target{
 				labels: labels.FromStrings(test.discoveryLabels...),
 			}
 
-			sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-			sl.enableNativeHistogramScraping = test.enableNativeHistogramsIngestion
-			sl.sampleMutator = func(l labels.Labels) labels.Labels {
-				return mutateSampleLabels(l, discoveryLabels, false, nil)
-			}
-			sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
-				return mutateReportSampleLabels(l, discoveryLabels)
-			}
-			sl.alwaysScrapeClassicHist = test.alwaysScrapeClassicHist
+			appTest := teststorage.NewAppendable()
+			sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+				sl.appendable = appTest
+				sl.enableNativeHistogramScraping = test.enableNativeHistogramsIngestion
+				sl.sampleMutator = func(l labels.Labels) labels.Labels {
+					return mutateSampleLabels(l, discoveryLabels, false, nil)
+				}
+				sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
+					return mutateReportSampleLabels(l, discoveryLabels)
+				}
+				sl.alwaysScrapeClassicHist = test.alwaysScrapeClassicHist
+				// This test does not care about metadata. Having this true would mean we need to add metadata to sample
+				// expectations.
+				sl.appendMetadataToWAL = false
+			})
+			app := sl.appender()
 
 			now := time.Now()
 
-			for i := range test.floats {
-				if test.floats[i].t != 0 {
+			for i := range test.samples {
+				if test.samples[i].T != 0 {
 					continue
 				}
-				test.floats[i].t = timestamp.FromTime(now)
-			}
+				test.samples[i].T = timestamp.FromTime(now)
 
-			// We need to set the timestamp for expected exemplars that does not have a timestamp.
-			for i := range test.exemplars {
-				if test.exemplars[i].Ts == 0 {
-					test.exemplars[i].Ts = timestamp.FromTime(now)
+				// We need to set the timestamp for expected exemplars that does not have a timestamp.
+				for j := range test.samples[i].ES {
+					if test.samples[i].ES[j].Ts == 0 {
+						test.samples[i].ES[j].Ts = timestamp.FromTime(now)
+					}
 				}
 			}
 
@@ -3018,12 +2884,10 @@ metric: <
 				buf.WriteString(test.scrapeText)
 			}
 
-			_, _, _, err := sl.append(app, buf.Bytes(), test.contentType, now)
+			_, _, _, err := app.append(buf.Bytes(), test.contentType, now)
 			require.NoError(t, err)
 			require.NoError(t, app.Commit())
-			requireEqual(t, test.floats, app.resultFloats)
-			requireEqual(t, test.histograms, app.resultHistograms)
-			requireEqual(t, test.exemplars, app.resultExemplars)
+			requireEqual(t, test.samples, appTest.ResultSamples())
 		})
 	}
 }
@@ -3052,152 +2916,136 @@ func TestScrapeLoopAppendExemplarSeries(t *testing.T) {
 	scrapeText := []string{`metric_total{n="1"} 1 # {t="1"} 1.0 10000
 # EOF`, `metric_total{n="1"} 2 # {t="2"} 2.0 20000
 # EOF`}
-	samples := []floatSample{{
-		metric: labels.FromStrings("__name__", "metric_total", "n", "1"),
-		f:      1,
+	samples := []sample{{
+		L: labels.FromStrings("__name__", "metric_total", "n", "1"),
+		V: 1,
+		ES: []exemplar.Exemplar{
+			{Labels: labels.FromStrings("t", "1"), Value: 1, Ts: 10000000, HasTs: true},
+		},
 	}, {
-		metric: labels.FromStrings("__name__", "metric_total", "n", "1"),
-		f:      2,
+		L: labels.FromStrings("__name__", "metric_total", "n", "1"),
+		V: 2,
+		ES: []exemplar.Exemplar{
+			{Labels: labels.FromStrings("t", "2"), Value: 2, Ts: 20000000, HasTs: true},
+		},
 	}}
-	exemplars := []exemplar.Exemplar{
-		{Labels: labels.FromStrings("t", "1"), Value: 1, Ts: 10000000, HasTs: true},
-		{Labels: labels.FromStrings("t", "2"), Value: 2, Ts: 20000000, HasTs: true},
-	}
 	discoveryLabels := &Target{
 		labels: labels.FromStrings(),
 	}
 
-	app := &collectResultAppender{}
-
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-	sl.sampleMutator = func(l labels.Labels) labels.Labels {
-		return mutateSampleLabels(l, discoveryLabels, false, nil)
-	}
-	sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
-		return mutateReportSampleLabels(l, discoveryLabels)
-	}
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.sampleMutator = func(l labels.Labels) labels.Labels {
+			return mutateSampleLabels(l, discoveryLabels, false, nil)
+		}
+		sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
+			return mutateReportSampleLabels(l, discoveryLabels)
+		}
+		// This test does not care about metadata. Having this true would mean we need to add metadata to sample
+		// expectations.
+		sl.appendMetadataToWAL = false
+	})
 
 	now := time.Now()
-
 	for i := range samples {
 		ts := now.Add(time.Second * time.Duration(i))
-		samples[i].t = timestamp.FromTime(ts)
-	}
-
-	// We need to set the timestamp for expected exemplars that does not have a timestamp.
-	for i := range exemplars {
-		if exemplars[i].Ts == 0 {
-			ts := now.Add(time.Second * time.Duration(i))
-			exemplars[i].Ts = timestamp.FromTime(ts)
-		}
+		samples[i].T = timestamp.FromTime(ts)
 	}
 
 	for i, st := range scrapeText {
-		_, _, _, err := sl.append(app, []byte(st), "application/openmetrics-text", timestamp.Time(samples[i].t))
+		app := sl.appender()
+		_, _, _, err := app.append([]byte(st), "application/openmetrics-text", timestamp.Time(samples[i].T))
 		require.NoError(t, err)
 		require.NoError(t, app.Commit())
 	}
 
-	requireEqual(t, samples, app.resultFloats)
-	requireEqual(t, exemplars, app.resultExemplars)
+	requireEqual(t, samples, appTest.ResultSamples())
 }
 
 func TestScrapeLoopRunReportsTargetDownOnScrapeError(t *testing.T) {
-	var (
-		scraper  = &testScraper{}
-		appender = &collectResultAppender{}
-		app      = func(context.Context) storage.Appender { return appender }
-	)
-
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, scraper, app, 10*time.Millisecond)
-
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = appTest
+	})
 	scraper.scrapeFunc = func(context.Context, io.Writer) error {
 		cancel()
 		return errors.New("scrape failed")
 	}
 
 	sl.run(nil)
-	require.Equal(t, 0.0, appender.resultFloats[0].f, "bad 'up' value")
+	require.Equal(t, 0.0, appTest.ResultSamples()[0].V, "bad 'up' value")
 }
 
 func TestScrapeLoopRunReportsTargetDownOnInvalidUTF8(t *testing.T) {
-	var (
-		scraper  = &testScraper{}
-		appender = &collectResultAppender{}
-		app      = func(context.Context) storage.Appender { return appender }
-	)
-
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, scraper, app, 10*time.Millisecond)
-
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = appTest
+	})
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		cancel()
-		w.Write([]byte("a{l=\"\xff\"} 1\n"))
+		_, _ = w.Write([]byte("a{l=\"\xff\"} 1\n"))
 		return nil
 	}
 
 	sl.run(nil)
-	require.Equal(t, 0.0, appender.resultFloats[0].f, "bad 'up' value")
-}
-
-type errorAppender struct {
-	collectResultAppender
-}
-
-func (app *errorAppender) Append(ref storage.SeriesRef, lset labels.Labels, t int64, v float64) (storage.SeriesRef, error) {
-	switch lset.Get(model.MetricNameLabel) {
-	case "out_of_order":
-		return 0, storage.ErrOutOfOrderSample
-	case "amend":
-		return 0, storage.ErrDuplicateSampleForTimestamp
-	case "out_of_bounds":
-		return 0, storage.ErrOutOfBounds
-	default:
-		return app.collectResultAppender.Append(ref, lset, t, v)
-	}
+	require.Equal(t, 0.0, appTest.ResultSamples()[0].V, "bad 'up' value")
 }
 
 func TestScrapeLoopAppendGracefullyIfAmendOrOutOfOrderOrOutOfBounds(t *testing.T) {
-	app := &errorAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
+	appTest := teststorage.NewAppendable().WithErrs(
+		func(ls labels.Labels) error {
+			switch ls.Get(model.MetricNameLabel) {
+			case "out_of_order":
+				return storage.ErrOutOfOrderSample
+			case "amend":
+				return storage.ErrDuplicateSampleForTimestamp
+			case "out_of_bounds":
+				return storage.ErrOutOfBounds
+			default:
+				return nil
+			}
+		}, nil, nil)
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	now := time.Unix(1, 0)
-	slApp := sl.appender(context.Background())
-	total, added, seriesAdded, err := sl.append(slApp, []byte("out_of_order 1\namend 1\nnormal 1\nout_of_bounds 1\n"), "text/plain", now)
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("out_of_order 1\namend 1\nnormal 1\nout_of_bounds 1\n"), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "normal"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "normal"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 	}
-	requireEqual(t, want, app.resultFloats, "Appended samples not as expected:\n%s", appender)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 	require.Equal(t, 4, total)
 	require.Equal(t, 4, added)
 	require.Equal(t, 1, seriesAdded)
 }
 
 func TestScrapeLoopOutOfBoundsTimeError(t *testing.T) {
-	app := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil,
-		func(context.Context) storage.Appender {
+	sl, _ := newTestScrapeLoop(t, withAppendable(
+		appendableFunc(func(ctx context.Context) storage.Appender {
 			return &timeLimitAppender{
-				Appender: app,
+				Appender: teststorage.NewAppendable().Appender(ctx),
 				maxTime:  timestamp.FromTime(time.Now().Add(10 * time.Minute)),
 			}
-		},
-		0,
-	)
+		}),
+	))
 
 	now := time.Now().Add(20 * time.Minute)
-	slApp := sl.appender(context.Background())
-	total, added, seriesAdded, err := sl.append(slApp, []byte("normal 1\n"), "text/plain", now)
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("normal 1\n"), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 1, total)
 	require.Equal(t, 1, added)
 	require.Equal(t, 0, seriesAdded)
@@ -3292,7 +3140,7 @@ func TestRequestTraceparentHeader(t *testing.T) {
 	resp, err := ts.scrape(context.Background())
 	require.NoError(t, err)
 	require.NotNil(t, resp)
-	defer resp.Body.Close()
+	t.Cleanup(func() { _ = resp.Body.Close() })
 }
 
 func TestTargetScraperScrapeOK(t *testing.T) {
@@ -3339,7 +3187,7 @@ func TestTargetScraperScrapeOK(t *testing.T) {
 			} else {
 				w.Header().Set("Content-Type", `text/plain; version=0.0.4`)
 			}
-			w.Write([]byte("metric_a 1\nmetric_b 2\n"))
+			_, _ = w.Write([]byte("metric_a 1\nmetric_b 2\n"))
 		}),
 	)
 	defer server.Close()
@@ -3454,9 +3302,9 @@ func TestTargetScrapeScrapeCancel(t *testing.T) {
 		_, err := ts.scrape(ctx)
 		switch {
 		case err == nil:
-			errc <- errors.New("Expected error but got nil")
+			errc <- errors.New("expected error but got nil")
 		case !errors.Is(ctx.Err(), context.Canceled):
-			errc <- fmt.Errorf("Expected context cancellation error but got: %w", ctx.Err())
+			errc <- fmt.Errorf("expected context cancellation error but got: %w", ctx.Err())
 		default:
 			close(errc)
 		}
@@ -3516,11 +3364,11 @@ func TestTargetScraperBodySizeLimit(t *testing.T) {
 			if gzipResponse {
 				w.Header().Set("Content-Encoding", "gzip")
 				gw := gzip.NewWriter(w)
-				defer gw.Close()
-				gw.Write([]byte(responseBody))
+				defer func() { _ = gw.Close() }()
+				_, _ = gw.Write([]byte(responseBody))
 				return
 			}
-			w.Write([]byte(responseBody))
+			_, _ = w.Write([]byte(responseBody))
 		}),
 	)
 	defer server.Close()
@@ -3614,87 +3462,84 @@ func (ts *testScraper) readResponse(ctx context.Context, _ *http.Response, w io.
 
 func TestScrapeLoop_RespectTimestamps(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	app := s.Appender(context.Background())
-	capp := &collectResultAppender{next: app}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return capp }, 0)
+	appTest := teststorage.NewAppendable().Then(s)
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte(`metric_a{a="1",b="1"} 1 0`), "text/plain", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(`metric_a{a="1",b="1"} 1 0`), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
-			t:      0,
-			f:      1,
+			L: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
+			T: 0,
+			V: 1,
 		},
 	}
-	require.Equal(t, want, capp.resultFloats, "Appended samples not as expected:\n%s", appender)
+	require.Equal(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoop_DiscardTimestamps(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	app := s.Appender(context.Background())
-
-	capp := &collectResultAppender{next: app}
-
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return capp }, 0)
-	sl.honorTimestamps = false
+	appTest := teststorage.NewAppendable().Then(s)
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.honorTimestamps = false
+	})
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte(`metric_a{a="1",b="1"} 1 0`), "text/plain", now)
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(`metric_a{a="1",b="1"} 1 0`), "text/plain", now)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings("__name__", "metric_a", "a", "1", "b", "1"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 	}
-	require.Equal(t, want, capp.resultFloats, "Appended samples not as expected:\n%s", appender)
+	require.Equal(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
 func TestScrapeLoopDiscardDuplicateLabels(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, &testScraper{}, s.Appender, 0)
-	defer cancel()
+	appTest := teststorage.NewAppendable().Then(s)
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 
 	// We add a good and a bad metric to check that both are discarded.
-	slApp := sl.appender(ctx)
-	_, _, _, err := sl.append(slApp, []byte("test_metric{le=\"500\"} 1\ntest_metric{le=\"600\",le=\"700\"} 1\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("test_metric{le=\"500\"} 1\ntest_metric{le=\"600\",le=\"700\"} 1\n"), "text/plain", time.Time{})
 	require.Error(t, err)
-	require.NoError(t, slApp.Rollback())
-	// We need to cycle staleness cache maps after a manual rollback. Otherwise they will have old entries in them,
+	require.NoError(t, app.Rollback())
+	// We need to cycle staleness cache maps after a manual rollback. Otherwise, they will have old entries in them,
 	// which would cause ErrDuplicateSampleForTimestamp errors on the next append.
 	sl.cache.iterDone(true)
 
 	q, err := s.Querier(time.Time{}.UnixNano(), 0)
 	require.NoError(t, err)
-	series := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*"))
+	series := q.Select(sl.ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*"))
 	require.False(t, series.Next(), "series found in tsdb")
 	require.NoError(t, series.Err())
 
 	// We add a good metric to check that it is recorded.
-	slApp = sl.appender(ctx)
-	_, _, _, err = sl.append(slApp, []byte("test_metric{le=\"500\"} 1\n"), "text/plain", time.Time{})
+	app = sl.appender()
+	_, _, _, err = app.append([]byte("test_metric{le=\"500\"} 1\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
 	q, err = s.Querier(time.Time{}.UnixNano(), 0)
 	require.NoError(t, err)
-	series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "le", "500"))
+	series = q.Select(sl.ctx, false, nil, labels.MustNewMatcher(labels.MatchEqual, "le", "500"))
 	require.True(t, series.Next(), "series not found in tsdb")
 	require.NoError(t, series.Err())
 	require.False(t, series.Next(), "more than one series found in tsdb")
@@ -3702,29 +3547,28 @@ func TestScrapeLoopDiscardDuplicateLabels(t *testing.T) {
 
 func TestScrapeLoopDiscardUnnamedMetrics(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	app := s.Appender(context.Background())
-
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, context.Background(), &testScraper{}, func(context.Context) storage.Appender { return app }, 0)
-	sl.sampleMutator = func(l labels.Labels) labels.Labels {
-		if l.Has("drop") {
-			return labels.FromStrings("no", "name") // This label set will trigger an error.
+	appTest := teststorage.NewAppendable().Then(s)
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.sampleMutator = func(l labels.Labels) labels.Labels {
+			if l.Has("drop") {
+				return labels.FromStrings("no", "name") // This label set will trigger an error.
+			}
+			return l
 		}
-		return l
-	}
-	defer cancel()
+	})
 
-	slApp := sl.appender(context.Background())
-	_, _, _, err := sl.append(slApp, []byte("nok 1\nnok2{drop=\"drop\"} 1\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("nok 1\nnok2{drop=\"drop\"} 1\n"), "text/plain", time.Time{})
 	require.Error(t, err)
-	require.NoError(t, slApp.Rollback())
+	require.NoError(t, app.Rollback())
 	require.Equal(t, errNameLabelMandatory, err)
 
 	q, err := s.Querier(time.Time{}.UnixNano(), 0)
 	require.NoError(t, err)
-	series := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*"))
+	series := q.Select(sl.ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*"))
 	require.False(t, series.Next(), "series found in tsdb")
 	require.NoError(t, series.Err())
 }
@@ -3798,7 +3642,7 @@ func TestReusableConfig(t *testing.T) {
 
 func TestReuseScrapeCache(t *testing.T) {
 	var (
-		app = &nopAppendable{}
+		app = teststorage.NewAppendable()
 		cfg = &config.ScrapeConfig{
 			JobName:                    "Prometheus",
 			ScrapeTimeout:              model.Duration(5 * time.Second),
@@ -3964,7 +3808,7 @@ func TestReuseScrapeCache(t *testing.T) {
 
 	for i, s := range steps {
 		initCacheAddr := cacheAddr(sp)
-		sp.reload(s.newConfig)
+		require.NoError(t, sp.reload(s.newConfig))
 		for fp, newCacheAddr := range cacheAddr(sp) {
 			if s.keep {
 				require.Equal(t, initCacheAddr[fp], newCacheAddr, "step %d: old cache and new cache are not the same", i)
@@ -3973,7 +3817,7 @@ func TestReuseScrapeCache(t *testing.T) {
 			}
 		}
 		initCacheAddr = cacheAddr(sp)
-		sp.reload(s.newConfig)
+		require.NoError(t, sp.reload(s.newConfig))
 		for fp, newCacheAddr := range cacheAddr(sp) {
 			require.Equal(t, initCacheAddr[fp], newCacheAddr, "step %d: reloading the exact config invalidates the cache", i)
 		}
@@ -3982,16 +3826,14 @@ func TestReuseScrapeCache(t *testing.T) {
 
 func TestScrapeAddFast(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	ctx, cancel := context.WithCancel(context.Background())
-	sl := newBasicScrapeLoop(t, ctx, &testScraper{}, s.Appender, 0)
-	defer cancel()
+	sl, _ := newTestScrapeLoop(t, withAppendable(s))
 
-	slApp := sl.appender(ctx)
-	_, _, _, err := sl.append(slApp, []byte("up 1\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	_, _, _, err := app.append([]byte("up 1\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 
 	// Poison the cache. There is just one entry, and one series in the
 	// storage. Changing the ref will create a 'not found' error.
@@ -3999,15 +3841,14 @@ func TestScrapeAddFast(t *testing.T) {
 		v.ref++
 	}
 
-	slApp = sl.appender(ctx)
-	_, _, _, err = sl.append(slApp, []byte("up 1\n"), "text/plain", time.Time{}.Add(time.Second))
+	app = sl.appender()
+	_, _, _, err = app.append([]byte("up 1\n"), "text/plain", time.Time{}.Add(time.Second))
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 }
 
 func TestReuseCacheRace(t *testing.T) {
 	var (
-		app = &nopAppendable{}
 		cfg = &config.ScrapeConfig{
 			JobName:                    "Prometheus",
 			ScrapeTimeout:              model.Duration(5 * time.Second),
@@ -4017,7 +3858,7 @@ func TestReuseCacheRace(t *testing.T) {
 			MetricNameEscapingScheme:   model.AllowUTF8,
 		}
 		buffers = pool.New(1e3, 100e6, 3, func(sz int) any { return make([]byte, 0, sz) })
-		sp, _   = newScrapePool(cfg, app, 0, nil, buffers, &Options{}, newTestScrapeMetrics(t))
+		sp, _   = newScrapePool(cfg, teststorage.NewAppendable(), 0, nil, buffers, &Options{}, newTestScrapeMetrics(t))
 		t1      = &Target{
 			labels:       labels.FromStrings("labelNew", "nameNew"),
 			scrapeConfig: &config.ScrapeConfig{},
@@ -4031,7 +3872,7 @@ func TestReuseCacheRace(t *testing.T) {
 		if time.Since(start) > 5*time.Second {
 			break
 		}
-		sp.reload(&config.ScrapeConfig{
+		require.NoError(t, sp.reload(&config.ScrapeConfig{
 			JobName:                    "Prometheus",
 			ScrapeTimeout:              model.Duration(1 * time.Millisecond),
 			ScrapeInterval:             model.Duration(1 * time.Millisecond),
@@ -4039,39 +3880,42 @@ func TestReuseCacheRace(t *testing.T) {
 			SampleLimit:                i,
 			MetricNameValidationScheme: model.UTF8Validation,
 			MetricNameEscapingScheme:   model.AllowUTF8,
-		})
+		}))
 	}
 }
 
 func TestCheckAddError(t *testing.T) {
 	var appErrs appendErrors
-	sl := scrapeLoop{l: promslog.NewNopLogger(), metrics: newTestScrapeMetrics(t)}
-	sl.checkAddError(nil, storage.ErrOutOfOrderSample, nil, nil, &appErrs)
+	sl, _ := newTestScrapeLoop(t)
+	// TODO: Check err etc
+	_, _ = sl.checkAddError(nil, storage.ErrOutOfOrderSample, nil, nil, &appErrs)
 	require.Equal(t, 1, appErrs.numOutOfOrder)
+
+	// TODO(bwplotka): Test partial error check and other cases
 }
 
 func TestScrapeReportSingleAppender(t *testing.T) {
 	t.Parallel()
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
-	var (
-		signal  = make(chan struct{}, 1)
-		scraper = &testScraper{}
-	)
+	signal := make(chan struct{}, 1)
 
-	ctx, cancel := context.WithCancel(context.Background())
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, ctx, scraper, s.Appender, 10*time.Millisecond, "text/plain")
+	ctx, cancel := context.WithCancel(t.Context())
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = s
+		// Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+	})
 
 	numScrapes := 0
-
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		numScrapes++
 		if numScrapes%4 == 0 {
 			return errors.New("scrape failed")
 		}
-		w.Write([]byte("metric_a 44\nmetric_b 44\nmetric_c 44\nmetric_d 44\n"))
+		_, _ = w.Write([]byte("metric_a 44\nmetric_b 44\nmetric_c 44\nmetric_d 44\n"))
 		return nil
 	}
 
@@ -4095,7 +3939,7 @@ func TestScrapeReportSingleAppender(t *testing.T) {
 		}
 
 		require.Equal(t, 0, c%9, "Appended samples not as expected: %d", c)
-		q.Close()
+		require.NoError(t, q.Close())
 	}
 	cancel()
 
@@ -4108,7 +3952,7 @@ func TestScrapeReportSingleAppender(t *testing.T) {
 
 func TestScrapeReportLimit(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
 	cfg := &config.ScrapeConfig{
 		JobName:                    "test",
@@ -4146,7 +3990,7 @@ func TestScrapeReportLimit(t *testing.T) {
 	ctx := t.Context()
 	q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
 	require.NoError(t, err)
-	defer q.Close()
+	t.Cleanup(func() { _ = q.Close() })
 	series := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", "up"))
 
 	var found bool
@@ -4164,7 +4008,7 @@ func TestScrapeReportLimit(t *testing.T) {
 
 func TestScrapeUTF8(t *testing.T) {
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 
 	cfg := &config.ScrapeConfig{
 		JobName:                    "test",
@@ -4200,7 +4044,7 @@ func TestScrapeUTF8(t *testing.T) {
 	ctx := t.Context()
 	q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
 	require.NoError(t, err)
-	defer q.Close()
+	t.Cleanup(func() { _ = q.Close() })
 	series := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", "with.dots"))
 
 	require.True(t, series.Next(), "series not found in tsdb")
@@ -4272,30 +4116,29 @@ func TestScrapeLoopLabelLimit(t *testing.T) {
 	}
 
 	for _, test := range tests {
-		app := &collectResultAppender{}
-
 		discoveryLabels := &Target{
 			labels: labels.FromStrings(test.discoveryLabels...),
 		}
 
-		sl := newBasicScrapeLoop(t, context.Background(), nil, func(context.Context) storage.Appender { return app }, 0)
-		sl.sampleMutator = func(l labels.Labels) labels.Labels {
-			return mutateSampleLabels(l, discoveryLabels, false, nil)
-		}
-		sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
-			return mutateReportSampleLabels(l, discoveryLabels)
-		}
-		sl.labelLimits = &test.labelLimits
+		sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+			sl.sampleMutator = func(l labels.Labels) labels.Labels {
+				return mutateSampleLabels(l, discoveryLabels, false, nil)
+			}
+			sl.reportSampleMutator = func(l labels.Labels) labels.Labels {
+				return mutateReportSampleLabels(l, discoveryLabels)
+			}
+			sl.labelLimits = &test.labelLimits
+		})
 
-		slApp := sl.appender(context.Background())
-		_, _, _, err := sl.append(slApp, []byte(test.scrapeLabels), "text/plain", time.Now())
+		app := sl.appender()
+		_, _, _, err := app.append([]byte(test.scrapeLabels), "text/plain", time.Now())
 
 		t.Logf("Test:%s", test.title)
 		if test.expectErr {
 			require.Error(t, err)
 		} else {
 			require.NoError(t, err)
-			require.NoError(t, slApp.Commit())
+			require.NoError(t, app.Commit())
 		}
 	}
 }
@@ -4303,7 +4146,7 @@ func TestScrapeLoopLabelLimit(t *testing.T) {
 func TestTargetScrapeIntervalAndTimeoutRelabel(t *testing.T) {
 	interval, _ := model.ParseDuration("2s")
 	timeout, _ := model.ParseDuration("500ms")
-	config := &config.ScrapeConfig{
+	cfg := &config.ScrapeConfig{
 		ScrapeInterval:             interval,
 		ScrapeTimeout:              timeout,
 		MetricNameValidationScheme: model.UTF8Validation,
@@ -4327,7 +4170,7 @@ func TestTargetScrapeIntervalAndTimeoutRelabel(t *testing.T) {
 			},
 		},
 	}
-	sp, _ := newScrapePool(config, &nopAppendable{}, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+	sp, _ := newScrapePool(cfg, teststorage.NewAppendable(), 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 	tgts := []*targetgroup.Group{
 		{
 			Targets: []model.LabelSet{{model.AddressLabel: "127.0.0.1:9090"}},
@@ -4343,10 +4186,10 @@ func TestTargetScrapeIntervalAndTimeoutRelabel(t *testing.T) {
 
 // Testing whether we can remove trailing .0 from histogram 'le' and summary 'quantile' labels.
 func TestLeQuantileReLabel(t *testing.T) {
-	simpleStorage := teststorage.New(t)
-	defer simpleStorage.Close()
+	s := teststorage.New(t)
+	t.Cleanup(func() { _ = s.Close() })
 
-	config := &config.ScrapeConfig{
+	cfg := &config.ScrapeConfig{
 		JobName: "test",
 		MetricRelabelConfigs: []*relabel.Config{
 			{
@@ -4413,7 +4256,7 @@ test_summary_count 199
 	ts, scrapedTwice := newScrapableServer(metricsText)
 	defer ts.Close()
 
-	sp, err := newScrapePool(config, simpleStorage, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+	sp, err := newScrapePool(cfg, s, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 	require.NoError(t, err)
 	defer sp.stop()
 
@@ -4433,9 +4276,9 @@ test_summary_count 199
 	}
 
 	ctx := t.Context()
-	q, err := simpleStorage.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
+	q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
 	require.NoError(t, err)
-	defer q.Close()
+	t.Cleanup(func() { _ = q.Close() })
 
 	checkValues := func(labelName string, expectedValues []string, series storage.SeriesSet) {
 		foundLeValues := map[string]bool{}
@@ -4463,30 +4306,22 @@ test_summary_count 199
 // Testing whether we can automatically convert scraped classic histograms into native histograms with custom buckets.
 func TestConvertClassicHistogramsToNHCB(t *testing.T) {
 	t.Parallel()
-	genTestCounterText := func(name string, value int, withMetadata bool) string {
-		if withMetadata {
-			return fmt.Sprintf(`
+
+	genTestCounterText := func(name string) string {
+		return fmt.Sprintf(`
 # HELP %s some help text
 # TYPE %s counter
-%s{address="0.0.0.0",port="5001"} %d
-`, name, name, name, value)
-		}
-		return fmt.Sprintf(`
-%s %d
-`, name, value)
+%s{address="0.0.0.0",port="5001"} 1
+`, name, name, name)
 	}
-	genTestHistText := func(name string, withMetadata bool) string {
+	genTestHistText := func(name string) string {
 		data := map[string]any{
 			"name": name,
 		}
 		b := &bytes.Buffer{}
-		if withMetadata {
-			template.Must(template.New("").Parse(`
+		require.NoError(t, template.Must(template.New("").Parse(`
 # HELP {{.name}} This is a histogram with default buckets
 # TYPE {{.name}} histogram
-`)).Execute(b, data)
-		}
-		template.Must(template.New("").Parse(`
 {{.name}}_bucket{address="0.0.0.0",port="5001",le="0.005"} 0
 {{.name}}_bucket{address="0.0.0.0",port="5001",le="0.01"} 0
 {{.name}}_bucket{address="0.0.0.0",port="5001",le="0.025"} 0
@@ -4501,10 +4336,10 @@ func TestConvertClassicHistogramsToNHCB(t *testing.T) {
 {{.name}}_bucket{address="0.0.0.0",port="5001",le="+Inf"} 1
 {{.name}}_sum{address="0.0.0.0",port="5001"} 10
 {{.name}}_count{address="0.0.0.0",port="5001"} 1
-`)).Execute(b, data)
+`)).Execute(b, data))
 		return b.String()
 	}
-	genTestCounterProto := func(name string, value int) string {
+	genTestCounterProto := func(name string) string {
 		return fmt.Sprintf(`
 name: "%s"
 help: "some help text"
@@ -4522,7 +4357,7 @@ metric: <
 		value: %d
 	>
 >
-`, name, value)
+`, name, 1)
 	}
 	genTestHistProto := func(name string, hasClassic, hasExponential bool) string {
 		var classic string
@@ -4616,60 +4451,60 @@ metric: <
 	}{
 		"text": {
 			text: []string{
-				genTestCounterText("test_metric_1", 1, true),
-				genTestCounterText("test_metric_1_count", 1, true),
-				genTestCounterText("test_metric_1_sum", 1, true),
-				genTestCounterText("test_metric_1_bucket", 1, true),
-				genTestHistText("test_histogram_1", true),
-				genTestCounterText("test_metric_2", 1, true),
-				genTestCounterText("test_metric_2_count", 1, true),
-				genTestCounterText("test_metric_2_sum", 1, true),
-				genTestCounterText("test_metric_2_bucket", 1, true),
-				genTestHistText("test_histogram_2", true),
-				genTestCounterText("test_metric_3", 1, true),
-				genTestCounterText("test_metric_3_count", 1, true),
-				genTestCounterText("test_metric_3_sum", 1, true),
-				genTestCounterText("test_metric_3_bucket", 1, true),
-				genTestHistText("test_histogram_3", true),
+				genTestCounterText("test_metric_1"),
+				genTestCounterText("test_metric_1_count"),
+				genTestCounterText("test_metric_1_sum"),
+				genTestCounterText("test_metric_1_bucket"),
+				genTestHistText("test_histogram_1"),
+				genTestCounterText("test_metric_2"),
+				genTestCounterText("test_metric_2_count"),
+				genTestCounterText("test_metric_2_sum"),
+				genTestCounterText("test_metric_2_bucket"),
+				genTestHistText("test_histogram_2"),
+				genTestCounterText("test_metric_3"),
+				genTestCounterText("test_metric_3_count"),
+				genTestCounterText("test_metric_3_sum"),
+				genTestCounterText("test_metric_3_bucket"),
+				genTestHistText("test_histogram_3"),
 			},
 			hasClassic: true,
 		},
 		"text, in different order": {
 			text: []string{
-				genTestCounterText("test_metric_1", 1, true),
-				genTestCounterText("test_metric_1_count", 1, true),
-				genTestCounterText("test_metric_1_sum", 1, true),
-				genTestCounterText("test_metric_1_bucket", 1, true),
-				genTestHistText("test_histogram_1", true),
-				genTestCounterText("test_metric_2", 1, true),
-				genTestCounterText("test_metric_2_count", 1, true),
-				genTestCounterText("test_metric_2_sum", 1, true),
-				genTestCounterText("test_metric_2_bucket", 1, true),
-				genTestHistText("test_histogram_2", true),
-				genTestHistText("test_histogram_3", true),
-				genTestCounterText("test_metric_3", 1, true),
-				genTestCounterText("test_metric_3_count", 1, true),
-				genTestCounterText("test_metric_3_sum", 1, true),
-				genTestCounterText("test_metric_3_bucket", 1, true),
+				genTestCounterText("test_metric_1"),
+				genTestCounterText("test_metric_1_count"),
+				genTestCounterText("test_metric_1_sum"),
+				genTestCounterText("test_metric_1_bucket"),
+				genTestHistText("test_histogram_1"),
+				genTestCounterText("test_metric_2"),
+				genTestCounterText("test_metric_2_count"),
+				genTestCounterText("test_metric_2_sum"),
+				genTestCounterText("test_metric_2_bucket"),
+				genTestHistText("test_histogram_2"),
+				genTestHistText("test_histogram_3"),
+				genTestCounterText("test_metric_3"),
+				genTestCounterText("test_metric_3_count"),
+				genTestCounterText("test_metric_3_sum"),
+				genTestCounterText("test_metric_3_bucket"),
 			},
 			hasClassic: true,
 		},
 		"protobuf": {
 			text: []string{
-				genTestCounterProto("test_metric_1", 1),
-				genTestCounterProto("test_metric_1_count", 1),
-				genTestCounterProto("test_metric_1_sum", 1),
-				genTestCounterProto("test_metric_1_bucket", 1),
+				genTestCounterProto("test_metric_1"),
+				genTestCounterProto("test_metric_1_count"),
+				genTestCounterProto("test_metric_1_sum"),
+				genTestCounterProto("test_metric_1_bucket"),
 				genTestHistProto("test_histogram_1", true, false),
-				genTestCounterProto("test_metric_2", 1),
-				genTestCounterProto("test_metric_2_count", 1),
-				genTestCounterProto("test_metric_2_sum", 1),
-				genTestCounterProto("test_metric_2_bucket", 1),
+				genTestCounterProto("test_metric_2"),
+				genTestCounterProto("test_metric_2_count"),
+				genTestCounterProto("test_metric_2_sum"),
+				genTestCounterProto("test_metric_2_bucket"),
 				genTestHistProto("test_histogram_2", true, false),
-				genTestCounterProto("test_metric_3", 1),
-				genTestCounterProto("test_metric_3_count", 1),
-				genTestCounterProto("test_metric_3_sum", 1),
-				genTestCounterProto("test_metric_3_bucket", 1),
+				genTestCounterProto("test_metric_3"),
+				genTestCounterProto("test_metric_3_count"),
+				genTestCounterProto("test_metric_3_sum"),
+				genTestCounterProto("test_metric_3_bucket"),
 				genTestHistProto("test_histogram_3", true, false),
 			},
 			contentType: "application/vnd.google.protobuf",
@@ -4678,40 +4513,40 @@ metric: <
 		"protobuf, in different order": {
 			text: []string{
 				genTestHistProto("test_histogram_1", true, false),
-				genTestCounterProto("test_metric_1", 1),
-				genTestCounterProto("test_metric_1_count", 1),
-				genTestCounterProto("test_metric_1_sum", 1),
-				genTestCounterProto("test_metric_1_bucket", 1),
+				genTestCounterProto("test_metric_1"),
+				genTestCounterProto("test_metric_1_count"),
+				genTestCounterProto("test_metric_1_sum"),
+				genTestCounterProto("test_metric_1_bucket"),
 				genTestHistProto("test_histogram_2", true, false),
-				genTestCounterProto("test_metric_2", 1),
-				genTestCounterProto("test_metric_2_count", 1),
-				genTestCounterProto("test_metric_2_sum", 1),
-				genTestCounterProto("test_metric_2_bucket", 1),
+				genTestCounterProto("test_metric_2"),
+				genTestCounterProto("test_metric_2_count"),
+				genTestCounterProto("test_metric_2_sum"),
+				genTestCounterProto("test_metric_2_bucket"),
 				genTestHistProto("test_histogram_3", true, false),
-				genTestCounterProto("test_metric_3", 1),
-				genTestCounterProto("test_metric_3_count", 1),
-				genTestCounterProto("test_metric_3_sum", 1),
-				genTestCounterProto("test_metric_3_bucket", 1),
+				genTestCounterProto("test_metric_3"),
+				genTestCounterProto("test_metric_3_count"),
+				genTestCounterProto("test_metric_3_sum"),
+				genTestCounterProto("test_metric_3_bucket"),
 			},
 			contentType: "application/vnd.google.protobuf",
 			hasClassic:  true,
 		},
 		"protobuf, with additional native exponential histogram": {
 			text: []string{
-				genTestCounterProto("test_metric_1", 1),
-				genTestCounterProto("test_metric_1_count", 1),
-				genTestCounterProto("test_metric_1_sum", 1),
-				genTestCounterProto("test_metric_1_bucket", 1),
+				genTestCounterProto("test_metric_1"),
+				genTestCounterProto("test_metric_1_count"),
+				genTestCounterProto("test_metric_1_sum"),
+				genTestCounterProto("test_metric_1_bucket"),
 				genTestHistProto("test_histogram_1", true, true),
-				genTestCounterProto("test_metric_2", 1),
-				genTestCounterProto("test_metric_2_count", 1),
-				genTestCounterProto("test_metric_2_sum", 1),
-				genTestCounterProto("test_metric_2_bucket", 1),
+				genTestCounterProto("test_metric_2"),
+				genTestCounterProto("test_metric_2_count"),
+				genTestCounterProto("test_metric_2_sum"),
+				genTestCounterProto("test_metric_2_bucket"),
 				genTestHistProto("test_histogram_2", true, true),
-				genTestCounterProto("test_metric_3", 1),
-				genTestCounterProto("test_metric_3_count", 1),
-				genTestCounterProto("test_metric_3_sum", 1),
-				genTestCounterProto("test_metric_3_bucket", 1),
+				genTestCounterProto("test_metric_3"),
+				genTestCounterProto("test_metric_3_count"),
+				genTestCounterProto("test_metric_3_sum"),
+				genTestCounterProto("test_metric_3_bucket"),
 				genTestHistProto("test_histogram_3", true, true),
 			},
 			contentType:    "application/vnd.google.protobuf",
@@ -4720,20 +4555,20 @@ metric: <
 		},
 		"protobuf, with only native exponential histogram": {
 			text: []string{
-				genTestCounterProto("test_metric_1", 1),
-				genTestCounterProto("test_metric_1_count", 1),
-				genTestCounterProto("test_metric_1_sum", 1),
-				genTestCounterProto("test_metric_1_bucket", 1),
+				genTestCounterProto("test_metric_1"),
+				genTestCounterProto("test_metric_1_count"),
+				genTestCounterProto("test_metric_1_sum"),
+				genTestCounterProto("test_metric_1_bucket"),
 				genTestHistProto("test_histogram_1", false, true),
-				genTestCounterProto("test_metric_2", 1),
-				genTestCounterProto("test_metric_2_count", 1),
-				genTestCounterProto("test_metric_2_sum", 1),
-				genTestCounterProto("test_metric_2_bucket", 1),
+				genTestCounterProto("test_metric_2"),
+				genTestCounterProto("test_metric_2_count"),
+				genTestCounterProto("test_metric_2_sum"),
+				genTestCounterProto("test_metric_2_bucket"),
 				genTestHistProto("test_histogram_2", false, true),
-				genTestCounterProto("test_metric_3", 1),
-				genTestCounterProto("test_metric_3_count", 1),
-				genTestCounterProto("test_metric_3_sum", 1),
-				genTestCounterProto("test_metric_3_bucket", 1),
+				genTestCounterProto("test_metric_3"),
+				genTestCounterProto("test_metric_3_count"),
+				genTestCounterProto("test_metric_3_sum"),
+				genTestCounterProto("test_metric_3_bucket"),
 				genTestHistProto("test_histogram_3", false, true),
 			},
 			contentType:    "application/vnd.google.protobuf",
@@ -4741,7 +4576,7 @@ metric: <
 		},
 	}
 
-	checkBucketValues := func(expectedCount int, series storage.SeriesSet) {
+	checkBucketValues := func(t testing.TB, expectedCount int, series storage.SeriesSet) {
 		labelName := "le"
 		var expectedValues []string
 		if expectedCount > 0 {
@@ -4763,7 +4598,7 @@ metric: <
 	}
 
 	// Checks that the expected series is present and runs a basic sanity check of the float values.
-	checkFloatSeries := func(series storage.SeriesSet, expectedCount int, expectedFloat float64) {
+	checkFloatSeries := func(t testing.TB, series storage.SeriesSet, expectedCount int, expectedFloat float64) {
 		count := 0
 		for series.Next() {
 			i := series.At().Iterator(nil)
@@ -4789,7 +4624,7 @@ metric: <
 	}
 
 	// Checks that the expected series is present and runs a basic sanity check of the histogram values.
-	checkHistSeries := func(series storage.SeriesSet, expectedCount int, expectedSchema int32) {
+	checkHistSeries := func(t testing.TB, series storage.SeriesSet, expectedCount int, expectedSchema int32) {
 		count := 0
 		for series.Next() {
 			i := series.At().Iterator(nil)
@@ -4871,14 +4706,15 @@ metric: <
 
 			t.Run(fmt.Sprintf("%s with %s", name, metricsTextName), func(t *testing.T) {
 				t.Parallel()
-				simpleStorage := teststorage.New(t)
-				defer simpleStorage.Close()
+				s := teststorage.New(t)
+				t.Cleanup(func() { _ = s.Close() })
 
-				sl := newBasicScrapeLoop(t, context.Background(), nil, func(ctx context.Context) storage.Appender { return simpleStorage.Appender(ctx) }, 0)
-				sl.alwaysScrapeClassicHist = tc.alwaysScrapeClassicHistograms
-				sl.convertClassicHistToNHCB = tc.convertClassicHistToNHCB
-				sl.enableNativeHistogramScraping = true
-				app := simpleStorage.Appender(context.Background())
+				sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+					sl.appendable = s
+					sl.alwaysScrapeClassicHist = tc.alwaysScrapeClassicHistograms
+					sl.convertClassicHistToNHCB = tc.convertClassicHistToNHCB
+					sl.enableNativeHistogramScraping = true
+				})
 
 				var content []byte
 				contentType := metricsText.contentType
@@ -4902,47 +4738,50 @@ metric: <
 				default:
 					t.Error("unexpected content type")
 				}
-				sl.append(app, content, contentType, time.Now())
+				now := time.Now()
+				app := sl.appender()
+				_, _, _, err := app.append(content, contentType, now)
+				require.NoError(t, err)
 				require.NoError(t, app.Commit())
 
+				var expectedSchema int32
+				if expectCustomBuckets {
+					expectedSchema = histogram.CustomBucketsSchema
+				} else {
+					expectedSchema = 3
+				}
+
+				// Validated what was appended can be queried.
 				ctx := t.Context()
-				q, err := simpleStorage.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
+				q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
 				require.NoError(t, err)
-				defer q.Close()
+				t.Cleanup(func() { _ = q.Close() })
 
 				var series storage.SeriesSet
-
 				for i := 1; i <= 3; i++ {
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_metric_%d", i)))
-					checkFloatSeries(series, 1, 1.)
+					checkFloatSeries(t, series, 1, 1.)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_metric_%d_count", i)))
-					checkFloatSeries(series, 1, 1.)
+					checkFloatSeries(t, series, 1, 1.)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_metric_%d_sum", i)))
-					checkFloatSeries(series, 1, 1.)
+					checkFloatSeries(t, series, 1, 1.)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_metric_%d_bucket", i)))
-					checkFloatSeries(series, 1, 1.)
+					checkFloatSeries(t, series, 1, 1.)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_histogram_%d_count", i)))
-					checkFloatSeries(series, expectedClassicHistCount, 1.)
+					checkFloatSeries(t, series, expectedClassicHistCount, 1.)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_histogram_%d_sum", i)))
-					checkFloatSeries(series, expectedClassicHistCount, 10.)
+					checkFloatSeries(t, series, expectedClassicHistCount, 10.)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_histogram_%d_bucket", i)))
-					checkBucketValues(expectedClassicHistCount, series)
+					checkBucketValues(t, expectedClassicHistCount, series)
 
 					series = q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", fmt.Sprintf("test_histogram_%d", i)))
-
-					var expectedSchema int32
-					if expectCustomBuckets {
-						expectedSchema = histogram.CustomBucketsSchema
-					} else {
-						expectedSchema = 3
-					}
-					checkHistSeries(series, expectedNativeHistCount, expectedSchema)
+					checkHistSeries(t, series, expectedNativeHistCount, expectedSchema)
 				}
 			})
 		}
@@ -4950,10 +4789,10 @@ metric: <
 }
 
 func TestTypeUnitReLabel(t *testing.T) {
-	simpleStorage := teststorage.New(t)
-	defer simpleStorage.Close()
+	s := teststorage.New(t)
+	t.Cleanup(func() { _ = s.Close() })
 
-	config := &config.ScrapeConfig{
+	cfg := &config.ScrapeConfig{
 		JobName: "test",
 		MetricRelabelConfigs: []*relabel.Config{
 			{
@@ -4998,7 +4837,7 @@ disk_usage_bytes 456
 	ts, scrapedTwice := newScrapableServer(metricsText)
 	defer ts.Close()
 
-	sp, err := newScrapePool(config, simpleStorage, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+	sp, err := newScrapePool(cfg, s, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 	require.NoError(t, err)
 	defer sp.stop()
 
@@ -5018,9 +4857,9 @@ disk_usage_bytes 456
 	}
 
 	ctx := t.Context()
-	q, err := simpleStorage.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
+	q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
 	require.NoError(t, err)
-	defer q.Close()
+	t.Cleanup(func() { _ = q.Close() })
 
 	series := q.Select(ctx, false, nil, labels.MustNewMatcher(labels.MatchRegexp, "__name__", ".*_total$"))
 	for series.Next() {
@@ -5036,26 +4875,25 @@ disk_usage_bytes 456
 }
 
 func TestScrapeLoopRunCreatesStaleMarkersOnFailedScrapeForTimestampedMetrics(t *testing.T) {
-	appender := &collectResultAppender{}
-	var (
-		signal  = make(chan struct{}, 1)
-		scraper = &testScraper{}
-		app     = func(context.Context) storage.Appender { return appender }
-	)
+	signal := make(chan struct{}, 1)
+
+	ctx, cancel := context.WithCancel(t.Context())
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.ctx = ctx
+		sl.appendable = appTest // Since we're writing samples directly below we need to provide a protocol fallback.
+		sl.fallbackScrapeProtocol = "text/plain"
+		sl.trackTimestampsStaleness = true
+	})
 
-	ctx, cancel := context.WithCancel(context.Background())
-	// Since we're writing samples directly below we need to provide a protocol fallback.
-	sl := newBasicScrapeLoopWithFallback(t, ctx, scraper, app, 10*time.Millisecond, "text/plain")
-	sl.trackTimestampsStaleness = true
 	// Succeed once, several failures, then stop.
 	numScrapes := 0
-
 	scraper.scrapeFunc = func(_ context.Context, w io.Writer) error {
 		numScrapes++
 
 		switch numScrapes {
 		case 1:
-			fmt.Fprintf(w, "metric_a 42 %d\n", time.Now().UnixNano()/int64(time.Millisecond))
+			_, _ = fmt.Fprintf(w, "metric_a 42 %d\n", time.Now().UnixNano()/int64(time.Millisecond))
 			return nil
 		case 5:
 			cancel()
@@ -5073,17 +4911,19 @@ func TestScrapeLoopRunCreatesStaleMarkersOnFailedScrapeForTimestampedMetrics(t *
 	case <-time.After(5 * time.Second):
 		t.Fatalf("Scrape wasn't stopped.")
 	}
+
+	got := appTest.ResultSamples()
 	// 1 successfully scraped sample, 1 stale marker after first fail, 5 report samples for
 	// each scrape successful or not.
-	require.Len(t, appender.resultFloats, 27, "Appended samples not as expected:\n%s", appender)
-	require.Equal(t, 42.0, appender.resultFloats[0].f, "Appended first sample not as expected")
-	require.True(t, value.IsStaleNaN(appender.resultFloats[6].f),
-		"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(appender.resultFloats[6].f))
+	require.Len(t, got, 27, "Appended samples not as expected:\n%s", appTest)
+	require.Equal(t, 42.0, got[0].V, "Appended first sample not as expected")
+	require.True(t, value.IsStaleNaN(got[6].V),
+		"Appended second sample not as expected. Wanted: stale NaN Got: %x", math.Float64bits(got[6].V))
 }
 
 func TestScrapeLoopCompression(t *testing.T) {
-	simpleStorage := teststorage.New(t)
-	defer simpleStorage.Close()
+	s := teststorage.New(t)
+	t.Cleanup(func() { _ = s.Close() })
 
 	metricsText := makeTestGauges(10)
 
@@ -5105,12 +4945,12 @@ func TestScrapeLoopCompression(t *testing.T) {
 
 			ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 				require.Equal(t, tc.acceptEncoding, r.Header.Get("Accept-Encoding"), "invalid value of the Accept-Encoding header")
-				fmt.Fprint(w, string(metricsText))
+				_, _ = fmt.Fprint(w, string(metricsText))
 				close(scraped)
 			}))
 			defer ts.Close()
 
-			config := &config.ScrapeConfig{
+			cfg := &config.ScrapeConfig{
 				JobName:                    "test",
 				SampleLimit:                100,
 				Scheme:                     "http",
@@ -5121,7 +4961,7 @@ func TestScrapeLoopCompression(t *testing.T) {
 				MetricNameEscapingScheme:   model.AllowUTF8,
 			}
 
-			sp, err := newScrapePool(config, simpleStorage, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+			sp, err := newScrapePool(cfg, s, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 			require.NoError(t, err)
 			defer sp.stop()
 
@@ -5231,11 +5071,11 @@ func BenchmarkTargetScraperGzip(b *testing.B) {
 		gw := gzip.NewWriter(&buf)
 		for j := 0; j < scenarios[i].metricsCount; j++ {
 			name = fmt.Sprintf("go_memstats_alloc_bytes_total_%d", j)
-			fmt.Fprintf(gw, "# HELP %s Total number of bytes allocated, even if freed.\n", name)
-			fmt.Fprintf(gw, "# TYPE %s counter\n", name)
-			fmt.Fprintf(gw, "%s %d\n", name, i*j)
+			_, _ = fmt.Fprintf(gw, "# HELP %s Total number of bytes allocated, even if freed.\n", name)
+			_, _ = fmt.Fprintf(gw, "# TYPE %s counter\n", name)
+			_, _ = fmt.Fprintf(gw, "%s %d\n", name, i*j)
 		}
-		gw.Close()
+		require.NoError(b, gw.Close())
 		scenarios[i].body = buf.Bytes()
 	}
 
@@ -5244,7 +5084,7 @@ func BenchmarkTargetScraperGzip(b *testing.B) {
 		w.Header().Set("Content-Encoding", "gzip")
 		for _, scenario := range scenarios {
 			if strconv.Itoa(scenario.metricsCount) == r.URL.Query()["count"][0] {
-				w.Write(scenario.body)
+				_, _ = w.Write(scenario.body)
 				return
 			}
 		}
@@ -5293,31 +5133,31 @@ func BenchmarkTargetScraperGzip(b *testing.B) {
 // When a scrape contains multiple instances for the same time series we should increment
 // prometheus_target_scrapes_sample_duplicate_timestamp_total metric.
 func TestScrapeLoopSeriesAddedDuplicates(t *testing.T) {
-	ctx, sl := simpleTestScrapeLoop(t)
+	sl, _ := newTestScrapeLoop(t)
 
-	slApp := sl.appender(ctx)
-	total, added, seriesAdded, err := sl.append(slApp, []byte("test_metric 1\ntest_metric 2\ntest_metric 3\n"), "text/plain", time.Time{})
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("test_metric 1\ntest_metric 2\ntest_metric 3\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
 	require.Equal(t, 1, seriesAdded)
 	require.Equal(t, 2.0, prom_testutil.ToFloat64(sl.metrics.targetScrapeSampleDuplicate))
 
-	slApp = sl.appender(ctx)
-	total, added, seriesAdded, err = sl.append(slApp, []byte("test_metric 1\ntest_metric 1\ntest_metric 1\n"), "text/plain", time.Time{})
+	app = sl.appender()
+	total, added, seriesAdded, err = app.append([]byte("test_metric 1\ntest_metric 1\ntest_metric 1\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
 	require.Equal(t, 0, seriesAdded)
 	require.Equal(t, 4.0, prom_testutil.ToFloat64(sl.metrics.targetScrapeSampleDuplicate))
 
 	// When different timestamps are supplied, multiple samples are accepted.
-	slApp = sl.appender(ctx)
-	total, added, seriesAdded, err = sl.append(slApp, []byte("test_metric 1 1001\ntest_metric 1 1002\ntest_metric 1 1003\n"), "text/plain", time.Time{})
+	app = sl.appender()
+	total, added, seriesAdded, err = app.append([]byte("test_metric 1 1001\ntest_metric 1 1002\ntest_metric 1 1003\n"), "text/plain", time.Time{})
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 3, total)
 	require.Equal(t, 3, added)
 	require.Equal(t, 0, seriesAdded)
@@ -5365,7 +5205,7 @@ func testNativeHistogramMaxSchemaSet(t *testing.T, minBucketFactor string, expec
 		},
 	)
 	registry := prometheus.NewRegistry()
-	registry.Register(nativeHistogram)
+	require.NoError(t, registry.Register(nativeHistogram))
 	nativeHistogram.Observe(1.0)
 	nativeHistogram.Observe(1.0)
 	nativeHistogram.Observe(1.0)
@@ -5379,10 +5219,10 @@ func testNativeHistogramMaxSchemaSet(t *testing.T, minBucketFactor string, expec
 	histogramMetricFamily := gathered[0]
 	buffer := protoMarshalDelimited(t, histogramMetricFamily)
 
-	// Create a HTTP server to serve /metrics via ProtoBuf
+	// Create an HTTP server to serve /metrics via ProtoBuf
 	metricsServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
 		w.Header().Set("Content-Type", `application/vnd.google.protobuf; proto=io.prometheus.client.MetricFamily; encoding=delimited`)
-		w.Write(buffer)
+		_, _ = w.Write(buffer)
 	}))
 	defer metricsServer.Close()
 
@@ -5401,18 +5241,17 @@ scrape_configs:
 `, minBucketFactor, strings.ReplaceAll(metricsServer.URL, "http://", ""))
 
 	s := teststorage.New(t)
-	defer s.Close()
+	t.Cleanup(func() { _ = s.Close() })
 	reg := prometheus.NewRegistry()
 
 	mng, err := NewManager(&Options{DiscoveryReloadInterval: model.Duration(10 * time.Millisecond)}, nil, nil, s, reg)
 	require.NoError(t, err)
 	cfg, err := config.Load(configStr, promslog.NewNopLogger())
 	require.NoError(t, err)
-	mng.ApplyConfig(cfg)
+	require.NoError(t, mng.ApplyConfig(cfg))
 	tsets := make(chan map[string][]*targetgroup.Group)
 	go func() {
-		err = mng.Run(tsets)
-		require.NoError(t, err)
+		require.NoError(t, mng.Run(tsets))
 	}()
 	defer mng.Stop()
 
@@ -5441,7 +5280,7 @@ scrape_configs:
 	q, err := s.Querier(0, math.MaxInt64)
 	require.NoError(t, err)
 	seriesS := q.Select(context.Background(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "__name__", "testing_example_native_histogram"))
-	histogramSamples := []*histogram.Histogram{}
+	var histogramSamples []*histogram.Histogram
 	for seriesS.Next() {
 		series := seriesS.At()
 		it := series.Iterator(nil)
@@ -5487,7 +5326,7 @@ func TestTargetScrapeConfigWithLabels(t *testing.T) {
 				require.Equal(t, expectedPath, r.URL.Path)
 
 				w.Header().Set("Content-Type", `text/plain; version=0.0.4`)
-				w.Write([]byte("metric_a 1\nmetric_b 2\n"))
+				_, _ = w.Write([]byte("metric_a 1\nmetric_b 2\n"))
 			}),
 		)
 		t.Cleanup(server.Close)
@@ -5507,7 +5346,7 @@ func TestTargetScrapeConfigWithLabels(t *testing.T) {
 			}
 		}
 
-		sp, err := newScrapePool(cfg, &nopAppendable{}, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+		sp, err := newScrapePool(cfg, teststorage.NewAppendable(), 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 		require.NoError(t, err)
 		t.Cleanup(sp.stop)
 
@@ -5635,7 +5474,7 @@ func newScrapableServer(scrapeText string) (s *httptest.Server, scrapedTwice cha
 	scrapedTwice = make(chan bool)
 
 	return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
-		fmt.Fprint(w, scrapeText)
+		_, _ = fmt.Fprint(w, scrapeText)
 		scrapes++
 		if scrapes == 2 {
 			close(scrapedTwice)
@@ -5647,7 +5486,7 @@ func newScrapableServer(scrapeText string) (s *httptest.Server, scrapedTwice cha
 func TestScrapePoolScrapeAfterReload(t *testing.T) {
 	h := httptest.NewServer(http.HandlerFunc(
 		func(w http.ResponseWriter, _ *http.Request) {
-			w.Write([]byte{0x42, 0x42})
+			_, _ = w.Write([]byte{0x42, 0x42})
 		},
 	))
 	t.Cleanup(h.Close)
@@ -5670,7 +5509,7 @@ func TestScrapePoolScrapeAfterReload(t *testing.T) {
 		},
 	}
 
-	p, err := newScrapePool(cfg, &nopAppendable{}, 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
+	p, err := newScrapePool(cfg, teststorage.NewAppendable(), 0, nil, nil, &Options{}, newTestScrapeMetrics(t))
 	require.NoError(t, err)
 	t.Cleanup(p.stop)
 
@@ -5697,103 +5536,105 @@ func TestScrapeAppendWithParseError(t *testing.T) {
 # EOF`
 	)
 
-	sl := newBasicScrapeLoop(t, context.Background(), nil, nil, 0)
-	sl.cache = newScrapeCache(sl.metrics)
-
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, withAppendable(appTest))
 	now := time.Now()
-	capp := &collectResultAppender{next: nopAppender{}}
-	_, _, _, err := sl.append(capp, []byte(scrape1), "application/openmetrics-text", now)
+
+	app := sl.appender()
+	_, _, _, err := app.append([]byte(scrape1), "application/openmetrics-text", now)
 	require.Error(t, err)
-	_, _, _, err = sl.append(capp, nil, "application/openmetrics-text", now)
-	require.NoError(t, err)
-	require.Empty(t, capp.resultFloats)
+	require.NoError(t, app.Rollback())
 
-	capp = &collectResultAppender{next: nopAppender{}}
-	_, _, _, err = sl.append(capp, []byte(scrape2), "application/openmetrics-text", now.Add(15*time.Second))
+	app = sl.appender()
+	_, _, _, err = app.append(nil, "application/openmetrics-text", now)
 	require.NoError(t, err)
-	require.NoError(t, capp.Commit())
+	require.NoError(t, app.Commit())
+	require.Empty(t, appTest.ResultSamples())
 
-	want := []floatSample{
+	app = sl.appender()
+	_, _, _, err = app.append([]byte(scrape2), "application/openmetrics-text", now.Add(15*time.Second))
+	require.NoError(t, err)
+	require.NoError(t, app.Commit())
+
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now.Add(15 * time.Second)),
-			f:      11,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now.Add(15 * time.Second)),
+			V: 11,
 		},
 	}
-	requireEqual(t, want, capp.resultFloats, "Appended samples not as expected:\n%s", capp)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", appTest)
 }
 
-// This test covers a case where there's a target with sample_limit set and the some of exporter samples
+// This test covers a case where there's a target with sample_limit set and some samples
 // changes between scrapes.
 func TestScrapeLoopAppendSampleLimitWithDisappearingSeries(t *testing.T) {
 	const sampleLimit = 4
-	resApp := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(_ context.Context) storage.Appender {
-		return resApp
-	}, 0)
-	sl.sampleLimit = sampleLimit
+
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.sampleLimit = sampleLimit
+	})
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	samplesScraped, samplesAfterRelabel, createdSeries, err := sl.append(
-		slApp,
+	app := sl.appender()
+	samplesScraped, samplesAfterRelabel, createdSeries, err := app.append(
 		// Start with 3 samples, all accepted.
 		[]byte("metric_a 1\nmetric_b 1\nmetric_c 1\n"),
 		"text/plain",
 		now,
 	)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 3, samplesScraped)      // All on scrape.
 	require.Equal(t, 3, samplesAfterRelabel) // This is series after relabeling.
 	require.Equal(t, 3, createdSeries)       // Newly added to TSDB.
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_b"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_b"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_c"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_c"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 	}
-	requireEqual(t, want, resApp.resultFloats, "Appended samples not as expected:\n%s", slApp)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", app)
 
 	now = now.Add(time.Minute)
-	slApp = sl.appender(context.Background())
-	samplesScraped, samplesAfterRelabel, createdSeries, err = sl.append(
-		slApp,
+	app = sl.appender()
+	samplesScraped, samplesAfterRelabel, createdSeries, err = app.append(
 		// Start exporting 3 more samples, so we're over the limit now.
 		[]byte("metric_a 1\nmetric_b 1\nmetric_c 1\nmetric_d 1\nmetric_e 1\nmetric_f 1\n"),
 		"text/plain",
 		now,
 	)
 	require.ErrorIs(t, err, errSampleLimit)
-	require.NoError(t, slApp.Rollback())
+	require.NoError(t, app.Rollback())
 	require.Equal(t, 6, samplesScraped)
 	require.Equal(t, 6, samplesAfterRelabel)
 	require.Equal(t, 1, createdSeries) // We've added one series before hitting the limit.
-	requireEqual(t, want, resApp.resultFloats, "Appended samples not as expected:\n%s", slApp)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", app)
 	sl.cache.iterDone(false)
 
 	now = now.Add(time.Minute)
-	slApp = sl.appender(context.Background())
-	samplesScraped, samplesAfterRelabel, createdSeries, err = sl.append(
-		slApp,
+	app = sl.appender()
+	samplesScraped, samplesAfterRelabel, createdSeries, err = app.append(
 		// Remove all samples except first 2.
 		[]byte("metric_a 1\nmetric_b 1\n"),
 		"text/plain",
 		now,
 	)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 2, samplesScraped)
 	require.Equal(t, 2, samplesAfterRelabel)
 	require.Equal(t, 0, createdSeries)
@@ -5802,152 +5643,147 @@ func TestScrapeLoopAppendSampleLimitWithDisappearingSeries(t *testing.T) {
 	// - Append with stale markers for metric_c - this series was added during first scrape but disappeared during last scrape.
 	// - Append with stale marker for metric_d - this series was added during second scrape before we hit the sample_limit.
 	// We should NOT see:
-	// - Appends with stale markers for metric_e & metric_f - both over the limit during second scrape and so they never made it into TSDB.
-	want = append(want, []floatSample{
+	// - Appends with stale markers for metric_e & metric_f - both over the limit during second scrape, and so they never made it into TSDB.
+	want = append(want, []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_b"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_b"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_c"),
-			t:      timestamp.FromTime(now),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_c"),
+			T: timestamp.FromTime(now),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_d"),
-			t:      timestamp.FromTime(now),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_d"),
+			T: timestamp.FromTime(now),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 	}...)
-	requireEqual(t, want, resApp.resultFloats, "Appended samples not as expected:\n%s", slApp)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", app)
 }
 
 // This test covers a case where there's a target with sample_limit set and each scrape sees a completely
 // different set of samples.
 func TestScrapeLoopAppendSampleLimitReplaceAllSamples(t *testing.T) {
 	const sampleLimit = 4
-	resApp := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(_ context.Context) storage.Appender {
-		return resApp
-	}, 0)
-	sl.sampleLimit = sampleLimit
+
+	appTest := teststorage.NewAppendable()
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.appendable = appTest
+		sl.sampleLimit = sampleLimit
+	})
 
 	now := time.Now()
-	slApp := sl.appender(context.Background())
-	samplesScraped, samplesAfterRelabel, createdSeries, err := sl.append(
-		slApp,
+	app := sl.appender()
+	samplesScraped, samplesAfterRelabel, createdSeries, err := app.append(
 		// Start with 4 samples, all accepted.
 		[]byte("metric_a 1\nmetric_b 1\nmetric_c 1\nmetric_d 1\n"),
 		"text/plain",
 		now,
 	)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 4, samplesScraped)      // All on scrape.
 	require.Equal(t, 4, samplesAfterRelabel) // This is series after relabeling.
 	require.Equal(t, 4, createdSeries)       // Newly added to TSDB.
-	want := []floatSample{
+	want := []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_b"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_b"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_c"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_c"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_d"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_d"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 	}
-	requireEqual(t, want, resApp.resultFloats, "Appended samples not as expected:\n%s", slApp)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", app)
 
 	now = now.Add(time.Minute)
-	slApp = sl.appender(context.Background())
-	samplesScraped, samplesAfterRelabel, createdSeries, err = sl.append(
-		slApp,
+	app = sl.appender()
+	samplesScraped, samplesAfterRelabel, createdSeries, err = app.append(
 		// Replace all samples with new time series.
 		[]byte("metric_e 1\nmetric_f 1\nmetric_g 1\nmetric_h 1\n"),
 		"text/plain",
 		now,
 	)
 	require.NoError(t, err)
-	require.NoError(t, slApp.Commit())
+	require.NoError(t, app.Commit())
 	require.Equal(t, 4, samplesScraped)
 	require.Equal(t, 4, samplesAfterRelabel)
 	require.Equal(t, 4, createdSeries)
 	// We replaced all samples from first scrape with new set of samples.
-	// We expect to see:
+	// We expected to see:
 	// - 4 appends for new samples.
 	// - 4 appends with staleness markers for old samples.
-	want = append(want, []floatSample{
+	want = append(want, []sample{
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_e"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_e"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_f"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_f"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_g"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_g"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_h"),
-			t:      timestamp.FromTime(now),
-			f:      1,
+			L: labels.FromStrings(model.MetricNameLabel, "metric_h"),
+			T: timestamp.FromTime(now),
+			V: 1,
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_a"),
-			t:      timestamp.FromTime(now),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_a"),
+			T: timestamp.FromTime(now),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_b"),
-			t:      timestamp.FromTime(now),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_b"),
+			T: timestamp.FromTime(now),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_c"),
-			t:      timestamp.FromTime(now),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_c"),
+			T: timestamp.FromTime(now),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 		{
-			metric: labels.FromStrings(model.MetricNameLabel, "metric_d"),
-			t:      timestamp.FromTime(now),
-			f:      math.Float64frombits(value.StaleNaN),
+			L: labels.FromStrings(model.MetricNameLabel, "metric_d"),
+			T: timestamp.FromTime(now),
+			V: math.Float64frombits(value.StaleNaN),
 		},
 	}...)
-	requireEqual(t, want, resApp.resultFloats, "Appended samples not as expected:\n%s", slApp)
+	requireEqual(t, want, appTest.ResultSamples(), "Appended samples not as expected:\n%s", app)
 }
 
 func TestScrapeLoopDisableStalenessMarkerInjection(t *testing.T) {
-	var (
-		loopDone = atomic.NewBool(false)
-		appender = &collectResultAppender{}
-		scraper  = &testScraper{}
-		app      = func(_ context.Context) storage.Appender { return appender }
-	)
+	loopDone := atomic.NewBool(false)
 
-	sl := newBasicScrapeLoop(t, context.Background(), scraper, app, 10*time.Millisecond)
+	appTest := teststorage.NewAppendable()
+	sl, scraper := newTestScrapeLoop(t, withAppendable(appTest))
 	scraper.scrapeFunc = func(ctx context.Context, w io.Writer) error {
 		if _, err := w.Write([]byte("metric_a 42\n")); err != nil {
 			return err
@@ -5963,9 +5799,7 @@ func TestScrapeLoopDisableStalenessMarkerInjection(t *testing.T) {
 
 	// Wait for some samples to be appended.
 	require.Eventually(t, func() bool {
-		appender.mtx.Lock()
-		defer appender.mtx.Unlock()
-		return len(appender.resultFloats) > 2
+		return len(appTest.ResultSamples()) > 2
 	}, 5*time.Second, 100*time.Millisecond, "Scrape loop didn't append any samples.")
 
 	// Disable end of run staleness markers and stop the loop.
@@ -5976,9 +5810,46 @@ func TestScrapeLoopDisableStalenessMarkerInjection(t *testing.T) {
 	}, 5*time.Second, 100*time.Millisecond, "Scrape loop didn't stop.")
 
 	// No stale markers should be appended, since they were disabled.
-	for _, s := range appender.resultFloats {
-		if value.IsStaleNaN(s.f) {
-			t.Fatalf("Got stale NaN samples while end of run staleness is disabled: %x", math.Float64bits(s.f))
+	for _, s := range appTest.ResultSamples() {
+		if value.IsStaleNaN(s.V) {
+			t.Fatalf("Got stale NaN samples while end of run staleness is disabled: %x", math.Float64bits(s.V))
 		}
 	}
 }
+
+// Recommended CLI invocation:
+/*
+	export bench=restartLoops && go test ./scrape/... \
+		-run '^$' -bench '^BenchmarkScrapePoolRestartLoops' \
+		-benchtime 5s -count 6 -cpu 2 -timeout 999m \
+		| tee ${bench}.txt
+*/
+func BenchmarkScrapePoolRestartLoops(b *testing.B) {
+	sp, err := newScrapePool(
+		&config.ScrapeConfig{
+			MetricNameValidationScheme: model.UTF8Validation,
+			ScrapeInterval:             model.Duration(1 * time.Hour),
+			ScrapeTimeout:              model.Duration(1 * time.Hour),
+		},
+		nil,
+		0,
+		nil,
+		nil,
+		&Options{},
+		newTestScrapeMetrics(b),
+	)
+	require.NoError(b, err)
+	b.Cleanup(sp.stop)
+
+	for i := range 1000 {
+		sp.activeTargets[uint64(i)] = &Target{scrapeConfig: &config.ScrapeConfig{}}
+		sp.loops[uint64(i)] = noopLoop() // First restart will supplement those with proper scrapeLoops.
+	}
+	sp.restartLoops(true)
+
+	b.ReportAllocs()
+	b.ResetTimer()
+	for b.Loop() {
+		sp.restartLoops(true)
+	}
+}
diff --git a/scrape/target.go b/scrape/target.go
index 2aabff20e2..4265f9e782 100644
--- a/scrape/target.go
+++ b/scrape/target.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/scrape/target_test.go b/scrape/target_test.go
index 582b198c79..06227da816 100644
--- a/scrape/target_test.go
+++ b/scrape/target_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -14,7 +14,6 @@
 package scrape
 
 import (
-	"context"
 	"crypto/tls"
 	"crypto/x509"
 	"fmt"
@@ -36,7 +35,7 @@ import (
 	"github.com/prometheus/prometheus/model/histogram"
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/model/timestamp"
-	"github.com/prometheus/prometheus/storage"
+	"github.com/prometheus/prometheus/util/teststorage"
 )
 
 const (
@@ -611,12 +610,12 @@ func TestBucketLimitAppender(t *testing.T) {
 		},
 	}
 
-	resApp := &collectResultAppender{}
+	appTest := teststorage.NewAppendable()
 
 	for _, c := range cases {
 		for _, floatHisto := range []bool{true, false} {
 			t.Run(fmt.Sprintf("floatHistogram=%t", floatHisto), func(t *testing.T) {
-				app := &bucketLimitAppender{Appender: resApp, limit: c.limit}
+				app := &bucketLimitAppender{Appender: appTest.Appender(t.Context()), limit: c.limit}
 				ts := int64(10 * time.Minute / time.Millisecond)
 				lbls := labels.FromStrings("__name__", "sparse_histogram_series")
 				var err error
@@ -697,12 +696,12 @@ func TestMaxSchemaAppender(t *testing.T) {
 		},
 	}
 
-	resApp := &collectResultAppender{}
+	appTest := teststorage.NewAppendable()
 
 	for _, c := range cases {
 		for _, floatHisto := range []bool{true, false} {
 			t.Run(fmt.Sprintf("floatHistogram=%t", floatHisto), func(t *testing.T) {
-				app := &maxSchemaAppender{Appender: resApp, maxSchema: c.maxSchema}
+				app := &maxSchemaAppender{Appender: appTest.Appender(t.Context()), maxSchema: c.maxSchema}
 				ts := int64(10 * time.Minute / time.Millisecond)
 				lbls := labels.FromStrings("__name__", "sparse_histogram_series")
 				var err error
@@ -723,17 +722,12 @@ func TestMaxSchemaAppender(t *testing.T) {
 	}
 }
 
-// Test sample_limit when a scrape containst Native Histograms.
+// Test sample_limit when a scrape contains Native Histograms.
 func TestAppendWithSampleLimitAndNativeHistogram(t *testing.T) {
-	const sampleLimit = 2
-	resApp := &collectResultAppender{}
-	sl := newBasicScrapeLoop(t, context.Background(), nil, func(_ context.Context) storage.Appender {
-		return resApp
-	}, 0)
-	sl.sampleLimit = sampleLimit
+	appTest := teststorage.NewAppendable()
 
 	now := time.Now()
-	app := appender(sl.appender(context.Background()), sl.sampleLimit, sl.bucketLimit, sl.maxSchema)
+	app := appenderWithLimits(appTest.Appender(t.Context()), 2, 0, histogram.ExponentialSchemaMax)
 
 	// sample_limit is set to 2, so first two scrapes should work
 	_, err := app.Append(0, labels.FromStrings(model.MetricNameLabel, "foo"), timestamp.FromTime(now), 1)
diff --git a/util/teststorage/appender.go b/util/teststorage/appender.go
new file mode 100644
index 0000000000..058a09561c
--- /dev/null
+++ b/util/teststorage/appender.go
@@ -0,0 +1,399 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package teststorage
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"math"
+	"slices"
+	"strings"
+	"sync"
+
+	"github.com/prometheus/common/model"
+	"go.uber.org/atomic"
+
+	"github.com/prometheus/prometheus/model/exemplar"
+	"github.com/prometheus/prometheus/model/histogram"
+	"github.com/prometheus/prometheus/model/labels"
+	"github.com/prometheus/prometheus/model/metadata"
+	"github.com/prometheus/prometheus/storage"
+)
+
+// Sample represents test, combined sample for mocking storage.AppenderV2.
+type Sample struct {
+	MF    string
+	L     labels.Labels
+	M     metadata.Metadata
+	ST, T int64
+	V     float64
+	H     *histogram.Histogram
+	FH    *histogram.FloatHistogram
+	ES    []exemplar.Exemplar
+}
+
+func (s Sample) String() string {
+	// Attempting to format similar to ~ OpenMetrics 2.0 for readability.
+	b := strings.Builder{}
+	if s.M.Help != "" {
+		b.WriteString("HELP ")
+		b.WriteString(s.M.Help)
+		b.WriteString("\n")
+	}
+	if s.M.Type != model.MetricTypeUnknown && s.M.Type != "" {
+		b.WriteString("type@")
+		b.WriteString(string(s.M.Type))
+		b.WriteString(" ")
+	}
+	if s.M.Unit != "" {
+		b.WriteString("unit@")
+		b.WriteString(s.M.Unit)
+		b.WriteString(" ")
+	}
+	// Print all value types on purpose, to catch bugs for appending multiple sample types at once.
+	h := ""
+	if s.H != nil {
+		h = s.H.String()
+	}
+	fh := ""
+	if s.FH != nil {
+		fh = s.FH.String()
+	}
+	b.WriteString(fmt.Sprintf("%s %v%v%v st@%v t@%v\n", s.L.String(), s.V, h, fh, s.ST, s.T))
+	return b.String()
+}
+
+func (s Sample) Equals(other Sample) bool {
+	return strings.Compare(s.MF, other.MF) == 0 &&
+		labels.Equal(s.L, other.L) &&
+		s.M.Equals(other.M) &&
+		s.ST == other.ST &&
+		s.T == other.T &&
+		math.Float64bits(s.V) == math.Float64bits(other.V) && // Compare Float64bits so NaN values which are exactly the same will compare equal.
+		s.H.Equals(other.H) &&
+		s.FH.Equals(other.FH) &&
+		slices.EqualFunc(s.ES, other.ES, exemplar.Exemplar.Equals)
+}
+
+// Appendable is a storage.Appendable mock.
+// It allows recording all samples that were added through the appender and injecting errors.
+// Appendable will panic if more than one Appender is open.
+type Appendable struct {
+	appendErrFn          func(ls labels.Labels) error // If non-nil, inject appender error on every Append, AppendHistogram and ST zero calls.
+	appendExemplarsError error                        // If non-nil, inject exemplar error.
+	commitErr            error                        // If non-nil, inject commit error.
+
+	mtx           sync.Mutex
+	openAppenders atomic.Int32 // Guard against multi-appender use.
+
+	// Recorded results.
+	pendingSamples    []Sample
+	resultSamples     []Sample
+	rolledbackSamples []Sample
+
+	// Optional chain (Appender will collect samples, then run next).
+	next storage.Appendable
+}
+
+// NewAppendable returns mock Appendable.
+func NewAppendable() *Appendable {
+	return &Appendable{}
+}
+
+// Then chains another appender from the provided appendable for the Appender calls.
+func (a *Appendable) Then(appendable storage.Appendable) *Appendable {
+	a.next = appendable
+	return a
+}
+
+// WithErrs allows injecting errors to the appender.
+func (a *Appendable) WithErrs(appendErrFn func(ls labels.Labels) error, appendExemplarsError, commitErr error) *Appendable {
+	a.appendErrFn = appendErrFn
+	a.appendExemplarsError = appendExemplarsError
+	a.commitErr = commitErr
+	return a
+}
+
+// PendingSamples returns pending samples (samples appended without commit).
+func (a *Appendable) PendingSamples() []Sample {
+	a.mtx.Lock()
+	defer a.mtx.Unlock()
+
+	ret := make([]Sample, len(a.pendingSamples))
+	copy(ret, a.pendingSamples)
+	return ret
+}
+
+// ResultSamples returns committed samples.
+func (a *Appendable) ResultSamples() []Sample {
+	a.mtx.Lock()
+	defer a.mtx.Unlock()
+
+	ret := make([]Sample, len(a.resultSamples))
+	copy(ret, a.resultSamples)
+	return ret
+}
+
+// RolledbackSamples returns rolled back samples.
+func (a *Appendable) RolledbackSamples() []Sample {
+	a.mtx.Lock()
+	defer a.mtx.Unlock()
+
+	ret := make([]Sample, len(a.rolledbackSamples))
+	copy(ret, a.rolledbackSamples)
+	return ret
+}
+
+func (a *Appendable) ResultReset() {
+	a.mtx.Lock()
+	defer a.mtx.Unlock()
+
+	a.pendingSamples = a.pendingSamples[:0]
+	a.resultSamples = a.resultSamples[:0]
+	a.rolledbackSamples = a.rolledbackSamples[:0]
+}
+
+// ResultMetadata returns resultSamples with samples only containing L and M.
+// This is for compatibility with tests that only focus on metadata.
+//
+// TODO: Rewrite tests to test metadata on resultSamples instead.
+func (a *Appendable) ResultMetadata() []Sample {
+	a.mtx.Lock()
+	defer a.mtx.Unlock()
+
+	var ret []Sample
+	for _, s := range a.resultSamples {
+		if s.M.IsEmpty() {
+			continue
+		}
+		ret = append(ret, Sample{L: s.L, M: s.M})
+	}
+	return ret
+}
+
+func (a *Appendable) String() string {
+	var sb strings.Builder
+	sb.WriteString("committed:\n")
+	for _, s := range a.resultSamples {
+		sb.WriteString("\n")
+		sb.WriteString(s.String())
+	}
+	sb.WriteString("pending:\n")
+	for _, s := range a.pendingSamples {
+		sb.WriteString("\n")
+		sb.WriteString(s.String())
+	}
+	sb.WriteString("rolledback:\n")
+	for _, s := range a.rolledbackSamples {
+		sb.WriteString("\n")
+		sb.WriteString(s.String())
+	}
+	return sb.String()
+}
+
+var errClosedAppender = errors.New("appender was already committed/rolledback")
+
+type appender struct {
+	err  error
+	next storage.Appender
+
+	a *Appendable
+}
+
+func (a *appender) checkErr() error {
+	a.a.mtx.Lock()
+	defer a.a.mtx.Unlock()
+
+	return a.err
+}
+
+func (a *Appendable) Appender(ctx context.Context) storage.Appender {
+	ret := &appender{a: a}
+	if a.openAppenders.Inc() > 1 {
+		ret.err = errors.New("teststorage.Appendable.Appender() concurrent use is not supported; attempted opening new Appender() without Commit/Rollback of the previous one. Extend the implementation if concurrent mock is needed")
+	}
+
+	if a.next != nil {
+		ret.next = a.next.Appender(ctx)
+	}
+	return ret
+}
+
+func (*appender) SetOptions(*storage.AppendOptions) {}
+
+func (a *appender) Append(ref storage.SeriesRef, ls labels.Labels, t int64, v float64) (storage.SeriesRef, error) {
+	if err := a.checkErr(); err != nil {
+		return 0, err
+	}
+
+	if a.a.appendErrFn != nil {
+		if err := a.a.appendErrFn(ls); err != nil {
+			return 0, err
+		}
+	}
+
+	a.a.mtx.Lock()
+	a.a.pendingSamples = append(a.a.pendingSamples, Sample{L: ls, T: t, V: v})
+	a.a.mtx.Unlock()
+
+	if a.next != nil {
+		return a.next.Append(ref, ls, t, v)
+	}
+
+	return computeOrCheckRef(ref, ls)
+}
+
+func computeOrCheckRef(ref storage.SeriesRef, ls labels.Labels) (storage.SeriesRef, error) {
+	h := ls.Hash()
+	if ref == 0 {
+		// Use labels hash as a stand-in for unique series reference, to avoid having to track all series.
+		return storage.SeriesRef(h), nil
+	}
+
+	if storage.SeriesRef(h) != ref {
+		// Check for buggy ref while we at it.
+		return 0, errors.New("teststorage.appender: found input ref not matching labels; potential bug in Appendable user")
+	}
+	return ref, nil
+}
+
+func (a *appender) AppendHistogram(ref storage.SeriesRef, ls labels.Labels, t int64, h *histogram.Histogram, fh *histogram.FloatHistogram) (storage.SeriesRef, error) {
+	if err := a.checkErr(); err != nil {
+		return 0, err
+	}
+	if a.a.appendErrFn != nil {
+		if err := a.a.appendErrFn(ls); err != nil {
+			return 0, err
+		}
+	}
+
+	a.a.mtx.Lock()
+	a.a.pendingSamples = append(a.a.pendingSamples, Sample{L: ls, T: t, H: h, FH: fh})
+	a.a.mtx.Unlock()
+
+	if a.next != nil {
+		return a.next.AppendHistogram(ref, ls, t, h, fh)
+	}
+
+	return computeOrCheckRef(ref, ls)
+}
+
+func (a *appender) AppendExemplar(ref storage.SeriesRef, l labels.Labels, e exemplar.Exemplar) (storage.SeriesRef, error) {
+	if err := a.checkErr(); err != nil {
+		return 0, err
+	}
+	if a.a.appendExemplarsError != nil {
+		return 0, a.a.appendExemplarsError
+	}
+
+	a.a.mtx.Lock()
+	// NOTE(bwplotka): Eventually exemplar has to be attached to a series and soon
+	// the AppenderV2 will guarantee that for TSDB. Assume this from the mock perspective
+	// with the naive attaching. See: https://github.com/prometheus/prometheus/issues/17632
+	i := len(a.a.pendingSamples) - 1
+	for ; i >= 0; i-- { // Attach exemplars to the last matching sample.
+		if ref == storage.SeriesRef(a.a.pendingSamples[i].L.Hash()) {
+			a.a.pendingSamples[i].ES = append(a.a.pendingSamples[i].ES, e)
+			break
+		}
+	}
+	a.a.mtx.Unlock()
+	if i < 0 {
+		return 0, fmt.Errorf("teststorage.appender: exemplar appender without series; ref %v; l %v; exemplar: %v", ref, l, e)
+	}
+
+	if a.next != nil {
+		return a.next.AppendExemplar(ref, l, e)
+	}
+	return computeOrCheckRef(ref, l)
+}
+
+func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, _, st int64) (storage.SeriesRef, error) {
+	return a.Append(ref, l, st, 0.0) // This will change soon with AppenderV2, but we already report ST as 0 samples.
+}
+
+func (a *appender) AppendHistogramSTZeroSample(ref storage.SeriesRef, l labels.Labels, _, st int64, h *histogram.Histogram, _ *histogram.FloatHistogram) (storage.SeriesRef, error) {
+	if h != nil {
+		return a.AppendHistogram(ref, l, st, &histogram.Histogram{}, nil)
+	}
+	return a.AppendHistogram(ref, l, st, nil, &histogram.FloatHistogram{}) // This will change soon with AppenderV2, but we already report ST as 0 histograms.
+}
+
+func (a *appender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m metadata.Metadata) (storage.SeriesRef, error) {
+	if err := a.checkErr(); err != nil {
+		return 0, err
+	}
+
+	a.a.mtx.Lock()
+	// NOTE(bwplotka): Eventually metadata has to be attached to a series and soon
+	// the AppenderV2 will guarantee that for TSDB. Assume this from the mock perspective
+	// with the naive attaching. See: https://github.com/prometheus/prometheus/issues/17632
+	i := len(a.a.pendingSamples) - 1
+	for ; i >= 0; i-- { // Attach metadata to the last matching sample.
+		if ref == storage.SeriesRef(a.a.pendingSamples[i].L.Hash()) {
+			a.a.pendingSamples[i].M = m
+			break
+		}
+	}
+	a.a.mtx.Unlock()
+	if i < 0 {
+		return 0, fmt.Errorf("teststorage.appender: metadata update without series; ref %v; l %v; m: %v", ref, l, m)
+	}
+
+	if a.next != nil {
+		return a.next.UpdateMetadata(ref, l, m)
+	}
+	return computeOrCheckRef(ref, l)
+}
+
+func (a *appender) Commit() error {
+	if err := a.checkErr(); err != nil {
+		return err
+	}
+	defer a.a.openAppenders.Dec()
+
+	if a.a.commitErr != nil {
+		return a.a.commitErr
+	}
+
+	a.a.mtx.Lock()
+	a.a.resultSamples = append(a.a.resultSamples, a.a.pendingSamples...)
+	a.a.pendingSamples = a.a.pendingSamples[:0]
+	a.err = errClosedAppender
+	a.a.mtx.Unlock()
+
+	if a.a.next != nil {
+		return a.next.Commit()
+	}
+	return nil
+}
+
+func (a *appender) Rollback() error {
+	if err := a.checkErr(); err != nil {
+		return err
+	}
+	defer a.a.openAppenders.Dec()
+
+	a.a.mtx.Lock()
+	a.a.rolledbackSamples = append(a.a.rolledbackSamples, a.a.pendingSamples...)
+	a.a.pendingSamples = a.a.pendingSamples[:0]
+	a.err = errClosedAppender
+	a.a.mtx.Unlock()
+
+	if a.next != nil {
+		return a.next.Rollback()
+	}
+	return nil
+}
diff --git a/util/teststorage/appender_test.go b/util/teststorage/appender_test.go
new file mode 100644
index 0000000000..8c2a825c3a
--- /dev/null
+++ b/util/teststorage/appender_test.go
@@ -0,0 +1,131 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package teststorage
+
+import (
+	"errors"
+	"fmt"
+	"testing"
+
+	"github.com/google/go-cmp/cmp"
+	"github.com/stretchr/testify/require"
+
+	"github.com/prometheus/prometheus/model/exemplar"
+	"github.com/prometheus/prometheus/model/labels"
+	"github.com/prometheus/prometheus/model/metadata"
+	"github.com/prometheus/prometheus/util/testutil"
+)
+
+// TestSample_RequireEqual ensures standard testutil.RequireEqual is enough for comparisons.
+// This is thanks to the fact metadata has now Equals method.
+func TestSample_RequireEqual(t *testing.T) {
+	a := []Sample{
+		{},
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+	}
+	testutil.RequireEqual(t, a, a)
+
+	b1 := []Sample{
+		{},
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings("__name__", "test_metric2_diff", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123}, // test_metric2_diff is different.
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+	}
+	requireNotEqual(t, a, b1)
+
+	b2 := []Sample{
+		{},
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo2")}}}, // exemplar is different.
+	}
+	requireNotEqual(t, a, b2)
+
+	b3 := []Sample{
+		{},
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123, T: 123}, // Timestamp is different.
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+	}
+	requireNotEqual(t, a, b3)
+
+	b4 := []Sample{
+		{},
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 456.456}, // Value is different.
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+	}
+	requireNotEqual(t, a, b4)
+
+	b5 := []Sample{
+		{},
+		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter2", Unit: "metric", Help: "some help text"}}, // Different type.
+		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+	}
+	requireNotEqual(t, a, b5)
+}
+
+// TODO(bwplotka): While this mimick testutil.RequireEqual just making it negative, this does not literally test
+// testutil.RequireEqual. Either build test suita that mocks `testing.TB` or get rid of testutil.RequireEqual somehow.
+func requireNotEqual(t testing.TB, a, b any) {
+	t.Helper()
+	if !cmp.Equal(a, b, cmp.Comparer(labels.Equal)) {
+		return
+	}
+	require.Fail(t, fmt.Sprintf("Equal, but expected not: \n"+
+		"a: %s\n"+
+		"b: %s", a, b))
+}
+
+func TestConcurrentAppender_ReturnsErrAppender(t *testing.T) {
+	a := NewAppendable()
+
+	// Non-concurrent multiple use if fine.
+	app := a.Appender(t.Context())
+	require.Equal(t, int32(1), a.openAppenders.Load())
+	require.NoError(t, app.Commit())
+	// Repeated commit fails.
+	require.Error(t, app.Commit())
+
+	app = a.Appender(t.Context())
+	require.NoError(t, app.Rollback())
+	// Commit after rollback fails.
+	require.Error(t, app.Commit())
+
+	a.WithErrs(
+		nil,
+		nil,
+		errors.New("commit err"),
+	)
+	app = a.Appender(t.Context())
+	require.Error(t, app.Commit())
+
+	a.WithErrs(nil, nil, nil)
+	app = a.Appender(t.Context())
+	require.NoError(t, app.Commit())
+	require.Equal(t, int32(0), a.openAppenders.Load())
+
+	// Concurrent use should return appender that errors.
+	_ = a.Appender(t.Context())
+	app = a.Appender(t.Context())
+	_, err := app.Append(0, labels.EmptyLabels(), 0, 0)
+	require.Error(t, err)
+	_, err = app.AppendHistogram(0, labels.EmptyLabels(), 0, nil, nil)
+	require.Error(t, err)
+	require.Error(t, app.Commit())
+	require.Error(t, app.Rollback())
+}

From f0dfb9f8027111c00d86824eb29f0aaa3893771b Mon Sep 17 00:00:00 2001
From: Arve Knudsen 
Date: Mon, 22 Dec 2025 16:28:08 +0100
Subject: [PATCH 176/439] fix(scrape): use HonorLabels instead of
 HonorTimestamps in newScrapeLoop (#17731)

* fix(scrape): use HonorLabels instead of HonorTimestamps in newScrapeLoop

The sampleMutator closure in newScrapeLoop was incorrectly passing
HonorTimestamps to mutateSampleLabels instead of HonorLabels. This
caused honor_labels configuration to be ignored, with the behavior
incorrectly controlled by honor_timestamps instead.

Adding TestNewScrapeLoopHonorLabelsWiring integration test that exercises
the real newScrapeLoop constructor with HonorLabels and HonorTimestamps
set to opposite values to catch this class of wiring bug.

Signed-off-by: Arve Knudsen 

* Update scrape/scrape_test.go

Co-authored-by: George Krajcsovits 
Signed-off-by: Arve Knudsen 

* Add honor_labels=false test case

Signed-off-by: Arve Knudsen 

---------

Signed-off-by: Arve Knudsen 
Co-authored-by: George Krajcsovits 
---
 scrape/scrape.go      |  2 +-
 scrape/scrape_test.go | 79 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/scrape/scrape.go b/scrape/scrape.go
index 6be2525fe0..33683b4caf 100644
--- a/scrape/scrape.go
+++ b/scrape/scrape.go
@@ -1182,7 +1182,7 @@ func newScrapeLoop(opts scrapeLoopOptions) *scrapeLoop {
 		interval: opts.interval,
 		timeout:  opts.timeout,
 		sampleMutator: func(l labels.Labels) labels.Labels {
-			return mutateSampleLabels(l, opts.target, opts.sp.config.HonorTimestamps, opts.sp.config.MetricRelabelConfigs)
+			return mutateSampleLabels(l, opts.target, opts.sp.config.HonorLabels, opts.sp.config.MetricRelabelConfigs)
 		},
 		reportSampleMutator: func(l labels.Labels) labels.Labels { return mutateReportSampleLabels(l, opts.target) },
 		scraper:             opts.scraper,
diff --git a/scrape/scrape_test.go b/scrape/scrape_test.go
index ae004bbd56..7aa633d387 100644
--- a/scrape/scrape_test.go
+++ b/scrape/scrape_test.go
@@ -5853,3 +5853,82 @@ func BenchmarkScrapePoolRestartLoops(b *testing.B) {
 		sp.restartLoops(true)
 	}
 }
+
+// TestNewScrapeLoopHonorLabelsWiring verifies that newScrapeLoop correctly wires
+// HonorLabels (not HonorTimestamps) to the sampleMutator.
+func TestNewScrapeLoopHonorLabelsWiring(t *testing.T) {
+	// Scraped metric has label "lbl" with value "scraped".
+	// Discovery target has label "lbl" with value "discovery".
+	// With honor_labels=true, the scraped value should win.
+	// With honor_labels=false, the discovery value should win and scraped moves to exported_lbl.
+	testCases := []struct {
+		name           string
+		honorLabels    bool
+		expectedLbl    string
+		expectedExpLbl string // exported_lbl value, empty if not expected
+	}{
+		{
+			name:        "honor_labels=true",
+			honorLabels: true,
+			expectedLbl: "scraped",
+		},
+		{
+			name:           "honor_labels=false",
+			honorLabels:    false,
+			expectedLbl:    "discovery",
+			expectedExpLbl: "scraped",
+		},
+	}
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			ts, scrapedTwice := newScrapableServer(`metric{lbl="scraped"} 1`)
+			defer ts.Close()
+
+			testURL, err := url.Parse(ts.URL)
+			require.NoError(t, err)
+
+			s := teststorage.New(t)
+			defer s.Close()
+
+			cfg := &config.ScrapeConfig{
+				JobName:                    "test",
+				Scheme:                     "http",
+				HonorLabels:                tc.honorLabels,
+				HonorTimestamps:            !tc.honorLabels, // Opposite of HonorLabels to catch wiring bugs
+				ScrapeInterval:             model.Duration(1 * time.Second),
+				ScrapeTimeout:              model.Duration(100 * time.Millisecond),
+				MetricNameValidationScheme: model.UTF8Validation,
+			}
+
+			sp, err := newScrapePool(cfg, s, 0, nil, nil, &Options{skipOffsetting: true}, newTestScrapeMetrics(t))
+			require.NoError(t, err)
+			defer sp.stop()
+
+			// Sync with a target that has a conflicting label.
+			sp.Sync([]*targetgroup.Group{{
+				Targets: []model.LabelSet{{
+					model.AddressLabel: model.LabelValue(testURL.Host),
+					"lbl":              "discovery",
+				}},
+			}})
+			require.Len(t, sp.ActiveTargets(), 1)
+
+			// Wait for scrape to complete.
+			select {
+			case <-time.After(5 * time.Second):
+				t.Fatal("scrape did not complete in time")
+			case <-scrapedTwice:
+			}
+
+			// Query the storage to verify label values.
+			q, err := s.Querier(time.Time{}.UnixNano(), time.Now().UnixNano())
+			require.NoError(t, err)
+			defer q.Close()
+
+			series := q.Select(t.Context(), false, nil, labels.MustNewMatcher(labels.MatchEqual, "__name__", "metric"))
+			require.True(t, series.Next(), "metric series not found")
+			require.Equal(t, tc.expectedLbl, series.At().Labels().Get("lbl"))
+			require.Equal(t, tc.expectedExpLbl, series.At().Labels().Get("exported_lbl"))
+		})
+	}
+}

From e28d765d90437ca97c1a3e15c716b568e2e97bef Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 23 Dec 2025 10:12:24 +0100
Subject: [PATCH 177/439] chore(deps): update google/oss-fuzz digest to 4bf20ff
 (#17726)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 .github/workflows/fuzzing.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/fuzzing.yml b/.github/workflows/fuzzing.yml
index 24702c2920..60f643b4f0 100644
--- a/.github/workflows/fuzzing.yml
+++ b/.github/workflows/fuzzing.yml
@@ -10,12 +10,12 @@ jobs:
     steps:
       - name: Build Fuzzers
         id: build
-        uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@537c8005ba4c9de026b2fa3550663280d25d6175 # master
+        uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@4bf20ff8dfda18ad651583ebca9fb17a7ce1940a # master
         with:
           oss-fuzz-project-name: "prometheus"
           dry-run: false
       - name: Run Fuzzers
-        uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@537c8005ba4c9de026b2fa3550663280d25d6175 # master
+        uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@4bf20ff8dfda18ad651583ebca9fb17a7ce1940a # master
         # Note: Regularly check for updates to the pinned commit hash at:
         # https://github.com/google/oss-fuzz/tree/master/infra/cifuzz/actions/run_fuzzers
         with:

From 0bbf5c47ac675fadd7449e02512e016d56dab5e2 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 23 Dec 2025 10:17:55 +0100
Subject: [PATCH 178/439] chore(deps): update dependency ts-jest to v29.4.6
 (#17729)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 web/ui/package-lock.json | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json
index 883ee7aaee..5df415da49 100644
--- a/web/ui/package-lock.json
+++ b/web/ui/package-lock.json
@@ -24,7 +24,7 @@
     },
     "mantine-ui": {
       "name": "@prometheus-io/mantine-ui",
-      "version": "0.308.0",
+      "version": "0.308.1",
       "dependencies": {
         "@codemirror/autocomplete": "^6.19.1",
         "@codemirror/language": "^6.11.3",
@@ -42,7 +42,7 @@
         "@microsoft/fetch-event-source": "^2.0.1",
         "@nexucis/fuzzy": "^0.5.1",
         "@nexucis/kvsearch": "^0.9.1",
-        "@prometheus-io/codemirror-promql": "0.308.0",
+        "@prometheus-io/codemirror-promql": "0.308.1",
         "@reduxjs/toolkit": "^2.10.1",
         "@tabler/icons-react": "^3.35.0",
         "@tanstack/react-query": "^5.90.7",
@@ -88,10 +88,10 @@
     },
     "module/codemirror-promql": {
       "name": "@prometheus-io/codemirror-promql",
-      "version": "0.308.0",
+      "version": "0.308.1",
       "license": "Apache-2.0",
       "dependencies": {
-        "@prometheus-io/lezer-promql": "0.308.0",
+        "@prometheus-io/lezer-promql": "0.308.1",
         "lru-cache": "^11.2.2"
       },
       "devDependencies": {
@@ -121,7 +121,7 @@
     },
     "module/lezer-promql": {
       "name": "@prometheus-io/lezer-promql",
-      "version": "0.308.0",
+      "version": "0.308.1",
       "license": "Apache-2.0",
       "devDependencies": {
         "@lezer/generator": "^1.8.0",
@@ -8693,10 +8693,11 @@
       }
     },
     "node_modules/ts-jest": {
-      "version": "29.4.5",
-      "resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.4.5.tgz",
-      "integrity": "sha512-HO3GyiWn2qvTQA4kTgjDcXiMwYQt68a1Y8+JuLRVpdIzm+UOLSHgl/XqR4c6nzJkq5rOkjc02O2I7P7l/Yof0Q==",
+      "version": "29.4.6",
+      "resolved": "https://registry.npmjs.org/ts-jest/-/ts-jest-29.4.6.tgz",
+      "integrity": "sha512-fSpWtOO/1AjSNQguk43hb/JCo16oJDnMJf3CdEGNkqsEX3t0KX96xvyX1D7PfLCpVoKu4MfVrqUkFyblYoY4lA==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "bs-logger": "^0.2.6",
         "fast-json-stable-stringify": "^2.1.0",

From 71ffb19ef9e582ab4bfd2ceee1c45ed6c743f550 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 23 Dec 2025 10:18:05 +0100
Subject: [PATCH 179/439] chore(deps): update github/codeql-action action to
 v4.31.9 (#17730)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 .github/workflows/codeql-analysis.yml | 6 +++---
 .github/workflows/scorecards.yml      | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
index 02f92b7e17..8dfa6049f2 100644
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -29,12 +29,12 @@ jobs:
           persist-credentials: false
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # v4.31.4
+        uses: github/codeql-action/init@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
         with:
           languages: ${{ matrix.language }}
 
       - name: Autobuild
-        uses: github/codeql-action/autobuild@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # v4.31.4
+        uses: github/codeql-action/autobuild@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # v4.31.4
+        uses: github/codeql-action/analyze@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index c112b591dc..64a6365e48 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -45,6 +45,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@e12f0178983d466f2f6028f5cc7a6d786fd97f4b # tag=v4.31.4
+        uses: github/codeql-action/upload-sarif@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
         with:
           sarif_file: results.sarif

From 7acab416e5a1802166c7cd91e3572dc92431f75b Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 23 Dec 2025 09:20:38 +0000
Subject: [PATCH 180/439] chore(deps): update bufbuild/buf-push-action digest
 to 1c45f6a (#17725)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 .github/workflows/buf.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/buf.yml b/.github/workflows/buf.yml
index da3cf4952a..e65c14442d 100644
--- a/.github/workflows/buf.yml
+++ b/.github/workflows/buf.yml
@@ -25,7 +25,7 @@ jobs:
         with:
           input: 'prompb'
           against: 'https://github.com/prometheus/prometheus.git#branch=main,ref=HEAD~1,subdir=prompb'
-      - uses: bufbuild/buf-push-action@a654ff18effe4641ebea4a4ce242c49800728459 # v1.1.1
+      - uses: bufbuild/buf-push-action@1c45f6a21ec277ee4c1fa2772e49b9541ea17f38 # v1.1.1
         with:
           input: 'prompb'
           buf_token: ${{ secrets.BUF_TOKEN }}

From e7467319a4c86eab51097ce2133c7bcf14e7724a Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 23 Dec 2025 09:25:08 +0000
Subject: [PATCH 181/439] chore(deps): update actions/stale action to v10.1.1
 (#17728)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 .github/workflows/stale.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml
index 86deb94097..947e670fd8 100644
--- a/.github/workflows/stale.yml
+++ b/.github/workflows/stale.yml
@@ -11,7 +11,7 @@ jobs:
     if: github.repository_owner == 'prometheus' || github.repository_owner == 'prometheus-community' # Don't run this workflow on forks.
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@5f858e3efba33a5ca4407a664cc011ad407f2008 # v10.1.0
+      - uses: actions/stale@997185467fa4f803885201cee163a9f38240193d # v10.1.1
         with:
           repo-token: ${{ secrets.GITHUB_TOKEN }}
           # opt out of defaults to avoid marking issues as stale and closing them

From 041228bfcd27c392f8c0551bf3d750c853503227 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 23 Dec 2025 10:36:59 +0100
Subject: [PATCH 182/439] fix(deps): update github.com/hashicorp/nomad/api
 digest to 1355d4c (#17727)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/go.mod b/go.mod
index 6ebb6c46fe..67761b4dc4 100644
--- a/go.mod
+++ b/go.mod
@@ -39,7 +39,7 @@ require (
 	github.com/gophercloud/gophercloud/v2 v2.9.0
 	github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853
 	github.com/hashicorp/consul/api v1.32.1
-	github.com/hashicorp/nomad/api v0.0.0-20251216171439-1dee0671280e
+	github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671
 	github.com/hetznercloud/hcloud-go/v2 v2.32.0
 	github.com/ionos-cloud/sdk-go/v6 v6.3.5
 	github.com/json-iterator/go v1.1.12
diff --git a/go.sum b/go.sum
index b28b0eb3ff..6be018d24b 100644
--- a/go.sum
+++ b/go.sum
@@ -307,8 +307,8 @@ github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/
 github.com/hashicorp/memberlist v0.5.0/go.mod h1:yvyXLpo0QaGE59Y7hDTsTzDD25JYBZ4mHgHUZ8lrOI0=
 github.com/hashicorp/memberlist v0.5.3 h1:tQ1jOCypD0WvMemw/ZhhtH+PWpzcftQvgCorLu0hndk=
 github.com/hashicorp/memberlist v0.5.3/go.mod h1:h60o12SZn/ua/j0B6iKAZezA4eDaGsIuPO70eOaJ6WE=
-github.com/hashicorp/nomad/api v0.0.0-20251216171439-1dee0671280e h1:wGl06iy/H90NSbWjfXWeRwk9SJOks0u4voIryeJFlSA=
-github.com/hashicorp/nomad/api v0.0.0-20251216171439-1dee0671280e/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE=
+github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671 h1:4NbynIRljuOUvAQNLLJA1yuWcoL5EC3Qn4c7HCngUds=
+github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE=
 github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY=
 github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4=
 github.com/hetznercloud/hcloud-go/v2 v2.32.0 h1:BRe+k7ESdYv3xQLBGdKUfk+XBFRJNGKzq70nJI24ciM=

From bf7b83059c2b262abf881493064e53b6016540f1 Mon Sep 17 00:00:00 2001
From: Bryan Boreham 
Date: Tue, 23 Dec 2025 11:56:39 +0000
Subject: [PATCH 183/439] Prepare release candidate 3.9-rc.0 (#17716)

Signed-off-by: Bryan Boreham 
---
 CHANGELOG.md                                 | 38 ++++++++++++++++++--
 VERSION                                      |  2 +-
 web/ui/mantine-ui/package.json               |  4 +--
 web/ui/module/codemirror-promql/package.json |  4 +--
 web/ui/module/lezer-promql/package.json      |  2 +-
 web/ui/package-lock.json                     | 14 ++++----
 web/ui/package.json                          |  2 +-
 7 files changed, 50 insertions(+), 16 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 976be5f52f..05c9b71b0f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,8 +1,42 @@
 # Changelog
 
-## main / unreleased
+## 3.9.0-rc.0 / 2025-12-18
 
-* [BUGFIX] TSDB: Register `prometheus_tsdb_sample_ooo_delta` metric properly. #17477
+- [CHANGE] Native Histograms are no longer experimental! Make the `native-histogram` feature flag a no-op. Use `scrape_native_histograms` config option instead. #17528
+- [CHANGE] API: Add maximum limit of 10,000 sets of statistics to TSDB status endpoint. #17647
+- [FEATURE] API: Add /api/v1/features for clients to understand which features are supported. #17427
+- [FEATURE] Promtool: Add `start_timestamp` field for unit tests. #17636
+- [FEATURE] Promtool: Add `--format seriesjson` option to `tsdb dump` to output just series labels in JSON format. #13409
+- [FEATURE] Add `--storage.tsdb.delay-compact-file.path` flag for better interoperability with Thanos. #17435
+- [FEATURE] UI: Add an option on the query drop-down menu to duplicate that query panel. #17714
+- [ENHANCEMENT]: TSDB: add flag `--storage.tsdb.block-reload-interval` to configure TSDB Block Reload Interval. #16728
+- [ENHANCEMENT] UI: Add graph option to start the chart's Y axis at zero. #17565
+- [ENHANCEMENT] Scraping: Classic protobuf format no longer requires the unit in the metric name. #16834
+- [ENHANCEMENT] PromQL, Rules, SD, Scraping: Add native histograms to complement existing summaries. #17374
+- [ENHANCEMENT] Notifications: Add a histogram `prometheus_notifications_latency_histogram_seconds` to complement the existing summary. #16637
+- [ENHANCEMENT] Remote-write: Add custom scope support for AzureAD authentication. #17483
+- [ENHANCEMENT] SD: add a `config` label with job name for most `prometheus_sd_refresh` metrics. #17138
+- [ENHANCEMENT] TSDB: New histogram `prometheus_tsdb_sample_ooo_delta`, the distribution of out-of-order samples in seconds. Collected for all samples, accepted or not. #17477
+- [ENHANCEMENT] Remote-read: Validate histograms received via remote-read. #17561
+- [PERF] TSDB: Small optimizations to postings index. #17439
+- [PERF] Scraping: Speed up relabelling of series. #17530
+- [PERF] PromQL: Small optimisations in binary operators. #17524, #17519.
+- [BUGFIX] UI: PromQL autocomplete now shows the correct type and HELP text for OpenMetrics counters whose samples end in `_total`. #17682
+- [BUGFIX] UI: Fixed codemirror-promql incorrectly showing label completion suggestions after the closing curly brace of a vector selector. #17602
+- [BUGFIX] UI: Query editor no longer suggests a duration unit if one is already present after a number. #17605
+- [BUGFIX] PromQL: Fix some "vector cannot contain metrics with the same labelset" errors when experimental delayed name removal is enabled. #17678
+- [BUGFIX] PromQL: Fix possible corruption of PromQL text if the query had an empty `ignoring()` and non-empty grouping. #17643
+- [BUGFIX] PromQL: Fix resets/changes to return empty results for anchored selectors when all samples are outside the range. #17479
+- [BUGFIX] PromQL: Check more consistently for many-to-one matching in filter binary operators. #17668
+- [BUGFIX] PromQL: Fix collision in unary negation with non-overlapping series. #17708
+- [BUGFIX] PromQL: Fix collision in label_join and label_replace with non-overlapping series. #17703
+- [BUGFIX] PromQL: Fix bug with inconsistent results for queries with OR expression when experimental delayed name removal is enabled. #17161
+- [BUGFIX] PromQL: Ensure that `rate`/`increase`/`delta` of histograms results in a gauge histogram. #17608
+- [BUGFIX] PromQL: Do not panic while iterating over invalid histograms. #17559
+- [BUGFIX] TSDB: Reject chunk files whose encoded chunk length overflows int. #17533
+- [BUGFIX] TSDB: Do not panic during resolution reduction of invalid histograms. #17561
+- [BUGFIX] Remote-write Receive: Avoid duplicate labels when experimental type-and-unit-label feature is enabled. #17546
+- [BUGFIX] OTLP Receiver: Only write metadata to disk when experimental metadata-wal-records feature is enabled. #17472
 
 ## 3.8.1 / 2025-12-16
 
diff --git a/VERSION b/VERSION
index f280719674..44fc2364a9 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-3.8.1
+3.9.0-rc.0
diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json
index baf47d6f6b..7958d5db91 100644
--- a/web/ui/mantine-ui/package.json
+++ b/web/ui/mantine-ui/package.json
@@ -1,7 +1,7 @@
 {
   "name": "@prometheus-io/mantine-ui",
   "private": true,
-  "version": "0.308.1",
+  "version": "0.309.0-rc.0",
   "type": "module",
   "scripts": {
     "start": "vite",
@@ -28,7 +28,7 @@
     "@microsoft/fetch-event-source": "^2.0.1",
     "@nexucis/fuzzy": "^0.5.1",
     "@nexucis/kvsearch": "^0.9.1",
-    "@prometheus-io/codemirror-promql": "0.308.1",
+    "@prometheus-io/codemirror-promql": "0.309.0-rc.0",
     "@reduxjs/toolkit": "^2.10.1",
     "@tabler/icons-react": "^3.35.0",
     "@tanstack/react-query": "^5.90.7",
diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json
index 5f632320bd..6ad2116497 100644
--- a/web/ui/module/codemirror-promql/package.json
+++ b/web/ui/module/codemirror-promql/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@prometheus-io/codemirror-promql",
-  "version": "0.308.1",
+  "version": "0.309.0-rc.0",
   "description": "a CodeMirror mode for the PromQL language",
   "types": "dist/esm/index.d.ts",
   "module": "dist/esm/index.js",
@@ -29,7 +29,7 @@
   },
   "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md",
   "dependencies": {
-    "@prometheus-io/lezer-promql": "0.308.1",
+    "@prometheus-io/lezer-promql": "0.309.0-rc.0",
     "lru-cache": "^11.2.2"
   },
   "devDependencies": {
diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json
index 85cc4c50ed..d83e1a6488 100644
--- a/web/ui/module/lezer-promql/package.json
+++ b/web/ui/module/lezer-promql/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@prometheus-io/lezer-promql",
-  "version": "0.308.1",
+  "version": "0.309.0-rc.0",
   "description": "lezer-based PromQL grammar",
   "main": "dist/index.cjs",
   "type": "module",
diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json
index 883ee7aaee..23ae580c20 100644
--- a/web/ui/package-lock.json
+++ b/web/ui/package-lock.json
@@ -1,12 +1,12 @@
 {
   "name": "prometheus-io",
-  "version": "0.308.1",
+  "version": "0.309.0-rc.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "prometheus-io",
-      "version": "0.308.1",
+      "version": "0.309.0-rc.0",
       "workspaces": [
         "mantine-ui",
         "module/*"
@@ -24,7 +24,7 @@
     },
     "mantine-ui": {
       "name": "@prometheus-io/mantine-ui",
-      "version": "0.308.0",
+      "version": "0.309.0-rc.0",
       "dependencies": {
         "@codemirror/autocomplete": "^6.19.1",
         "@codemirror/language": "^6.11.3",
@@ -42,7 +42,7 @@
         "@microsoft/fetch-event-source": "^2.0.1",
         "@nexucis/fuzzy": "^0.5.1",
         "@nexucis/kvsearch": "^0.9.1",
-        "@prometheus-io/codemirror-promql": "0.308.0",
+        "@prometheus-io/codemirror-promql": "0.309.0-rc.0",
         "@reduxjs/toolkit": "^2.10.1",
         "@tabler/icons-react": "^3.35.0",
         "@tanstack/react-query": "^5.90.7",
@@ -88,10 +88,10 @@
     },
     "module/codemirror-promql": {
       "name": "@prometheus-io/codemirror-promql",
-      "version": "0.308.0",
+      "version": "0.309.0-rc.0",
       "license": "Apache-2.0",
       "dependencies": {
-        "@prometheus-io/lezer-promql": "0.308.0",
+        "@prometheus-io/lezer-promql": "0.309.0-rc.0",
         "lru-cache": "^11.2.2"
       },
       "devDependencies": {
@@ -121,7 +121,7 @@
     },
     "module/lezer-promql": {
       "name": "@prometheus-io/lezer-promql",
-      "version": "0.308.0",
+      "version": "0.309.0-rc.0",
       "license": "Apache-2.0",
       "devDependencies": {
         "@lezer/generator": "^1.8.0",
diff --git a/web/ui/package.json b/web/ui/package.json
index 44d0b52ce0..dd7d25628a 100644
--- a/web/ui/package.json
+++ b/web/ui/package.json
@@ -1,7 +1,7 @@
 {
   "name": "prometheus-io",
   "description": "Monorepo for the Prometheus UI",
-  "version": "0.308.1",
+  "version": "0.309.0-rc.0",
   "private": true,
   "scripts": {
     "build": "bash build_ui.sh --all",

From 71c4e69a083d50d39d78f4695e4ed80aeae04c7e Mon Sep 17 00:00:00 2001
From: Arve Knudsen 
Date: Wed, 24 Dec 2025 13:16:18 +0100
Subject: [PATCH 184/439] fix(config): check all fields in
 GlobalConfig.isZero()

The isZero() method was missing checks for 9 fields that exist in the
GlobalConfig struct. This caused the method to incorrectly return true
when only these fields had non-zero values, resulting in user
configurations being silently overwritten with defaults during YAML
unmarshaling.

Added checks for: BodySizeLimit, SampleLimit, TargetLimit, LabelLimit,
LabelNameLengthLimit, LabelValueLengthLimit, KeepDroppedTargets,
MetricNameValidationScheme, and MetricNameEscapingScheme.

Consolidated TestEmptyGlobalBlock and new isZero tests under
TestGlobalConfig.

Signed-off-by: Arve Knudsen 
---
 config/config.go      | 11 +++++-
 config/config_test.go | 87 ++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 91 insertions(+), 7 deletions(-)

diff --git a/config/config.go b/config/config.go
index 113942b61a..51a8cefe3b 100644
--- a/config/config.go
+++ b/config/config.go
@@ -687,7 +687,16 @@ func (c *GlobalConfig) isZero() bool {
 		c.ScrapeProtocols == nil &&
 		c.ScrapeNativeHistograms == nil &&
 		!c.ConvertClassicHistogramsToNHCB &&
-		!c.AlwaysScrapeClassicHistograms
+		!c.AlwaysScrapeClassicHistograms &&
+		c.BodySizeLimit == 0 &&
+		c.SampleLimit == 0 &&
+		c.TargetLimit == 0 &&
+		c.LabelLimit == 0 &&
+		c.LabelNameLengthLimit == 0 &&
+		c.LabelValueLengthLimit == 0 &&
+		c.KeepDroppedTargets == 0 &&
+		c.MetricNameValidationScheme == model.UnsetValidation &&
+		c.MetricNameEscapingScheme == ""
 }
 
 const DefaultGoGCPercentage = 75
diff --git a/config/config_test.go b/config/config_test.go
index 28c8f2196d..1804f4925e 100644
--- a/config/config_test.go
+++ b/config/config_test.go
@@ -2663,12 +2663,87 @@ func TestAgentMode(t *testing.T) {
 	)
 }
 
-func TestEmptyGlobalBlock(t *testing.T) {
-	c, err := Load("global:\n", promslog.NewNopLogger())
-	require.NoError(t, err)
-	exp := DefaultConfig
-	exp.loaded = true
-	require.Equal(t, exp, *c)
+func TestGlobalConfig(t *testing.T) {
+	t.Run("empty block restores defaults", func(t *testing.T) {
+		c, err := Load("global:\n", promslog.NewNopLogger())
+		require.NoError(t, err)
+		exp := DefaultConfig
+		exp.loaded = true
+		require.Equal(t, exp, *c)
+	})
+
+	// Verify that isZero() correctly identifies non-zero configurations for all
+	// fields in GlobalConfig. This is important because isZero() is used during
+	// YAML unmarshaling to detect empty global blocks that should be replaced
+	// with defaults.
+	t.Run("isZero", func(t *testing.T) {
+		for _, tc := range []struct {
+			name       string
+			config     GlobalConfig
+			expectZero bool
+		}{
+			{
+				name:       "empty GlobalConfig",
+				config:     GlobalConfig{},
+				expectZero: true,
+			},
+			{
+				name:       "ScrapeInterval set",
+				config:     GlobalConfig{ScrapeInterval: model.Duration(30 * time.Second)},
+				expectZero: false,
+			},
+			{
+				name:       "BodySizeLimit set",
+				config:     GlobalConfig{BodySizeLimit: 1 * units.MiB},
+				expectZero: false,
+			},
+			{
+				name:       "SampleLimit set",
+				config:     GlobalConfig{SampleLimit: 1000},
+				expectZero: false,
+			},
+			{
+				name:       "TargetLimit set",
+				config:     GlobalConfig{TargetLimit: 500},
+				expectZero: false,
+			},
+			{
+				name:       "LabelLimit set",
+				config:     GlobalConfig{LabelLimit: 100},
+				expectZero: false,
+			},
+			{
+				name:       "LabelNameLengthLimit set",
+				config:     GlobalConfig{LabelNameLengthLimit: 50},
+				expectZero: false,
+			},
+			{
+				name:       "LabelValueLengthLimit set",
+				config:     GlobalConfig{LabelValueLengthLimit: 200},
+				expectZero: false,
+			},
+			{
+				name:       "KeepDroppedTargets set",
+				config:     GlobalConfig{KeepDroppedTargets: 10},
+				expectZero: false,
+			},
+			{
+				name:       "MetricNameValidationScheme set",
+				config:     GlobalConfig{MetricNameValidationScheme: model.LegacyValidation},
+				expectZero: false,
+			},
+			{
+				name:       "MetricNameEscapingScheme set",
+				config:     GlobalConfig{MetricNameEscapingScheme: model.EscapeUnderscores},
+				expectZero: false,
+			},
+		} {
+			t.Run(tc.name, func(t *testing.T) {
+				result := tc.config.isZero()
+				require.Equal(t, tc.expectZero, result)
+			})
+		}
+	})
 }
 
 // ScrapeConfigOptions contains options for creating a scrape config.

From 9b6e244b83571e81dab0e5a14330a4e58b156c80 Mon Sep 17 00:00:00 2001
From: matt-gp 
Date: Fri, 21 Nov 2025 22:23:08 +0000
Subject: [PATCH 185/439] AWS SD: ECS Bridge Mode

Previously the AWS SD ECS Role only discovered instances that used
`awsvpc` network mode, which attaches a dedicated Elastic Network
Interface (ENI). This change adds in additional logic so that we
discover instances that are using `host` and `bridge` networking modes,
where the IP address is that of the EC2 instance that is hosting the
container. Also this change exposes a number of additional labels that
relate to the EC2 instance when the launch type is `EC2`.

Signed-off-by: matt-gp 
---
 discovery/aws/ecs.go                | 349 +++++++++++++++++++---
 discovery/aws/ecs_test.go           | 440 +++++++++++++++++++++++++++-
 docs/configuration/configuration.md |  24 +-
 3 files changed, 766 insertions(+), 47 deletions(-)

diff --git a/discovery/aws/ecs.go b/discovery/aws/ecs.go
index d6b36a7980..1d5ff366de 100644
--- a/discovery/aws/ecs.go
+++ b/discovery/aws/ecs.go
@@ -28,6 +28,7 @@ import (
 	"github.com/aws/aws-sdk-go-v2/credentials"
 	"github.com/aws/aws-sdk-go-v2/credentials/stscreds"
 	"github.com/aws/aws-sdk-go-v2/feature/ec2/imds"
+	"github.com/aws/aws-sdk-go-v2/service/ec2"
 	"github.com/aws/aws-sdk-go-v2/service/ecs"
 	"github.com/aws/aws-sdk-go-v2/service/ecs/types"
 	"github.com/aws/aws-sdk-go-v2/service/sts"
@@ -44,31 +45,37 @@ import (
 )
 
 const (
-	ecsLabel                 = model.MetaLabelPrefix + "ecs_"
-	ecsLabelCluster          = ecsLabel + "cluster"
-	ecsLabelClusterARN       = ecsLabel + "cluster_arn"
-	ecsLabelService          = ecsLabel + "service"
-	ecsLabelServiceARN       = ecsLabel + "service_arn"
-	ecsLabelServiceStatus    = ecsLabel + "service_status"
-	ecsLabelTaskGroup        = ecsLabel + "task_group"
-	ecsLabelTaskARN          = ecsLabel + "task_arn"
-	ecsLabelTaskDefinition   = ecsLabel + "task_definition"
-	ecsLabelRegion           = ecsLabel + "region"
-	ecsLabelAvailabilityZone = ecsLabel + "availability_zone"
-	ecsLabelAZID             = ecsLabel + "availability_zone_id"
-	ecsLabelSubnetID         = ecsLabel + "subnet_id"
-	ecsLabelIPAddress        = ecsLabel + "ip_address"
-	ecsLabelLaunchType       = ecsLabel + "launch_type"
-	ecsLabelDesiredStatus    = ecsLabel + "desired_status"
-	ecsLabelLastStatus       = ecsLabel + "last_status"
-	ecsLabelHealthStatus     = ecsLabel + "health_status"
-	ecsLabelPlatformFamily   = ecsLabel + "platform_family"
-	ecsLabelPlatformVersion  = ecsLabel + "platform_version"
-	ecsLabelTag              = ecsLabel + "tag_"
-	ecsLabelTagCluster       = ecsLabelTag + "cluster_"
-	ecsLabelTagService       = ecsLabelTag + "service_"
-	ecsLabelTagTask          = ecsLabelTag + "task_"
-	ecsLabelSeparator        = ","
+	ecsLabel                     = model.MetaLabelPrefix + "ecs_"
+	ecsLabelCluster              = ecsLabel + "cluster"
+	ecsLabelClusterARN           = ecsLabel + "cluster_arn"
+	ecsLabelService              = ecsLabel + "service"
+	ecsLabelServiceARN           = ecsLabel + "service_arn"
+	ecsLabelServiceStatus        = ecsLabel + "service_status"
+	ecsLabelTaskGroup            = ecsLabel + "task_group"
+	ecsLabelTaskARN              = ecsLabel + "task_arn"
+	ecsLabelTaskDefinition       = ecsLabel + "task_definition"
+	ecsLabelRegion               = ecsLabel + "region"
+	ecsLabelAvailabilityZone     = ecsLabel + "availability_zone"
+	ecsLabelSubnetID             = ecsLabel + "subnet_id"
+	ecsLabelIPAddress            = ecsLabel + "ip_address"
+	ecsLabelLaunchType           = ecsLabel + "launch_type"
+	ecsLabelDesiredStatus        = ecsLabel + "desired_status"
+	ecsLabelLastStatus           = ecsLabel + "last_status"
+	ecsLabelHealthStatus         = ecsLabel + "health_status"
+	ecsLabelPlatformFamily       = ecsLabel + "platform_family"
+	ecsLabelPlatformVersion      = ecsLabel + "platform_version"
+	ecsLabelTag                  = ecsLabel + "tag_"
+	ecsLabelTagCluster           = ecsLabelTag + "cluster_"
+	ecsLabelTagService           = ecsLabelTag + "service_"
+	ecsLabelTagTask              = ecsLabelTag + "task_"
+	ecsLabelTagEC2               = ecsLabelTag + "ec2_"
+	ecsLabelNetworkMode          = ecsLabel + "network_mode"
+	ecsLabelContainerInstanceARN = ecsLabel + "container_instance_arn"
+	ecsLabelEC2InstanceID        = ecsLabel + "ec2_instance_id"
+	ecsLabelEC2InstanceType      = ecsLabel + "ec2_instance_type"
+	ecsLabelEC2InstancePrivateIP = ecsLabel + "ec2_instance_private_ip"
+	ecsLabelEC2InstancePublicIP  = ecsLabel + "ec2_instance_public_ip"
+	ecsLabelPublicIP             = ecsLabel + "public_ip"
 )
 
 // DefaultECSSDConfig is the default ECS SD configuration.
@@ -153,6 +160,12 @@ type ecsClient interface {
 	DescribeServices(context.Context, *ecs.DescribeServicesInput, ...func(*ecs.Options)) (*ecs.DescribeServicesOutput, error)
 	ListTasks(context.Context, *ecs.ListTasksInput, ...func(*ecs.Options)) (*ecs.ListTasksOutput, error)
 	DescribeTasks(context.Context, *ecs.DescribeTasksInput, ...func(*ecs.Options)) (*ecs.DescribeTasksOutput, error)
+	DescribeContainerInstances(context.Context, *ecs.DescribeContainerInstancesInput, ...func(*ecs.Options)) (*ecs.DescribeContainerInstancesOutput, error)
+}
+
+type ecsEC2Client interface {
+	DescribeInstances(context.Context, *ec2.DescribeInstancesInput, ...func(*ec2.Options)) (*ec2.DescribeInstancesOutput, error)
+	DescribeNetworkInterfaces(context.Context, *ec2.DescribeNetworkInterfacesInput, ...func(*ec2.Options)) (*ec2.DescribeNetworkInterfacesOutput, error)
 }
 
 // ECSDiscovery periodically performs ECS-SD requests. It implements
@@ -162,6 +175,7 @@ type ECSDiscovery struct {
 	logger *slog.Logger
 	cfg    *ECSSDConfig
 	ecs    ecsClient
+	ec2    ecsEC2Client
 }
 
 // NewECSDiscovery returns a new ECSDiscovery which periodically refreshes its targets.
@@ -191,7 +205,7 @@ func NewECSDiscovery(conf *ECSSDConfig, opts discovery.DiscovererOptions) (*ECSD
 }
 
 func (d *ECSDiscovery) initEcsClient(ctx context.Context) error {
-	if d.ecs != nil {
+	if d.ecs != nil && d.ec2 != nil {
 		return nil
 	}
 
@@ -240,6 +254,10 @@ func (d *ECSDiscovery) initEcsClient(ctx context.Context) error {
 		options.HTTPClient = client
 	})
 
+	d.ec2 = ec2.NewFromConfig(cfg, func(options *ec2.Options) {
+		options.HTTPClient = client
+	})
+
 	// Test credentials by making a simple API call
 	testCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
 	defer cancel()
@@ -458,6 +476,113 @@ func (d *ECSDiscovery) describeTasks(ctx context.Context, clusterARN string, tas
 	return tasks, errg.Wait()
 }
 
+// describeContainerInstances returns a map of container instance ARN to EC2 instance ID
+// Uses batching to respect AWS API limits (100 container instances per request).
+func (d *ECSDiscovery) describeContainerInstances(ctx context.Context, clusterARN string, containerInstanceARNs []string) (map[string]string, error) {
+	if len(containerInstanceARNs) == 0 {
+		return make(map[string]string), nil
+	}
+
+	containerInstToEC2 := make(map[string]string)
+	batchSize := 100 // AWS API limit
+
+	for _, batch := range batchSlice(containerInstanceARNs, batchSize) {
+		resp, err := d.ecs.DescribeContainerInstances(ctx, &ecs.DescribeContainerInstancesInput{
+			Cluster:            aws.String(clusterARN),
+			ContainerInstances: batch,
+		})
+		if err != nil {
+			return nil, fmt.Errorf("could not describe container instances: %w", err)
+		}
+
+		for _, ci := range resp.ContainerInstances {
+			if ci.ContainerInstanceArn != nil && ci.Ec2InstanceId != nil {
+				containerInstToEC2[*ci.ContainerInstanceArn] = *ci.Ec2InstanceId
+			}
+		}
+	}
+
+	return containerInstToEC2, nil
+}
+
+// ec2InstanceInfo holds information retrieved from EC2 DescribeInstances.
+type ec2InstanceInfo struct {
+	privateIP    string
+	publicIP     string
+	subnetID     string
+	instanceType string
+	tags         map[string]string
+}
+
+// describeEC2Instances returns a map of EC2 instance ID to instance information.
+func (d *ECSDiscovery) describeEC2Instances(ctx context.Context, instanceIDs []string) (map[string]ec2InstanceInfo, error) {
+	if len(instanceIDs) == 0 {
+		return make(map[string]ec2InstanceInfo), nil
+	}
+
+	instanceInfo := make(map[string]ec2InstanceInfo)
+
+	resp, err := d.ec2.DescribeInstances(ctx, &ec2.DescribeInstancesInput{
+		InstanceIds: instanceIDs,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("could not describe EC2 instances: %w", err)
+	}
+
+	for _, reservation := range resp.Reservations {
+		for _, instance := range reservation.Instances {
+			if instance.InstanceId != nil && instance.PrivateIpAddress != nil {
+				info := ec2InstanceInfo{
+					privateIP: *instance.PrivateIpAddress,
+					tags:      make(map[string]string),
+				}
+				if instance.PublicIpAddress != nil {
+					info.publicIP = *instance.PublicIpAddress
+				}
+				if instance.SubnetId != nil {
+					info.subnetID = *instance.SubnetId
+				}
+				if instance.InstanceType != "" {
+					info.instanceType = string(instance.InstanceType)
+				}
+				// Collect EC2 instance tags
+				for _, tag := range instance.Tags {
+					if tag.Key != nil && tag.Value != nil {
+						info.tags[*tag.Key] = *tag.Value
+					}
+				}
+				instanceInfo[*instance.InstanceId] = info
+			}
+		}
+	}
+
+	return instanceInfo, nil
+}
+
+// describeNetworkInterfaces returns a map of ENI ID to public IP address.
+func (d *ECSDiscovery) describeNetworkInterfaces(ctx context.Context, eniIDs []string) (map[string]string, error) {
+	if len(eniIDs) == 0 {
+		return make(map[string]string), nil
+	}
+
+	eniToPublicIP := make(map[string]string)
+
+	resp, err := d.ec2.DescribeNetworkInterfaces(ctx, &ec2.DescribeNetworkInterfacesInput{
+		NetworkInterfaceIds: eniIDs,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("could not describe network interfaces: %w", err)
+	}
+
+	for _, eni := range resp.NetworkInterfaces {
+		if eni.NetworkInterfaceId != nil && eni.Association != nil && eni.Association.PublicIp != nil {
+			eniToPublicIP[*eni.NetworkInterfaceId] = *eni.Association.PublicIp
+		}
+	}
+
+	return eniToPublicIP, nil
+}
+
 func batchSlice[T any](a []T, size int) [][]T {
 	batches := make([][]T, 0, len(a)/size+1)
 	for i := 0; i < len(a); i += size {
@@ -554,8 +679,76 @@ func (d *ECSDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, error
 					if tasks, exists := serviceTaskMap[serviceArn]; exists {
 						var serviceTargets []model.LabelSet
 
+						// Collect container instance ARNs for all EC2 tasks to get instance type
+						var containerInstanceARNs []string
+						taskToContainerInstance := make(map[string]string)
+						// Collect ENI IDs for awsvpc tasks to get public IPs
+						var eniIDs []string
+						taskToENI := make(map[string]string)
+
 						for _, task := range tasks {
-							// Find the ENI attachment to get the private IP address
+							// Collect container instance ARN for any task running on EC2
+							if task.ContainerInstanceArn != nil {
+								containerInstanceARNs = append(containerInstanceARNs, *task.ContainerInstanceArn)
+								taskToContainerInstance[*task.TaskArn] = *task.ContainerInstanceArn
+							}
+
+							// Collect ENI IDs from awsvpc tasks
+							for _, attachment := range task.Attachments {
+								if attachment.Type != nil && *attachment.Type == "ElasticNetworkInterface" {
+									for _, detail := range attachment.Details {
+										if detail.Name != nil && *detail.Name == "networkInterfaceId" && detail.Value != nil {
+											eniIDs = append(eniIDs, *detail.Value)
+											taskToENI[*task.TaskArn] = *detail.Value
+											break
+										}
+									}
+									break
+								}
+							}
+						}
+
+						// Batch describe container instances and EC2 instances to get instance type and other metadata
+						var containerInstToEC2 map[string]string
+						var ec2InstInfo map[string]ec2InstanceInfo
+						if len(containerInstanceARNs) > 0 {
+							var err error
+							containerInstToEC2, err = d.describeContainerInstances(ctx, clusterArn, containerInstanceARNs)
+							if err != nil {
+								d.logger.Error("Failed to describe container instances", "cluster", clusterArn, "error", err)
+								// Continue processing tasks
+							} else {
+								// Collect unique EC2 instance IDs
+								ec2InstanceIDs := make([]string, 0, len(containerInstToEC2))
+								for _, ec2ID := range containerInstToEC2 {
+									ec2InstanceIDs = append(ec2InstanceIDs, ec2ID)
+								}
+
+								// Batch describe EC2 instances
+								ec2InstInfo, err = d.describeEC2Instances(ctx, ec2InstanceIDs)
+								if err != nil {
+									d.logger.Error("Failed to describe EC2 instances", "cluster", clusterArn, "error", err)
+								}
+							}
+						}
+
+						// Batch describe ENIs to get public IPs for awsvpc tasks
+						var eniToPublicIP map[string]string
+						if len(eniIDs) > 0 {
+							var err error
+							eniToPublicIP, err = d.describeNetworkInterfaces(ctx, eniIDs)
+							if err != nil {
+								d.logger.Error("Failed to describe network interfaces", "cluster", clusterArn, "error", err)
+								// Continue processing without ENI public IPs
+							}
+						}
+
+						for _, task := range tasks {
+							var ipAddress, subnetID, publicIP string
+							var networkMode string
+							var ec2InstanceID, ec2InstanceType, ec2InstancePrivateIP, ec2InstancePublicIP string
+
+							// Try to get IP from ENI attachment (awsvpc mode)
 							var eniAttachment *types.Attachment
 							for _, attachment := range task.Attachments {
 								if attachment.Type != nil && *attachment.Type == "ElasticNetworkInterface" {
@@ -563,19 +756,65 @@ func (d *ECSDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, error
 									break
 								}
 							}
-							if eniAttachment == nil {
-								continue
-							}
 
-							var ipAddress, subnetID string
-							for _, detail := range eniAttachment.Details {
-								switch *detail.Name {
-								case "privateIPv4Address":
-									ipAddress = *detail.Value
-								case "subnetId":
-									subnetID = *detail.Value
+							if eniAttachment != nil {
+								// awsvpc networking mode - get IP from ENI
+								networkMode = "awsvpc"
+								for _, detail := range eniAttachment.Details {
+									switch *detail.Name {
+									case "privateIPv4Address":
+										ipAddress = *detail.Value
+									case "subnetId":
+										subnetID = *detail.Value
+									}
+								}
+								// Get public IP from ENI if available
+								if eniID, ok := taskToENI[*task.TaskArn]; ok {
+									if eniPublicIP, ok := eniToPublicIP[eniID]; ok {
+										publicIP = eniPublicIP
+									}
+								}
+							} else if task.ContainerInstanceArn != nil {
+								// bridge/host networking mode - need to get EC2 instance IP and subnet
+								networkMode = "bridge"
+								containerInstARN, ok := taskToContainerInstance[*task.TaskArn]
+								if ok {
+									ec2InstanceID, ok = containerInstToEC2[containerInstARN]
+									if ok {
+										info, ok := ec2InstInfo[ec2InstanceID]
+										if ok {
+											ipAddress = info.privateIP
+											publicIP = info.publicIP
+											subnetID = info.subnetID
+											ec2InstanceType = info.instanceType
+											ec2InstancePrivateIP = info.privateIP
+											ec2InstancePublicIP = info.publicIP
+										} else {
+											d.logger.Debug("EC2 instance info not found", "instance", ec2InstanceID, "task", *task.TaskArn)
+										}
+									} else {
+										d.logger.Debug("Container instance not found in map", "arn", containerInstARN, "task", *task.TaskArn)
+									}
 								}
 							}
+
+							// Get EC2 instance metadata for awsvpc tasks running on EC2
+							// We want the instance type and the host IPs for advanced use cases
+							if networkMode == "awsvpc" && task.ContainerInstanceArn != nil {
+								containerInstARN, ok := taskToContainerInstance[*task.TaskArn]
+								if ok {
+									ec2InstanceID, ok = containerInstToEC2[containerInstARN]
+									if ok {
+										info, ok := ec2InstInfo[ec2InstanceID]
+										if ok {
+											ec2InstanceType = info.instanceType
+											ec2InstancePrivateIP = info.privateIP
+											ec2InstancePublicIP = info.publicIP
+										}
+									}
+								}
+							}
+
 							if ipAddress == "" {
 								continue
 							}
@@ -589,13 +828,38 @@ func (d *ECSDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, error
 								ecsLabelTaskARN:          model.LabelValue(*task.TaskArn),
 								ecsLabelTaskDefinition:   model.LabelValue(*task.TaskDefinitionArn),
 								ecsLabelIPAddress:        model.LabelValue(ipAddress),
-								ecsLabelSubnetID:         model.LabelValue(subnetID),
 								ecsLabelRegion:           model.LabelValue(d.cfg.Region),
 								ecsLabelLaunchType:       model.LabelValue(task.LaunchType),
 								ecsLabelAvailabilityZone: model.LabelValue(*task.AvailabilityZone),
 								ecsLabelDesiredStatus:    model.LabelValue(*task.DesiredStatus),
 								ecsLabelLastStatus:       model.LabelValue(*task.LastStatus),
 								ecsLabelHealthStatus:     model.LabelValue(task.HealthStatus),
+								ecsLabelNetworkMode:      model.LabelValue(networkMode),
+							}
+
+							// Add subnet ID when available (awsvpc mode from ENI, bridge/host from EC2 instance)
+							if subnetID != "" {
+								labels[ecsLabelSubnetID] = model.LabelValue(subnetID)
+							}
+
+							// Add container instance and EC2 instance info for EC2 launch type
+							if task.ContainerInstanceArn != nil {
+								labels[ecsLabelContainerInstanceARN] = model.LabelValue(*task.ContainerInstanceArn)
+							}
+							if ec2InstanceID != "" {
+								labels[ecsLabelEC2InstanceID] = model.LabelValue(ec2InstanceID)
+							}
+							if ec2InstanceType != "" {
+								labels[ecsLabelEC2InstanceType] = model.LabelValue(ec2InstanceType)
+							}
+							if ec2InstancePrivateIP != "" {
+								labels[ecsLabelEC2InstancePrivateIP] = model.LabelValue(ec2InstancePrivateIP)
+							}
+							if ec2InstancePublicIP != "" {
+								labels[ecsLabelEC2InstancePublicIP] = model.LabelValue(ec2InstancePublicIP)
+							}
+							if publicIP != "" {
+								labels[ecsLabelPublicIP] = model.LabelValue(publicIP)
 							}
 
 							if task.PlatformFamily != nil {
@@ -634,6 +898,15 @@ func (d *ECSDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, error
 								}
 							}
 
+							// Add EC2 instance tags (if running on EC2)
+							if ec2InstanceID != "" {
+								if info, ok := ec2InstInfo[ec2InstanceID]; ok {
+									for tagKey, tagValue := range info.tags {
+										labels[model.LabelName(ecsLabelTagEC2+strutil.SanitizeLabelName(tagKey))] = model.LabelValue(tagValue)
+									}
+								}
+							}
+
 							serviceTargets = append(serviceTargets, labels)
 						}
 
diff --git a/discovery/aws/ecs_test.go b/discovery/aws/ecs_test.go
index 60138a01c7..1cb48b27fa 100644
--- a/discovery/aws/ecs_test.go
+++ b/discovery/aws/ecs_test.go
@@ -17,6 +17,8 @@ import (
 	"context"
 	"testing"
 
+	"github.com/aws/aws-sdk-go-v2/service/ec2"
+	ec2Types "github.com/aws/aws-sdk-go-v2/service/ec2/types"
 	"github.com/aws/aws-sdk-go-v2/service/ecs"
 	ecsTypes "github.com/aws/aws-sdk-go-v2/service/ecs/types"
 	"github.com/prometheus/common/model"
@@ -29,9 +31,12 @@ import (
 type ecsDataStore struct {
 	region string
 
-	clusters []ecsTypes.Cluster
-	services []ecsTypes.Service
-	tasks    []ecsTypes.Task
+	clusters           []ecsTypes.Cluster
+	services           []ecsTypes.Service
+	tasks              []ecsTypes.Task
+	containerInstances []ecsTypes.ContainerInstance
+	ec2Instances       map[string]ec2InstanceInfo // EC2 instance ID to instance info
+	eniPublicIPs       map[string]string          // ENI ID to public IP
 }
 
 func TestECSDiscoveryListClusterARNs(t *testing.T) {
@@ -716,6 +721,7 @@ func TestECSDiscoveryRefresh(t *testing.T) {
 								Details: []ecsTypes.KeyValuePair{
 									{Name: strptr("subnetId"), Value: strptr("subnet-12345")},
 									{Name: strptr("privateIPv4Address"), Value: strptr("10.0.1.100")},
+									{Name: strptr("networkInterfaceId"), Value: strptr("eni-fargate-123")},
 								},
 							},
 						},
@@ -724,6 +730,9 @@ func TestECSDiscoveryRefresh(t *testing.T) {
 						},
 					},
 				},
+				eniPublicIPs: map[string]string{
+					"eni-fargate-123": "52.1.2.3",
+				},
 			},
 			expected: []*targetgroup.Group{
 				{
@@ -749,6 +758,8 @@ func TestECSDiscoveryRefresh(t *testing.T) {
 							"__meta_ecs_health_status":           model.LabelValue("HEALTHY"),
 							"__meta_ecs_platform_family":         model.LabelValue("Linux"),
 							"__meta_ecs_platform_version":        model.LabelValue("1.4.0"),
+							"__meta_ecs_network_mode":            model.LabelValue("awsvpc"),
+							"__meta_ecs_public_ip":               model.LabelValue("52.1.2.3"),
 							"__meta_ecs_tag_cluster_Environment": model.LabelValue("test"),
 							"__meta_ecs_tag_service_App":         model.LabelValue("web"),
 							"__meta_ecs_tag_task_Version":        model.LabelValue("v1.0"),
@@ -825,14 +836,345 @@ func TestECSDiscoveryRefresh(t *testing.T) {
 				},
 			},
 		},
+		{
+			name: "TaskWithBridgeNetworking",
+			ecsData: &ecsDataStore{
+				region: "us-west-2",
+				clusters: []ecsTypes.Cluster{
+					{
+						ClusterName: strptr("test-cluster"),
+						ClusterArn:  strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"),
+						Status:      strptr("ACTIVE"),
+					},
+				},
+				services: []ecsTypes.Service{
+					{
+						ServiceName: strptr("bridge-service"),
+						ServiceArn:  strptr("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/bridge-service"),
+						ClusterArn:  strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"),
+						Status:      strptr("ACTIVE"),
+					},
+				},
+				tasks: []ecsTypes.Task{
+					{
+						TaskArn:              strptr("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-bridge"),
+						ClusterArn:           strptr("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"),
+						TaskDefinitionArn:    strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/bridge-task:1"),
+						Group:                strptr("service:bridge-service"),
+						LaunchType:           ecsTypes.LaunchTypeEc2,
+						LastStatus:           strptr("RUNNING"),
+						DesiredStatus:        strptr("RUNNING"),
+						HealthStatus:         ecsTypes.HealthStatusHealthy,
+						AvailabilityZone:     strptr("us-west-2a"),
+						ContainerInstanceArn: strptr("arn:aws:ecs:us-west-2:123456789012:container-instance/test-cluster/abc123"),
+						Attachments:          []ecsTypes.Attachment{},
+					},
+				},
+				containerInstances: []ecsTypes.ContainerInstance{
+					{
+						ContainerInstanceArn: strptr("arn:aws:ecs:us-west-2:123456789012:container-instance/test-cluster/abc123"),
+						Ec2InstanceId:        strptr("i-1234567890abcdef0"),
+						Status:               strptr("ACTIVE"),
+					},
+				},
+				ec2Instances: map[string]ec2InstanceInfo{
+					"i-1234567890abcdef0": {
+						privateIP:    "10.0.1.50",
+						publicIP:     "54.1.2.3",
+						subnetID:     "subnet-bridge-1",
+						instanceType: "t3.medium",
+						tags: map[string]string{
+							"Name":        "ecs-host-1",
+							"Environment": "production",
+						},
+					},
+				},
+			},
+			expected: []*targetgroup.Group{
+				{
+					Source: "us-west-2",
+					Targets: []model.LabelSet{
+						{
+							model.AddressLabel:                   model.LabelValue("10.0.1.50:80"),
+							"__meta_ecs_cluster":                 model.LabelValue("test-cluster"),
+							"__meta_ecs_cluster_arn":             model.LabelValue("arn:aws:ecs:us-west-2:123456789012:cluster/test-cluster"),
+							"__meta_ecs_service":                 model.LabelValue("bridge-service"),
+							"__meta_ecs_service_arn":             model.LabelValue("arn:aws:ecs:us-west-2:123456789012:service/test-cluster/bridge-service"),
+							"__meta_ecs_service_status":          model.LabelValue("ACTIVE"),
+							"__meta_ecs_task_group":              model.LabelValue("service:bridge-service"),
+							"__meta_ecs_task_arn":                model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task/test-cluster/task-bridge"),
+							"__meta_ecs_task_definition":         model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task-definition/bridge-task:1"),
+							"__meta_ecs_region":                  model.LabelValue("us-west-2"),
+							"__meta_ecs_availability_zone":       model.LabelValue("us-west-2a"),
+							"__meta_ecs_ip_address":              model.LabelValue("10.0.1.50"),
+							"__meta_ecs_subnet_id":               model.LabelValue("subnet-bridge-1"),
+							"__meta_ecs_launch_type":             model.LabelValue("EC2"),
+							"__meta_ecs_desired_status":          model.LabelValue("RUNNING"),
+							"__meta_ecs_last_status":             model.LabelValue("RUNNING"),
+							"__meta_ecs_health_status":           model.LabelValue("HEALTHY"),
+							"__meta_ecs_network_mode":            model.LabelValue("bridge"),
+							"__meta_ecs_container_instance_arn":  model.LabelValue("arn:aws:ecs:us-west-2:123456789012:container-instance/test-cluster/abc123"),
+							"__meta_ecs_ec2_instance_id":         model.LabelValue("i-1234567890abcdef0"),
+							"__meta_ecs_ec2_instance_type":       model.LabelValue("t3.medium"),
+							"__meta_ecs_ec2_instance_private_ip": model.LabelValue("10.0.1.50"),
+							"__meta_ecs_ec2_instance_public_ip":  model.LabelValue("54.1.2.3"),
+							"__meta_ecs_public_ip":               model.LabelValue("54.1.2.3"),
+							"__meta_ecs_tag_ec2_Name":            model.LabelValue("ecs-host-1"),
+							"__meta_ecs_tag_ec2_Environment":     model.LabelValue("production"),
+						},
+					},
+				},
+			},
+		},
+		{
+			name: "MixedNetworkingModes",
+			ecsData: &ecsDataStore{
+				region: "us-west-2",
+				clusters: []ecsTypes.Cluster{
+					{
+						ClusterName: strptr("mixed-cluster"),
+						ClusterArn:  strptr("arn:aws:ecs:us-west-2:123456789012:cluster/mixed-cluster"),
+						Status:      strptr("ACTIVE"),
+					},
+				},
+				services: []ecsTypes.Service{
+					{
+						ServiceName: strptr("mixed-service"),
+						ServiceArn:  strptr("arn:aws:ecs:us-west-2:123456789012:service/mixed-cluster/mixed-service"),
+						ClusterArn:  strptr("arn:aws:ecs:us-west-2:123456789012:cluster/mixed-cluster"),
+						Status:      strptr("ACTIVE"),
+					},
+				},
+				tasks: []ecsTypes.Task{
+					{
+						TaskArn:           strptr("arn:aws:ecs:us-west-2:123456789012:task/mixed-cluster/task-awsvpc"),
+						ClusterArn:        strptr("arn:aws:ecs:us-west-2:123456789012:cluster/mixed-cluster"),
+						TaskDefinitionArn: strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/awsvpc-task:1"),
+						Group:             strptr("service:mixed-service"),
+						LaunchType:        ecsTypes.LaunchTypeFargate,
+						LastStatus:        strptr("RUNNING"),
+						DesiredStatus:     strptr("RUNNING"),
+						HealthStatus:      ecsTypes.HealthStatusHealthy,
+						AvailabilityZone:  strptr("us-west-2a"),
+						Attachments: []ecsTypes.Attachment{
+							{
+								Type: strptr("ElasticNetworkInterface"),
+								Details: []ecsTypes.KeyValuePair{
+									{Name: strptr("subnetId"), Value: strptr("subnet-12345")},
+									{Name: strptr("privateIPv4Address"), Value: strptr("10.0.2.100")},
+									{Name: strptr("networkInterfaceId"), Value: strptr("eni-mixed-awsvpc")},
+								},
+							},
+						},
+					},
+					{
+						TaskArn:              strptr("arn:aws:ecs:us-west-2:123456789012:task/mixed-cluster/task-bridge"),
+						ClusterArn:           strptr("arn:aws:ecs:us-west-2:123456789012:cluster/mixed-cluster"),
+						TaskDefinitionArn:    strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/bridge-task:1"),
+						Group:                strptr("service:mixed-service"),
+						LaunchType:           ecsTypes.LaunchTypeEc2,
+						LastStatus:           strptr("RUNNING"),
+						DesiredStatus:        strptr("RUNNING"),
+						HealthStatus:         ecsTypes.HealthStatusHealthy,
+						AvailabilityZone:     strptr("us-west-2b"),
+						ContainerInstanceArn: strptr("arn:aws:ecs:us-west-2:123456789012:container-instance/mixed-cluster/xyz789"),
+						Attachments:          []ecsTypes.Attachment{},
+					},
+				},
+				containerInstances: []ecsTypes.ContainerInstance{
+					{
+						ContainerInstanceArn: strptr("arn:aws:ecs:us-west-2:123456789012:container-instance/mixed-cluster/xyz789"),
+						Ec2InstanceId:        strptr("i-0987654321fedcba0"),
+						Status:               strptr("ACTIVE"),
+					},
+				},
+				ec2Instances: map[string]ec2InstanceInfo{
+					"i-0987654321fedcba0": {
+						privateIP:    "10.0.1.75",
+						publicIP:     "54.2.3.4",
+						subnetID:     "subnet-bridge-2",
+						instanceType: "t3.large",
+						tags: map[string]string{
+							"Name": "mixed-host",
+							"Team": "platform",
+						},
+					},
+				},
+				eniPublicIPs: map[string]string{
+					"eni-mixed-awsvpc": "52.2.3.4",
+				},
+			},
+			expected: []*targetgroup.Group{
+				{
+					Source: "us-west-2",
+					Targets: []model.LabelSet{
+						{
+							model.AddressLabel:             model.LabelValue("10.0.2.100:80"),
+							"__meta_ecs_cluster":           model.LabelValue("mixed-cluster"),
+							"__meta_ecs_cluster_arn":       model.LabelValue("arn:aws:ecs:us-west-2:123456789012:cluster/mixed-cluster"),
+							"__meta_ecs_service":           model.LabelValue("mixed-service"),
+							"__meta_ecs_service_arn":       model.LabelValue("arn:aws:ecs:us-west-2:123456789012:service/mixed-cluster/mixed-service"),
+							"__meta_ecs_service_status":    model.LabelValue("ACTIVE"),
+							"__meta_ecs_task_group":        model.LabelValue("service:mixed-service"),
+							"__meta_ecs_task_arn":          model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task/mixed-cluster/task-awsvpc"),
+							"__meta_ecs_task_definition":   model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task-definition/awsvpc-task:1"),
+							"__meta_ecs_region":            model.LabelValue("us-west-2"),
+							"__meta_ecs_availability_zone": model.LabelValue("us-west-2a"),
+							"__meta_ecs_ip_address":        model.LabelValue("10.0.2.100"),
+							"__meta_ecs_subnet_id":         model.LabelValue("subnet-12345"),
+							"__meta_ecs_launch_type":       model.LabelValue("FARGATE"),
+							"__meta_ecs_desired_status":    model.LabelValue("RUNNING"),
+							"__meta_ecs_last_status":       model.LabelValue("RUNNING"),
+							"__meta_ecs_health_status":     model.LabelValue("HEALTHY"),
+							"__meta_ecs_network_mode":      model.LabelValue("awsvpc"),
+							"__meta_ecs_public_ip":         model.LabelValue("52.2.3.4"),
+						},
+						{
+							model.AddressLabel:                   model.LabelValue("10.0.1.75:80"),
+							"__meta_ecs_cluster":                 model.LabelValue("mixed-cluster"),
+							"__meta_ecs_cluster_arn":             model.LabelValue("arn:aws:ecs:us-west-2:123456789012:cluster/mixed-cluster"),
+							"__meta_ecs_service":                 model.LabelValue("mixed-service"),
+							"__meta_ecs_service_arn":             model.LabelValue("arn:aws:ecs:us-west-2:123456789012:service/mixed-cluster/mixed-service"),
+							"__meta_ecs_service_status":          model.LabelValue("ACTIVE"),
+							"__meta_ecs_task_group":              model.LabelValue("service:mixed-service"),
+							"__meta_ecs_task_arn":                model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task/mixed-cluster/task-bridge"),
+							"__meta_ecs_task_definition":         model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task-definition/bridge-task:1"),
+							"__meta_ecs_region":                  model.LabelValue("us-west-2"),
+							"__meta_ecs_availability_zone":       model.LabelValue("us-west-2b"),
+							"__meta_ecs_ip_address":              model.LabelValue("10.0.1.75"),
+							"__meta_ecs_subnet_id":               model.LabelValue("subnet-bridge-2"),
+							"__meta_ecs_launch_type":             model.LabelValue("EC2"),
+							"__meta_ecs_desired_status":          model.LabelValue("RUNNING"),
+							"__meta_ecs_last_status":             model.LabelValue("RUNNING"),
+							"__meta_ecs_health_status":           model.LabelValue("HEALTHY"),
+							"__meta_ecs_network_mode":            model.LabelValue("bridge"),
+							"__meta_ecs_container_instance_arn":  model.LabelValue("arn:aws:ecs:us-west-2:123456789012:container-instance/mixed-cluster/xyz789"),
+							"__meta_ecs_ec2_instance_id":         model.LabelValue("i-0987654321fedcba0"),
+							"__meta_ecs_ec2_instance_type":       model.LabelValue("t3.large"),
+							"__meta_ecs_ec2_instance_private_ip": model.LabelValue("10.0.1.75"),
+							"__meta_ecs_ec2_instance_public_ip":  model.LabelValue("54.2.3.4"),
+							"__meta_ecs_public_ip":               model.LabelValue("54.2.3.4"),
+							"__meta_ecs_tag_ec2_Name":            model.LabelValue("mixed-host"),
+							"__meta_ecs_tag_ec2_Team":            model.LabelValue("platform"),
+						},
+					},
+				},
+			},
+		},
+		{
+			name: "EC2WithAwsvpcNetworking",
+			ecsData: &ecsDataStore{
+				region: "us-west-2",
+				clusters: []ecsTypes.Cluster{
+					{
+						ClusterName: strptr("ec2-awsvpc-cluster"),
+						ClusterArn:  strptr("arn:aws:ecs:us-west-2:123456789012:cluster/ec2-awsvpc-cluster"),
+						Status:      strptr("ACTIVE"),
+					},
+				},
+				services: []ecsTypes.Service{
+					{
+						ServiceName: strptr("ec2-awsvpc-service"),
+						ServiceArn:  strptr("arn:aws:ecs:us-west-2:123456789012:service/ec2-awsvpc-cluster/ec2-awsvpc-service"),
+						ClusterArn:  strptr("arn:aws:ecs:us-west-2:123456789012:cluster/ec2-awsvpc-cluster"),
+						Status:      strptr("ACTIVE"),
+					},
+				},
+				tasks: []ecsTypes.Task{
+					{
+						TaskArn:              strptr("arn:aws:ecs:us-west-2:123456789012:task/ec2-awsvpc-cluster/task-ec2-awsvpc"),
+						ClusterArn:           strptr("arn:aws:ecs:us-west-2:123456789012:cluster/ec2-awsvpc-cluster"),
+						TaskDefinitionArn:    strptr("arn:aws:ecs:us-west-2:123456789012:task-definition/ec2-awsvpc-task:1"),
+						Group:                strptr("service:ec2-awsvpc-service"),
+						LaunchType:           ecsTypes.LaunchTypeEc2,
+						LastStatus:           strptr("RUNNING"),
+						DesiredStatus:        strptr("RUNNING"),
+						HealthStatus:         ecsTypes.HealthStatusHealthy,
+						AvailabilityZone:     strptr("us-west-2c"),
+						ContainerInstanceArn: strptr("arn:aws:ecs:us-west-2:123456789012:container-instance/ec2-awsvpc-cluster/def456"),
+						// Has BOTH ENI attachment AND container instance ARN - should use ENI
+						Attachments: []ecsTypes.Attachment{
+							{
+								Type: strptr("ElasticNetworkInterface"),
+								Details: []ecsTypes.KeyValuePair{
+									{Name: strptr("subnetId"), Value: strptr("subnet-99999")},
+									{Name: strptr("privateIPv4Address"), Value: strptr("10.0.3.200")},
+									{Name: strptr("networkInterfaceId"), Value: strptr("eni-ec2-awsvpc")},
+								},
+							},
+						},
+					},
+				},
+				eniPublicIPs: map[string]string{
+					"eni-ec2-awsvpc": "52.3.4.5",
+				},
+				// Container instance data - IP should NOT be used, but instance type SHOULD be used
+				containerInstances: []ecsTypes.ContainerInstance{
+					{
+						ContainerInstanceArn: strptr("arn:aws:ecs:us-west-2:123456789012:container-instance/ec2-awsvpc-cluster/def456"),
+						Ec2InstanceId:        strptr("i-ec2awsvpcinstance"),
+						Status:               strptr("ACTIVE"),
+					},
+				},
+				ec2Instances: map[string]ec2InstanceInfo{
+					"i-ec2awsvpcinstance": {
+						privateIP:    "10.0.9.99",    // This IP should NOT be used (ENI IP is used instead)
+						publicIP:     "54.3.4.5",     // This public IP SHOULD be exposed
+						subnetID:     "subnet-wrong", // This subnet should NOT be used (ENI subnet is used instead)
+						instanceType: "c5.2xlarge",   // This instance type SHOULD be used
+						tags: map[string]string{
+							"Name":  "ec2-awsvpc-host",
+							"Owner": "team-a",
+						},
+					},
+				},
+			},
+			expected: []*targetgroup.Group{
+				{
+					Source: "us-west-2",
+					Targets: []model.LabelSet{
+						{
+							model.AddressLabel:                   model.LabelValue("10.0.3.200:80"),
+							"__meta_ecs_cluster":                 model.LabelValue("ec2-awsvpc-cluster"),
+							"__meta_ecs_cluster_arn":             model.LabelValue("arn:aws:ecs:us-west-2:123456789012:cluster/ec2-awsvpc-cluster"),
+							"__meta_ecs_service":                 model.LabelValue("ec2-awsvpc-service"),
+							"__meta_ecs_service_arn":             model.LabelValue("arn:aws:ecs:us-west-2:123456789012:service/ec2-awsvpc-cluster/ec2-awsvpc-service"),
+							"__meta_ecs_service_status":          model.LabelValue("ACTIVE"),
+							"__meta_ecs_task_group":              model.LabelValue("service:ec2-awsvpc-service"),
+							"__meta_ecs_task_arn":                model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task/ec2-awsvpc-cluster/task-ec2-awsvpc"),
+							"__meta_ecs_task_definition":         model.LabelValue("arn:aws:ecs:us-west-2:123456789012:task-definition/ec2-awsvpc-task:1"),
+							"__meta_ecs_region":                  model.LabelValue("us-west-2"),
+							"__meta_ecs_availability_zone":       model.LabelValue("us-west-2c"),
+							"__meta_ecs_ip_address":              model.LabelValue("10.0.3.200"),
+							"__meta_ecs_subnet_id":               model.LabelValue("subnet-99999"),
+							"__meta_ecs_launch_type":             model.LabelValue("EC2"),
+							"__meta_ecs_desired_status":          model.LabelValue("RUNNING"),
+							"__meta_ecs_last_status":             model.LabelValue("RUNNING"),
+							"__meta_ecs_health_status":           model.LabelValue("HEALTHY"),
+							"__meta_ecs_network_mode":            model.LabelValue("awsvpc"),
+							"__meta_ecs_container_instance_arn":  model.LabelValue("arn:aws:ecs:us-west-2:123456789012:container-instance/ec2-awsvpc-cluster/def456"),
+							"__meta_ecs_ec2_instance_id":         model.LabelValue("i-ec2awsvpcinstance"),
+							"__meta_ecs_ec2_instance_type":       model.LabelValue("c5.2xlarge"),
+							"__meta_ecs_ec2_instance_private_ip": model.LabelValue("10.0.9.99"),
+							"__meta_ecs_ec2_instance_public_ip":  model.LabelValue("54.3.4.5"),
+							"__meta_ecs_public_ip":               model.LabelValue("52.3.4.5"),
+							"__meta_ecs_tag_ec2_Name":            model.LabelValue("ec2-awsvpc-host"),
+							"__meta_ecs_tag_ec2_Owner":           model.LabelValue("team-a"),
+						},
+					},
+				},
+			},
+		},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			client := newMockECSClient(tt.ecsData)
+			ecsClient := newMockECSClient(tt.ecsData)
+			ec2Client := newMockECSEC2Client(tt.ecsData.ec2Instances, tt.ecsData.eniPublicIPs)
 
 			d := &ECSDiscovery{
-				ecs: client,
+				ecs: ecsClient,
+				ec2: ec2Client,
 				cfg: &ECSSDConfig{
 					Region:             tt.ecsData.region,
 					Port:               80,
@@ -951,3 +1293,91 @@ func (m *mockECSClient) DescribeTasks(_ context.Context, input *ecs.DescribeTask
 		Tasks: tasks,
 	}, nil
 }
+
+func (m *mockECSClient) DescribeContainerInstances(_ context.Context, input *ecs.DescribeContainerInstancesInput, _ ...func(*ecs.Options)) (*ecs.DescribeContainerInstancesOutput, error) {
+	var containerInstances []ecsTypes.ContainerInstance
+	for _, ciArn := range input.ContainerInstances {
+		for _, ci := range m.ecsData.containerInstances {
+			if *ci.ContainerInstanceArn == ciArn {
+				containerInstances = append(containerInstances, ci)
+				break
+			}
+		}
+	}
+
+	return &ecs.DescribeContainerInstancesOutput{
+		ContainerInstances: containerInstances,
+	}, nil
+}
+
+// Mock EC2 client wrapper for ECS tests.
+type mockECSEC2Client struct {
+	ec2Instances map[string]ec2InstanceInfo
+	eniPublicIPs map[string]string
+}
+
+func newMockECSEC2Client(ec2Instances map[string]ec2InstanceInfo, eniPublicIPs map[string]string) *mockECSEC2Client {
+	return &mockECSEC2Client{
+		ec2Instances: ec2Instances,
+		eniPublicIPs: eniPublicIPs,
+	}
+}
+
+func (m *mockECSEC2Client) DescribeInstances(_ context.Context, input *ec2.DescribeInstancesInput, _ ...func(*ec2.Options)) (*ec2.DescribeInstancesOutput, error) {
+	var reservations []ec2Types.Reservation
+
+	for _, instanceID := range input.InstanceIds {
+		if info, ok := m.ec2Instances[instanceID]; ok {
+			instance := ec2Types.Instance{
+				InstanceId:       &instanceID,
+				PrivateIpAddress: &info.privateIP,
+			}
+			if info.publicIP != "" {
+				instance.PublicIpAddress = &info.publicIP
+			}
+			if info.subnetID != "" {
+				instance.SubnetId = &info.subnetID
+			}
+			if info.instanceType != "" {
+				instance.InstanceType = ec2Types.InstanceType(info.instanceType)
+			}
+			// Add tags
+			for tagKey, tagValue := range info.tags {
+				instance.Tags = append(instance.Tags, ec2Types.Tag{
+					Key:   &tagKey,
+					Value: &tagValue,
+				})
+			}
+			reservation := ec2Types.Reservation{
+				Instances: []ec2Types.Instance{instance},
+			}
+			reservations = append(reservations, reservation)
+		}
+	}
+
+	return &ec2.DescribeInstancesOutput{
+		Reservations: reservations,
+	}, nil
+}
+
+func (m *mockECSEC2Client) DescribeNetworkInterfaces(_ context.Context, input *ec2.DescribeNetworkInterfacesInput, _ ...func(*ec2.Options)) (*ec2.DescribeNetworkInterfacesOutput, error) {
+	var networkInterfaces []ec2Types.NetworkInterface
+
+	for _, eniID := range input.NetworkInterfaceIds {
+		if publicIP, ok := m.eniPublicIPs[eniID]; ok {
+			eni := ec2Types.NetworkInterface{
+				NetworkInterfaceId: &eniID,
+			}
+			if publicIP != "" {
+				eni.Association = &ec2Types.NetworkInterfaceAssociation{
+					PublicIp: &publicIP,
+				}
+			}
+			networkInterfaces = append(networkInterfaces, eni)
+		}
+	}
+
+	return &ec2.DescribeNetworkInterfacesOutput{
+		NetworkInterfaces: networkInterfaces,
+	}, nil
+}
diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md
index a539ee9461..3b71f26fc2 100644
--- a/docs/configuration/configuration.md
+++ b/docs/configuration/configuration.md
@@ -919,11 +919,16 @@ The following meta labels are available on targets during [relabeling](#relabel_
 
 #### `ecs`
 
-The `ecs` role discovers targets from AWS ECS containers. The private IP address is used by default, but may be changed to
-the public IP address with relabeling.
+The `ecs` role discovers targets from AWS ECS containers. 
 
-The IAM credentials used must have the following permissions to discover
-scrape targets:
+ECS service discovery supports all ECS networking modes:
+- **awsvpc mode** (Fargate and EC2 with ENI): Uses the task's private IP address from its elastic network interface
+- **bridge mode** (EC2): Uses the EC2 host instance's private IP address
+- **host mode** (EC2): Uses the EC2 host instance's private IP address
+
+The private IP address is used by default, but may be changed to the public IP address with relabeling.
+
+The IAM credentials used must have the following permissions to discover scrape targets:
 
 - `ecs:ListClusters`
 - `ecs:DescribeClusters`
@@ -931,6 +936,9 @@ scrape targets:
 - `ecs:DescribeServices`
 - `ecs:ListTasks`
 - `ecs:DescribeTasks`
+- `ecs:DescribeContainerInstances` (required for EC2 launch type tasks)
+- `ec2:DescribeInstances` (required for EC2 launch type tasks)
+- `ec2:DescribeNetworkInterfaces` (required to get public IP for awsvpc mode tasks)
 
 The following meta labels are available on targets during [relabeling](#relabel_config):
 
@@ -952,9 +960,17 @@ The following meta labels are available on targets during [relabeling](#relabel_
 * `__meta_ecs_subnet_id`: the subnet ID where the task is running
 * `__meta_ecs_availability_zone`: the availability zone where the task is running
 * `__meta_ecs_region`: the AWS region
+* `__meta_ecs_public_ip`: the public IP address (from ENI for awsvpc mode, from EC2 instance for bridge/host mode), if available
+* `__meta_ecs_network_mode`: the network mode of the task (awsvpc or bridge)
+* `__meta_ecs_container_instance_arn`: the ARN of the container instance (EC2 launch type only)
+* `__meta_ecs_ec2_instance_id`: the EC2 instance ID (EC2 launch type only)
+* `__meta_ecs_ec2_instance_type`: the EC2 instance type (EC2 launch type only)
+* `__meta_ecs_ec2_instance_private_ip`: the private IP address of the EC2 instance (EC2 launch type only)
+* `__meta_ecs_ec2_instance_public_ip`: the public IP address of the EC2 instance, if available (EC2 launch type only)
 * `__meta_ecs_tag_cluster_`: each cluster tag value, keyed by tag name
 * `__meta_ecs_tag_service_`: each service tag value, keyed by tag name
 * `__meta_ecs_tag_task_`: each task tag value, keyed by tag name
+* `__meta_ecs_tag_ec2_`: each EC2 instance tag value, keyed by tag name (EC2 launch type only)
 
 See below for the configuration options for AWS discovery:
 

From ec539d0929e1dc9e9fb632cde654cf1a56711637 Mon Sep 17 00:00:00 2001
From: Joshua Shanks 
Date: Sat, 27 Dec 2025 04:50:19 -0800
Subject: [PATCH 186/439] docs: Fix inaccuracies in root markdown files
 (#17433)

* docs: Fix inaccuracies in root markdown files

- Update npm requirement from v8 to v10 in README
- Remove obsolete GO111MODULE references
- Correct formatting issues and typos in CHANGELOG
- Fix maintainer entry formatting in MAINTAINERS
- Update outdated URLs in CONTRIBUTING

---------

Signed-off-by: Joshua Shanks 
Co-authored-by: Arve Knudsen 
---
 CHANGELOG.md    | 10 +++++-----
 CONTRIBUTING.md |  5 ++---
 MAINTAINERS.md  |  4 ++--
 README.md       | 10 +++++-----
 4 files changed, 14 insertions(+), 15 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 976be5f52f..6c9d621aee 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -240,7 +240,7 @@
 
 ## 3.2.1 / 2025-02-25
 
-* [BUGFIX] Don't send Accept` header `escape=allow-utf-8` when `metric_name_validation_scheme: legacy` is configured. #16061
+* [BUGFIX] Don't send `Accept` header `escape=allow-utf-8` when `metric_name_validation_scheme: legacy` is configured. #16061
 
 ## 3.2.0 / 2025-02-17
 
@@ -251,10 +251,10 @@
 * [ENHANCEMENT] scrape: Add metadata for automatic metrics to WAL for `metadata-wal-records` feature. #15837
 * [ENHANCEMENT] promtool: Support linting of scrape interval, through lint option `too-long-scrape-interval`. #15719
 * [ENHANCEMENT] promtool: Add --ignore-unknown-fields option. #15706
-* [ENHANCEMENT] ui: Make "hide empty rules" and hide empty rules" persistent #15807
+* [ENHANCEMENT] ui: Make "hide empty rules" and "hide empty rules" persistent #15807
 * [ENHANCEMENT] web/api: Add a limit parameter to `/query` and `/query_range`. #15552
 * [ENHANCEMENT] api: Add fields Node and ServerTime to `/status`. #15784
-* [PERF] Scraping: defer computing labels for dropped targets until they are needed by the UI.  #15261
+* [PERF] Scraping: defer computing labels for dropped targets until they are needed by the UI. #15261
 * [BUGFIX] remotewrite2: Fix invalid metadata bug for metrics without metadata. #15829
 * [BUGFIX] remotewrite2: Fix the unit field propagation. #15825
 * [BUGFIX] scrape: Fix WAL metadata for histograms and summaries. #15832
@@ -271,9 +271,9 @@
  * [ENHANCEMENT] TSDB: Improve calculation of space used by labels. #13880
  * [ENHANCEMENT] Rules: new metric rule_group_last_rule_duration_sum_seconds. #15672
  * [ENHANCEMENT] Observability: Export 'go_sync_mutex_wait_total_seconds_total' metric. #15339
- * [ENHANCEMEN] Remote-Write: optionally use a DNS resolver that picks a random IP. #15329
+ * [ENHANCEMENT] Remote-Write: optionally use a DNS resolver that picks a random IP. #15329
  * [PERF] Optimize `l=~".+"` matcher. #15474, #15684
- * [PERF] TSDB: Cache all symbols for compaction . #15455
+ * [PERF] TSDB: Cache all symbols for compaction. #15455
  * [PERF] TSDB: MemPostings: keep a map of label values slices. #15426
  * [PERF] Remote-Write: Remove interning hook. #15456
  * [PERF] Scrape: optimize string manipulation for experimental native histograms with custom buckets. #15453
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 9b1b286ccf..cfb346e4d0 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -14,7 +14,7 @@ Prometheus uses GitHub to manage reviews of pull requests.
   of inspiration. Also please see our [non-goals issue](https://github.com/prometheus/docs/issues/149) on areas that the Prometheus community doesn't plan to work on.
 
 * Relevant coding style guidelines are the [Go Code Review
-  Comments](https://code.google.com/p/go-wiki/wiki/CodeReviewComments)
+  Comments](https://go.dev/wiki/CodeReviewComments)
   and the _Formatting and style_ section of Peter Bourgon's [Go: Best
   Practices for Production
   Environments](https://peter.bourgon.org/go-in-production/#formatting-and-style).
@@ -78,8 +78,7 @@ go get example.com/some/module/pkg@vX.Y.Z
 Tidy up the `go.mod` and `go.sum` files:
 
 ```bash
-# The GO111MODULE variable can be omitted when the code isn't located in GOPATH.
-GO111MODULE=on go mod tidy
+go mod tidy
 ```
 
 You have to commit the changes to `go.mod` and `go.sum` before submitting the pull request.
diff --git a/MAINTAINERS.md b/MAINTAINERS.md
index c91b270bc6..f23c7fbd63 100644
--- a/MAINTAINERS.md
+++ b/MAINTAINERS.md
@@ -18,12 +18,12 @@ Maintainers for specific parts of the codebase:
 * `model/histogram` and other code related to native histograms: Björn Rabenstein ( / @beorn7),
 George Krajcsovits ( / @krajorama)
 * `storage`
-  * `remote`: Callum Styan ( / @cstyan), Bartłomiej Płotka ( / @bwplotka), Tom Wilkie (tom.wilkie@gmail.com / @tomwilkie), Nicolás Pazos (  / @npazosmendez), Alex Greenbank (  / @alexgreenbank)
+  * `remote`: Callum Styan ( / @cstyan), Bartłomiej Płotka ( / @bwplotka), Tom Wilkie (tom.wilkie@gmail.com / @tomwilkie), Nicolás Pazos ( / @npazosmendez), Alex Greenbank ( / @alexgreenbank)
     * `otlptranslator`: Arthur Silva Sens ( / @ArthurSens), Arve Knudsen ( / @aknuds1), Jesús Vázquez ( / @jesusvazquez)
 * `tsdb`: Ganesh Vernekar ( / @codesome), Bartłomiej Płotka ( / @bwplotka), Jesús Vázquez ( / @jesusvazquez), George Krajcsovits ( / @krajorama)
 * `web`
   * `ui`: Julius Volz ( / @juliusv)
-    * `module`: Augustin Husson ( @nexucis)
+    * `module`: Augustin Husson ( / @nexucis)
 * `Makefile` and related build configuration: Simon Pasquier ( / @simonpasquier), Ben Kochie ( / @SuperQ)
 
 For the sake of brevity, not all subtrees are explicitly listed. Due to the
diff --git a/README.md b/README.md
index 08355649f3..ae4ae50431 100644
--- a/README.md
+++ b/README.md
@@ -69,7 +69,7 @@ To build Prometheus from source code, You need:
 
 * Go: Version specified in [go.mod](./go.mod) or greater.
 * NodeJS: Version specified in [.nvmrc](./web/ui/.nvmrc) or greater.
-* npm: Version 8 or greater (check with `npm --version` and [here](https://www.npmjs.com/)).
+* npm: Version 10 or greater (check with `npm --version` and [here](https://www.npmjs.com/)).
 
 Start by cloning the repository:
 
@@ -87,10 +87,10 @@ prometheus --config.file=your_config.yml
 ```
 
 *However*, when using `go install` to build Prometheus, Prometheus will expect to be able to
-read its web assets from local filesystem directories under `web/ui/static` and
-`web/ui/templates`. In order for these assets to be found, you will have to run Prometheus
-from the root of the cloned repository. Note also that these directories do not include the
-React UI unless it has been built explicitly using `make assets` or `make build`.
+read its web assets from local filesystem directories under `web/ui/static`. In order for
+these assets to be found, you will have to run Prometheus from the root of the cloned
+repository. Note also that this directory does not include the React UI unless it has been
+built explicitly using `make assets` or `make build`.
 
 An example of the above configuration file can be found [here.](https://github.com/prometheus/prometheus/blob/main/documentation/examples/prometheus.yml)
 

From 31f046f416c8249d67c4a2e71e805714fa27a268 Mon Sep 17 00:00:00 2001
From: Ben Blackmore 
Date: Fri, 26 Dec 2025 12:03:35 +0100
Subject: [PATCH 187/439] feat: add destroy() method to PromQLExtension for
 memory leak prevention

When React components mount/unmount repeatedly, each creating a new
PromQLExtension, memory leaks occur due to LRU caches with ttlAutopurge
timers keeping references alive and in-flight HTTP requests holding
closure references. This adds destroy() methods throughout the class
hierarchy to properly release resources on unmount.

Changes:
- Add destroy() to PrometheusClient interface (optional)
- HTTPPrometheusClient: track AbortControllers and abort pending requests
- Cache: clear all LRU caches and reset cached data
- CachedPrometheusClient: delegate to cache and underlying client
- HybridComplete: delegate to prometheusClient
- CompleteStrategy: add optional destroy() method
- PromQLExtension: delegate to complete strategy

Signed-off-by: Ben Blackmore 
---
 .../src/client/prometheus.test.ts             | 97 +++++++++++++++++++
 .../src/client/prometheus.ts                  | 24 ++++-
 .../codemirror-promql/src/complete/hybrid.ts  |  4 +
 .../codemirror-promql/src/complete/index.ts   |  1 +
 .../codemirror-promql/src/promql.test.ts      | 58 +++++++++++
 web/ui/module/codemirror-promql/src/promql.ts |  4 +
 6 files changed, 187 insertions(+), 1 deletion(-)
 create mode 100644 web/ui/module/codemirror-promql/src/client/prometheus.test.ts
 create mode 100644 web/ui/module/codemirror-promql/src/promql.test.ts

diff --git a/web/ui/module/codemirror-promql/src/client/prometheus.test.ts b/web/ui/module/codemirror-promql/src/client/prometheus.test.ts
new file mode 100644
index 0000000000..c872edbb69
--- /dev/null
+++ b/web/ui/module/codemirror-promql/src/client/prometheus.test.ts
@@ -0,0 +1,97 @@
+// Copyright 2025 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import { HTTPPrometheusClient, CachedPrometheusClient } from './prometheus';
+
+describe('HTTPPrometheusClient destroy', () => {
+  it('should be safe to call destroy multiple times', () => {
+    const client = new HTTPPrometheusClient({ url: 'http://localhost:8080' });
+    // First call
+    client.destroy();
+    // Second call should not throw
+    expect(() => client.destroy()).not.toThrow();
+  });
+
+  it('should abort in-flight requests when destroy is called', async () => {
+    let abortSignal: AbortSignal | null | undefined;
+
+    const mockFetch = (_url: RequestInfo, init?: RequestInit): Promise => {
+      abortSignal = init?.signal;
+      // Return a promise that never resolves to simulate an in-flight request
+      return new Promise(() => {});
+    };
+
+    const client = new HTTPPrometheusClient({
+      url: 'http://localhost:8080',
+      fetchFn: mockFetch,
+    });
+
+    // Start a request (don't await it)
+    client.labelNames();
+
+    // Verify the signal was captured and not aborted yet
+    expect(abortSignal).toBeDefined();
+    expect(abortSignal?.aborted).toBe(false);
+
+    // Destroy the client
+    client.destroy();
+
+    // Verify the request was aborted
+    expect(abortSignal?.aborted).toBe(true);
+  });
+});
+
+describe('CachedPrometheusClient destroy', () => {
+  it('should be safe to call destroy multiple times', () => {
+    const httpClient = new HTTPPrometheusClient({ url: 'http://localhost:8080' });
+    const cachedClient = new CachedPrometheusClient(httpClient);
+
+    // First call
+    cachedClient.destroy();
+    // Second call should not throw
+    expect(() => cachedClient.destroy()).not.toThrow();
+  });
+
+  it('should call destroy on the underlying HTTPPrometheusClient', () => {
+    const httpClient = new HTTPPrometheusClient({ url: 'http://localhost:8080' });
+
+    let destroyCalled = false;
+    const originalDestroy = httpClient.destroy.bind(httpClient);
+    httpClient.destroy = () => {
+      destroyCalled = true;
+      originalDestroy();
+    };
+
+    const cachedClient = new CachedPrometheusClient(httpClient);
+    cachedClient.destroy();
+
+    expect(destroyCalled).toBe(true);
+  });
+
+  it('should handle underlying clients without destroy method', () => {
+    // Create a minimal PrometheusClient without destroy
+    const minimalClient = {
+      labelNames: () => Promise.resolve([]),
+      labelValues: () => Promise.resolve([]),
+      metricMetadata: () => Promise.resolve({}),
+      series: () => Promise.resolve([]),
+      metricNames: () => Promise.resolve([]),
+      flags: () => Promise.resolve({}),
+    };
+
+    const cachedClient = new CachedPrometheusClient(minimalClient);
+
+    // Should not throw even though underlying client has no destroy
+    expect(() => cachedClient.destroy()).not.toThrow();
+  });
+});
diff --git a/web/ui/module/codemirror-promql/src/client/prometheus.ts b/web/ui/module/codemirror-promql/src/client/prometheus.ts
index 165549ac82..91de148f3c 100644
--- a/web/ui/module/codemirror-promql/src/client/prometheus.ts
+++ b/web/ui/module/codemirror-promql/src/client/prometheus.ts
@@ -39,6 +39,9 @@ export interface PrometheusClient {
 
   // flags returns flag values that prometheus was configured with.
   flags(): Promise>;
+
+  // destroy is called to release all resources held by this client
+  destroy?(): void;
 }
 
 export interface CacheConfig {
@@ -88,6 +91,7 @@ export class HTTPPrometheusClient implements PrometheusClient {
   // when calling it, thus the indirection via another function wrapper.
   private readonly fetchFn: FetchFn = (input: RequestInfo, init?: RequestInit): Promise => fetch(input, init);
   private requestHeaders: Headers = new Headers();
+  private readonly abortControllers: Set = new Set();
 
   constructor(config: PrometheusConfig) {
     this.url = config.url ? config.url : '';
@@ -199,11 +203,22 @@ export class HTTPPrometheusClient implements PrometheusClient {
     });
   }
 
+  destroy(): void {
+    for (const controller of this.abortControllers) {
+      controller.abort();
+    }
+    this.abortControllers.clear();
+  }
+
   private fetchAPI(resource: string, init?: RequestInit): Promise {
+    const controller = new AbortController();
+    this.abortControllers.add(controller);
+
     if (init) {
       init.headers = this.requestHeaders;
+      init.signal = controller.signal;
     } else {
-      init = { headers: this.requestHeaders };
+      init = { headers: this.requestHeaders, signal: controller.signal };
     }
     return this.fetchFn(this.url + resource, init)
       .then((res) => {
@@ -221,6 +236,9 @@ export class HTTPPrometheusClient implements PrometheusClient {
           throw new Error('missing "data" field in response JSON');
         }
         return apiRes.data;
+      })
+      .finally(() => {
+        this.abortControllers.delete(controller);
       });
   }
 
@@ -448,4 +466,8 @@ export class CachedPrometheusClient implements PrometheusClient {
       return flags;
     });
   }
+
+  destroy(): void {
+    this.client.destroy?.();
+  }
 }
diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
index bb5f4d9d36..fc79b6fcd6 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
@@ -575,6 +575,10 @@ export class HybridComplete implements CompleteStrategy {
     return this.prometheusClient;
   }
 
+  destroy(): void {
+    this.prometheusClient?.destroy?.();
+  }
+
   promQL(context: CompletionContext): Promise | CompletionResult | null {
     const { state, pos } = context;
     const tree = syntaxTree(state).resolve(pos, -1);
diff --git a/web/ui/module/codemirror-promql/src/complete/index.ts b/web/ui/module/codemirror-promql/src/complete/index.ts
index b3902c3b6b..dd73857639 100644
--- a/web/ui/module/codemirror-promql/src/complete/index.ts
+++ b/web/ui/module/codemirror-promql/src/complete/index.ts
@@ -19,6 +19,7 @@ import { CompletionContext, CompletionResult } from '@codemirror/autocomplete';
 // Every different completion mode must implement this interface.
 export interface CompleteStrategy {
   promQL(context: CompletionContext): Promise | CompletionResult | null;
+  destroy?(): void;
 }
 
 // CompleteConfiguration should be used to customize the autocompletion.
diff --git a/web/ui/module/codemirror-promql/src/promql.test.ts b/web/ui/module/codemirror-promql/src/promql.test.ts
new file mode 100644
index 0000000000..787747cc5e
--- /dev/null
+++ b/web/ui/module/codemirror-promql/src/promql.test.ts
@@ -0,0 +1,58 @@
+// Copyright 2025 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import { PromQLExtension } from './promql';
+import { CompleteStrategy } from './complete';
+import { CompletionResult } from '@codemirror/autocomplete';
+
+describe('PromQLExtension destroy', () => {
+  it('should be safe to call destroy multiple times', () => {
+    const extension = new PromQLExtension();
+    // First call
+    extension.destroy();
+    // Second call should not throw
+    expect(() => extension.destroy()).not.toThrow();
+  });
+
+  it('should call destroy on the complete strategy if available', () => {
+    const extension = new PromQLExtension();
+
+    // Set up a mock complete strategy with destroy
+    let destroyCalled = false;
+    const mockCompleteStrategy: CompleteStrategy = {
+      promQL: (): CompletionResult | null => null,
+      destroy: () => {
+        destroyCalled = true;
+      },
+    };
+
+    extension.setComplete({ completeStrategy: mockCompleteStrategy });
+    extension.destroy();
+
+    expect(destroyCalled).toBe(true);
+  });
+
+  it('should handle complete strategies without destroy method', () => {
+    const extension = new PromQLExtension();
+
+    // Set up a mock complete strategy without destroy
+    const mockCompleteStrategy: CompleteStrategy = {
+      promQL: (): CompletionResult | null => null,
+    };
+
+    extension.setComplete({ completeStrategy: mockCompleteStrategy });
+
+    // Should not throw even though complete strategy has no destroy
+    expect(() => extension.destroy()).not.toThrow();
+  });
+});
diff --git a/web/ui/module/codemirror-promql/src/promql.ts b/web/ui/module/codemirror-promql/src/promql.ts
index 506cd1348b..859442559f 100644
--- a/web/ui/module/codemirror-promql/src/promql.ts
+++ b/web/ui/module/codemirror-promql/src/promql.ts
@@ -79,6 +79,10 @@ export class PromQLExtension {
     return this;
   }
 
+  destroy(): void {
+    this.complete.destroy?.();
+  }
+
   asExtension(languageType = LanguageType.PromQL): Extension {
     const language = promQLLanguage(languageType);
     let extension: Extension = [language];

From 6b5bc170429cabbef43a467303eb024e514246e9 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Wed, 31 Dec 2025 12:00:37 +0100
Subject: [PATCH 188/439] fix(deps): update module
 github.com/grpc-ecosystem/grpc-gateway/v2 to v2.27.4 (#17746)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 internal/tools/go.mod | 14 +++++++-------
 internal/tools/go.sum | 28 ++++++++++++++--------------
 2 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/internal/tools/go.mod b/internal/tools/go.mod
index b31cd5bc3e..84b540df4b 100644
--- a/internal/tools/go.mod
+++ b/internal/tools/go.mod
@@ -6,7 +6,7 @@ require (
 	github.com/bufbuild/buf v1.61.0
 	github.com/daixiang0/gci v0.13.7
 	github.com/gogo/protobuf v1.3.2
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4
 )
 
 require (
@@ -98,14 +98,14 @@ require (
 	golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b // indirect
 	golang.org/x/mod v0.30.0 // indirect
 	golang.org/x/net v0.47.0 // indirect
-	golang.org/x/sync v0.18.0 // indirect
+	golang.org/x/sync v0.19.0 // indirect
 	golang.org/x/sys v0.38.0 // indirect
 	golang.org/x/term v0.37.0 // indirect
-	golang.org/x/text v0.31.0 // indirect
-	google.golang.org/genproto/googleapis/api v0.0.0-20251007200510-49b9836ed3ff // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff // indirect
-	google.golang.org/grpc v1.75.1 // indirect
-	google.golang.org/protobuf v1.36.10 // indirect
+	golang.org/x/text v0.32.0 // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20251222181119-0a764e51fe1b // indirect
+	google.golang.org/grpc v1.78.0 // indirect
+	google.golang.org/protobuf v1.36.11 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	pluginrpc.com/pluginrpc v0.5.0 // indirect
 )
diff --git a/internal/tools/go.sum b/internal/tools/go.sum
index 7f1161148b..7c06edc7ed 100644
--- a/internal/tools/go.sum
+++ b/internal/tools/go.sum
@@ -106,8 +106,8 @@ github.com/google/go-containerregistry v0.20.6 h1:cvWX87UxxLgaH76b4hIvya6Dzz9qHB
 github.com/google/go-containerregistry v0.20.6/go.mod h1:T0x8MuoAoKX/873bkeSfLD2FAkwCDf9/HZgsFJ02E2Y=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 h1:NmZ1PKzSTQbuGHw9DGPFomqkkLWMC+vZCkfs+FHv1Vg=
-github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3/go.mod h1:zQrxl1YP88HQlA6i9c63DSVPFklWpGX4OWAc9bFuaH4=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4 h1:kEISI/Gx67NzH3nJxAmY/dGac80kKZgZt134u7Y/k1s=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4/go.mod h1:6Nz966r3vQYCqIzWsuEl9d7cf7mRhtDmm++sOxlnfxI=
 github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
 github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
@@ -256,8 +256,8 @@ golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
-golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
+golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
+golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -270,8 +270,8 @@ golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=
 golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
-golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
+golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
+golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
 golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
 golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -284,14 +284,14 @@ golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8T
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-google.golang.org/genproto/googleapis/api v0.0.0-20251007200510-49b9836ed3ff h1:8Zg5TdmcbU8A7CXGjGXF1Slqu/nIFCRaR3S5gT2plIA=
-google.golang.org/genproto/googleapis/api v0.0.0-20251007200510-49b9836ed3ff/go.mod h1:dbWfpVPvW/RqafStmRWBUpMN14puDezDMHxNYiRfQu0=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff h1:A90eA31Wq6HOMIQlLfzFwzqGKBTuaVztYu/g8sn+8Zc=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
-google.golang.org/grpc v1.75.1 h1:/ODCNEuf9VghjgO3rqLcfg8fiOP0nSluljWFlDxELLI=
-google.golang.org/grpc v1.75.1/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ=
-google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
-google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
+google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b h1:uA40e2M6fYRBf0+8uN5mLlqUtV192iiksiICIBkYJ1E=
+google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b/go.mod h1:Xa7le7qx2vmqB/SzWUBa7KdMjpdpAHlh5QCSnjessQk=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20251222181119-0a764e51fe1b h1:Mv8VFug0MP9e5vUxfBcE3vUkV6CImK3cMNMIDFjmzxU=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20251222181119-0a764e51fe1b/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=
+google.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc=
+google.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U=
+google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
+google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=

From 6a5a25953242161a136c71b7f70d0da4dc5ade02 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Wed, 31 Dec 2025 12:06:53 +0100
Subject: [PATCH 189/439] fix(deps): update module
 github.com/ionos-cloud/sdk-go/v6 to v6.3.6 (#17747)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/go.mod b/go.mod
index 67761b4dc4..6a50512c6f 100644
--- a/go.mod
+++ b/go.mod
@@ -41,7 +41,7 @@ require (
 	github.com/hashicorp/consul/api v1.32.1
 	github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671
 	github.com/hetznercloud/hcloud-go/v2 v2.32.0
-	github.com/ionos-cloud/sdk-go/v6 v6.3.5
+	github.com/ionos-cloud/sdk-go/v6 v6.3.6
 	github.com/json-iterator/go v1.1.12
 	github.com/klauspost/compress v1.18.2
 	github.com/kolo/xmlrpc v0.0.0-20220921171641-a4b6fa1dd06b
diff --git a/go.sum b/go.sum
index 6be018d24b..aaa3958613 100644
--- a/go.sum
+++ b/go.sum
@@ -313,8 +313,8 @@ github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY
 github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4=
 github.com/hetznercloud/hcloud-go/v2 v2.32.0 h1:BRe+k7ESdYv3xQLBGdKUfk+XBFRJNGKzq70nJI24ciM=
 github.com/hetznercloud/hcloud-go/v2 v2.32.0/go.mod h1:hAanyyfn9M0cMmZ68CXzPCF54KRb9EXd8eiE2FHKGIE=
-github.com/ionos-cloud/sdk-go/v6 v6.3.5 h1:6fHArdV1lf50iRhCkCP7wkvGwWzVwi+l9w1t5mwkOa8=
-github.com/ionos-cloud/sdk-go/v6 v6.3.5/go.mod h1:nUGHP4kZHAZngCVr4v6C8nuargFrtvt7GrzH/hqn7c4=
+github.com/ionos-cloud/sdk-go/v6 v6.3.6 h1:l/TtKgdQ1wUH3DDe2SfFD78AW+TJWdEbDpQhHkWd6CM=
+github.com/ionos-cloud/sdk-go/v6 v6.3.6/go.mod h1:nUGHP4kZHAZngCVr4v6C8nuargFrtvt7GrzH/hqn7c4=
 github.com/jarcoal/httpmock v1.4.1 h1:0Ju+VCFuARfFlhVXFc2HxlcQkfB+Xq12/EotHko+x2A=
 github.com/jarcoal/httpmock v1.4.1/go.mod h1:ftW1xULwo+j0R0JJkJIIi7UKigZUXCLLanykgjwBXL0=
 github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA=

From c7285093f818cb4a45eb82b357ec953a36f789f0 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Wed, 31 Dec 2025 12:08:46 +0100
Subject: [PATCH 190/439] chore(deps): update actions/setup-node action to
 v6.1.0 (#17749)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 .github/workflows/ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 48611c1973..18b05660df 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -308,7 +308,7 @@ jobs:
           persist-credentials: false
       - uses: prometheus/promci@c0916f0a41f13444612a8f0f5e700ea34edd7c19 # v0.5.3
       - name: Install nodejs
-        uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
+        uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
         with:
           node-version-file: "web/ui/.nvmrc"
           registry-url: "https://registry.npmjs.org"

From d2ad86fb738c99435370ae88b4d86b83daa17cdd Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Wed, 31 Dec 2025 12:12:22 +0100
Subject: [PATCH 191/439] fix(deps): update module
 github.com/scaleway/scaleway-sdk-go to v1.0.0-beta.36 (#17748)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 go.mod |  2 +-
 go.sum | 10 ++++++----
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/go.mod b/go.mod
index 6a50512c6f..734133b1df 100644
--- a/go.mod
+++ b/go.mod
@@ -62,7 +62,7 @@ require (
 	github.com/prometheus/common/assets v0.2.0
 	github.com/prometheus/exporter-toolkit v0.15.0
 	github.com/prometheus/sigv4 v0.3.0
-	github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35
+	github.com/scaleway/scaleway-sdk-go v1.0.0-beta.36
 	github.com/shurcooL/httpfs v0.0.0-20230704072500-f1e31cf0ba5c
 	github.com/stackitcloud/stackit-sdk-go/core v0.20.1
 	github.com/stretchr/testify v1.11.1
diff --git a/go.sum b/go.sum
index aaa3958613..78d759a2b7 100644
--- a/go.sum
+++ b/go.sum
@@ -116,8 +116,6 @@ github.com/digitalocean/godo v1.171.0 h1:QwpkwWKr3v7yxc8D4NQG973NoR9APCEWjYnLOQe
 github.com/digitalocean/godo v1.171.0/go.mod h1:xQsWpVCCbkDrWisHA72hPzPlnC+4W5w/McZY5ij9uvU=
 github.com/distribution/reference v0.5.0 h1:/FUIFXtfc/x2gpa5/VGfiGLuOIdYa1t65IKK2OFGvA0=
 github.com/distribution/reference v0.5.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
-github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
-github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
 github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM=
 github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
 github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
@@ -493,8 +491,8 @@ github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPK
 github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
 github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
 github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
-github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35 h1:8xfn1RzeI9yoCUuEwDy08F+No6PcKZGEDOQ6hrRyLts=
-github.com/scaleway/scaleway-sdk-go v1.0.0-beta.35/go.mod h1:47B1d/YXmSAxlJxUJxClzHR6b3T4M1WyCvwENPQNBWc=
+github.com/scaleway/scaleway-sdk-go v1.0.0-beta.36 h1:ObX9hZmK+VmijreZO/8x9pQ8/P/ToHD/bdSb4Eg4tUo=
+github.com/scaleway/scaleway-sdk-go v1.0.0-beta.36/go.mod h1:LEsDu4BubxK7/cWhtlQWfuxwL4rf/2UEpxXz1o1EMtM=
 github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 h1:nn5Wsu0esKSJiIVhscUtVbo7ada43DJhG55ua/hjS5I=
 github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
 github.com/shoenig/test v1.12.2 h1:ZVT8NeIUwGWpZcKaepPmFMoNQ3sVpxvqUh/MAqwFiJI=
@@ -617,6 +615,8 @@ go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
 go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
 go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
+go.yaml.in/yaml/v4 v4.0.0-rc.3 h1:3h1fjsh1CTAPjW7q/EMe+C8shx5d8ctzZTrLcs/j8Go=
+go.yaml.in/yaml/v4 v4.0.0-rc.3/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
 golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY=
@@ -723,6 +723,8 @@ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8
 gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
+gopkg.in/dnaeon/go-vcr.v4 v4.0.6 h1:PiJkrakkmzc5s7EfBnZOnyiLwi7o7A9fwPzN0X2uwe0=
+gopkg.in/dnaeon/go-vcr.v4 v4.0.6/go.mod h1:sbq5oMEcM4PXngbcNbHhzfCP9OdZodLhrbRYoyg09HY=
 gopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4=
 gopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
 gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=

From 74ee4f99607ae30c215e6a5478daf20bbe4bac24 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Wed, 31 Dec 2025 11:22:01 +0000
Subject: [PATCH 192/439] fix(deps): update
 google.golang.org/genproto/googleapis/api digest to 0a764e5 (#17745)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 go.mod | 4 ++--
 go.sum | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/go.mod b/go.mod
index 734133b1df..53df4909fc 100644
--- a/go.mod
+++ b/go.mod
@@ -91,7 +91,7 @@ require (
 	golang.org/x/sys v0.39.0
 	golang.org/x/text v0.32.0
 	google.golang.org/api v0.257.0
-	google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2
+	google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b
 	google.golang.org/grpc v1.77.0
 	google.golang.org/protobuf v1.36.11
 	gopkg.in/yaml.v3 v3.0.1
@@ -234,7 +234,7 @@ require (
 	golang.org/x/term v0.38.0 // indirect
 	golang.org/x/time v0.14.0 // indirect
 	golang.org/x/tools v0.39.0 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2 // indirect
 	gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
diff --git a/go.sum b/go.sum
index 78d759a2b7..b61713fb9a 100644
--- a/go.sum
+++ b/go.sum
@@ -710,10 +710,10 @@ google.golang.org/api v0.257.0 h1:8Y0lzvHlZps53PEaw+G29SsQIkuKrumGWs9puiexNAA=
 google.golang.org/api v0.257.0/go.mod h1:4eJrr+vbVaZSqs7vovFd1Jb/A6ml6iw2e6FBYf3GAO4=
 google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4=
 google.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s=
-google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2 h1:7LRqPCEdE4TP4/9psdaB7F2nhZFfBiGJomA5sojLWdU=
-google.golang.org/genproto/googleapis/api v0.0.0-20251213004720-97cd9d5aeac2/go.mod h1:+rXWjjaukWZun3mLfjmVnQi18E1AsFbDN9QdJ5YXLto=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
+google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b h1:uA40e2M6fYRBf0+8uN5mLlqUtV192iiksiICIBkYJ1E=
+google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b/go.mod h1:Xa7le7qx2vmqB/SzWUBa7KdMjpdpAHlh5QCSnjessQk=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2 h1:2I6GHUeJ/4shcDpoUlLs/2WPnhg7yJwvXtqcMJt9liA=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
 google.golang.org/grpc v1.77.0 h1:wVVY6/8cGA6vvffn+wWK5ToddbgdU3d8MNENr4evgXM=
 google.golang.org/grpc v1.77.0/go.mod h1:z0BY1iVj0q8E1uSQCjL9cppRj+gnZjzDnzV0dHhrNig=
 google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=

From 8c38d1914fe291b264a9b13f26be94297484233e Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Fri, 2 Jan 2026 09:16:21 +0100
Subject: [PATCH 193/439] chore(deps): bump actions/upload-artifact from 5.0.0
 to 6.0.0 (#17758)

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5.0.0 to 6.0.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/330a01c490aca151604b8cf639adc76d48f6c5d4...b7c566a772e6b6bfb58ed0dc250532a479d7789f)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/fuzzing.yml    | 2 +-
 .github/workflows/scorecards.yml | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/fuzzing.yml b/.github/workflows/fuzzing.yml
index 60f643b4f0..f9f7abafd6 100644
--- a/.github/workflows/fuzzing.yml
+++ b/.github/workflows/fuzzing.yml
@@ -23,7 +23,7 @@ jobs:
           fuzz-seconds: 600
           dry-run: false
       - name: Upload Crash
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
         if: failure() && steps.build.outcome == 'success'
         with:
           name: artifacts
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index 64a6365e48..81dcbf5c2a 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -37,7 +37,7 @@ jobs:
       # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
       # format to the repository Actions tab.
       - name: "Upload artifact"
-        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # tag=v5.0.0
+        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # tag=v6.0.0
         with:
           name: SARIF file
           path: results.sarif

From e7f29066a00f04a15385388460c8038a75247a6e Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Fri, 2 Jan 2026 09:17:39 +0100
Subject: [PATCH 194/439] chore(deps): bump dessant/lock-threads from 5.0.1 to
 6.0.0 (#17755)

Bumps [dessant/lock-threads](https://github.com/dessant/lock-threads) from 5.0.1 to 6.0.0.
- [Release notes](https://github.com/dessant/lock-threads/releases)
- [Changelog](https://github.com/dessant/lock-threads/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dessant/lock-threads/compare/1bf7ec25051fe7c00bdd17e6a7cf3d7bfb7dc771...7266a7ce5c1df01b1c6db85bf8cd86c737dadbe7)

---
updated-dependencies:
- dependency-name: dessant/lock-threads
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/lock.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/lock.yml b/.github/workflows/lock.yml
index e7e813e3b6..8f34aad204 100644
--- a/.github/workflows/lock.yml
+++ b/.github/workflows/lock.yml
@@ -16,7 +16,7 @@ jobs:
     runs-on: ubuntu-latest
     if: github.repository_owner == 'prometheus'
     steps:
-      - uses: dessant/lock-threads@1bf7ec25051fe7c00bdd17e6a7cf3d7bfb7dc771 # v5.0.1
+      - uses: dessant/lock-threads@7266a7ce5c1df01b1c6db85bf8cd86c737dadbe7 # v6.0.0
         with:
           process-only: 'issues'
           issue-inactive-days: '180'

From 5bd6809161e133fa700de10b19c3d59af3648115 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Fri, 2 Jan 2026 09:40:02 +0100
Subject: [PATCH 195/439] chore(deps): bump actions/cache from 4.3.0 to 5.0.1
 (#17754)

Bumps [actions/cache](https://github.com/actions/cache) from 4.3.0 to 5.0.1.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/0057852bfaa89a56745cba8c7296529d2fc39830...9255dc7a253b0ccc959486e2bca901246202afeb)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: 5.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 18b05660df..50a5b2a43c 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -312,7 +312,7 @@ jobs:
         with:
           node-version-file: "web/ui/.nvmrc"
           registry-url: "https://registry.npmjs.org"
-      - uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+      - uses: actions/cache@9255dc7a253b0ccc959486e2bca901246202afeb # v5.0.1
         with:
           path: ~/.npm
           key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}

From a2ad371b03a9540cfeb722fed303d2197fc1a181 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Fri, 2 Jan 2026 10:15:43 +0100
Subject: [PATCH 196/439] chore(deps): bump bufbuild/buf-push-action from 1.1.1
 to 1.2.0 (#17756)

Bumps [bufbuild/buf-push-action](https://github.com/bufbuild/buf-push-action) from 1.1.1 to 1.2.0.
- [Release notes](https://github.com/bufbuild/buf-push-action/releases)
- [Commits](https://github.com/bufbuild/buf-push-action/compare/1c45f6a21ec277ee4c1fa2772e49b9541ea17f38...a654ff18effe4641ebea4a4ce242c49800728459)

---
updated-dependencies:
- dependency-name: bufbuild/buf-push-action
  dependency-version: 1.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/buf.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/buf.yml b/.github/workflows/buf.yml
index e65c14442d..5de3c133b9 100644
--- a/.github/workflows/buf.yml
+++ b/.github/workflows/buf.yml
@@ -25,7 +25,7 @@ jobs:
         with:
           input: 'prompb'
           against: 'https://github.com/prometheus/prometheus.git#branch=main,ref=HEAD~1,subdir=prompb'
-      - uses: bufbuild/buf-push-action@1c45f6a21ec277ee4c1fa2772e49b9541ea17f38 # v1.1.1
+      - uses: bufbuild/buf-push-action@a654ff18effe4641ebea4a4ce242c49800728459 # v1.2.0
         with:
           input: 'prompb'
           buf_token: ${{ secrets.BUF_TOKEN }}

From bbd1e6378359eb118a3e0bc26259ae4bba26bc95 Mon Sep 17 00:00:00 2001
From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Date: Fri, 2 Jan 2026 17:29:17 +0100
Subject: [PATCH 197/439] docs: Update API documentation for missing features

- Add stats parameter documentation for query endpoints.
- Add documentation for new /api/v1/scrape_pools endpoint (added in v2.42).

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---
 docs/querying/api.md | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/docs/querying/api.md b/docs/querying/api.md
index 4cd5e175fd..4891db8980 100644
--- a/docs/querying/api.md
+++ b/docs/querying/api.md
@@ -84,8 +84,9 @@ URL query parameters:
 - `time=`: Evaluation timestamp. Optional.
 - `timeout=`: Evaluation timeout. Optional. Defaults to and
    is capped by the value of the `-query.timeout` flag.
-- `limit=`: Maximum number of returned series. Doesn’t affect scalars or strings but truncates the number of series for matrices and vectors. Optional. 0 means disabled.
+- `limit=`: Maximum number of returned series. Doesn't affect scalars or strings but truncates the number of series for matrices and vectors. Optional. 0 means disabled.
 - `lookback_delta=`: Override the the [lookback period](#staleness) just for this query. Optional.
+- `stats=`: Include query statistics in the response. If set to `all`, includes detailed statistics. Optional.
 
 The current server time is used if the `time` parameter is omitted.
 
@@ -159,6 +160,7 @@ URL query parameters:
    is capped by the value of the `-query.timeout` flag.
 - `limit=`: Maximum number of returned series. Optional. 0 means disabled.
 - `lookback_delta=`: Override the the [lookback period](#staleness) just for this query. Optional.
+- `stats=`: Include query statistics in the response. If set to `all`, includes detailed statistics. Optional.
 
 You can URL-encode these parameters directly in the request body by using the `POST` method and
 `Content-Type: application/x-www-form-urlencoded` header. This is useful when specifying a large
@@ -670,6 +672,35 @@ Note that with the currently implemented bucket schemas, positive buckets are
 “open left”, negative buckets are “open right”, and the zero bucket (with a
 negative left boundary and a positive right boundary) is “closed both”.
 
+## Scrape pools
+
+The following endpoint returns a list of all configured scrape pools:
+
+```
+GET /api/v1/scrape_pools
+```
+
+The `data` section of the JSON response is a list of string scrape pool names.
+
+```bash
+curl http://localhost:9090/api/v1/scrape_pools
+```
+
+```json
+{
+  "status": "success",
+  "data": {
+    "scrapePools": [
+      "prometheus",
+      "node_exporter",
+      "blackbox"
+    ]
+  }
+}
+```
+
+*New in v2.42*
+
 ## Targets
 
 The following endpoint returns an overview of the current state of the

From bd7ed84a39c1da29f4fc0f085aa919ed4c4dcc8c Mon Sep 17 00:00:00 2001
From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Date: Fri, 2 Jan 2026 17:52:54 +0100
Subject: [PATCH 198/439] Remove obsolete /classic/static route

The /classic/static/* route was added to serve vendor JavaScript and CSS
files (jQuery, Bootstrap, etc.) for console templates. These vendor assets
were removed in #14807 due to security vulnerabilities, making this route
obsolete as it now serves an empty directory.

The console feature remains functional via --web.console.templates and
--web.console.libraries flags. Users who need JavaScript/CSS libraries in
their custom console templates must provide these assets within the
directory specified by --web.console.libraries.

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---
 web/web.go | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/web/web.go b/web/web.go
index 2d216502c1..e787cbb4ac 100644
--- a/web/web.go
+++ b/web/web.go
@@ -455,13 +455,6 @@ func New(logger *slog.Logger, o *Options) *Handler {
 		reactAssetsRoot = "/static/react-app"
 	}
 
-	// The console library examples at 'console_libraries/prom.lib' still depend on old asset files being served under `classic`.
-	router.Get("/classic/static/*filepath", func(w http.ResponseWriter, r *http.Request) {
-		r.URL.Path = path.Join("/static", route.Param(r.Context(), "filepath"))
-		fs := server.StaticFileServer(ui.Assets)
-		fs.ServeHTTP(w, r)
-	})
-
 	router.Get("/version", h.version)
 	router.Get("/metrics", promhttp.Handler().ServeHTTP)
 

From 44ed09336dcb591de0deb0063f94c85ec8a5cd90 Mon Sep 17 00:00:00 2001
From: Ben Kochie 
Date: Sun, 4 Jan 2026 15:18:48 +0100
Subject: [PATCH 199/439] Update golangci-lint (#17767)

Update golangci-lint to latest.
* Update revive config to ignore package name rules.

Signed-off-by: SuperQ 
---
 .golangci.yml   | 5 +++++
 Makefile.common | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/.golangci.yml b/.golangci.yml
index 6dbbcc433d..0c866611e9 100644
--- a/.golangci.yml
+++ b/.golangci.yml
@@ -182,6 +182,11 @@ linters:
         - name: unused-receiver
         - name: var-declaration
         - name: var-naming
+          # TODO(SuperQ): See: https://github.com/prometheus/prometheus/issues/17766
+          arguments:
+            - []
+            - []
+            - - skip-package-name-checks: true
     testifylint:
       disable:
         - float-compare
diff --git a/Makefile.common b/Makefile.common
index 840bc0ea71..998da23093 100644
--- a/Makefile.common
+++ b/Makefile.common
@@ -61,7 +61,7 @@ PROMU_URL     := https://github.com/prometheus/promu/releases/download/v$(PROMU_
 SKIP_GOLANGCI_LINT :=
 GOLANGCI_LINT :=
 GOLANGCI_LINT_OPTS ?=
-GOLANGCI_LINT_VERSION ?= v2.6.2
+GOLANGCI_LINT_VERSION ?= v2.7.2
 GOLANGCI_FMT_OPTS ?=
 # golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64.
 # windows isn't included here because of the path separator being different.

From 1b88f2a98e4849b26a6fb9b5630a6541474ffe42 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 4 Jan 2026 15:44:42 +0100
Subject: [PATCH 200/439] chore(deps): bump google.golang.org/grpc from 1.77.0
 to 1.78.0 (#17763)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.77.0 to 1.78.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.77.0...v1.78.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.78.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/go.mod b/go.mod
index 53df4909fc..11606a2b1d 100644
--- a/go.mod
+++ b/go.mod
@@ -92,7 +92,7 @@ require (
 	golang.org/x/text v0.32.0
 	google.golang.org/api v0.257.0
 	google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b
-	google.golang.org/grpc v1.77.0
+	google.golang.org/grpc v1.78.0
 	google.golang.org/protobuf v1.36.11
 	gopkg.in/yaml.v3 v3.0.1
 	k8s.io/api v0.34.3
diff --git a/go.sum b/go.sum
index b61713fb9a..061a249f1d 100644
--- a/go.sum
+++ b/go.sum
@@ -714,8 +714,8 @@ google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b h1:
 google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b/go.mod h1:Xa7le7qx2vmqB/SzWUBa7KdMjpdpAHlh5QCSnjessQk=
 google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2 h1:2I6GHUeJ/4shcDpoUlLs/2WPnhg7yJwvXtqcMJt9liA=
 google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
-google.golang.org/grpc v1.77.0 h1:wVVY6/8cGA6vvffn+wWK5ToddbgdU3d8MNENr4evgXM=
-google.golang.org/grpc v1.77.0/go.mod h1:z0BY1iVj0q8E1uSQCjL9cppRj+gnZjzDnzV0dHhrNig=
+google.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc=
+google.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U=
 google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
 google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
 gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=

From 7521fdda738892dda79ab242d4ae87296d98b4b0 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 4 Jan 2026 15:45:28 +0100
Subject: [PATCH 201/439] chore(deps): bump the aws group across 1 directory
 with 2 updates (#17760)

Bumps the aws group with 2 updates in the / directory: [github.com/aws/aws-sdk-go-v2/service/ec2](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/service/ecs](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/ec2` from 1.277.0 to 1.279.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/ec2/v1.277.0...service/ec2/v1.279.0)

Updates `github.com/aws/aws-sdk-go-v2/service/ecs` from 1.69.5 to 1.70.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/ecs/v1.69.5...service/s3/v1.70.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ec2
  dependency-version: 1.279.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ecs
  dependency-version: 1.70.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 documentation/examples/remote_storage/go.mod | 25 +++++-----
 documentation/examples/remote_storage/go.sum | 50 ++++++++++----------
 go.mod                                       |  4 +-
 go.sum                                       |  8 ++--
 4 files changed, 45 insertions(+), 42 deletions(-)

diff --git a/documentation/examples/remote_storage/go.mod b/documentation/examples/remote_storage/go.mod
index e7f9551290..b77f248bf5 100644
--- a/documentation/examples/remote_storage/go.mod
+++ b/documentation/examples/remote_storage/go.mod
@@ -23,19 +23,20 @@ require (
 	github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 // indirect
 	github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect
 	github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
-	github.com/aws/aws-sdk-go-v2 v1.39.6 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.31.17 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.18.21 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.41.0 // indirect
+	github.com/aws/aws-sdk-go-v2/config v1.32.6 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.19.6 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 // indirect
-	github.com/aws/smithy-go v1.23.2 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.0.4 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.30.8 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.41.5 // indirect
+	github.com/aws/smithy-go v1.24.0 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
diff --git a/documentation/examples/remote_storage/go.sum b/documentation/examples/remote_storage/go.sum
index 692f9f5abf..1a3e86ff22 100644
--- a/documentation/examples/remote_storage/go.sum
+++ b/documentation/examples/remote_storage/go.sum
@@ -33,38 +33,40 @@ github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7D
 github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk=
 github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA=
 github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4=
-github.com/aws/aws-sdk-go-v2 v1.39.6 h1:2JrPCVgWJm7bm83BDwY5z8ietmeJUbh3O2ACnn+Xsqk=
-github.com/aws/aws-sdk-go-v2 v1.39.6/go.mod h1:c9pm7VwuW0UPxAEYGyTmyurVcNrbF6Rt/wixFqDhcjE=
-github.com/aws/aws-sdk-go-v2/config v1.31.17 h1:QFl8lL6RgakNK86vusim14P2k8BFSxjvUkcWLDjgz9Y=
-github.com/aws/aws-sdk-go-v2/config v1.31.17/go.mod h1:V8P7ILjp/Uef/aX8TjGk6OHZN6IKPM5YW6S78QnRD5c=
-github.com/aws/aws-sdk-go-v2/credentials v1.18.21 h1:56HGpsgnmD+2/KpG0ikvvR8+3v3COCwaF4r+oWwOeNA=
-github.com/aws/aws-sdk-go-v2/credentials v1.18.21/go.mod h1:3YELwedmQbw7cXNaII2Wywd+YY58AmLPwX4LzARgmmA=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 h1:T1brd5dR3/fzNFAQch/iBKeX07/ffu/cLu+q+RuzEWk=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13/go.mod h1:Peg/GBAQ6JDt+RoBf4meB1wylmAipb7Kg2ZFakZTlwk=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 h1:a+8/MLcWlIxo1lF9xaGt3J/u3yOZx+CdSveSNwjhD40=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13/go.mod h1:oGnKwIYZ4XttyU2JWxFrwvhF6YKiK/9/wmE3v3Iu9K8=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 h1:HBSI2kDkMdWz4ZM7FjwE7e/pWDEZ+nR95x8Ztet1ooY=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13/go.mod h1:YE94ZoDArI7awZqJzBAZ3PDD2zSfuP7w6P2knOzIn8M=
+github.com/aws/aws-sdk-go-v2 v1.41.0 h1:tNvqh1s+v0vFYdA1xq0aOJH+Y5cRyZ5upu6roPgPKd4=
+github.com/aws/aws-sdk-go-v2 v1.41.0/go.mod h1:MayyLB8y+buD9hZqkCW3kX1AKq07Y5pXxtgB+rRFhz0=
+github.com/aws/aws-sdk-go-v2/config v1.32.6 h1:hFLBGUKjmLAekvi1evLi5hVvFQtSo3GYwi+Bx4lpJf8=
+github.com/aws/aws-sdk-go-v2/config v1.32.6/go.mod h1:lcUL/gcd8WyjCrMnxez5OXkO3/rwcNmvfno62tnXNcI=
+github.com/aws/aws-sdk-go-v2/credentials v1.19.6 h1:F9vWao2TwjV2MyiyVS+duza0NIRtAslgLUM0vTA1ZaE=
+github.com/aws/aws-sdk-go-v2/credentials v1.19.6/go.mod h1:SgHzKjEVsdQr6Opor0ihgWtkWdfRAIwxYzSJ8O85VHY=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 h1:80+uETIWS1BqjnN9uJ0dBUaETh+P1XwFy5vwHwK5r9k=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16/go.mod h1:wOOsYuxYuB/7FlnVtzeBYRcjSRtQpAW0hCP7tIULMwo=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 h1:rgGwPzb82iBYSvHMHXc8h9mRoOUBZIGFgKb9qniaZZc=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16/go.mod h1:L/UxsGeKpGoIj6DxfhOWHWQ/kGKcd4I1VncE4++IyKA=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 h1:1jtGzuV7c82xnqOVfx2F0xmJcOw5374L7N6juGW6x6U=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16/go.mod h1:M2E5OQf+XLe+SZGmmpaI2yy+J326aFf6/+54PoxSANc=
 github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk=
 github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc=
 github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0 h1:5qBb1XV/D18qtCHd3bmmxoVglI+fZ4QWuS/EB8kIXYQ=
 github.com/aws/aws-sdk-go-v2/service/ec2 v1.262.0/go.mod h1:NDdDLLW5PtLLXN661gKcvJvqAH5OBXsfhMlmKVu1/pY=
 github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2 h1:oeICOX/+D0XXV1aMYJPXVe3CO37zYr7fB6HFgxchleU=
 github.com/aws/aws-sdk-go-v2/service/ecs v1.67.2/go.mod h1:rrhqfkXfa2DSNq0RyFhnnFEAyI+yJB4+2QlZKeJvMjs=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 h1:x2Ibm/Af8Fi+BH+Hsn9TXGdT+hKbDd5XOTZxTMxDk7o=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3/go.mod h1:IW1jwyrQgMdhisceG8fQLmQIydcT/jWY21rFhzgaKwo=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 h1:kDqdFvMY4AtKoACfzIGD8A0+hbT41KTKF//gq7jITfM=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13/go.mod h1:lmKuogqSU3HzQCwZ9ZtcqOc5XGMqtDK7OIc2+DxiUEg=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 h1:0ryTNEdJbzUCEWkVXEXoqlXV72J5keC1GvILMOuD00E=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4/go.mod h1:HQ4qwNZh32C3CBeO6iJLQlgtMzqeG17ziAA/3KDJFow=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 h1:oHjJHeUy0ImIV0bsrX0X91GkV5nJAyv1l1CC9lnO0TI=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16/go.mod h1:iRSNGgOYmiYwSCXxXaKb9HfOEj40+oTKn8pTxMlYkRM=
 github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4 h1:/1o2AYwHJojUDeMvQNyJiKZwcWCc3e4kQuTXqRLuThc=
 github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.4/go.mod h1:Nn2xx6HojGuNMtUFxxz/nyNLSS+tHMRsMhe3+W3wB5k=
-github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 h1:0JPwLz1J+5lEOfy/g0SURC9cxhbQ1lIMHMa+AHZSzz0=
-github.com/aws/aws-sdk-go-v2/service/sso v1.30.1/go.mod h1:fKvyjJcz63iL/ftA6RaM8sRCtN4r4zl4tjL3qw5ec7k=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 h1:OWs0/j2UYR5LOGi88sD5/lhN6TDLG6SfA7CqsQO9zF0=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5/go.mod h1:klO+ejMvYsB4QATfEOIXk8WAEwN4N0aBfJpvC+5SZBo=
-github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 h1:mLlUgHn02ue8whiR4BmxxGJLR2gwU6s6ZzJ5wDamBUs=
-github.com/aws/aws-sdk-go-v2/service/sts v1.39.1/go.mod h1:E19xDjpzPZC7LS2knI9E6BaRFDK43Eul7vd6rSq2HWk=
-github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM=
-github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
+github.com/aws/aws-sdk-go-v2/service/signin v1.0.4 h1:HpI7aMmJ+mm1wkSHIA2t5EaFFv5EFYXePW30p1EIrbQ=
+github.com/aws/aws-sdk-go-v2/service/signin v1.0.4/go.mod h1:C5RdGMYGlfM0gYq/tifqgn4EbyX99V15P2V3R+VHbQU=
+github.com/aws/aws-sdk-go-v2/service/sso v1.30.8 h1:aM/Q24rIlS3bRAhTyFurowU8A0SMyGDtEOY/l/s/1Uw=
+github.com/aws/aws-sdk-go-v2/service/sso v1.30.8/go.mod h1:+fWt2UHSb4kS7Pu8y+BMBvJF0EWx+4H0hzNwtDNRTrg=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 h1:AHDr0DaHIAo8c9t1emrzAlVDFp+iMMKnPdYy6XO4MCE=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12/go.mod h1:GQ73XawFFiWxyWXMHWfhiomvP3tXtdNar/fi8z18sx0=
+github.com/aws/aws-sdk-go-v2/service/sts v1.41.5 h1:SciGFVNZ4mHdm7gpD1dgZYnCuVdX1s+lFTg4+4DOy70=
+github.com/aws/aws-sdk-go-v2/service/sts v1.41.5/go.mod h1:iW40X4QBmUxdP+fZNOpfmkdMZqsovezbAeO+Ubiv2pk=
+github.com/aws/smithy-go v1.24.0 h1:LpilSUItNPFr1eY85RYgTIg5eIEPtvFbskaFcmmIUnk=
+github.com/aws/smithy-go v1.24.0/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
 github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3 h1:6df1vn4bBlDDo4tARvBm7l6KA9iVMnE3NWizDeWSrps=
 github.com/bboreham/go-loser v0.0.0-20230920113527-fcc2c21820a3/go.mod h1:CIWtjkly68+yqLPbvwwR/fjNJA/idrtULjZWh2v1ys0=
 github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
diff --git a/go.mod b/go.mod
index 11606a2b1d..c8d1250cca 100644
--- a/go.mod
+++ b/go.mod
@@ -14,8 +14,8 @@ require (
 	github.com/aws/aws-sdk-go-v2 v1.41.0
 	github.com/aws/aws-sdk-go-v2/config v1.32.6
 	github.com/aws/aws-sdk-go-v2/credentials v1.19.6
-	github.com/aws/aws-sdk-go-v2/service/ec2 v1.277.0
-	github.com/aws/aws-sdk-go-v2/service/ecs v1.69.5
+	github.com/aws/aws-sdk-go-v2/service/ec2 v1.279.0
+	github.com/aws/aws-sdk-go-v2/service/ecs v1.70.0
 	github.com/aws/aws-sdk-go-v2/service/lightsail v1.50.10
 	github.com/aws/aws-sdk-go-v2/service/sts v1.41.5
 	github.com/aws/smithy-go v1.24.0
diff --git a/go.sum b/go.sum
index 061a249f1d..823a33c02d 100644
--- a/go.sum
+++ b/go.sum
@@ -61,10 +61,10 @@ github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 h1:1jtGzuV7c82xnqOVfx
 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16/go.mod h1:M2E5OQf+XLe+SZGmmpaI2yy+J326aFf6/+54PoxSANc=
 github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk=
 github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc=
-github.com/aws/aws-sdk-go-v2/service/ec2 v1.277.0 h1:RHJSkRXDGkAKrV4CTEsZsZkOmSpxXKO4aKx4rXd94K4=
-github.com/aws/aws-sdk-go-v2/service/ec2 v1.277.0/go.mod h1:Wg68QRgy2gEGGdmTPU/UbVpdv8sM14bUZmF64KFwAsY=
-github.com/aws/aws-sdk-go-v2/service/ecs v1.69.5 h1:5nkhwt0d/gjuT3AQ2LUK0aFRNB3MGlzB2elqy/ZsKP4=
-github.com/aws/aws-sdk-go-v2/service/ecs v1.69.5/go.mod h1:LQMlcWBoiFVD3vUVEz42ST0yTiaDujv2dRE6sXt1yPE=
+github.com/aws/aws-sdk-go-v2/service/ec2 v1.279.0 h1:o7eJKe6VYAnqERPlLAvDW5VKXV6eTKv1oxTpMoDP378=
+github.com/aws/aws-sdk-go-v2/service/ec2 v1.279.0/go.mod h1:Wg68QRgy2gEGGdmTPU/UbVpdv8sM14bUZmF64KFwAsY=
+github.com/aws/aws-sdk-go-v2/service/ecs v1.70.0 h1:IZpZatHsscdOKjwmDXC6idsCXmm3F/obutAUNjnX+OM=
+github.com/aws/aws-sdk-go-v2/service/ecs v1.70.0/go.mod h1:LQMlcWBoiFVD3vUVEz42ST0yTiaDujv2dRE6sXt1yPE=
 github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 h1:0ryTNEdJbzUCEWkVXEXoqlXV72J5keC1GvILMOuD00E=
 github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4/go.mod h1:HQ4qwNZh32C3CBeO6iJLQlgtMzqeG17ziAA/3KDJFow=
 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 h1:oHjJHeUy0ImIV0bsrX0X91GkV5nJAyv1l1CC9lnO0TI=

From ff22779966cbe7006b842c3e4888ffaf5e54b3aa Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 4 Jan 2026 15:50:55 +0100
Subject: [PATCH 202/439] chore(deps): bump github.com/bufbuild/buf in
 /internal/tools (#17764)

Bumps [github.com/bufbuild/buf](https://github.com/bufbuild/buf) from 1.61.0 to 1.62.1.
- [Release notes](https://github.com/bufbuild/buf/releases)
- [Changelog](https://github.com/bufbuild/buf/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bufbuild/buf/compare/v1.61.0...v1.62.1)

---
updated-dependencies:
- dependency-name: github.com/bufbuild/buf
  dependency-version: 1.62.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 internal/tools/go.mod |  61 +++++++++---------
 internal/tools/go.sum | 146 +++++++++++++++++++++---------------------
 2 files changed, 105 insertions(+), 102 deletions(-)

diff --git a/internal/tools/go.mod b/internal/tools/go.mod
index 84b540df4b..a7a1ebec54 100644
--- a/internal/tools/go.mod
+++ b/internal/tools/go.mod
@@ -3,42 +3,43 @@ module github.com/prometheus/prometheus/internal/tools
 go 1.24.9
 
 require (
-	github.com/bufbuild/buf v1.61.0
+	github.com/bufbuild/buf v1.62.1
 	github.com/daixiang0/gci v0.13.7
 	github.com/gogo/protobuf v1.3.2
 	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4
 )
 
 require (
-	buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.10-20250718181942-e35f9b667443.1 // indirect
-	buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.10-20250109164928-1da0de137947.1 // indirect
-	buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.10-20250912141014-52f32327d4b0.1 // indirect
-	buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251027152159-f1066ce064ca.2 // indirect
-	buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.10-20251027152159-f1066ce064ca.1 // indirect
-	buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.10-20241007202033-cf42259fcbfc.1 // indirect
+	buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.11-20250718181942-e35f9b667443.1 // indirect
+	buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.11-20250109164928-1da0de137947.1 // indirect
+	buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.11-20251209175733-2a1774d88802.1 // indirect
+	buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251202164234-62b14f0b533c.2 // indirect
+	buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.11-20251202164234-62b14f0b533c.1 // indirect
+	buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.11-20241007202033-cf42259fcbfc.1 // indirect
 	buf.build/go/app v0.2.0 // indirect
 	buf.build/go/bufplugin v0.9.0 // indirect
 	buf.build/go/bufprivateusage v0.1.0 // indirect
 	buf.build/go/interrupt v1.1.0 // indirect
-	buf.build/go/protovalidate v1.0.1 // indirect
+	buf.build/go/protovalidate v1.1.0 // indirect
 	buf.build/go/protoyaml v0.6.0 // indirect
 	buf.build/go/spdx v0.2.0 // indirect
 	buf.build/go/standard v0.1.0 // indirect
-	cel.dev/expr v0.24.0 // indirect
+	cel.dev/expr v0.25.1 // indirect
 	connectrpc.com/connect v1.19.1 // indirect
 	connectrpc.com/otelconnect v0.8.0 // indirect
 	github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
 	github.com/Microsoft/go-winio v0.6.2 // indirect
 	github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
-	github.com/bufbuild/protocompile v0.14.2-0.20251120233202-3f9009bcd6c8 // indirect
+	github.com/bufbuild/protocompile v0.14.2-0.20251223142729-db46c1b9d34e // indirect
 	github.com/bufbuild/protoplugin v0.0.0-20250218205857-750e09ce93e1 // indirect
+	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/cli/browser v1.3.0 // indirect
 	github.com/containerd/errdefs v1.0.0 // indirect
 	github.com/containerd/errdefs/pkg v0.3.0 // indirect
-	github.com/containerd/stargz-snapshotter/estargz v0.17.0 // indirect
+	github.com/containerd/stargz-snapshotter/estargz v0.18.1 // indirect
 	github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
 	github.com/distribution/reference v0.6.0 // indirect
-	github.com/docker/cli v28.5.1+incompatible // indirect
+	github.com/docker/cli v29.1.3+incompatible // indirect
 	github.com/docker/distribution v2.8.3+incompatible // indirect
 	github.com/docker/docker v28.5.2+incompatible // indirect
 	github.com/docker/docker-credential-helpers v0.9.4 // indirect
@@ -50,57 +51,57 @@ require (
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/gofrs/flock v0.13.0 // indirect
 	github.com/google/cel-go v0.26.1 // indirect
-	github.com/google/go-containerregistry v0.20.6 // indirect
+	github.com/google/go-containerregistry v0.20.7 // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/hexops/gotextdiff v1.0.3 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/jdx/go-netrc v1.0.0 // indirect
-	github.com/klauspost/compress v1.18.1 // indirect
+	github.com/klauspost/compress v1.18.2 // indirect
 	github.com/klauspost/pgzip v1.2.6 // indirect
 	github.com/mattn/go-colorable v0.1.14 // indirect
 	github.com/mattn/go-isatty v0.0.20 // indirect
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/moby/docker-image-spec v1.3.1 // indirect
 	github.com/moby/term v0.5.2 // indirect
-	github.com/morikuni/aec v1.0.0 // indirect
+	github.com/morikuni/aec v1.1.0 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.1 // indirect
-	github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 // indirect
+	github.com/petermattis/goid v0.0.0-20251121121749-a11dd1a45f9a // indirect
 	github.com/pkg/errors v0.9.1 // indirect
 	github.com/quic-go/qpack v0.6.0 // indirect
-	github.com/quic-go/quic-go v0.57.0 // indirect
+	github.com/quic-go/quic-go v0.58.0 // indirect
 	github.com/rivo/uniseg v0.4.7 // indirect
 	github.com/rs/cors v1.11.1 // indirect
 	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/segmentio/asm v1.2.1 // indirect
 	github.com/segmentio/encoding v0.5.3 // indirect
 	github.com/sirupsen/logrus v1.9.3 // indirect
-	github.com/spf13/cobra v1.10.1 // indirect
+	github.com/spf13/cobra v1.10.2 // indirect
 	github.com/spf13/pflag v1.0.10 // indirect
 	github.com/stoewer/go-strcase v1.3.1 // indirect
-	github.com/tetratelabs/wazero v1.9.0 // indirect
+	github.com/tetratelabs/wazero v1.11.0 // indirect
 	github.com/tidwall/btree v1.8.1 // indirect
-	github.com/vbatts/tar-split v0.12.1 // indirect
+	github.com/vbatts/tar-split v0.12.2 // indirect
 	go.lsp.dev/jsonrpc2 v0.10.0 // indirect
 	go.lsp.dev/pkg v0.0.0-20210717090340-384b27a52fb2 // indirect
 	go.lsp.dev/protocol v0.12.0 // indirect
 	go.lsp.dev/uri v0.3.0 // indirect
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 // indirect
-	go.opentelemetry.io/otel v1.38.0 // indirect
-	go.opentelemetry.io/otel/metric v1.38.0 // indirect
-	go.opentelemetry.io/otel/trace v1.38.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 // indirect
+	go.opentelemetry.io/otel v1.39.0 // indirect
+	go.opentelemetry.io/otel/metric v1.39.0 // indirect
+	go.opentelemetry.io/otel/trace v1.39.0 // indirect
 	go.uber.org/mock v0.6.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.uber.org/zap v1.27.1 // indirect
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/crypto v0.45.0 // indirect
-	golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b // indirect
-	golang.org/x/mod v0.30.0 // indirect
-	golang.org/x/net v0.47.0 // indirect
+	golang.org/x/crypto v0.46.0 // indirect
+	golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 // indirect
+	golang.org/x/mod v0.31.0 // indirect
+	golang.org/x/net v0.48.0 // indirect
 	golang.org/x/sync v0.19.0 // indirect
-	golang.org/x/sys v0.38.0 // indirect
-	golang.org/x/term v0.37.0 // indirect
+	golang.org/x/sys v0.39.0 // indirect
+	golang.org/x/term v0.38.0 // indirect
 	golang.org/x/text v0.32.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20251222181119-0a764e51fe1b // indirect
diff --git a/internal/tools/go.sum b/internal/tools/go.sum
index 7c06edc7ed..df735a5536 100644
--- a/internal/tools/go.sum
+++ b/internal/tools/go.sum
@@ -1,15 +1,15 @@
-buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.10-20250718181942-e35f9b667443.1 h1:FzJGrb8r7vir+P3zJ5Ebey8p54LYTYtQsrM/U35YO9Q=
-buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.10-20250718181942-e35f9b667443.1/go.mod h1:E6HwqUm4Ag7bXtg/tX7jHWO7CgpknbmeACgDax0icV0=
-buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.10-20250109164928-1da0de137947.1 h1:9hkMnVoImDlY7rTlAWIWXdkGUKOjf3YlyZeSbYT29uA=
-buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.10-20250109164928-1da0de137947.1/go.mod h1:/AouMCAeQ+kB7+RRFpdUlZe3503p18VoUNcU2AFqZXM=
-buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.10-20250912141014-52f32327d4b0.1 h1:31on4W/yPcV4nZHL4+UCiCvLPsMqe/vJcNg8Rci0scc=
-buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.10-20250912141014-52f32327d4b0.1/go.mod h1:fUl8CEN/6ZAMk6bP8ahBJPUJw7rbp+j4x+wCcYi2IG4=
-buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251027152159-f1066ce064ca.2 h1:Dbh4Edwy5qHlz1/boPAQ7T5Q7ZDMgEuQlEbXa94+JEo=
-buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251027152159-f1066ce064ca.2/go.mod h1:SqqTA3aiYVDkpDINxgbxDT6QBjkVjdqUXtbiz6DiWIg=
-buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.10-20251027152159-f1066ce064ca.1 h1:5tUFlRgcC+N2JJtjwlwyb2J4bBk/bJYLXk50zlewtzk=
-buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.10-20251027152159-f1066ce064ca.1/go.mod h1:AaYXXeRvnOc151wEuupAmn58Mh9bccKce2kk3QKMIrQ=
-buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.10-20241007202033-cf42259fcbfc.1 h1:CzM0kZcoaIr8+R4i8QVorUNRM/CqMr87i3j+w2pdpCc=
-buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.10-20241007202033-cf42259fcbfc.1/go.mod h1:bG+Fa7tcA+4pW0JdOh4h7iKjleyZIKhfVzVS10qfrnk=
+buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.11-20250718181942-e35f9b667443.1 h1:zQ9C3e6FtwSZUFuKAQfpIKGFk5ZuRoGt5g35Bix55sI=
+buf.build/gen/go/bufbuild/bufplugin/protocolbuffers/go v1.36.11-20250718181942-e35f9b667443.1/go.mod h1:1Znr6gmYBhbxWUPRrrVnSLXQsz8bvFVw1HHJq2bI3VQ=
+buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.11-20250109164928-1da0de137947.1 h1:HwzzCRS4ZrEm1++rzSDxHnO0DOjiT1b8I/24e8a4exY=
+buf.build/gen/go/bufbuild/protodescriptor/protocolbuffers/go v1.36.11-20250109164928-1da0de137947.1/go.mod h1:8PRKXhgNes29Tjrnv8KdZzg3I1QceOkzibW1QK7EXv0=
+buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.11-20251209175733-2a1774d88802.1 h1:j9yeqTWEFrtimt8Nng2MIeRrpoCvQzM9/g25XTvqUGg=
+buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.11-20251209175733-2a1774d88802.1/go.mod h1:tvtbpgaVXZX4g6Pn+AnzFycuRK3MOz5HJfEGeEllXYM=
+buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251202164234-62b14f0b533c.2 h1:eQ6XRVUaYYZFOZvBsyrOYLWbw6464s5dVnHscxa0b8w=
+buf.build/gen/go/bufbuild/registry/connectrpc/go v1.19.1-20251202164234-62b14f0b533c.2/go.mod h1:omxVRch3jEPMINnUipLsuRWoEhND6LPXELKBG7xzyDw=
+buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.11-20251202164234-62b14f0b533c.1 h1:PdfIJUbUVKdajMVYuMdvr2Wvo+wmzGnlPEYA4bhFaWI=
+buf.build/gen/go/bufbuild/registry/protocolbuffers/go v1.36.11-20251202164234-62b14f0b533c.1/go.mod h1:1JJi9jvOqRxSMa+JxiZSm57doB+db/1WYCIa2lHfc40=
+buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.11-20241007202033-cf42259fcbfc.1 h1:iGPvEJltOXUMANWf0zajcRcbiOXLD90ZwPUFvbcuv6Q=
+buf.build/gen/go/pluginrpc/pluginrpc/protocolbuffers/go v1.36.11-20241007202033-cf42259fcbfc.1/go.mod h1:nWVKKRA29zdt4uvkjka3i/y4mkrswyWwiu0TbdX0zts=
 buf.build/go/app v0.2.0 h1:NYaH13A+RzPb7M5vO8uZYZ2maBZI5+MS9A9tQm66fy8=
 buf.build/go/app v0.2.0/go.mod h1:0XVOYemubVbxNXVY0DnsVgWeGkcbbAvjDa1fmhBC+Wo=
 buf.build/go/bufplugin v0.9.0 h1:ktZJNP3If7ldcWVqh46XKeiYJVPxHQxCfjzVQDzZ/lo=
@@ -18,16 +18,16 @@ buf.build/go/bufprivateusage v0.1.0 h1:SzCoCcmzS3zyXHEXHeSQhGI7OTkgtljoknLzsUz9G
 buf.build/go/bufprivateusage v0.1.0/go.mod h1:GlCCJ3VVF7EqqU0CoRmo1FzAwwaKymEWSr+ty69xU5w=
 buf.build/go/interrupt v1.1.0 h1:olBuhgv9Sav4/9pkSLoxgiOsZDgM5VhRhvRpn3DL0lE=
 buf.build/go/interrupt v1.1.0/go.mod h1:ql56nXPG1oHlvZa6efNC7SKAQ/tUjS6z0mhJl0gyeRM=
-buf.build/go/protovalidate v1.0.1 h1:Fwmf08OOUuKVeMvEnDmcKxQam4PJc/zFgvVX64BhTms=
-buf.build/go/protovalidate v1.0.1/go.mod h1:SoZmvk/3ZzOVg9YSkTdm4grMAByjf8zgZq4ZNaLZXoQ=
+buf.build/go/protovalidate v1.1.0 h1:pQqEQRpOo4SqS60qkvmhLTTQU9JwzEvdyiqAtXa5SeY=
+buf.build/go/protovalidate v1.1.0/go.mod h1:bGZcPiAQDC3ErCHK3t74jSoJDFOs2JH3d7LWuTEIdss=
 buf.build/go/protoyaml v0.6.0 h1:Nzz1lvcXF8YgNZXk+voPPwdU8FjDPTUV4ndNTXN0n2w=
 buf.build/go/protoyaml v0.6.0/go.mod h1:RgUOsBu/GYKLDSIRgQXniXbNgFlGEZnQpRAUdLAFV2Q=
 buf.build/go/spdx v0.2.0 h1:IItqM0/cMxvFJJumcBuP8NrsIzMs/UYjp/6WSpq8LTw=
 buf.build/go/spdx v0.2.0/go.mod h1:bXdwQFem9Si3nsbNy8aJKGPoaPi5DKwdeEp5/ArZ6w8=
 buf.build/go/standard v0.1.0 h1:g98T9IyvAl0vS3Pq8iVk6Cvj2ZiFvoUJRtfyGa0120U=
 buf.build/go/standard v0.1.0/go.mod h1:PiqpHz/7ZFq+kqvYhc/SK3lxFIB9N/aiH2CFC2JHIQg=
-cel.dev/expr v0.24.0 h1:56OvJKSH3hDGL0ml5uSxZmz3/3Pq4tJ+fb1unVLAFcY=
-cel.dev/expr v0.24.0/go.mod h1:hLPLo1W4QUmuYdA72RBX06QTs6MXw941piREPl3Yfiw=
+cel.dev/expr v0.25.1 h1:1KrZg61W6TWSxuNZ37Xy49ps13NUovb66QLprthtwi4=
+cel.dev/expr v0.25.1/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=
 connectrpc.com/connect v1.19.1 h1:R5M57z05+90EfEvCY1b7hBxDVOUl45PrtXtAV2fOC14=
 connectrpc.com/connect v1.19.1/go.mod h1:tN20fjdGlewnSFeZxLKb0xwIZ6ozc3OQs2hTXy4du9w=
 connectrpc.com/otelconnect v0.8.0 h1:a4qrN4H8aEE2jAoCxheZYYfEjXMgVPyL9OzPQLBEFXU=
@@ -42,14 +42,16 @@ github.com/bmatcuk/doublestar/v4 v4.9.1 h1:X8jg9rRZmJd4yRy7ZeNDRnM+T3ZfHv15JiBJ/
 github.com/bmatcuk/doublestar/v4 v4.9.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
 github.com/brianvoe/gofakeit/v6 v6.28.0 h1:Xib46XXuQfmlLS2EXRuJpqcw8St6qSZz75OUo0tgAW4=
 github.com/brianvoe/gofakeit/v6 v6.28.0/go.mod h1:Xj58BMSnFqcn/fAQeSK+/PLtC5kSb7FJIq4JyGa8vEs=
-github.com/bufbuild/buf v1.61.0 h1:JPaK/RM2eoheyzznW+1LxaFgN6xjBCi8s25q2kUbH9A=
-github.com/bufbuild/buf v1.61.0/go.mod h1:Xs3leBmxjL5tTnSVYfNwNXHXD1k5et3fR/tJyIyQl4s=
-github.com/bufbuild/protocompile v0.14.2-0.20251120233202-3f9009bcd6c8 h1:l4PKzJ7Usff8j5/e+YaWZPaM+rJHIghgDxRn8vDNxNo=
-github.com/bufbuild/protocompile v0.14.2-0.20251120233202-3f9009bcd6c8/go.mod h1:HKN246DRQwavs64sr2xYmSL+RFOFxmLti+WGCZ2jh9U=
+github.com/bufbuild/buf v1.62.1 h1:QdYB6JDW7dP+5H7sKx0lN1raxnuUJDDlEJtPHDYKB0g=
+github.com/bufbuild/buf v1.62.1/go.mod h1:igMN/6U32/GDzyfkmn0VfIaKoeOnWTTizEf5CG0/87k=
+github.com/bufbuild/protocompile v0.14.2-0.20251223142729-db46c1b9d34e h1:LQA+1MyiPkolGHJGC2GMDC5Xu+0RDVH6jGMKech7Exs=
+github.com/bufbuild/protocompile v0.14.2-0.20251223142729-db46c1b9d34e/go.mod h1:5UUj46Eu+U+C59C5N6YilaMI7WWfP2bW9xGcOkme2DI=
 github.com/bufbuild/protoplugin v0.0.0-20250218205857-750e09ce93e1 h1:V1xulAoqLqVg44rY97xOR+mQpD2N+GzhMHVwJ030WEU=
 github.com/bufbuild/protoplugin v0.0.0-20250218205857-750e09ce93e1/go.mod h1:c5D8gWRIZ2HLWO3gXYTtUfw/hbJyD8xikv2ooPxnklQ=
 github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
 github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
+github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
+github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
 github.com/cli/browser v1.3.0 h1:LejqCrpWr+1pRqmEPDGnTZOjsMe7sehifLynZJuqJpo=
 github.com/cli/browser v1.3.0/go.mod h1:HH8s+fOAxjhQoBUAsKuPCbqUuxZDhQ2/aD+SzsEfBTk=
 github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
@@ -58,8 +60,8 @@ github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151X
 github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
 github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
 github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
-github.com/containerd/stargz-snapshotter/estargz v0.17.0 h1:+TyQIsR/zSFI1Rm31EQBwpAA1ovYgIKHy7kctL3sLcE=
-github.com/containerd/stargz-snapshotter/estargz v0.17.0/go.mod h1:s06tWAiJcXQo9/8AReBCIo/QxcXFZ2n4qfsRnpl71SM=
+github.com/containerd/stargz-snapshotter/estargz v0.18.1 h1:cy2/lpgBXDA3cDKSyEfNOFMA/c10O1axL69EU7iirO8=
+github.com/containerd/stargz-snapshotter/estargz v0.18.1/go.mod h1:ALIEqa7B6oVDsrF37GkGN20SuvG/pIMm7FwP7ZmRb0Q=
 github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
 github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo=
 github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
@@ -72,8 +74,8 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
 github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
-github.com/docker/cli v28.5.1+incompatible h1:ESutzBALAD6qyCLqbQSEf1a/U8Ybms5agw59yGVc+yY=
-github.com/docker/cli v28.5.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
+github.com/docker/cli v29.1.3+incompatible h1:+kz9uDWgs+mAaIZojWfFt4d53/jv0ZUOOoSh5ZnH36c=
+github.com/docker/cli v29.1.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
 github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
 github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
 github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM=
@@ -102,8 +104,8 @@ github.com/google/cel-go v0.26.1/go.mod h1:A9O8OU9rdvrK5MQyrqfIxo1a0u4g3sF8KB6PU
 github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
 github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
-github.com/google/go-containerregistry v0.20.6 h1:cvWX87UxxLgaH76b4hIvya6Dzz9qHB31qAwjAohdSTU=
-github.com/google/go-containerregistry v0.20.6/go.mod h1:T0x8MuoAoKX/873bkeSfLD2FAkwCDf9/HZgsFJ02E2Y=
+github.com/google/go-containerregistry v0.20.7 h1:24VGNpS0IwrOZ2ms2P1QE3Xa5X9p4phx0aUgzYzHW6I=
+github.com/google/go-containerregistry v0.20.7/go.mod h1:Lx5LCZQjLH1QBaMPeGwsME9biPeo1lPx6lbGj/UmzgM=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4 h1:kEISI/Gx67NzH3nJxAmY/dGac80kKZgZt134u7Y/k1s=
@@ -118,8 +120,8 @@ github.com/jhump/protoreflect/v2 v2.0.0-beta.2 h1:qZU+rEZUOYTz1Bnhi3xbwn+VxdXkLV
 github.com/jhump/protoreflect/v2 v2.0.0-beta.2/go.mod h1:4tnOYkB/mq7QTyS3YKtVtNrJv4Psqout8HA1U+hZtgM=
 github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
-github.com/klauspost/compress v1.18.1 h1:bcSGx7UbpBqMChDtsF28Lw6v/G94LPrrbMbdC3JH2co=
-github.com/klauspost/compress v1.18.1/go.mod h1:ZQFFVG+MdnR0P+l6wpXgIL4NTtwiKIdBnrBd8Nrxr+0=
+github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
+github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
 github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
 github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
 github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
@@ -140,14 +142,14 @@ github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7z
 github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko=
 github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ=
 github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc=
-github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
-github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
+github.com/morikuni/aec v1.1.0 h1:vBBl0pUnvi/Je71dsRrhMBtreIqNMYErSAbEeb8jrXQ=
+github.com/morikuni/aec v1.1.0/go.mod h1:xDRgiq/iw5l+zkao76YTKzKttOp2cwPEne25HDkJnBw=
 github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
 github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
 github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
 github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
-github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490 h1:QTvNkZ5ylY0PGgA+Lih+GdboMLY/G9SEGLMEGVjTVA4=
-github.com/petermattis/goid v0.0.0-20250904145737-900bdf8bb490/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
+github.com/petermattis/goid v0.0.0-20251121121749-a11dd1a45f9a h1:VweslR2akb/ARhXfqSfRbj1vpWwYXf3eeAUyw/ndms0=
+github.com/petermattis/goid v0.0.0-20251121121749-a11dd1a45f9a/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -156,8 +158,8 @@ github.com/protocolbuffers/protoscope v0.0.0-20221109213918-8e7a6aafa2c9 h1:arwj
 github.com/protocolbuffers/protoscope v0.0.0-20221109213918-8e7a6aafa2c9/go.mod h1:SKZx6stCn03JN3BOWTwvVIO2ajMkb/zQdTceXYhKw/4=
 github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
 github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
-github.com/quic-go/quic-go v0.57.0 h1:AsSSrrMs4qI/hLrKlTH/TGQeTMY0ib1pAOX7vA3AdqE=
-github.com/quic-go/quic-go v0.57.0/go.mod h1:ly4QBAjHA2VhdnxhojRsCUOeJwKYg+taDlos92xb1+s=
+github.com/quic-go/quic-go v0.58.0 h1:ggY2pvZaVdB9EyojxL1p+5mptkuHyX5MOSv4dgWF4Ug=
+github.com/quic-go/quic-go v0.58.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
 github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
 github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
 github.com/rodaine/protogofakeit v0.1.1 h1:ZKouljuRM3A+TArppfBqnH8tGZHOwM/pjvtXe9DaXH8=
@@ -174,8 +176,8 @@ github.com/segmentio/encoding v0.5.3 h1:OjMgICtcSFuNvQCdwqMCv9Tg7lEOXGwm1J5RPQcc
 github.com/segmentio/encoding v0.5.3/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
 github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
 github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
-github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
-github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
+github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
+github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
 github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
 github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
@@ -190,12 +192,12 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
 github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
 github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
 github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
-github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I=
-github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM=
+github.com/tetratelabs/wazero v1.11.0 h1:+gKemEuKCTevU4d7ZTzlsvgd1uaToIDtlQlmNbwqYhA=
+github.com/tetratelabs/wazero v1.11.0/go.mod h1:eV28rsN8Q+xwjogd7f4/Pp4xFxO7uOGbLcD/LzB1wiU=
 github.com/tidwall/btree v1.8.1 h1:27ehoXvm5AG/g+1VxLS1SD3vRhp/H7LuEfwNvddEdmA=
 github.com/tidwall/btree v1.8.1/go.mod h1:jBbTdUWhSZClZWoDg54VnvV7/54modSOzDN7VXftj1A=
-github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo=
-github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
+github.com/vbatts/tar-split v0.12.2 h1:w/Y6tjxpeiFMR47yzZPlPj/FcPLpXbTUi/9H7d3CPa4=
+github.com/vbatts/tar-split v0.12.2/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 go.lsp.dev/jsonrpc2 v0.10.0 h1:Pr/YcXJoEOTMc/b6OTmcR1DPJ3mSWl/SWiU1Cct6VmI=
@@ -208,24 +210,24 @@ go.lsp.dev/uri v0.3.0 h1:KcZJmh6nFIBeJzTugn5JTU6OOyG0lDOo3R9KwTxTYbo=
 go.lsp.dev/uri v0.3.0/go.mod h1:P5sbO1IQR+qySTWOCnhnK7phBx+W3zbLqSMDJNTw88I=
 go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
 go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
-go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18=
-go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg=
-go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8=
-go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 h1:GqRJVj7UmLjCVyVJ3ZFLdPRmhDUp2zFmQe3RHIOsw24=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0/go.mod h1:ri3aaHSmCTVYu2AWv44YMauwAQc0aqI9gHKIcSbI1pU=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 h1:ssfIgGNANqpVFCndZvcuyKbl0g+UAVcbBcqGkG28H0Y=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0/go.mod h1:GQ/474YrbE4Jx8gZ4q5I4hrhUzM6UPzyrqJYV2AqPoQ=
+go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48=
+go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.39.0 h1:f0cb2XPmrqn4XMy9PNliTgRKJgS5WcL/u0/WRYGz4t0=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.39.0/go.mod h1:vnakAaFckOMiMtOIhFI2MNH4FYrZzXCYxmb1LlhoGz8=
 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.33.0 h1:wpMfgF8E1rkrT1Z6meFh1NDtownE9Ii3n3X2GJYjsaU=
 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.33.0/go.mod h1:wAy0T/dUbs468uOlkT31xjvqQgEVXv58BRFWEgn5v/0=
-go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA=
-go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI=
-go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E=
-go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg=
-go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM=
-go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA=
-go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE=
-go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs=
-go.opentelemetry.io/proto/otlp v1.8.0 h1:fRAZQDcAFHySxpJ1TwlA1cJ4tvcrw7nXl9xWWC8N5CE=
-go.opentelemetry.io/proto/otlp v1.8.0/go.mod h1:tIeYOeNBU4cvmPqpaji1P+KbB4Oloai8wN4rWzRrFF0=
+go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0=
+go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs=
+go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18=
+go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE=
+go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8=
+go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew=
+go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI=
+go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA=
+go.opentelemetry.io/proto/otlp v1.9.0 h1:l706jCMITVouPOqEnii2fIAuO3IVGBRPV5ICjceRb/A=
+go.opentelemetry.io/proto/otlp v1.9.0/go.mod h1:xE+Cx5E/eEHw+ISFkwPLwCZefwVjY+pqKg1qcK03+/4=
 go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
 go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
 go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
@@ -239,20 +241,20 @@ go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
-golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
-golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
-golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b h1:18qgiDvlvH7kk8Ioa8Ov+K6xCi0GMvmGfGW0sgd/SYA=
-golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70=
+golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
+golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
+golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 h1:fQsdNF2N+/YewlRZiricy4P1iimyPKZ/xwniHj8Q2a0=
+golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93/go.mod h1:EPRbTFwzwjXj9NpYyyrvenVh9Y+GFeEvMNh7Xuz7xgU=
 golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk=
-golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc=
+golang.org/x/mod v0.31.0 h1:HaW9xtz0+kOcWKwli0ZXy79Ix+UW/vOfmWI5QVd2tgI=
+golang.org/x/mod v0.31.0/go.mod h1:43JraMp9cGx1Rx3AqioxrbrhNsLl2l/iNAvuBkrezpg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
-golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
+golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
+golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -264,22 +266,22 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
-golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
-golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=
-golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=
+golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk=
+golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
+golang.org/x/term v0.38.0 h1:PQ5pkm/rLO6HnxFR7N2lJHOZX6Kez5Y1gDSJla6jo7Q=
+golang.org/x/term v0.38.0/go.mod h1:bSEAKrOT1W+VSu9TSCMtoGEOUcKxOKgl3LE5QEF/xVg=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
 golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
-golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
-golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
+golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs=
+golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ=
-golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ=
+golang.org/x/tools v0.40.0 h1:yLkxfA+Qnul4cs9QA3KnlFu0lVmd8JJfoq+E41uSutA=
+golang.org/x/tools v0.40.0/go.mod h1:Ik/tzLRlbscWpqqMRjyWYDisX8bG13FrdXp3o4Sr9lc=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

From 79b0b86560b5617010c11a0083a383e46b66f471 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 4 Jan 2026 14:53:47 +0000
Subject: [PATCH 203/439] chore(deps): bump golangci/golangci-lint-action in
 /scripts (#17759)

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 9.0.0 to 9.2.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/0a35821d5c230e903fcfe077583637dea1b27b47...1e7e51e771db61008b38414a730f564565cf7c20)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-version: 9.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 scripts/golangci-lint.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/golangci-lint.yml b/scripts/golangci-lint.yml
index 2736e69b78..ae5fdc80ec 100644
--- a/scripts/golangci-lint.yml
+++ b/scripts/golangci-lint.yml
@@ -38,7 +38,7 @@ jobs:
         id: golangci-lint-version
         run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT
       - name: Lint
-        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
+        uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0
         with:
           args: --verbose
           version: ${{ steps.golangci-lint-version.outputs.version }}

From a946d2c8effb14d6a99a2c5a57a27973288a443d Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 4 Jan 2026 15:46:03 +0000
Subject: [PATCH 204/439] chore(deps): bump golangci/golangci-lint-action from
 9.0.0 to 9.2.0 (#17757)

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 9.0.0 to 9.2.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/0a35821d5c230e903fcfe077583637dea1b27b47...1e7e51e771db61008b38414a730f564565cf7c20)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-version: 9.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 .github/workflows/ci.yml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 50a5b2a43c..22b8b55a26 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -237,18 +237,18 @@ jobs:
         id: golangci-lint-version
         run: echo "version=$(make print-golangci-lint-version)" >> $GITHUB_OUTPUT
       - name: Lint
-        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
+        uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0
         with:
           args: --verbose
           version: ${{ steps.golangci-lint-version.outputs.version }}
       - name: Lint with slicelabels
-        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
+        uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0
         with:
           # goexperiment.synctest to ensure we don't miss files that depend on it.
           args: --verbose --build-tags=slicelabels,goexperiment.synctest
           version: ${{ steps.golangci-lint-version.outputs.version }}
       - name: Lint with dedupelabels
-        uses: golangci/golangci-lint-action@0a35821d5c230e903fcfe077583637dea1b27b47 # v9.0.0
+        uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0
         with:
           args: --verbose --build-tags=dedupelabels
           version: ${{ steps.golangci-lint-version.outputs.version }}

From 87401302f6009350e5ce7c8d9ed402453965dc31 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 4 Jan 2026 15:47:58 +0000
Subject: [PATCH 205/439] chore(deps): bump google.golang.org/api from 0.257.0
 to 0.258.0 (#17761)

Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.257.0 to 0.258.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.257.0...v0.258.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.258.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---
 go.mod                                |  2 +-
 go.sum                                |  4 ++--
 storage/remote/googleiam/googleiam.go | 11 ++++++++++-
 3 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/go.mod b/go.mod
index c8d1250cca..808b391c45 100644
--- a/go.mod
+++ b/go.mod
@@ -90,7 +90,7 @@ require (
 	golang.org/x/sync v0.19.0
 	golang.org/x/sys v0.39.0
 	golang.org/x/text v0.32.0
-	google.golang.org/api v0.257.0
+	google.golang.org/api v0.258.0
 	google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b
 	google.golang.org/grpc v1.78.0
 	google.golang.org/protobuf v1.36.11
diff --git a/go.sum b/go.sum
index 823a33c02d..bbe0ea9129 100644
--- a/go.sum
+++ b/go.sum
@@ -706,8 +706,8 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T
 golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
 gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
-google.golang.org/api v0.257.0 h1:8Y0lzvHlZps53PEaw+G29SsQIkuKrumGWs9puiexNAA=
-google.golang.org/api v0.257.0/go.mod h1:4eJrr+vbVaZSqs7vovFd1Jb/A6ml6iw2e6FBYf3GAO4=
+google.golang.org/api v0.258.0 h1:IKo1j5FBlN74fe5isA2PVozN3Y5pwNKriEgAXPOkDAc=
+google.golang.org/api v0.258.0/go.mod h1:qhOMTQEZ6lUps63ZNq9jhODswwjkjYYguA7fA3TBFww=
 google.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4=
 google.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s=
 google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b h1:uA40e2M6fYRBf0+8uN5mLlqUtV192iiksiICIBkYJ1E=
diff --git a/storage/remote/googleiam/googleiam.go b/storage/remote/googleiam/googleiam.go
index acf3bd5a68..0555458d69 100644
--- a/storage/remote/googleiam/googleiam.go
+++ b/storage/remote/googleiam/googleiam.go
@@ -19,6 +19,7 @@ import (
 	"context"
 	"fmt"
 	"net/http"
+	"os"
 
 	"golang.org/x/oauth2/google"
 	"google.golang.org/api/option"
@@ -41,7 +42,15 @@ func NewRoundTripper(cfg *Config, next http.RoundTripper) (http.RoundTripper, er
 		option.WithScopes(scopes),
 	}
 	if cfg.CredentialsFile != "" {
-		opts = append(opts, option.WithCredentialsFile(cfg.CredentialsFile))
+		credBytes, err := os.ReadFile(cfg.CredentialsFile)
+		if err != nil {
+			return nil, fmt.Errorf("error reading Google credentials file: %w", err)
+		}
+		creds, err := google.CredentialsFromJSON(ctx, credBytes, scopes)
+		if err != nil {
+			return nil, fmt.Errorf("error parsing Google credentials file: %w", err)
+		}
+		opts = append(opts, option.WithCredentials(creds))
 	} else {
 		creds, err := google.FindDefaultCredentials(ctx, scopes)
 		if err != nil {

From a35e19e6cff5878ca0384f9a8da98b112429c45c Mon Sep 17 00:00:00 2001
From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Date: Mon, 5 Jan 2026 11:00:44 +0100
Subject: [PATCH 206/439] Makefile.common: Add check for future copyright years

Add validation in common-check_license to detect and reject copyright
headers with years 2026 or later. This enforces the removal of copyright
dates as per https://github.com/prometheus/proposals/blob/main/proposals/0050-remove-copyright-dates.md

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---
 Makefile.common | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Makefile.common b/Makefile.common
index 998da23093..f70443272f 100644
--- a/Makefile.common
+++ b/Makefile.common
@@ -129,6 +129,12 @@ common-check_license:
                echo "license header checking failed:"; echo "$${licRes}"; \
                exit 1; \
        fi
+	@echo ">> checking for copyright years 2026 or later"
+	@futureYearRes=$$(git grep -E 'Copyright (202[6-9]|20[3-9][0-9])' -- '*.go' ':!:vendor/*' || true); \
+	if [ -n "$${futureYearRes}" ]; then \
+		echo "Files with copyright year 2026 or later found (should use 'Copyright The Prometheus Authors'):"; echo "$${futureYearRes}"; \
+		exit 1; \
+	fi
 
 .PHONY: common-deps
 common-deps:

From e14795bbf4fbd1837a9e3428aafb40b4b1dda99d Mon Sep 17 00:00:00 2001
From: Ben Kochie 
Date: Mon, 5 Jan 2026 13:46:21 +0100
Subject: [PATCH 207/439] Remove copyright date from headers (#17785)

Remove copyright dates from various files as part of [PROM-50].

[PROM-50]: https://github.com/prometheus/proposals/blob/main/proposals/0050-remove-copyright-dates.md

Signed-off-by: SuperQ 
---
 Makefile                                                      | 2 +-
 Makefile.common                                               | 2 +-
 cmd/prometheus/main.go                                        | 2 +-
 cmd/prometheus/main_test.go                                   | 2 +-
 cmd/prometheus/main_unix_test.go                              | 2 +-
 cmd/prometheus/query_log_test.go                              | 2 +-
 cmd/prometheus/reload_test.go                                 | 2 +-
 cmd/prometheus/scrape_failure_log_test.go                     | 2 +-
 cmd/prometheus/upload_test.go                                 | 2 +-
 cmd/promtool/analyze.go                                       | 2 +-
 cmd/promtool/analyze_test.go                                  | 2 +-
 cmd/promtool/archive.go                                       | 2 +-
 cmd/promtool/backfill.go                                      | 2 +-
 cmd/promtool/backfill_test.go                                 | 2 +-
 cmd/promtool/debug.go                                         | 2 +-
 cmd/promtool/main.go                                          | 2 +-
 cmd/promtool/main_test.go                                     | 2 +-
 cmd/promtool/metrics.go                                       | 2 +-
 cmd/promtool/metrics_test.go                                  | 2 +-
 cmd/promtool/query.go                                         | 2 +-
 cmd/promtool/rules.go                                         | 2 +-
 cmd/promtool/rules_test.go                                    | 2 +-
 cmd/promtool/sd.go                                            | 2 +-
 cmd/promtool/sd_test.go                                       | 2 +-
 cmd/promtool/tsdb.go                                          | 2 +-
 cmd/promtool/tsdb_posix_test.go                               | 2 +-
 cmd/promtool/tsdb_test.go                                     | 2 +-
 cmd/promtool/unittest.go                                      | 2 +-
 cmd/promtool/unittest_test.go                                 | 2 +-
 config/config.go                                              | 2 +-
 config/config_default_test.go                                 | 2 +-
 config/config_test.go                                         | 2 +-
 config/config_windows_test.go                                 | 2 +-
 config/reload.go                                              | 2 +-
 config/reload_test.go                                         | 2 +-
 discovery/aws/ec2.go                                          | 2 +-
 discovery/aws/ec2_test.go                                     | 2 +-
 discovery/aws/lightsail.go                                    | 2 +-
 discovery/aws/metrics_ec2.go                                  | 2 +-
 discovery/aws/metrics_lightsail.go                            | 2 +-
 discovery/azure/azure.go                                      | 2 +-
 discovery/azure/azure_test.go                                 | 2 +-
 discovery/azure/metrics.go                                    | 2 +-
 discovery/consul/consul.go                                    | 2 +-
 discovery/consul/consul_test.go                               | 2 +-
 discovery/consul/metrics.go                                   | 2 +-
 discovery/digitalocean/digitalocean.go                        | 2 +-
 discovery/digitalocean/digitalocean_test.go                   | 2 +-
 discovery/digitalocean/metrics.go                             | 2 +-
 discovery/digitalocean/mock_test.go                           | 2 +-
 discovery/discoverer_metrics_noop.go                          | 2 +-
 discovery/discovery.go                                        | 2 +-
 discovery/discovery_test.go                                   | 2 +-
 discovery/dns/dns.go                                          | 2 +-
 discovery/dns/dns_test.go                                     | 2 +-
 discovery/dns/metrics.go                                      | 2 +-
 discovery/eureka/client.go                                    | 2 +-
 discovery/eureka/client_test.go                               | 2 +-
 discovery/eureka/eureka.go                                    | 2 +-
 discovery/eureka/eureka_test.go                               | 2 +-
 discovery/eureka/metrics.go                                   | 2 +-
 discovery/file/file.go                                        | 2 +-
 discovery/file/file_test.go                                   | 2 +-
 discovery/file/metrics.go                                     | 2 +-
 discovery/gce/gce.go                                          | 2 +-
 discovery/gce/metrics.go                                      | 2 +-
 discovery/hetzner/hcloud.go                                   | 2 +-
 discovery/hetzner/hcloud_test.go                              | 2 +-
 discovery/hetzner/hetzner.go                                  | 2 +-
 discovery/hetzner/metrics.go                                  | 2 +-
 discovery/hetzner/mock_test.go                                | 2 +-
 discovery/hetzner/robot.go                                    | 2 +-
 discovery/hetzner/robot_test.go                               | 2 +-
 discovery/http/http.go                                        | 2 +-
 discovery/http/http_test.go                                   | 2 +-
 discovery/http/metrics.go                                     | 2 +-
 discovery/install/install.go                                  | 2 +-
 discovery/ionos/ionos.go                                      | 2 +-
 discovery/ionos/metrics.go                                    | 2 +-
 discovery/ionos/server.go                                     | 2 +-
 discovery/ionos/server_test.go                                | 2 +-
 discovery/kubernetes/endpoints.go                             | 2 +-
 discovery/kubernetes/endpoints_test.go                        | 2 +-
 discovery/kubernetes/endpointslice.go                         | 2 +-
 discovery/kubernetes/endpointslice_test.go                    | 2 +-
 discovery/kubernetes/ingress.go                               | 2 +-
 discovery/kubernetes/ingress_test.go                          | 2 +-
 discovery/kubernetes/kubernetes.go                            | 2 +-
 discovery/kubernetes/kubernetes_test.go                       | 2 +-
 discovery/kubernetes/metrics.go                               | 2 +-
 discovery/kubernetes/node.go                                  | 2 +-
 discovery/kubernetes/node_test.go                             | 2 +-
 discovery/kubernetes/pod.go                                   | 2 +-
 discovery/kubernetes/pod_test.go                              | 2 +-
 discovery/kubernetes/service.go                               | 2 +-
 discovery/kubernetes/service_test.go                          | 2 +-
 discovery/linode/linode.go                                    | 2 +-
 discovery/linode/linode_test.go                               | 2 +-
 discovery/linode/metrics.go                                   | 2 +-
 discovery/linode/mock_test.go                                 | 2 +-
 discovery/manager.go                                          | 2 +-
 discovery/manager_test.go                                     | 2 +-
 discovery/marathon/marathon.go                                | 2 +-
 discovery/marathon/marathon_test.go                           | 2 +-
 discovery/marathon/metrics.go                                 | 2 +-
 discovery/metrics.go                                          | 2 +-
 discovery/metrics_k8s_client.go                               | 2 +-
 discovery/metrics_refresh.go                                  | 2 +-
 discovery/moby/docker.go                                      | 2 +-
 discovery/moby/docker_test.go                                 | 2 +-
 discovery/moby/dockerswarm.go                                 | 2 +-
 discovery/moby/metrics_docker.go                              | 2 +-
 discovery/moby/metrics_dockerswarm.go                         | 2 +-
 discovery/moby/mock_test.go                                   | 2 +-
 discovery/moby/network.go                                     | 2 +-
 discovery/moby/nodes.go                                       | 2 +-
 discovery/moby/nodes_test.go                                  | 2 +-
 discovery/moby/services.go                                    | 2 +-
 discovery/moby/services_test.go                               | 2 +-
 discovery/moby/tasks.go                                       | 2 +-
 discovery/moby/tasks_test.go                                  | 2 +-
 discovery/nomad/metrics.go                                    | 2 +-
 discovery/nomad/nomad.go                                      | 2 +-
 discovery/nomad/nomad_test.go                                 | 2 +-
 discovery/openstack/hypervisor.go                             | 2 +-
 discovery/openstack/hypervisor_test.go                        | 2 +-
 discovery/openstack/instance.go                               | 2 +-
 discovery/openstack/instance_test.go                          | 2 +-
 discovery/openstack/loadbalancer.go                           | 2 +-
 discovery/openstack/loadbalancer_test.go                      | 2 +-
 discovery/openstack/metrics.go                                | 2 +-
 discovery/openstack/mock_test.go                              | 2 +-
 discovery/openstack/openstack.go                              | 2 +-
 discovery/ovhcloud/dedicated_server.go                        | 2 +-
 discovery/ovhcloud/dedicated_server_test.go                   | 2 +-
 discovery/ovhcloud/metrics.go                                 | 2 +-
 discovery/ovhcloud/ovhcloud.go                                | 2 +-
 discovery/ovhcloud/ovhcloud_test.go                           | 2 +-
 discovery/ovhcloud/vps.go                                     | 2 +-
 discovery/ovhcloud/vps_test.go                                | 2 +-
 discovery/puppetdb/metrics.go                                 | 2 +-
 discovery/puppetdb/puppetdb.go                                | 2 +-
 discovery/puppetdb/puppetdb_test.go                           | 2 +-
 discovery/puppetdb/resources.go                               | 2 +-
 discovery/refresh/refresh.go                                  | 2 +-
 discovery/refresh/refresh_test.go                             | 2 +-
 discovery/registry.go                                         | 2 +-
 discovery/scaleway/baremetal.go                               | 2 +-
 discovery/scaleway/instance.go                                | 2 +-
 discovery/scaleway/instance_test.go                           | 2 +-
 discovery/scaleway/metrics.go                                 | 2 +-
 discovery/scaleway/scaleway.go                                | 2 +-
 discovery/stackit/metrics.go                                  | 2 +-
 discovery/stackit/mock_test.go                                | 2 +-
 discovery/stackit/server.go                                   | 2 +-
 discovery/stackit/server_test.go                              | 2 +-
 discovery/stackit/stackit.go                                  | 2 +-
 discovery/stackit/types.go                                    | 2 +-
 discovery/targetgroup/targetgroup.go                          | 2 +-
 discovery/targetgroup/targetgroup_test.go                     | 2 +-
 discovery/triton/metrics.go                                   | 2 +-
 discovery/triton/triton.go                                    | 2 +-
 discovery/triton/triton_test.go                               | 2 +-
 discovery/util.go                                             | 2 +-
 discovery/uyuni/metrics.go                                    | 2 +-
 discovery/uyuni/uyuni.go                                      | 2 +-
 discovery/uyuni/uyuni_test.go                                 | 2 +-
 discovery/vultr/metrics.go                                    | 2 +-
 discovery/vultr/mock_test.go                                  | 2 +-
 discovery/vultr/vultr.go                                      | 2 +-
 discovery/vultr/vultr_test.go                                 | 2 +-
 discovery/xds/client.go                                       | 2 +-
 discovery/xds/client_test.go                                  | 2 +-
 discovery/xds/kuma.go                                         | 2 +-
 discovery/xds/kuma_mads.pb.go                                 | 2 +-
 discovery/xds/kuma_test.go                                    | 2 +-
 discovery/xds/metrics.go                                      | 2 +-
 discovery/xds/xds.go                                          | 2 +-
 discovery/xds/xds_test.go                                     | 2 +-
 discovery/zookeeper/zookeeper.go                              | 2 +-
 discovery/zookeeper/zookeeper_test.go                         | 2 +-
 documentation/examples/Makefile                               | 2 +-
 documentation/examples/custom-sd/adapter-usage/main.go        | 2 +-
 documentation/examples/custom-sd/adapter/adapter.go           | 2 +-
 documentation/examples/custom-sd/adapter/adapter_test.go      | 2 +-
 documentation/examples/remote_storage/Makefile                | 2 +-
 .../examples/remote_storage/example_write_adapter/server.go   | 2 +-
 .../remote_storage/remote_storage_adapter/graphite/client.go  | 2 +-
 .../remote_storage_adapter/graphite/client_test.go            | 2 +-
 .../remote_storage/remote_storage_adapter/graphite/escape.go  | 2 +-
 .../remote_storage/remote_storage_adapter/influxdb/client.go  | 2 +-
 .../remote_storage_adapter/influxdb/client_test.go            | 2 +-
 .../examples/remote_storage/remote_storage_adapter/main.go    | 2 +-
 .../remote_storage/remote_storage_adapter/opentsdb/client.go  | 2 +-
 .../remote_storage_adapter/opentsdb/client_test.go            | 2 +-
 .../remote_storage_adapter/opentsdb/tagvalue.go               | 2 +-
 .../remote_storage_adapter/opentsdb/tagvalue_test.go          | 2 +-
 internal/tools/tools.go                                       | 2 +-
 model/exemplar/exemplar.go                                    | 2 +-
 model/histogram/float_histogram.go                            | 2 +-
 model/histogram/float_histogram_test.go                       | 2 +-
 model/histogram/generic.go                                    | 2 +-
 model/histogram/generic_test.go                               | 2 +-
 model/histogram/histogram.go                                  | 2 +-
 model/histogram/histogram_test.go                             | 2 +-
 model/histogram/test_utils.go                                 | 2 +-
 model/labels/labels_common.go                                 | 2 +-
 model/labels/labels_dedupelabels.go                           | 2 +-
 model/labels/labels_dedupelabels_test.go                      | 2 +-
 model/labels/labels_slicelabels.go                            | 2 +-
 model/labels/labels_slicelabels_test.go                       | 2 +-
 model/labels/labels_stringlabels.go                           | 2 +-
 model/labels/labels_stringlabels_test.go                      | 2 +-
 model/labels/labels_test.go                                   | 2 +-
 model/labels/matcher.go                                       | 2 +-
 model/labels/matcher_test.go                                  | 2 +-
 model/labels/regexp.go                                        | 2 +-
 model/labels/regexp_test.go                                   | 2 +-
 model/labels/sharding.go                                      | 2 +-
 model/labels/sharding_dedupelabels.go                         | 2 +-
 model/labels/sharding_stringlabels.go                         | 2 +-
 model/labels/sharding_test.go                                 | 2 +-
 model/labels/test_utils.go                                    | 2 +-
 model/relabel/relabel.go                                      | 2 +-
 model/relabel/relabel_test.go                                 | 2 +-
 model/rulefmt/rulefmt.go                                      | 2 +-
 model/rulefmt/rulefmt_test.go                                 | 2 +-
 model/textparse/benchmark_test.go                             | 2 +-
 model/textparse/interface.go                                  | 2 +-
 model/textparse/interface_test.go                             | 2 +-
 model/textparse/nhcbparse.go                                  | 2 +-
 model/textparse/nhcbparse_test.go                             | 2 +-
 model/textparse/openmetricsparse.go                           | 2 +-
 model/textparse/openmetricsparse_test.go                      | 2 +-
 model/textparse/promparse.go                                  | 2 +-
 model/textparse/promparse_test.go                             | 2 +-
 model/textparse/protobufparse.go                              | 2 +-
 model/textparse/protobufparse_test.go                         | 2 +-
 model/timestamp/timestamp.go                                  | 2 +-
 model/value/value.go                                          | 2 +-
 notifier/alert.go                                             | 2 +-
 notifier/alertmanager.go                                      | 2 +-
 notifier/alertmanager_test.go                                 | 2 +-
 notifier/alertmanagerset.go                                   | 2 +-
 notifier/manager.go                                           | 2 +-
 notifier/manager_test.go                                      | 2 +-
 notifier/metric.go                                            | 2 +-
 notifier/util.go                                              | 2 +-
 notifier/util_test.go                                         | 2 +-
 plugins/generate.go                                           | 4 ++--
 plugins/minimum.go                                            | 2 +-
 plugins/plugins.go                                            | 2 +-
 prompb/codec.go                                               | 2 +-
 prompb/custom.go                                              | 2 +-
 prompb/io/prometheus/client/decoder.go                        | 2 +-
 prompb/io/prometheus/client/decoder_test.go                   | 2 +-
 prompb/io/prometheus/write/v2/codec.go                        | 2 +-
 prompb/io/prometheus/write/v2/custom.go                       | 2 +-
 prompb/io/prometheus/write/v2/custom_test.go                  | 2 +-
 prompb/io/prometheus/write/v2/symbols.go                      | 2 +-
 prompb/io/prometheus/write/v2/symbols_test.go                 | 2 +-
 prompb/io/prometheus/write/v2/types_test.go                   | 2 +-
 prompb/rwcommon/codec_test.go                                 | 2 +-
 promql/bench_test.go                                          | 2 +-
 promql/durations.go                                           | 2 +-
 promql/durations_test.go                                      | 2 +-
 promql/engine.go                                              | 2 +-
 promql/engine_internal_test.go                                | 2 +-
 promql/engine_test.go                                         | 2 +-
 promql/functions.go                                           | 2 +-
 promql/functions_internal_test.go                             | 2 +-
 promql/functions_test.go                                      | 2 +-
 promql/fuzz.go                                                | 2 +-
 promql/fuzz_test.go                                           | 2 +-
 promql/histogram_stats_iterator.go                            | 2 +-
 promql/histogram_stats_iterator_test.go                       | 2 +-
 promql/info.go                                                | 2 +-
 promql/parser/ast.go                                          | 2 +-
 promql/parser/functions.go                                    | 2 +-
 promql/parser/lex.go                                          | 2 +-
 promql/parser/lex_test.go                                     | 2 +-
 promql/parser/parse.go                                        | 2 +-
 promql/parser/parse_test.go                                   | 2 +-
 promql/parser/posrange/posrange.go                            | 2 +-
 promql/parser/prettier.go                                     | 2 +-
 promql/parser/prettier_test.go                                | 2 +-
 promql/parser/printer.go                                      | 2 +-
 promql/parser/printer_test.go                                 | 2 +-
 promql/parser/value.go                                        | 2 +-
 promql/promql_test.go                                         | 2 +-
 promql/promqltest/cmd/migrate/main.go                         | 2 +-
 promql/promqltest/test.go                                     | 2 +-
 promql/promqltest/test_migrate.go                             | 2 +-
 promql/promqltest/test_migrate_test.go                        | 2 +-
 promql/promqltest/test_test.go                                | 2 +-
 promql/quantile.go                                            | 2 +-
 promql/quantile_test.go                                       | 2 +-
 promql/query_logger.go                                        | 2 +-
 promql/query_logger_test.go                                   | 2 +-
 promql/value.go                                               | 2 +-
 promql/value_test.go                                          | 2 +-
 rules/alerting.go                                             | 2 +-
 rules/alerting_test.go                                        | 2 +-
 rules/group.go                                                | 2 +-
 rules/group_test.go                                           | 2 +-
 rules/manager.go                                              | 2 +-
 rules/manager_test.go                                         | 2 +-
 rules/origin.go                                               | 2 +-
 rules/origin_test.go                                          | 2 +-
 rules/recording.go                                            | 2 +-
 rules/recording_test.go                                       | 2 +-
 rules/rule.go                                                 | 2 +-
 schema/labels.go                                              | 2 +-
 schema/labels_test.go                                         | 2 +-
 scrape/clientprotobuf.go                                      | 2 +-
 scrape/metrics.go                                             | 2 +-
 storage/buffer.go                                             | 2 +-
 storage/buffer_test.go                                        | 2 +-
 storage/errors.go                                             | 2 +-
 storage/errors_test.go                                        | 2 +-
 storage/fanout.go                                             | 2 +-
 storage/fanout_test.go                                        | 2 +-
 storage/generic.go                                            | 2 +-
 storage/interface.go                                          | 2 +-
 storage/interface_test.go                                     | 2 +-
 storage/lazy.go                                               | 2 +-
 storage/memoized_iterator.go                                  | 2 +-
 storage/memoized_iterator_test.go                             | 2 +-
 storage/merge.go                                              | 2 +-
 storage/merge_test.go                                         | 2 +-
 storage/noop.go                                               | 2 +-
 storage/remote/azuread/azuread.go                             | 2 +-
 storage/remote/azuread/azuread_test.go                        | 2 +-
 storage/remote/chunked.go                                     | 2 +-
 storage/remote/chunked_test.go                                | 2 +-
 storage/remote/client.go                                      | 2 +-
 storage/remote/client_test.go                                 | 2 +-
 storage/remote/codec.go                                       | 2 +-
 storage/remote/codec_test.go                                  | 2 +-
 storage/remote/dial_context.go                                | 2 +-
 storage/remote/dial_context_test.go                           | 2 +-
 storage/remote/ewma.go                                        | 2 +-
 storage/remote/googleiam/googleiam.go                         | 2 +-
 storage/remote/intern.go                                      | 2 +-
 storage/remote/intern_test.go                                 | 2 +-
 storage/remote/max_timestamp.go                               | 2 +-
 storage/remote/metadata_watcher.go                            | 2 +-
 storage/remote/metadata_watcher_test.go                       | 2 +-
 .../prometheusremotewrite/combined_appender_test.go           | 2 +-
 .../remote/otlptranslator/prometheusremotewrite/context.go    | 2 +-
 .../otlptranslator/prometheusremotewrite/context_test.go      | 2 +-
 storage/remote/otlptranslator/prometheusremotewrite/helper.go | 2 +-
 .../otlptranslator/prometheusremotewrite/helper_test.go       | 2 +-
 .../remote/otlptranslator/prometheusremotewrite/histograms.go | 2 +-
 .../otlptranslator/prometheusremotewrite/histograms_test.go   | 2 +-
 .../otlptranslator/prometheusremotewrite/metrics_to_prw.go    | 2 +-
 .../prometheusremotewrite/metrics_to_prw_test.go              | 2 +-
 .../prometheusremotewrite/number_data_points.go               | 2 +-
 .../prometheusremotewrite/number_data_points_test.go          | 2 +-
 .../prometheusremotewrite/otlp_to_openmetrics_metadata.go     | 2 +-
 .../otlptranslator/prometheusremotewrite/testutil_test.go     | 2 +-
 storage/remote/queue_manager.go                               | 2 +-
 storage/remote/queue_manager_test.go                          | 2 +-
 storage/remote/read.go                                        | 2 +-
 storage/remote/read_handler.go                                | 2 +-
 storage/remote/read_handler_test.go                           | 2 +-
 storage/remote/read_test.go                                   | 2 +-
 storage/remote/stats.go                                       | 2 +-
 storage/remote/storage.go                                     | 2 +-
 storage/remote/storage_test.go                                | 2 +-
 storage/remote/write.go                                       | 2 +-
 storage/remote/write_handler.go                               | 2 +-
 storage/remote/write_handler_test.go                          | 2 +-
 storage/remote/write_test.go                                  | 2 +-
 storage/secondary.go                                          | 2 +-
 storage/series.go                                             | 2 +-
 storage/series_test.go                                        | 2 +-
 template/template.go                                          | 2 +-
 template/template_amd64_test.go                               | 2 +-
 template/template_test.go                                     | 2 +-
 tracing/tracing.go                                            | 2 +-
 tracing/tracing_test.go                                       | 2 +-
 tsdb/agent/db.go                                              | 2 +-
 tsdb/agent/db_test.go                                         | 2 +-
 tsdb/agent/series.go                                          | 2 +-
 tsdb/agent/series_test.go                                     | 2 +-
 tsdb/block.go                                                 | 2 +-
 tsdb/block_test.go                                            | 2 +-
 tsdb/blockwriter.go                                           | 2 +-
 tsdb/blockwriter_test.go                                      | 2 +-
 tsdb/chunkenc/bstream.go                                      | 2 +-
 tsdb/chunkenc/bstream_test.go                                 | 2 +-
 tsdb/chunkenc/chunk.go                                        | 2 +-
 tsdb/chunkenc/chunk_test.go                                   | 2 +-
 tsdb/chunkenc/float_histogram.go                              | 2 +-
 tsdb/chunkenc/float_histogram_test.go                         | 2 +-
 tsdb/chunkenc/histogram.go                                    | 2 +-
 tsdb/chunkenc/histogram_meta.go                               | 2 +-
 tsdb/chunkenc/histogram_meta_test.go                          | 2 +-
 tsdb/chunkenc/histogram_test.go                               | 2 +-
 tsdb/chunkenc/varbit.go                                       | 2 +-
 tsdb/chunkenc/varbit_test.go                                  | 2 +-
 tsdb/chunkenc/xor.go                                          | 2 +-
 tsdb/chunkenc/xor_test.go                                     | 2 +-
 tsdb/chunks/chunk_write_queue.go                              | 2 +-
 tsdb/chunks/chunk_write_queue_test.go                         | 2 +-
 tsdb/chunks/chunks.go                                         | 2 +-
 tsdb/chunks/chunks_test.go                                    | 2 +-
 tsdb/chunks/head_chunks.go                                    | 2 +-
 tsdb/chunks/head_chunks_other.go                              | 2 +-
 tsdb/chunks/head_chunks_test.go                               | 2 +-
 tsdb/chunks/head_chunks_windows.go                            | 2 +-
 tsdb/chunks/queue.go                                          | 2 +-
 tsdb/chunks/queue_test.go                                     | 2 +-
 tsdb/chunks/samples.go                                        | 2 +-
 tsdb/compact.go                                               | 2 +-
 tsdb/compact_test.go                                          | 2 +-
 tsdb/db.go                                                    | 2 +-
 tsdb/db_test.go                                               | 2 +-
 tsdb/encoding/encoding.go                                     | 2 +-
 tsdb/errors/errors.go                                         | 2 +-
 tsdb/errors/errors_test.go                                    | 2 +-
 tsdb/example_test.go                                          | 2 +-
 tsdb/exemplar.go                                              | 2 +-
 tsdb/exemplar_test.go                                         | 2 +-
 tsdb/fileutil/dir.go                                          | 2 +-
 tsdb/fileutil/dir_unix.go                                     | 2 +-
 tsdb/fileutil/dir_windows.go                                  | 2 +-
 tsdb/fileutil/direct_io.go                                    | 2 +-
 tsdb/fileutil/direct_io_force.go                              | 2 +-
 tsdb/fileutil/direct_io_linux.go                              | 2 +-
 tsdb/fileutil/direct_io_unsupported.go                        | 2 +-
 tsdb/fileutil/direct_io_writer.go                             | 2 +-
 tsdb/fileutil/direct_io_writer_test.go                        | 2 +-
 tsdb/fileutil/fileutil.go                                     | 2 +-
 tsdb/fileutil/flock.go                                        | 2 +-
 tsdb/fileutil/flock_js.go                                     | 2 +-
 tsdb/fileutil/flock_plan9.go                                  | 2 +-
 tsdb/fileutil/flock_solaris.go                                | 2 +-
 tsdb/fileutil/flock_test.go                                   | 2 +-
 tsdb/fileutil/flock_unix.go                                   | 2 +-
 tsdb/fileutil/flock_windows.go                                | 2 +-
 tsdb/fileutil/mmap.go                                         | 2 +-
 tsdb/fileutil/mmap_386.go                                     | 2 +-
 tsdb/fileutil/mmap_amd64.go                                   | 2 +-
 tsdb/fileutil/mmap_arm64.go                                   | 2 +-
 tsdb/fileutil/mmap_js.go                                      | 2 +-
 tsdb/fileutil/mmap_unix.go                                    | 2 +-
 tsdb/fileutil/mmap_windows.go                                 | 2 +-
 tsdb/fileutil/preallocate.go                                  | 2 +-
 tsdb/fileutil/preallocate_darwin.go                           | 2 +-
 tsdb/fileutil/preallocate_linux.go                            | 2 +-
 tsdb/fileutil/preallocate_other.go                            | 2 +-
 tsdb/fileutil/sync.go                                         | 2 +-
 tsdb/fileutil/sync_darwin.go                                  | 2 +-
 tsdb/fileutil/sync_linux.go                                   | 2 +-
 tsdb/goversion/goversion.go                                   | 2 +-
 tsdb/goversion/goversion_test.go                              | 2 +-
 tsdb/goversion/init.go                                        | 2 +-
 tsdb/head.go                                                  | 2 +-
 tsdb/head_append.go                                           | 2 +-
 tsdb/head_bench_test.go                                       | 2 +-
 tsdb/head_dedupelabels.go                                     | 2 +-
 tsdb/head_other.go                                            | 2 +-
 tsdb/head_read.go                                             | 2 +-
 tsdb/head_read_test.go                                        | 2 +-
 tsdb/head_test.go                                             | 2 +-
 tsdb/head_wal.go                                              | 2 +-
 tsdb/index/index.go                                           | 2 +-
 tsdb/index/index_test.go                                      | 2 +-
 tsdb/index/postings.go                                        | 2 +-
 tsdb/index/postings_test.go                                   | 2 +-
 tsdb/index/postingsstats.go                                   | 2 +-
 tsdb/index/postingsstats_test.go                              | 2 +-
 tsdb/isolation.go                                             | 2 +-
 tsdb/isolation_test.go                                        | 2 +-
 tsdb/mocks_test.go                                            | 2 +-
 tsdb/ooo_head.go                                              | 2 +-
 tsdb/ooo_head_read.go                                         | 2 +-
 tsdb/ooo_head_read_test.go                                    | 2 +-
 tsdb/ooo_head_test.go                                         | 2 +-
 tsdb/ooo_isolation.go                                         | 2 +-
 tsdb/ooo_isolation_test.go                                    | 2 +-
 tsdb/querier.go                                               | 2 +-
 tsdb/querier_bench_test.go                                    | 2 +-
 tsdb/querier_test.go                                          | 2 +-
 tsdb/record/record.go                                         | 2 +-
 tsdb/record/record_test.go                                    | 2 +-
 tsdb/repair.go                                                | 2 +-
 tsdb/repair_test.go                                           | 2 +-
 tsdb/testutil.go                                              | 2 +-
 tsdb/tombstones/tombstones.go                                 | 2 +-
 tsdb/tombstones/tombstones_test.go                            | 2 +-
 tsdb/tsdbblockutil.go                                         | 2 +-
 tsdb/tsdbutil/dir_locker.go                                   | 2 +-
 tsdb/tsdbutil/dir_locker_test.go                              | 2 +-
 tsdb/tsdbutil/dir_locker_testutil.go                          | 2 +-
 tsdb/tsdbutil/histogram.go                                    | 2 +-
 tsdb/wlog/checkpoint.go                                       | 2 +-
 tsdb/wlog/checkpoint_test.go                                  | 2 +-
 tsdb/wlog/live_reader.go                                      | 2 +-
 tsdb/wlog/reader.go                                           | 2 +-
 tsdb/wlog/reader_test.go                                      | 2 +-
 tsdb/wlog/watcher.go                                          | 2 +-
 tsdb/wlog/watcher_test.go                                     | 2 +-
 tsdb/wlog/wlog.go                                             | 2 +-
 tsdb/wlog/wlog_test.go                                        | 2 +-
 util/almost/almost.go                                         | 2 +-
 util/almost/almost_test.go                                    | 2 +-
 util/annotations/annotations.go                               | 2 +-
 util/compression/buffers.go                                   | 2 +-
 util/compression/compression.go                               | 2 +-
 util/compression/compression_test.go                          | 2 +-
 util/convertnhcb/convertnhcb.go                               | 2 +-
 util/convertnhcb/convertnhcb_test.go                          | 2 +-
 util/documentcli/documentcli.go                               | 2 +-
 util/fmtutil/format.go                                        | 2 +-
 util/fmtutil/format_test.go                                   | 2 +-
 util/gate/gate.go                                             | 2 +-
 util/httputil/compression.go                                  | 2 +-
 util/httputil/compression_test.go                             | 2 +-
 util/httputil/context.go                                      | 2 +-
 util/httputil/cors.go                                         | 2 +-
 util/httputil/cors_test.go                                    | 2 +-
 util/jsonutil/marshal.go                                      | 2 +-
 util/junitxml/junitxml.go                                     | 2 +-
 util/junitxml/junitxml_test.go                                | 2 +-
 util/logging/dedupe.go                                        | 2 +-
 util/logging/dedupe_test.go                                   | 2 +-
 util/logging/file.go                                          | 2 +-
 util/logging/file_test.go                                     | 2 +-
 util/namevalidationutil/namevalidationutil.go                 | 2 +-
 util/namevalidationutil/namevalidationutil_test.go            | 2 +-
 util/netconnlimit/netconnlimit.go                             | 2 +-
 util/netconnlimit/netconnlimit_test.go                        | 2 +-
 util/notifications/notifications.go                           | 2 +-
 util/notifications/notifications_test.go                      | 2 +-
 util/osutil/hostname.go                                       | 2 +-
 util/pool/pool.go                                             | 2 +-
 util/pool/pool_test.go                                        | 2 +-
 util/runtime/limits_default.go                                | 2 +-
 util/runtime/limits_windows.go                                | 2 +-
 util/runtime/statfs.go                                        | 2 +-
 util/runtime/statfs_default.go                                | 2 +-
 util/runtime/statfs_linux_386.go                              | 2 +-
 util/runtime/statfs_uint32.go                                 | 2 +-
 util/runtime/uname_default.go                                 | 2 +-
 util/runtime/uname_linux.go                                   | 2 +-
 util/runtime/vmlimits_default.go                              | 2 +-
 util/runtime/vmlimits_openbsd.go                              | 2 +-
 util/runutil/runutil.go                                       | 2 +-
 util/stats/query_stats.go                                     | 2 +-
 util/stats/stats_test.go                                      | 2 +-
 util/stats/timer.go                                           | 2 +-
 util/strutil/quote.go                                         | 2 +-
 util/strutil/quote_test.go                                    | 2 +-
 util/strutil/strconv.go                                       | 2 +-
 util/strutil/strconv_test.go                                  | 2 +-
 util/teststorage/storage.go                                   | 2 +-
 util/testutil/cmp.go                                          | 2 +-
 util/testutil/context.go                                      | 2 +-
 util/testutil/directory.go                                    | 2 +-
 util/testutil/port.go                                         | 2 +-
 util/testutil/roundtrip.go                                    | 2 +-
 util/testutil/synctest/disabled.go                            | 2 +-
 util/testutil/synctest/enabled.go                             | 2 +-
 util/testutil/synctest/synctest.go                            | 2 +-
 util/treecache/treecache.go                                   | 2 +-
 util/zeropool/pool.go                                         | 2 +-
 util/zeropool/pool_test.go                                    | 2 +-
 web/api/v1/api.go                                             | 2 +-
 web/api/v1/api_test.go                                        | 2 +-
 web/api/v1/codec.go                                           | 2 +-
 web/api/v1/codec_test.go                                      | 2 +-
 web/api/v1/errors_test.go                                     | 2 +-
 web/api/v1/json_codec.go                                      | 2 +-
 web/api/v1/json_codec_test.go                                 | 2 +-
 web/api/v1/translate_ast.go                                   | 2 +-
 web/federate.go                                               | 2 +-
 web/federate_test.go                                          | 2 +-
 web/ui/assets_embed.go                                        | 2 +-
 web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go | 2 +-
 web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go | 2 +-
 web/ui/module/lezer-promql/src/highlight.js                   | 2 +-
 web/ui/module/lezer-promql/src/tokens.js                      | 2 +-
 web/ui/ui.go                                                  | 2 +-
 web/web.go                                                    | 2 +-
 web/web_test.go                                               | 2 +-
 588 files changed, 589 insertions(+), 589 deletions(-)

diff --git a/Makefile b/Makefile
index 197fd17c19..bc5d67da6b 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,4 @@
-# Copyright 2018 The Prometheus Authors
+# Copyright The Prometheus Authors
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
diff --git a/Makefile.common b/Makefile.common
index f70443272f..7beae6e58f 100644
--- a/Makefile.common
+++ b/Makefile.common
@@ -1,4 +1,4 @@
-# Copyright 2018 The Prometheus Authors
+# Copyright The Prometheus Authors
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go
index cb6541607e..c330671b1e 100644
--- a/cmd/prometheus/main.go
+++ b/cmd/prometheus/main.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/prometheus/main_test.go b/cmd/prometheus/main_test.go
index 2a1c9816b8..6765bae900 100644
--- a/cmd/prometheus/main_test.go
+++ b/cmd/prometheus/main_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/prometheus/main_unix_test.go b/cmd/prometheus/main_unix_test.go
index 66bfe9b60a..ea130b3bf9 100644
--- a/cmd/prometheus/main_unix_test.go
+++ b/cmd/prometheus/main_unix_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/prometheus/query_log_test.go b/cmd/prometheus/query_log_test.go
index 645ac31145..5e5a9ac3b7 100644
--- a/cmd/prometheus/query_log_test.go
+++ b/cmd/prometheus/query_log_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/prometheus/reload_test.go b/cmd/prometheus/reload_test.go
index 6feb2bf3a5..bbe108c9a6 100644
--- a/cmd/prometheus/reload_test.go
+++ b/cmd/prometheus/reload_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/prometheus/scrape_failure_log_test.go b/cmd/prometheus/scrape_failure_log_test.go
index f35cb7bee6..c3f459f601 100644
--- a/cmd/prometheus/scrape_failure_log_test.go
+++ b/cmd/prometheus/scrape_failure_log_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/prometheus/upload_test.go b/cmd/prometheus/upload_test.go
index 565531b016..97a98351a7 100644
--- a/cmd/prometheus/upload_test.go
+++ b/cmd/prometheus/upload_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/analyze.go b/cmd/promtool/analyze.go
index aea72a193b..a725772f5d 100644
--- a/cmd/promtool/analyze.go
+++ b/cmd/promtool/analyze.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/analyze_test.go b/cmd/promtool/analyze_test.go
index 3de4283a15..d2e81da2c8 100644
--- a/cmd/promtool/analyze_test.go
+++ b/cmd/promtool/analyze_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/archive.go b/cmd/promtool/archive.go
index 7b565c57cc..23baea2700 100644
--- a/cmd/promtool/archive.go
+++ b/cmd/promtool/archive.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/backfill.go b/cmd/promtool/backfill.go
index 47de3b5c1c..f04a76b0a5 100644
--- a/cmd/promtool/backfill.go
+++ b/cmd/promtool/backfill.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/backfill_test.go b/cmd/promtool/backfill_test.go
index 8a599510a9..499b90e99a 100644
--- a/cmd/promtool/backfill_test.go
+++ b/cmd/promtool/backfill_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/debug.go b/cmd/promtool/debug.go
index 6383aaface..b6e82ef981 100644
--- a/cmd/promtool/debug.go
+++ b/cmd/promtool/debug.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/main.go b/cmd/promtool/main.go
index d379d6e587..16cc40233a 100644
--- a/cmd/promtool/main.go
+++ b/cmd/promtool/main.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/main_test.go b/cmd/promtool/main_test.go
index 094852a01b..4f4ca3de71 100644
--- a/cmd/promtool/main_test.go
+++ b/cmd/promtool/main_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/metrics.go b/cmd/promtool/metrics.go
index c21ef15fd8..b1a2beb72e 100644
--- a/cmd/promtool/metrics.go
+++ b/cmd/promtool/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/metrics_test.go b/cmd/promtool/metrics_test.go
index 938f1cadfd..d5a3bf63cc 100644
--- a/cmd/promtool/metrics_test.go
+++ b/cmd/promtool/metrics_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/query.go b/cmd/promtool/query.go
index 0d7cb12cf4..1342f148f8 100644
--- a/cmd/promtool/query.go
+++ b/cmd/promtool/query.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/rules.go b/cmd/promtool/rules.go
index 98f2c38b58..3960206f6b 100644
--- a/cmd/promtool/rules.go
+++ b/cmd/promtool/rules.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/rules_test.go b/cmd/promtool/rules_test.go
index 6fe7d8c5a1..678e2b4d50 100644
--- a/cmd/promtool/rules_test.go
+++ b/cmd/promtool/rules_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/sd.go b/cmd/promtool/sd.go
index 884864205c..6b844c699a 100644
--- a/cmd/promtool/sd.go
+++ b/cmd/promtool/sd.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/sd_test.go b/cmd/promtool/sd_test.go
index e41c9893b2..9f43764f55 100644
--- a/cmd/promtool/sd_test.go
+++ b/cmd/promtool/sd_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/tsdb.go b/cmd/promtool/tsdb.go
index 914d13289a..9ccd1da714 100644
--- a/cmd/promtool/tsdb.go
+++ b/cmd/promtool/tsdb.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/tsdb_posix_test.go b/cmd/promtool/tsdb_posix_test.go
index 8a83aead70..9d0034844f 100644
--- a/cmd/promtool/tsdb_posix_test.go
+++ b/cmd/promtool/tsdb_posix_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/tsdb_test.go b/cmd/promtool/tsdb_test.go
index 286456fee3..3a2a5aff72 100644
--- a/cmd/promtool/tsdb_test.go
+++ b/cmd/promtool/tsdb_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/unittest.go b/cmd/promtool/unittest.go
index 944ffc9d7c..105e626eba 100644
--- a/cmd/promtool/unittest.go
+++ b/cmd/promtool/unittest.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/cmd/promtool/unittest_test.go b/cmd/promtool/unittest_test.go
index bf4de02ccd..32886fc4df 100644
--- a/cmd/promtool/unittest_test.go
+++ b/cmd/promtool/unittest_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/config/config.go b/config/config.go
index 51a8cefe3b..cce8fc4168 100644
--- a/config/config.go
+++ b/config/config.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/config/config_default_test.go b/config/config_default_test.go
index e5f43e1f50..91c290ae4e 100644
--- a/config/config_default_test.go
+++ b/config/config_default_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/config/config_test.go b/config/config_test.go
index 1804f4925e..aefdd5248c 100644
--- a/config/config_test.go
+++ b/config/config_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/config/config_windows_test.go b/config/config_windows_test.go
index 9d338b99e7..72a56ff41a 100644
--- a/config/config_windows_test.go
+++ b/config/config_windows_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/config/reload.go b/config/reload.go
index 07a077a6a9..a250693169 100644
--- a/config/reload.go
+++ b/config/reload.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/config/reload_test.go b/config/reload_test.go
index 3e77260ab3..cb60d47651 100644
--- a/config/reload_test.go
+++ b/config/reload_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/aws/ec2.go b/discovery/aws/ec2.go
index 0aae35d75d..19ecebd491 100644
--- a/discovery/aws/ec2.go
+++ b/discovery/aws/ec2.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/aws/ec2_test.go b/discovery/aws/ec2_test.go
index 46ab8e771d..bd1047ffc0 100644
--- a/discovery/aws/ec2_test.go
+++ b/discovery/aws/ec2_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/aws/lightsail.go b/discovery/aws/lightsail.go
index c9ca3eaee9..b13f26cc5f 100644
--- a/discovery/aws/lightsail.go
+++ b/discovery/aws/lightsail.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/aws/metrics_ec2.go b/discovery/aws/metrics_ec2.go
index 45227c3534..1a37347b40 100644
--- a/discovery/aws/metrics_ec2.go
+++ b/discovery/aws/metrics_ec2.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/aws/metrics_lightsail.go b/discovery/aws/metrics_lightsail.go
index 4dfe14c60c..40f7639459 100644
--- a/discovery/aws/metrics_lightsail.go
+++ b/discovery/aws/metrics_lightsail.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/azure/azure.go b/discovery/azure/azure.go
index 3c38bbf3e6..32fc97fdfa 100644
--- a/discovery/azure/azure.go
+++ b/discovery/azure/azure.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/azure/azure_test.go b/discovery/azure/azure_test.go
index a6e3a6713b..23c120ac6b 100644
--- a/discovery/azure/azure_test.go
+++ b/discovery/azure/azure_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/azure/metrics.go b/discovery/azure/metrics.go
index 3e3dbdbfbb..dc0291cdb8 100644
--- a/discovery/azure/metrics.go
+++ b/discovery/azure/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/consul/consul.go b/discovery/consul/consul.go
index 74b5d0724e..1004d0941a 100644
--- a/discovery/consul/consul.go
+++ b/discovery/consul/consul.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/consul/consul_test.go b/discovery/consul/consul_test.go
index a6ff4a625e..feec5d4747 100644
--- a/discovery/consul/consul_test.go
+++ b/discovery/consul/consul_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/consul/metrics.go b/discovery/consul/metrics.go
index b49509bd8f..903fba5cef 100644
--- a/discovery/consul/metrics.go
+++ b/discovery/consul/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/digitalocean/digitalocean.go b/discovery/digitalocean/digitalocean.go
index d2fbee1d94..0a185c2915 100644
--- a/discovery/digitalocean/digitalocean.go
+++ b/discovery/digitalocean/digitalocean.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/digitalocean/digitalocean_test.go b/discovery/digitalocean/digitalocean_test.go
index ca99e83b20..560d8d533a 100644
--- a/discovery/digitalocean/digitalocean_test.go
+++ b/discovery/digitalocean/digitalocean_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/digitalocean/metrics.go b/discovery/digitalocean/metrics.go
index 7f68b39e56..4b11b825e5 100644
--- a/discovery/digitalocean/metrics.go
+++ b/discovery/digitalocean/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/digitalocean/mock_test.go b/discovery/digitalocean/mock_test.go
index 62d963c3b3..d5703d7702 100644
--- a/discovery/digitalocean/mock_test.go
+++ b/discovery/digitalocean/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/discoverer_metrics_noop.go b/discovery/discoverer_metrics_noop.go
index 4321204b6c..b75474dfec 100644
--- a/discovery/discoverer_metrics_noop.go
+++ b/discovery/discoverer_metrics_noop.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/discovery.go b/discovery/discovery.go
index e643cb10af..c4f8c8d458 100644
--- a/discovery/discovery.go
+++ b/discovery/discovery.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/discovery_test.go b/discovery/discovery_test.go
index 116095fd62..53539b6d40 100644
--- a/discovery/discovery_test.go
+++ b/discovery/discovery_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/dns/dns.go b/discovery/dns/dns.go
index 1e0a78698b..4d9200d734 100644
--- a/discovery/dns/dns.go
+++ b/discovery/dns/dns.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/dns/dns_test.go b/discovery/dns/dns_test.go
index 4a7170cc7d..eeb1137878 100644
--- a/discovery/dns/dns_test.go
+++ b/discovery/dns/dns_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/dns/metrics.go b/discovery/dns/metrics.go
index 27c96b53e0..b65db5e6c0 100644
--- a/discovery/dns/metrics.go
+++ b/discovery/dns/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/eureka/client.go b/discovery/eureka/client.go
index e4b54faae6..252b152637 100644
--- a/discovery/eureka/client.go
+++ b/discovery/eureka/client.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/eureka/client_test.go b/discovery/eureka/client_test.go
index f85409a11e..19812b1f5d 100644
--- a/discovery/eureka/client_test.go
+++ b/discovery/eureka/client_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/eureka/eureka.go b/discovery/eureka/eureka.go
index 6d726966bc..0d46667437 100644
--- a/discovery/eureka/eureka.go
+++ b/discovery/eureka/eureka.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/eureka/eureka_test.go b/discovery/eureka/eureka_test.go
index def6126e86..69612fedb7 100644
--- a/discovery/eureka/eureka_test.go
+++ b/discovery/eureka/eureka_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/eureka/metrics.go b/discovery/eureka/metrics.go
index 72cfe47096..5a0720a8d5 100644
--- a/discovery/eureka/metrics.go
+++ b/discovery/eureka/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/file/file.go b/discovery/file/file.go
index e0225891ce..c654297e0a 100644
--- a/discovery/file/file.go
+++ b/discovery/file/file.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/file/file_test.go b/discovery/file/file_test.go
index c80744f8c3..d8a36df399 100644
--- a/discovery/file/file_test.go
+++ b/discovery/file/file_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/file/metrics.go b/discovery/file/metrics.go
index 3e3df7bbf6..0371338d46 100644
--- a/discovery/file/metrics.go
+++ b/discovery/file/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/gce/gce.go b/discovery/gce/gce.go
index 106028ff93..96eed2b27b 100644
--- a/discovery/gce/gce.go
+++ b/discovery/gce/gce.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/gce/metrics.go b/discovery/gce/metrics.go
index 7ea69b1a89..c4020f0a53 100644
--- a/discovery/gce/metrics.go
+++ b/discovery/gce/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/hcloud.go b/discovery/hetzner/hcloud.go
index 88fe09bd3e..61869459a3 100644
--- a/discovery/hetzner/hcloud.go
+++ b/discovery/hetzner/hcloud.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/hcloud_test.go b/discovery/hetzner/hcloud_test.go
index fa8291625a..3f20bcb86c 100644
--- a/discovery/hetzner/hcloud_test.go
+++ b/discovery/hetzner/hcloud_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/hetzner.go b/discovery/hetzner/hetzner.go
index 8e52d21e39..932cfc8c93 100644
--- a/discovery/hetzner/hetzner.go
+++ b/discovery/hetzner/hetzner.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/metrics.go b/discovery/hetzner/metrics.go
index 0023018194..cab1d66a3e 100644
--- a/discovery/hetzner/metrics.go
+++ b/discovery/hetzner/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/mock_test.go b/discovery/hetzner/mock_test.go
index d192a4eae9..5f1e9c036b 100644
--- a/discovery/hetzner/mock_test.go
+++ b/discovery/hetzner/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/robot.go b/discovery/hetzner/robot.go
index 33aa2abcd8..ef5de1a30c 100644
--- a/discovery/hetzner/robot.go
+++ b/discovery/hetzner/robot.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/hetzner/robot_test.go b/discovery/hetzner/robot_test.go
index 2618bd097c..0e8b7954cc 100644
--- a/discovery/hetzner/robot_test.go
+++ b/discovery/hetzner/robot_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/http/http.go b/discovery/http/http.go
index d792bdacd7..fa9c7208fa 100644
--- a/discovery/http/http.go
+++ b/discovery/http/http.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/http/http_test.go b/discovery/http/http_test.go
index c553c21504..50a5800fc6 100644
--- a/discovery/http/http_test.go
+++ b/discovery/http/http_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/http/metrics.go b/discovery/http/metrics.go
index b1f8b84433..57fbcac15a 100644
--- a/discovery/http/metrics.go
+++ b/discovery/http/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/install/install.go b/discovery/install/install.go
index 9c397f9d36..05598347c1 100644
--- a/discovery/install/install.go
+++ b/discovery/install/install.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ionos/ionos.go b/discovery/ionos/ionos.go
index c74013d109..93d57654e8 100644
--- a/discovery/ionos/ionos.go
+++ b/discovery/ionos/ionos.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ionos/metrics.go b/discovery/ionos/metrics.go
index e79bded695..7fc78fdfa5 100644
--- a/discovery/ionos/metrics.go
+++ b/discovery/ionos/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ionos/server.go b/discovery/ionos/server.go
index 81bb497277..bd351625db 100644
--- a/discovery/ionos/server.go
+++ b/discovery/ionos/server.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ionos/server_test.go b/discovery/ionos/server_test.go
index 30f358e325..28fd285f67 100644
--- a/discovery/ionos/server_test.go
+++ b/discovery/ionos/server_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/endpoints.go b/discovery/kubernetes/endpoints.go
index 21c401da2c..4edcf9d4fa 100644
--- a/discovery/kubernetes/endpoints.go
+++ b/discovery/kubernetes/endpoints.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/endpoints_test.go b/discovery/kubernetes/endpoints_test.go
index aa0e432bfd..0ac472324d 100644
--- a/discovery/kubernetes/endpoints_test.go
+++ b/discovery/kubernetes/endpoints_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/endpointslice.go b/discovery/kubernetes/endpointslice.go
index 85b579438f..a6cfb0706a 100644
--- a/discovery/kubernetes/endpointslice.go
+++ b/discovery/kubernetes/endpointslice.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/endpointslice_test.go b/discovery/kubernetes/endpointslice_test.go
index cfd6be709e..b4dc0c36ce 100644
--- a/discovery/kubernetes/endpointslice_test.go
+++ b/discovery/kubernetes/endpointslice_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/ingress.go b/discovery/kubernetes/ingress.go
index 551453e513..985cc8f138 100644
--- a/discovery/kubernetes/ingress.go
+++ b/discovery/kubernetes/ingress.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/ingress_test.go b/discovery/kubernetes/ingress_test.go
index 76c9ff9036..15fa28002a 100644
--- a/discovery/kubernetes/ingress_test.go
+++ b/discovery/kubernetes/ingress_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/kubernetes.go b/discovery/kubernetes/kubernetes.go
index 1a6f965ecd..678f287ef5 100644
--- a/discovery/kubernetes/kubernetes.go
+++ b/discovery/kubernetes/kubernetes.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/kubernetes_test.go b/discovery/kubernetes/kubernetes_test.go
index f8edec23cb..a68a7c9a43 100644
--- a/discovery/kubernetes/kubernetes_test.go
+++ b/discovery/kubernetes/kubernetes_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/metrics.go b/discovery/kubernetes/metrics.go
index ba3cb1d32a..cdf158a032 100644
--- a/discovery/kubernetes/metrics.go
+++ b/discovery/kubernetes/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/node.go b/discovery/kubernetes/node.go
index 131cdcc9e7..cbc69dd0ca 100644
--- a/discovery/kubernetes/node.go
+++ b/discovery/kubernetes/node.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/node_test.go b/discovery/kubernetes/node_test.go
index bc17efdc01..9e56b95bb9 100644
--- a/discovery/kubernetes/node_test.go
+++ b/discovery/kubernetes/node_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/pod.go b/discovery/kubernetes/pod.go
index 03089e39d4..1fed78b3a7 100644
--- a/discovery/kubernetes/pod.go
+++ b/discovery/kubernetes/pod.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/pod_test.go b/discovery/kubernetes/pod_test.go
index 2cf336774a..db5db546d0 100644
--- a/discovery/kubernetes/pod_test.go
+++ b/discovery/kubernetes/pod_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/service.go b/discovery/kubernetes/service.go
index d676490d6c..ac2d42fc7c 100644
--- a/discovery/kubernetes/service.go
+++ b/discovery/kubernetes/service.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/kubernetes/service_test.go b/discovery/kubernetes/service_test.go
index 43c2b7922d..56a785d9c2 100644
--- a/discovery/kubernetes/service_test.go
+++ b/discovery/kubernetes/service_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/linode/linode.go b/discovery/linode/linode.go
index 2dc4d5f796..a5f05600c1 100644
--- a/discovery/linode/linode.go
+++ b/discovery/linode/linode.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/linode/linode_test.go b/discovery/linode/linode_test.go
index 533bc0fb62..d795d29698 100644
--- a/discovery/linode/linode_test.go
+++ b/discovery/linode/linode_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/linode/metrics.go b/discovery/linode/metrics.go
index 8f81389226..5bc805a60e 100644
--- a/discovery/linode/metrics.go
+++ b/discovery/linode/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/linode/mock_test.go b/discovery/linode/mock_test.go
index 50f0572ecd..b8094ec211 100644
--- a/discovery/linode/mock_test.go
+++ b/discovery/linode/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/manager.go b/discovery/manager.go
index 431050aa0b..3f2b2db652 100644
--- a/discovery/manager.go
+++ b/discovery/manager.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/manager_test.go b/discovery/manager_test.go
index 5d34cb7ac0..162730d9aa 100644
--- a/discovery/manager_test.go
+++ b/discovery/manager_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/marathon/marathon.go b/discovery/marathon/marathon.go
index 438b8915df..878d404373 100644
--- a/discovery/marathon/marathon.go
+++ b/discovery/marathon/marathon.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/marathon/marathon_test.go b/discovery/marathon/marathon_test.go
index 53f7d3a1f9..71c7d73d7e 100644
--- a/discovery/marathon/marathon_test.go
+++ b/discovery/marathon/marathon_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/marathon/metrics.go b/discovery/marathon/metrics.go
index 40e2ade558..3d3d57d9ae 100644
--- a/discovery/marathon/metrics.go
+++ b/discovery/marathon/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/metrics.go b/discovery/metrics.go
index 356be1ddcb..2a3734fb2d 100644
--- a/discovery/metrics.go
+++ b/discovery/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/metrics_k8s_client.go b/discovery/metrics_k8s_client.go
index 19dfd4e247..3642eac568 100644
--- a/discovery/metrics_k8s_client.go
+++ b/discovery/metrics_k8s_client.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/metrics_refresh.go b/discovery/metrics_refresh.go
index 9f3eb27b49..11092d9f96 100644
--- a/discovery/metrics_refresh.go
+++ b/discovery/metrics_refresh.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/docker.go b/discovery/moby/docker.go
index ec1187278b..aa1cd2eb42 100644
--- a/discovery/moby/docker.go
+++ b/discovery/moby/docker.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/docker_test.go b/discovery/moby/docker_test.go
index 88c832db1b..effdf90b36 100644
--- a/discovery/moby/docker_test.go
+++ b/discovery/moby/docker_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/dockerswarm.go b/discovery/moby/dockerswarm.go
index 2761e891b5..5cb12279d8 100644
--- a/discovery/moby/dockerswarm.go
+++ b/discovery/moby/dockerswarm.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/metrics_docker.go b/discovery/moby/metrics_docker.go
index 716f52b60a..8c2518a75e 100644
--- a/discovery/moby/metrics_docker.go
+++ b/discovery/moby/metrics_docker.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/metrics_dockerswarm.go b/discovery/moby/metrics_dockerswarm.go
index 17dd30d1b3..e4682b032a 100644
--- a/discovery/moby/metrics_dockerswarm.go
+++ b/discovery/moby/metrics_dockerswarm.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/mock_test.go b/discovery/moby/mock_test.go
index 2450ca4436..e43319494d 100644
--- a/discovery/moby/mock_test.go
+++ b/discovery/moby/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/network.go b/discovery/moby/network.go
index ea1ca66bc7..02db2b8a12 100644
--- a/discovery/moby/network.go
+++ b/discovery/moby/network.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/nodes.go b/discovery/moby/nodes.go
index a11afeee25..76e090c803 100644
--- a/discovery/moby/nodes.go
+++ b/discovery/moby/nodes.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/nodes_test.go b/discovery/moby/nodes_test.go
index c65b9411ed..1f97016297 100644
--- a/discovery/moby/nodes_test.go
+++ b/discovery/moby/nodes_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/services.go b/discovery/moby/services.go
index 0698c01e6a..558d544e25 100644
--- a/discovery/moby/services.go
+++ b/discovery/moby/services.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/services_test.go b/discovery/moby/services_test.go
index 95702ced9b..eb5c75c71e 100644
--- a/discovery/moby/services_test.go
+++ b/discovery/moby/services_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/tasks.go b/discovery/moby/tasks.go
index 8a3dbe8101..d4e3678ee5 100644
--- a/discovery/moby/tasks.go
+++ b/discovery/moby/tasks.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/moby/tasks_test.go b/discovery/moby/tasks_test.go
index 3f38135096..60453990c4 100644
--- a/discovery/moby/tasks_test.go
+++ b/discovery/moby/tasks_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/nomad/metrics.go b/discovery/nomad/metrics.go
index 9707153d91..0e5dca4723 100644
--- a/discovery/nomad/metrics.go
+++ b/discovery/nomad/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/nomad/nomad.go b/discovery/nomad/nomad.go
index f2971fb01b..da558f54d9 100644
--- a/discovery/nomad/nomad.go
+++ b/discovery/nomad/nomad.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/nomad/nomad_test.go b/discovery/nomad/nomad_test.go
index 099a347cbf..3a4963e24b 100644
--- a/discovery/nomad/nomad_test.go
+++ b/discovery/nomad/nomad_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/hypervisor.go b/discovery/openstack/hypervisor.go
index e7a6362052..141b77c706 100644
--- a/discovery/openstack/hypervisor.go
+++ b/discovery/openstack/hypervisor.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/hypervisor_test.go b/discovery/openstack/hypervisor_test.go
index e4a97f32cf..afba84af2d 100644
--- a/discovery/openstack/hypervisor_test.go
+++ b/discovery/openstack/hypervisor_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/instance.go b/discovery/openstack/instance.go
index 58bf154555..2a6a777e9a 100644
--- a/discovery/openstack/instance.go
+++ b/discovery/openstack/instance.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/instance_test.go b/discovery/openstack/instance_test.go
index 0933b57067..aa202cddff 100644
--- a/discovery/openstack/instance_test.go
+++ b/discovery/openstack/instance_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/loadbalancer.go b/discovery/openstack/loadbalancer.go
index 254b713cdd..3b2def0d6a 100644
--- a/discovery/openstack/loadbalancer.go
+++ b/discovery/openstack/loadbalancer.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/loadbalancer_test.go b/discovery/openstack/loadbalancer_test.go
index eee21b9831..68be323a5a 100644
--- a/discovery/openstack/loadbalancer_test.go
+++ b/discovery/openstack/loadbalancer_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/metrics.go b/discovery/openstack/metrics.go
index 664f5ea6bc..01e7ab3add 100644
--- a/discovery/openstack/metrics.go
+++ b/discovery/openstack/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/mock_test.go b/discovery/openstack/mock_test.go
index 34e09c710f..c44dadfbc0 100644
--- a/discovery/openstack/mock_test.go
+++ b/discovery/openstack/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/openstack/openstack.go b/discovery/openstack/openstack.go
index 61dff847cf..ce365e6cd0 100644
--- a/discovery/openstack/openstack.go
+++ b/discovery/openstack/openstack.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/dedicated_server.go b/discovery/ovhcloud/dedicated_server.go
index 2035e92c91..e892607c34 100644
--- a/discovery/ovhcloud/dedicated_server.go
+++ b/discovery/ovhcloud/dedicated_server.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/dedicated_server_test.go b/discovery/ovhcloud/dedicated_server_test.go
index 686fa7ef3f..84fa2c4c12 100644
--- a/discovery/ovhcloud/dedicated_server_test.go
+++ b/discovery/ovhcloud/dedicated_server_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/metrics.go b/discovery/ovhcloud/metrics.go
index 18492c0ab4..dbcfe130e9 100644
--- a/discovery/ovhcloud/metrics.go
+++ b/discovery/ovhcloud/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/ovhcloud.go b/discovery/ovhcloud/ovhcloud.go
index df150b8ce4..863fcfeaf9 100644
--- a/discovery/ovhcloud/ovhcloud.go
+++ b/discovery/ovhcloud/ovhcloud.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/ovhcloud_test.go b/discovery/ovhcloud/ovhcloud_test.go
index 8f2272b746..acb1c43fad 100644
--- a/discovery/ovhcloud/ovhcloud_test.go
+++ b/discovery/ovhcloud/ovhcloud_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/vps.go b/discovery/ovhcloud/vps.go
index 4e71a877bc..4023c4ff49 100644
--- a/discovery/ovhcloud/vps.go
+++ b/discovery/ovhcloud/vps.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/ovhcloud/vps_test.go b/discovery/ovhcloud/vps_test.go
index 051d52e85e..d997f2bb0e 100644
--- a/discovery/ovhcloud/vps_test.go
+++ b/discovery/ovhcloud/vps_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/puppetdb/metrics.go b/discovery/puppetdb/metrics.go
index 83e7975ed5..5a8e9736c2 100644
--- a/discovery/puppetdb/metrics.go
+++ b/discovery/puppetdb/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/puppetdb/puppetdb.go b/discovery/puppetdb/puppetdb.go
index db5fc2e2fb..52a1cf73c6 100644
--- a/discovery/puppetdb/puppetdb.go
+++ b/discovery/puppetdb/puppetdb.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/puppetdb/puppetdb_test.go b/discovery/puppetdb/puppetdb_test.go
index a96310553b..b12835b47c 100644
--- a/discovery/puppetdb/puppetdb_test.go
+++ b/discovery/puppetdb/puppetdb_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/puppetdb/resources.go b/discovery/puppetdb/resources.go
index 487c471c1b..09aa43a776 100644
--- a/discovery/puppetdb/resources.go
+++ b/discovery/puppetdb/resources.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/refresh/refresh.go b/discovery/refresh/refresh.go
index 0613fd6c6d..3e766d1c84 100644
--- a/discovery/refresh/refresh.go
+++ b/discovery/refresh/refresh.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/refresh/refresh_test.go b/discovery/refresh/refresh_test.go
index 385c256932..e227d0abc9 100644
--- a/discovery/refresh/refresh_test.go
+++ b/discovery/refresh/refresh_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/registry.go b/discovery/registry.go
index b3b82cdeec..04145e72e4 100644
--- a/discovery/registry.go
+++ b/discovery/registry.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/scaleway/baremetal.go b/discovery/scaleway/baremetal.go
index 06f13532df..347ed40bab 100644
--- a/discovery/scaleway/baremetal.go
+++ b/discovery/scaleway/baremetal.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/scaleway/instance.go b/discovery/scaleway/instance.go
index 162a75e407..c0ed5853b3 100644
--- a/discovery/scaleway/instance.go
+++ b/discovery/scaleway/instance.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/scaleway/instance_test.go b/discovery/scaleway/instance_test.go
index b67b858ae0..2d0f7a67ff 100644
--- a/discovery/scaleway/instance_test.go
+++ b/discovery/scaleway/instance_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/scaleway/metrics.go b/discovery/scaleway/metrics.go
index d7a4e78556..5871f7e31b 100644
--- a/discovery/scaleway/metrics.go
+++ b/discovery/scaleway/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/scaleway/scaleway.go b/discovery/scaleway/scaleway.go
index 16a9835848..f8ef6c706c 100644
--- a/discovery/scaleway/scaleway.go
+++ b/discovery/scaleway/scaleway.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/stackit/metrics.go b/discovery/stackit/metrics.go
index 5ba565eb9c..a44d0728e3 100644
--- a/discovery/stackit/metrics.go
+++ b/discovery/stackit/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/stackit/mock_test.go b/discovery/stackit/mock_test.go
index 59641ce2bc..d1366508a3 100644
--- a/discovery/stackit/mock_test.go
+++ b/discovery/stackit/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/stackit/server.go b/discovery/stackit/server.go
index 1be834a689..c553d9b3f3 100644
--- a/discovery/stackit/server.go
+++ b/discovery/stackit/server.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/stackit/server_test.go b/discovery/stackit/server_test.go
index 117fbdd66d..afb9460851 100644
--- a/discovery/stackit/server_test.go
+++ b/discovery/stackit/server_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/stackit/stackit.go b/discovery/stackit/stackit.go
index 1f9bd22469..bae76c8897 100644
--- a/discovery/stackit/stackit.go
+++ b/discovery/stackit/stackit.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/stackit/types.go b/discovery/stackit/types.go
index 84b7d0266c..575acbbe56 100644
--- a/discovery/stackit/types.go
+++ b/discovery/stackit/types.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/targetgroup/targetgroup.go b/discovery/targetgroup/targetgroup.go
index 5c3b67d6e8..4b1670ae1b 100644
--- a/discovery/targetgroup/targetgroup.go
+++ b/discovery/targetgroup/targetgroup.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/targetgroup/targetgroup_test.go b/discovery/targetgroup/targetgroup_test.go
index d68e29644a..1c1583d33d 100644
--- a/discovery/targetgroup/targetgroup_test.go
+++ b/discovery/targetgroup/targetgroup_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/triton/metrics.go b/discovery/triton/metrics.go
index ea98eae452..2d4193ee1f 100644
--- a/discovery/triton/metrics.go
+++ b/discovery/triton/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/triton/triton.go b/discovery/triton/triton.go
index 209e1c4deb..b21beef9d0 100644
--- a/discovery/triton/triton.go
+++ b/discovery/triton/triton.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/triton/triton_test.go b/discovery/triton/triton_test.go
index 6cbc52d020..f2b6398bc8 100644
--- a/discovery/triton/triton_test.go
+++ b/discovery/triton/triton_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/util.go b/discovery/util.go
index 4e2a088518..064a5312a7 100644
--- a/discovery/util.go
+++ b/discovery/util.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/uyuni/metrics.go b/discovery/uyuni/metrics.go
index 85ea9d73d2..e1a9fd4db0 100644
--- a/discovery/uyuni/metrics.go
+++ b/discovery/uyuni/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/uyuni/uyuni.go b/discovery/uyuni/uyuni.go
index 0320a0490d..6f29fa130c 100644
--- a/discovery/uyuni/uyuni.go
+++ b/discovery/uyuni/uyuni.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/uyuni/uyuni_test.go b/discovery/uyuni/uyuni_test.go
index 4a73fa9ada..71f1c5afb1 100644
--- a/discovery/uyuni/uyuni_test.go
+++ b/discovery/uyuni/uyuni_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/vultr/metrics.go b/discovery/vultr/metrics.go
index 65b15eae2f..823fe4bdc0 100644
--- a/discovery/vultr/metrics.go
+++ b/discovery/vultr/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/vultr/mock_test.go b/discovery/vultr/mock_test.go
index bfc24d06fb..03e5952dd0 100644
--- a/discovery/vultr/mock_test.go
+++ b/discovery/vultr/mock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/vultr/vultr.go b/discovery/vultr/vultr.go
index 27f3e11064..b2f6bde52a 100644
--- a/discovery/vultr/vultr.go
+++ b/discovery/vultr/vultr.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/vultr/vultr_test.go b/discovery/vultr/vultr_test.go
index 8975cfb455..d116c419b7 100644
--- a/discovery/vultr/vultr_test.go
+++ b/discovery/vultr/vultr_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/client.go b/discovery/xds/client.go
index a27e060fbd..59485ffcba 100644
--- a/discovery/xds/client.go
+++ b/discovery/xds/client.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/client_test.go b/discovery/xds/client_test.go
index 7e3cd85b6c..e663902161 100644
--- a/discovery/xds/client_test.go
+++ b/discovery/xds/client_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/kuma.go b/discovery/xds/kuma.go
index 82ca8f2c9a..34bebe7765 100644
--- a/discovery/xds/kuma.go
+++ b/discovery/xds/kuma.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/kuma_mads.pb.go b/discovery/xds/kuma_mads.pb.go
index 210a5343a4..d234241453 100644
--- a/discovery/xds/kuma_mads.pb.go
+++ b/discovery/xds/kuma_mads.pb.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/kuma_test.go b/discovery/xds/kuma_test.go
index 3f8a769fe1..6620f9fac6 100644
--- a/discovery/xds/kuma_test.go
+++ b/discovery/xds/kuma_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/metrics.go b/discovery/xds/metrics.go
index bdc9598f2c..7e5be89bd3 100644
--- a/discovery/xds/metrics.go
+++ b/discovery/xds/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/xds.go b/discovery/xds/xds.go
index db55a2b6f7..29da7b7c89 100644
--- a/discovery/xds/xds.go
+++ b/discovery/xds/xds.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/xds/xds_test.go b/discovery/xds/xds_test.go
index 5a2e9d737b..c11cdd2c05 100644
--- a/discovery/xds/xds_test.go
+++ b/discovery/xds/xds_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/zookeeper/zookeeper.go b/discovery/zookeeper/zookeeper.go
index d5239324cb..6ac9b25cd6 100644
--- a/discovery/zookeeper/zookeeper.go
+++ b/discovery/zookeeper/zookeeper.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/discovery/zookeeper/zookeeper_test.go b/discovery/zookeeper/zookeeper_test.go
index de0d1f4924..ae2d23e607 100644
--- a/discovery/zookeeper/zookeeper_test.go
+++ b/discovery/zookeeper/zookeeper_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/Makefile b/documentation/examples/Makefile
index 4085155f80..8ed308899b 100644
--- a/documentation/examples/Makefile
+++ b/documentation/examples/Makefile
@@ -1,4 +1,4 @@
-# Copyright 2022 The Prometheus Authors
+# Copyright The Prometheus Authors
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
diff --git a/documentation/examples/custom-sd/adapter-usage/main.go b/documentation/examples/custom-sd/adapter-usage/main.go
index e7f7a69b5d..c0ce03cd0f 100644
--- a/documentation/examples/custom-sd/adapter-usage/main.go
+++ b/documentation/examples/custom-sd/adapter-usage/main.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/custom-sd/adapter/adapter.go b/documentation/examples/custom-sd/adapter/adapter.go
index b242c4eaa0..83f0e80c49 100644
--- a/documentation/examples/custom-sd/adapter/adapter.go
+++ b/documentation/examples/custom-sd/adapter/adapter.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/custom-sd/adapter/adapter_test.go b/documentation/examples/custom-sd/adapter/adapter_test.go
index 329ca8c29a..0ec69348d8 100644
--- a/documentation/examples/custom-sd/adapter/adapter_test.go
+++ b/documentation/examples/custom-sd/adapter/adapter_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/Makefile b/documentation/examples/remote_storage/Makefile
index e0dfd4d647..a6c8e48c45 100644
--- a/documentation/examples/remote_storage/Makefile
+++ b/documentation/examples/remote_storage/Makefile
@@ -1,4 +1,4 @@
-# Copyright 2022 The Prometheus Authors
+# Copyright The Prometheus Authors
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/example_write_adapter/server.go b/documentation/examples/remote_storage/example_write_adapter/server.go
index 21267c80e5..c2ec7184e3 100644
--- a/documentation/examples/remote_storage/example_write_adapter/server.go
+++ b/documentation/examples/remote_storage/example_write_adapter/server.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/graphite/client.go b/documentation/examples/remote_storage/remote_storage_adapter/graphite/client.go
index d04355a712..2e78354bd2 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/graphite/client.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/graphite/client.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/graphite/client_test.go b/documentation/examples/remote_storage/remote_storage_adapter/graphite/client_test.go
index 535027e076..8a96413443 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/graphite/client_test.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/graphite/client_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go b/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go
index 3793973b7b..e7357c001a 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/graphite/escape.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go b/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go
index ffd81802c1..ddf78283e7 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client_test.go b/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client_test.go
index f78d4db794..faf48045cb 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client_test.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/influxdb/client_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/main.go b/documentation/examples/remote_storage/remote_storage_adapter/main.go
index ffcbb5385a..ac891cca50 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/main.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/main.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client.go b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client.go
index ffc6c58b88..e2f64be5d8 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client_test.go b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client_test.go
index bc9703c88c..fa76cc334d 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client_test.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/client_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go
index c40f829a56..f822e37808 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue_test.go b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue_test.go
index 5adedb3248..071fd5a85a 100644
--- a/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue_test.go
+++ b/documentation/examples/remote_storage/remote_storage_adapter/opentsdb/tagvalue_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/internal/tools/tools.go b/internal/tools/tools.go
index e57e37186f..22e79a56f7 100644
--- a/internal/tools/tools.go
+++ b/internal/tools/tools.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/exemplar/exemplar.go b/model/exemplar/exemplar.go
index d03940f1b2..5db7c46a68 100644
--- a/model/exemplar/exemplar.go
+++ b/model/exemplar/exemplar.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/float_histogram.go b/model/histogram/float_histogram.go
index 0acf9cb28f..75021d2c62 100644
--- a/model/histogram/float_histogram.go
+++ b/model/histogram/float_histogram.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/float_histogram_test.go b/model/histogram/float_histogram_test.go
index e79f5a0f49..5c29544c8f 100644
--- a/model/histogram/float_histogram_test.go
+++ b/model/histogram/float_histogram_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/generic.go b/model/histogram/generic.go
index 649db769c7..61fc5067f2 100644
--- a/model/histogram/generic.go
+++ b/model/histogram/generic.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/generic_test.go b/model/histogram/generic_test.go
index 54324beaff..525c731571 100644
--- a/model/histogram/generic_test.go
+++ b/model/histogram/generic_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/histogram.go b/model/histogram/histogram.go
index aa9f696be6..5be60174fc 100644
--- a/model/histogram/histogram.go
+++ b/model/histogram/histogram.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/histogram_test.go b/model/histogram/histogram_test.go
index ae17f9be37..a2b4c7c0a8 100644
--- a/model/histogram/histogram_test.go
+++ b/model/histogram/histogram_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/histogram/test_utils.go b/model/histogram/test_utils.go
index a4871ada31..c86becdcf9 100644
--- a/model/histogram/test_utils.go
+++ b/model/histogram/test_utils.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_common.go b/model/labels/labels_common.go
index ab82ae6a8f..571064d6c4 100644
--- a/model/labels/labels_common.go
+++ b/model/labels/labels_common.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_dedupelabels.go b/model/labels/labels_dedupelabels.go
index 4518482c96..ae751fe34a 100644
--- a/model/labels/labels_dedupelabels.go
+++ b/model/labels/labels_dedupelabels.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_dedupelabels_test.go b/model/labels/labels_dedupelabels_test.go
index 229bb45a8e..b05d18e4cc 100644
--- a/model/labels/labels_dedupelabels_test.go
+++ b/model/labels/labels_dedupelabels_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_slicelabels.go b/model/labels/labels_slicelabels.go
index 71dbcd0044..2a9056e68f 100644
--- a/model/labels/labels_slicelabels.go
+++ b/model/labels/labels_slicelabels.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_slicelabels_test.go b/model/labels/labels_slicelabels_test.go
index 0e55730082..700e88fd13 100644
--- a/model/labels/labels_slicelabels_test.go
+++ b/model/labels/labels_slicelabels_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_stringlabels.go b/model/labels/labels_stringlabels.go
index 1460e7db93..c9be42bf74 100644
--- a/model/labels/labels_stringlabels.go
+++ b/model/labels/labels_stringlabels.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_stringlabels_test.go b/model/labels/labels_stringlabels_test.go
index 0704a2ff36..45b5a19f40 100644
--- a/model/labels/labels_stringlabels_test.go
+++ b/model/labels/labels_stringlabels_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/labels_test.go b/model/labels/labels_test.go
index 4be2eeb0b7..67614daf92 100644
--- a/model/labels/labels_test.go
+++ b/model/labels/labels_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/matcher.go b/model/labels/matcher.go
index a09c838e3f..6d22b1bf64 100644
--- a/model/labels/matcher.go
+++ b/model/labels/matcher.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/matcher_test.go b/model/labels/matcher_test.go
index 214bb37eff..11ed6dd29c 100644
--- a/model/labels/matcher_test.go
+++ b/model/labels/matcher_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/regexp.go b/model/labels/regexp.go
index 47b50e703a..5123bbc7dd 100644
--- a/model/labels/regexp.go
+++ b/model/labels/regexp.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/regexp_test.go b/model/labels/regexp_test.go
index 94ef14028b..2fb5e806f0 100644
--- a/model/labels/regexp_test.go
+++ b/model/labels/regexp_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/sharding.go b/model/labels/sharding.go
index ed05da675f..6394d0a01e 100644
--- a/model/labels/sharding.go
+++ b/model/labels/sharding.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/sharding_dedupelabels.go b/model/labels/sharding_dedupelabels.go
index 5bf41b05d6..11342146a8 100644
--- a/model/labels/sharding_dedupelabels.go
+++ b/model/labels/sharding_dedupelabels.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/sharding_stringlabels.go b/model/labels/sharding_stringlabels.go
index 4dcbaa21d1..776a58bb5e 100644
--- a/model/labels/sharding_stringlabels.go
+++ b/model/labels/sharding_stringlabels.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/sharding_test.go b/model/labels/sharding_test.go
index 78e3047509..8d094d780e 100644
--- a/model/labels/sharding_test.go
+++ b/model/labels/sharding_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/labels/test_utils.go b/model/labels/test_utils.go
index 66020799e9..21d1d71296 100644
--- a/model/labels/test_utils.go
+++ b/model/labels/test_utils.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/relabel/relabel.go b/model/relabel/relabel.go
index f7085037fd..6087253d11 100644
--- a/model/relabel/relabel.go
+++ b/model/relabel/relabel.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/relabel/relabel_test.go b/model/relabel/relabel_test.go
index 7ce3c86549..a3eb925995 100644
--- a/model/relabel/relabel_test.go
+++ b/model/relabel/relabel_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/rulefmt/rulefmt.go b/model/rulefmt/rulefmt.go
index 83203ba769..70541eb0d3 100644
--- a/model/rulefmt/rulefmt.go
+++ b/model/rulefmt/rulefmt.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/rulefmt/rulefmt_test.go b/model/rulefmt/rulefmt_test.go
index 45fc0f8227..ec16052bc0 100644
--- a/model/rulefmt/rulefmt_test.go
+++ b/model/rulefmt/rulefmt_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/benchmark_test.go b/model/textparse/benchmark_test.go
index 510da72c6c..cf63dad260 100644
--- a/model/textparse/benchmark_test.go
+++ b/model/textparse/benchmark_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/interface.go b/model/textparse/interface.go
index bbc52290ad..08d9a080a7 100644
--- a/model/textparse/interface.go
+++ b/model/textparse/interface.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/interface_test.go b/model/textparse/interface_test.go
index 7030544793..d0b6b293a9 100644
--- a/model/textparse/interface_test.go
+++ b/model/textparse/interface_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/nhcbparse.go b/model/textparse/nhcbparse.go
index 79441e1f75..13ce3ca988 100644
--- a/model/textparse/nhcbparse.go
+++ b/model/textparse/nhcbparse.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/nhcbparse_test.go b/model/textparse/nhcbparse_test.go
index 7e2f75ae63..9a27c16ea8 100644
--- a/model/textparse/nhcbparse_test.go
+++ b/model/textparse/nhcbparse_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/openmetricsparse.go b/model/textparse/openmetricsparse.go
index 207ceb4573..724c340546 100644
--- a/model/textparse/openmetricsparse.go
+++ b/model/textparse/openmetricsparse.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/openmetricsparse_test.go b/model/textparse/openmetricsparse_test.go
index f0bbab309e..8f6393cd53 100644
--- a/model/textparse/openmetricsparse_test.go
+++ b/model/textparse/openmetricsparse_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/promparse.go b/model/textparse/promparse.go
index 4a75bcd8d8..ada1b29013 100644
--- a/model/textparse/promparse.go
+++ b/model/textparse/promparse.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/promparse_test.go b/model/textparse/promparse_test.go
index 4e9406808f..a398067efe 100644
--- a/model/textparse/promparse_test.go
+++ b/model/textparse/promparse_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/protobufparse.go b/model/textparse/protobufparse.go
index a48aa4af69..637ae7b747 100644
--- a/model/textparse/protobufparse.go
+++ b/model/textparse/protobufparse.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/textparse/protobufparse_test.go b/model/textparse/protobufparse_test.go
index 6a16258f00..3a4f4abdda 100644
--- a/model/textparse/protobufparse_test.go
+++ b/model/textparse/protobufparse_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/timestamp/timestamp.go b/model/timestamp/timestamp.go
index 93458f644d..0f27314e57 100644
--- a/model/timestamp/timestamp.go
+++ b/model/timestamp/timestamp.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/model/value/value.go b/model/value/value.go
index 655ce852d5..fe8f50e002 100644
--- a/model/value/value.go
+++ b/model/value/value.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/alert.go b/notifier/alert.go
index 83e7a97fe0..5e6df2097b 100644
--- a/notifier/alert.go
+++ b/notifier/alert.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/alertmanager.go b/notifier/alertmanager.go
index 8bcf7954ec..a9c1e8669f 100644
--- a/notifier/alertmanager.go
+++ b/notifier/alertmanager.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/alertmanager_test.go b/notifier/alertmanager_test.go
index ea27f37be7..668271d267 100644
--- a/notifier/alertmanager_test.go
+++ b/notifier/alertmanager_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/alertmanagerset.go b/notifier/alertmanagerset.go
index b6d1b8c4aa..eca798e6f5 100644
--- a/notifier/alertmanagerset.go
+++ b/notifier/alertmanagerset.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/manager.go b/notifier/manager.go
index e37f59a250..a835cccffd 100644
--- a/notifier/manager.go
+++ b/notifier/manager.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/manager_test.go b/notifier/manager_test.go
index 64de020338..21ab0b28a1 100644
--- a/notifier/manager_test.go
+++ b/notifier/manager_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/metric.go b/notifier/metric.go
index 3f4abdda93..d10a02614c 100644
--- a/notifier/metric.go
+++ b/notifier/metric.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/util.go b/notifier/util.go
index c21c33a57b..cf9a53eda0 100644
--- a/notifier/util.go
+++ b/notifier/util.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/notifier/util_test.go b/notifier/util_test.go
index 2c1c7d241b..a9f0509ba1 100644
--- a/notifier/util_test.go
+++ b/notifier/util_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/plugins/generate.go b/plugins/generate.go
index 2c4ba410f2..c0e58ec83b 100644
--- a/plugins/generate.go
+++ b/plugins/generate.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -44,7 +44,7 @@ func main() {
 		log.Fatal(err)
 	}
 	defer f.Close()
-	_, err = f.WriteString(`// Copyright 2022 The Prometheus Authors
+	_, err = f.WriteString(`// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/plugins/minimum.go b/plugins/minimum.go
index 8541de922f..9797c2dbe2 100644
--- a/plugins/minimum.go
+++ b/plugins/minimum.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/plugins/plugins.go b/plugins/plugins.go
index 90b1407281..686fdfb325 100644
--- a/plugins/plugins.go
+++ b/plugins/plugins.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/codec.go b/prompb/codec.go
index 6cc0cdc861..9eb668a8e7 100644
--- a/prompb/codec.go
+++ b/prompb/codec.go
@@ -1,4 +1,4 @@
-// Copyright 2024 Prometheus Team
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/custom.go b/prompb/custom.go
index f73ddd446b..65f856a755 100644
--- a/prompb/custom.go
+++ b/prompb/custom.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/client/decoder.go b/prompb/io/prometheus/client/decoder.go
index 6bc9600ab6..de7184c4b5 100644
--- a/prompb/io/prometheus/client/decoder.go
+++ b/prompb/io/prometheus/client/decoder.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/client/decoder_test.go b/prompb/io/prometheus/client/decoder_test.go
index b28fe43db9..0b210c7c0f 100644
--- a/prompb/io/prometheus/client/decoder_test.go
+++ b/prompb/io/prometheus/client/decoder_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/write/v2/codec.go b/prompb/io/prometheus/write/v2/codec.go
index 71196edb88..ae4d0f635a 100644
--- a/prompb/io/prometheus/write/v2/codec.go
+++ b/prompb/io/prometheus/write/v2/codec.go
@@ -1,4 +1,4 @@
-// Copyright 2024 Prometheus Team
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/write/v2/custom.go b/prompb/io/prometheus/write/v2/custom.go
index 5721aec532..4063cf32ed 100644
--- a/prompb/io/prometheus/write/v2/custom.go
+++ b/prompb/io/prometheus/write/v2/custom.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/write/v2/custom_test.go b/prompb/io/prometheus/write/v2/custom_test.go
index 139cbfb225..30715477cb 100644
--- a/prompb/io/prometheus/write/v2/custom_test.go
+++ b/prompb/io/prometheus/write/v2/custom_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/write/v2/symbols.go b/prompb/io/prometheus/write/v2/symbols.go
index 7c7feca239..292801a185 100644
--- a/prompb/io/prometheus/write/v2/symbols.go
+++ b/prompb/io/prometheus/write/v2/symbols.go
@@ -1,4 +1,4 @@
-// Copyright 2024 Prometheus Team
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/write/v2/symbols_test.go b/prompb/io/prometheus/write/v2/symbols_test.go
index 7e7c7cb0bd..d0f335665a 100644
--- a/prompb/io/prometheus/write/v2/symbols_test.go
+++ b/prompb/io/prometheus/write/v2/symbols_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 Prometheus Team
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/io/prometheus/write/v2/types_test.go b/prompb/io/prometheus/write/v2/types_test.go
index 5b7622fc2f..12528943a1 100644
--- a/prompb/io/prometheus/write/v2/types_test.go
+++ b/prompb/io/prometheus/write/v2/types_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 Prometheus Team
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/prompb/rwcommon/codec_test.go b/prompb/rwcommon/codec_test.go
index 73a8196fa8..2e0a72eff9 100644
--- a/prompb/rwcommon/codec_test.go
+++ b/prompb/rwcommon/codec_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 Prometheus Team
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/bench_test.go b/promql/bench_test.go
index 37c8311305..f647b03600 100644
--- a/promql/bench_test.go
+++ b/promql/bench_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/durations.go b/promql/durations.go
index 216dd02725..c660dbf464 100644
--- a/promql/durations.go
+++ b/promql/durations.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/durations_test.go b/promql/durations_test.go
index 7a5e8f00a4..e9759af0dd 100644
--- a/promql/durations_test.go
+++ b/promql/durations_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/engine.go b/promql/engine.go
index a9f0dd2952..11a7ad22ec 100644
--- a/promql/engine.go
+++ b/promql/engine.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/engine_internal_test.go b/promql/engine_internal_test.go
index 4c5d532cbc..f040f53e61 100644
--- a/promql/engine_internal_test.go
+++ b/promql/engine_internal_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/engine_test.go b/promql/engine_test.go
index 208ac4f89d..7b7a67a54b 100644
--- a/promql/engine_test.go
+++ b/promql/engine_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/functions.go b/promql/functions.go
index f844bf5ada..3f2079aba0 100644
--- a/promql/functions.go
+++ b/promql/functions.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/functions_internal_test.go b/promql/functions_internal_test.go
index 24d9a44e04..e5cd839459 100644
--- a/promql/functions_internal_test.go
+++ b/promql/functions_internal_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/functions_test.go b/promql/functions_test.go
index 8dd91e7537..2566843092 100644
--- a/promql/functions_test.go
+++ b/promql/functions_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/fuzz.go b/promql/fuzz.go
index a71a63f8eb..f9cc4794a6 100644
--- a/promql/fuzz.go
+++ b/promql/fuzz.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/fuzz_test.go b/promql/fuzz_test.go
index 4a26798ded..a24da48e63 100644
--- a/promql/fuzz_test.go
+++ b/promql/fuzz_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/histogram_stats_iterator.go b/promql/histogram_stats_iterator.go
index e58cc7d848..87cc5acfbd 100644
--- a/promql/histogram_stats_iterator.go
+++ b/promql/histogram_stats_iterator.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/histogram_stats_iterator_test.go b/promql/histogram_stats_iterator_test.go
index 80bfee519d..cfea8a568e 100644
--- a/promql/histogram_stats_iterator_test.go
+++ b/promql/histogram_stats_iterator_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/info.go b/promql/info.go
index d5ffda6af2..ab4250104d 100644
--- a/promql/info.go
+++ b/promql/info.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/ast.go b/promql/parser/ast.go
index 8a1a094b79..130f9aefb7 100644
--- a/promql/parser/ast.go
+++ b/promql/parser/ast.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/functions.go b/promql/parser/functions.go
index a471cb3a6d..2f2b1c68e4 100644
--- a/promql/parser/functions.go
+++ b/promql/parser/functions.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/lex.go b/promql/parser/lex.go
index ad4b685150..b3a82dc0c6 100644
--- a/promql/parser/lex.go
+++ b/promql/parser/lex.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/lex_test.go b/promql/parser/lex_test.go
index f86f282089..5c915ec74f 100644
--- a/promql/parser/lex_test.go
+++ b/promql/parser/lex_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/parse.go b/promql/parser/parse.go
index bcd511f467..817e0d02d9 100644
--- a/promql/parser/parse.go
+++ b/promql/parser/parse.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/parse_test.go b/promql/parser/parse_test.go
index 62349efd93..ab5564f0ff 100644
--- a/promql/parser/parse_test.go
+++ b/promql/parser/parse_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/posrange/posrange.go b/promql/parser/posrange/posrange.go
index f883a91bbb..c5cdc4b91b 100644
--- a/promql/parser/posrange/posrange.go
+++ b/promql/parser/posrange/posrange.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/prettier.go b/promql/parser/prettier.go
index 90fb7a0cf9..a0ab9e1219 100644
--- a/promql/parser/prettier.go
+++ b/promql/parser/prettier.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/prettier_test.go b/promql/parser/prettier_test.go
index ea9a7a1a26..8ba5134d4a 100644
--- a/promql/parser/prettier_test.go
+++ b/promql/parser/prettier_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/printer.go b/promql/parser/printer.go
index cc4f1202df..01e2c46c1b 100644
--- a/promql/parser/printer.go
+++ b/promql/parser/printer.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/printer_test.go b/promql/parser/printer_test.go
index bce4302d83..4499fa7860 100644
--- a/promql/parser/printer_test.go
+++ b/promql/parser/printer_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/parser/value.go b/promql/parser/value.go
index f882f9f0be..3c1c8571dc 100644
--- a/promql/parser/value.go
+++ b/promql/parser/value.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/promql_test.go b/promql/promql_test.go
index 92d933f1ee..fc13f7e64f 100644
--- a/promql/promql_test.go
+++ b/promql/promql_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/promqltest/cmd/migrate/main.go b/promql/promqltest/cmd/migrate/main.go
index a506f084c5..b570b1dfaa 100644
--- a/promql/promqltest/cmd/migrate/main.go
+++ b/promql/promqltest/cmd/migrate/main.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go
index 83e47f1915..1c4226b461 100644
--- a/promql/promqltest/test.go
+++ b/promql/promqltest/test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/promqltest/test_migrate.go b/promql/promqltest/test_migrate.go
index 0b233e7592..693b773b7d 100644
--- a/promql/promqltest/test_migrate.go
+++ b/promql/promqltest/test_migrate.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/promqltest/test_migrate_test.go b/promql/promqltest/test_migrate_test.go
index fcf7e9db03..6c9784b56f 100644
--- a/promql/promqltest/test_migrate_test.go
+++ b/promql/promqltest/test_migrate_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/promqltest/test_test.go b/promql/promqltest/test_test.go
index f441d148d6..cbb73a5651 100644
--- a/promql/promqltest/test_test.go
+++ b/promql/promqltest/test_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/quantile.go b/promql/quantile.go
index 78df925c51..c44eb89e68 100644
--- a/promql/quantile.go
+++ b/promql/quantile.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/quantile_test.go b/promql/quantile_test.go
index a1047d73f4..c97ff7c3c4 100644
--- a/promql/quantile_test.go
+++ b/promql/quantile_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/query_logger.go b/promql/query_logger.go
index 5923223aa0..954f8b1a5b 100644
--- a/promql/query_logger.go
+++ b/promql/query_logger.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/query_logger_test.go b/promql/query_logger_test.go
index 47a6d1a25d..8c88757bd7 100644
--- a/promql/query_logger_test.go
+++ b/promql/query_logger_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/value.go b/promql/value.go
index b909085b17..02cb021024 100644
--- a/promql/value.go
+++ b/promql/value.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/promql/value_test.go b/promql/value_test.go
index 0017b41e2c..c7454284ff 100644
--- a/promql/value_test.go
+++ b/promql/value_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/alerting.go b/rules/alerting.go
index bb0763fbc6..d94113b46b 100644
--- a/rules/alerting.go
+++ b/rules/alerting.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/alerting_test.go b/rules/alerting_test.go
index b619d56b56..a2c7abcd56 100644
--- a/rules/alerting_test.go
+++ b/rules/alerting_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/group.go b/rules/group.go
index 47afe6f715..704fd13d85 100644
--- a/rules/group.go
+++ b/rules/group.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/group_test.go b/rules/group_test.go
index ff1ef3d6c1..a110c78510 100644
--- a/rules/group_test.go
+++ b/rules/group_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/manager.go b/rules/manager.go
index d610c154be..c835a7c6e8 100644
--- a/rules/manager.go
+++ b/rules/manager.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/manager_test.go b/rules/manager_test.go
index a88be1e5d1..0991e8198a 100644
--- a/rules/manager_test.go
+++ b/rules/manager_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/origin.go b/rules/origin.go
index 695fc5f838..683568c71f 100644
--- a/rules/origin.go
+++ b/rules/origin.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/origin_test.go b/rules/origin_test.go
index 16f87de716..55ad927fd9 100644
--- a/rules/origin_test.go
+++ b/rules/origin_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/recording.go b/rules/recording.go
index 1bc41b834a..61a27aceb6 100644
--- a/rules/recording.go
+++ b/rules/recording.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/recording_test.go b/rules/recording_test.go
index 44ef257f8f..1fee5ede72 100644
--- a/rules/recording_test.go
+++ b/rules/recording_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/rules/rule.go b/rules/rule.go
index 33f1755ac5..fc88e22840 100644
--- a/rules/rule.go
+++ b/rules/rule.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/schema/labels.go b/schema/labels.go
index 05329af7f6..c71e352640 100644
--- a/schema/labels.go
+++ b/schema/labels.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/schema/labels_test.go b/schema/labels_test.go
index ae1ec9e90b..c2ba576c4a 100644
--- a/schema/labels_test.go
+++ b/schema/labels_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/scrape/clientprotobuf.go b/scrape/clientprotobuf.go
index 6dc22c959f..d84d4bebfc 100644
--- a/scrape/clientprotobuf.go
+++ b/scrape/clientprotobuf.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/scrape/metrics.go b/scrape/metrics.go
index 634c52fb2d..4662a9fd9e 100644
--- a/scrape/metrics.go
+++ b/scrape/metrics.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/buffer.go b/storage/buffer.go
index bc27948fd0..223c4fa42b 100644
--- a/storage/buffer.go
+++ b/storage/buffer.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/buffer_test.go b/storage/buffer_test.go
index 259e54d6f7..fc6603d4a5 100644
--- a/storage/buffer_test.go
+++ b/storage/buffer_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/errors.go b/storage/errors.go
index dd48066db6..4dd61e2523 100644
--- a/storage/errors.go
+++ b/storage/errors.go
@@ -1,4 +1,4 @@
-// Copyright 2014 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/errors_test.go b/storage/errors_test.go
index b3e202b49b..0e7277bf8b 100644
--- a/storage/errors_test.go
+++ b/storage/errors_test.go
@@ -1,4 +1,4 @@
-// Copyright 2014 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/fanout.go b/storage/fanout.go
index a699a97b02..246a955b73 100644
--- a/storage/fanout.go
+++ b/storage/fanout.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/fanout_test.go b/storage/fanout_test.go
index b1762ec555..ed4cf17696 100644
--- a/storage/fanout_test.go
+++ b/storage/fanout_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/generic.go b/storage/generic.go
index e5f4b4d03a..e85ac77b9c 100644
--- a/storage/generic.go
+++ b/storage/generic.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/interface.go b/storage/interface.go
index ae8bec033e..23b8b48a0c 100644
--- a/storage/interface.go
+++ b/storage/interface.go
@@ -1,4 +1,4 @@
-// Copyright 2014 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/interface_test.go b/storage/interface_test.go
index ba60721736..d28e5177e3 100644
--- a/storage/interface_test.go
+++ b/storage/interface_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/lazy.go b/storage/lazy.go
index fab974c286..2851ba7135 100644
--- a/storage/lazy.go
+++ b/storage/lazy.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/memoized_iterator.go b/storage/memoized_iterator.go
index 273b3caa1d..b248bca641 100644
--- a/storage/memoized_iterator.go
+++ b/storage/memoized_iterator.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/memoized_iterator_test.go b/storage/memoized_iterator_test.go
index 81e517f96e..1a1a5f7680 100644
--- a/storage/memoized_iterator_test.go
+++ b/storage/memoized_iterator_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/merge.go b/storage/merge.go
index f8ba1ab76a..a86a26891f 100644
--- a/storage/merge.go
+++ b/storage/merge.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/merge_test.go b/storage/merge_test.go
index 90f2097054..6e2daaeb3a 100644
--- a/storage/merge_test.go
+++ b/storage/merge_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/noop.go b/storage/noop.go
index f5092da7c7..751e6304db 100644
--- a/storage/noop.go
+++ b/storage/noop.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/azuread/azuread.go b/storage/remote/azuread/azuread.go
index 638ba586fc..fe0c4f9e21 100644
--- a/storage/remote/azuread/azuread.go
+++ b/storage/remote/azuread/azuread.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/azuread/azuread_test.go b/storage/remote/azuread/azuread_test.go
index 986a01695c..857ecdba8a 100644
--- a/storage/remote/azuread/azuread_test.go
+++ b/storage/remote/azuread/azuread_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/chunked.go b/storage/remote/chunked.go
index aa5addd6aa..b6cadf8691 100644
--- a/storage/remote/chunked.go
+++ b/storage/remote/chunked.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/chunked_test.go b/storage/remote/chunked_test.go
index 82ed866345..7493d734a3 100644
--- a/storage/remote/chunked_test.go
+++ b/storage/remote/chunked_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/client.go b/storage/remote/client.go
index 0f2b5ddca6..78405b378e 100644
--- a/storage/remote/client.go
+++ b/storage/remote/client.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/client_test.go b/storage/remote/client_test.go
index 7fb670a24d..d5f126342a 100644
--- a/storage/remote/client_test.go
+++ b/storage/remote/client_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/codec.go b/storage/remote/codec.go
index 059d5e66ce..9f0fb7d92a 100644
--- a/storage/remote/codec.go
+++ b/storage/remote/codec.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/codec_test.go b/storage/remote/codec_test.go
index ba67ff33d9..e6e7813c7b 100644
--- a/storage/remote/codec_test.go
+++ b/storage/remote/codec_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/dial_context.go b/storage/remote/dial_context.go
index b842728e4c..f7a52442ed 100644
--- a/storage/remote/dial_context.go
+++ b/storage/remote/dial_context.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/dial_context_test.go b/storage/remote/dial_context_test.go
index 5a0cd7c88c..61b929401f 100644
--- a/storage/remote/dial_context_test.go
+++ b/storage/remote/dial_context_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/ewma.go b/storage/remote/ewma.go
index ea4472c494..27ba39c35d 100644
--- a/storage/remote/ewma.go
+++ b/storage/remote/ewma.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/googleiam/googleiam.go b/storage/remote/googleiam/googleiam.go
index 0555458d69..0ca7185ab7 100644
--- a/storage/remote/googleiam/googleiam.go
+++ b/storage/remote/googleiam/googleiam.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/intern.go b/storage/remote/intern.go
index 34edeb370e..193cdf96db 100644
--- a/storage/remote/intern.go
+++ b/storage/remote/intern.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/intern_test.go b/storage/remote/intern_test.go
index f992b2ada6..fd0ebed16f 100644
--- a/storage/remote/intern_test.go
+++ b/storage/remote/intern_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/max_timestamp.go b/storage/remote/max_timestamp.go
index bb67d9bb98..61dbda6bc6 100644
--- a/storage/remote/max_timestamp.go
+++ b/storage/remote/max_timestamp.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/metadata_watcher.go b/storage/remote/metadata_watcher.go
index b1f98038fc..f231691e30 100644
--- a/storage/remote/metadata_watcher.go
+++ b/storage/remote/metadata_watcher.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/metadata_watcher_test.go b/storage/remote/metadata_watcher_test.go
index 6c4608b3dd..f911a145bc 100644
--- a/storage/remote/metadata_watcher_test.go
+++ b/storage/remote/metadata_watcher_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go
index 753112cf82..a1a17fe82b 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/combined_appender_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/context.go b/storage/remote/otlptranslator/prometheusremotewrite/context.go
index 5c6dd20f18..db3c180036 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/context.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/context.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/context_test.go b/storage/remote/otlptranslator/prometheusremotewrite/context_test.go
index 4b47964313..8aa24a8110 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/context_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/context_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/helper.go b/storage/remote/otlptranslator/prometheusremotewrite/helper.go
index aa54433836..7e3c9d5021 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/helper.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/helper.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go b/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go
index 893fe97ec4..b06bf3d416 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/helper_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go
index c93a00db76..db7c0e1275 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/histograms.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/histograms.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go b/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go
index 22e654ab9c..644ec2e01b 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/histograms_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go
index f43e4964b1..41de42548a 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go
index e409b4e8b5..8eb0029dd7 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/metrics_to_prw_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go
index 8f30dbb6b6..e3814ce095 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go
index 32435020c5..77bc212c76 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/number_data_points_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/otlp_to_openmetrics_metadata.go b/storage/remote/otlptranslator/prometheusremotewrite/otlp_to_openmetrics_metadata.go
index 49f96e0019..0292790156 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/otlp_to_openmetrics_metadata.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/otlp_to_openmetrics_metadata.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/otlptranslator/prometheusremotewrite/testutil_test.go b/storage/remote/otlptranslator/prometheusremotewrite/testutil_test.go
index 187127fcb2..5194925cfe 100644
--- a/storage/remote/otlptranslator/prometheusremotewrite/testutil_test.go
+++ b/storage/remote/otlptranslator/prometheusremotewrite/testutil_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/queue_manager.go b/storage/remote/queue_manager.go
index 5fc5f5564b..2b26179e58 100644
--- a/storage/remote/queue_manager.go
+++ b/storage/remote/queue_manager.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/queue_manager_test.go b/storage/remote/queue_manager_test.go
index 704a5628d3..f1462b4406 100644
--- a/storage/remote/queue_manager_test.go
+++ b/storage/remote/queue_manager_test.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/read.go b/storage/remote/read.go
index e21d1538f5..70b55980b8 100644
--- a/storage/remote/read.go
+++ b/storage/remote/read.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/read_handler.go b/storage/remote/read_handler.go
index 3e315a6157..a628dd34ff 100644
--- a/storage/remote/read_handler.go
+++ b/storage/remote/read_handler.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/read_handler_test.go b/storage/remote/read_handler_test.go
index 355973e4be..255a037d1e 100644
--- a/storage/remote/read_handler_test.go
+++ b/storage/remote/read_handler_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/read_test.go b/storage/remote/read_test.go
index da0b7f81d4..49f29d9001 100644
--- a/storage/remote/read_test.go
+++ b/storage/remote/read_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/stats.go b/storage/remote/stats.go
index 89d00ffc31..3a1bfed805 100644
--- a/storage/remote/stats.go
+++ b/storage/remote/stats.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/storage.go b/storage/remote/storage.go
index 648c91c955..f482597249 100644
--- a/storage/remote/storage.go
+++ b/storage/remote/storage.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/storage_test.go b/storage/remote/storage_test.go
index f567c7a80b..416468cf79 100644
--- a/storage/remote/storage_test.go
+++ b/storage/remote/storage_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/write.go b/storage/remote/write.go
index 1a036c1795..92f447d624 100644
--- a/storage/remote/write.go
+++ b/storage/remote/write.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/write_handler.go b/storage/remote/write_handler.go
index 4291b0505a..c29896b843 100644
--- a/storage/remote/write_handler.go
+++ b/storage/remote/write_handler.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/write_handler_test.go b/storage/remote/write_handler_test.go
index 705c53a149..ac75d56095 100644
--- a/storage/remote/write_handler_test.go
+++ b/storage/remote/write_handler_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/remote/write_test.go b/storage/remote/write_test.go
index 2bf317465c..099a2f1cab 100644
--- a/storage/remote/write_test.go
+++ b/storage/remote/write_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/secondary.go b/storage/secondary.go
index 1cf8024b65..a071ddcfa3 100644
--- a/storage/secondary.go
+++ b/storage/secondary.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/series.go b/storage/series.go
index 2fff56785a..7e130d494d 100644
--- a/storage/series.go
+++ b/storage/series.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/storage/series_test.go b/storage/series_test.go
index 1ade558648..954d62f1b3 100644
--- a/storage/series_test.go
+++ b/storage/series_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/template/template.go b/template/template.go
index 572e8450d3..0ea7382ed3 100644
--- a/template/template.go
+++ b/template/template.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/template/template_amd64_test.go b/template/template_amd64_test.go
index 913a7e2b81..15db39b646 100644
--- a/template/template_amd64_test.go
+++ b/template/template_amd64_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/template/template_test.go b/template/template_test.go
index f3348caae6..073300a39b 100644
--- a/template/template_test.go
+++ b/template/template_test.go
@@ -1,4 +1,4 @@
-// Copyright 2014 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tracing/tracing.go b/tracing/tracing.go
index 91ac48007b..b35673b2b4 100644
--- a/tracing/tracing.go
+++ b/tracing/tracing.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tracing/tracing_test.go b/tracing/tracing_test.go
index e735e1a18a..0840abafdf 100644
--- a/tracing/tracing_test.go
+++ b/tracing/tracing_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/agent/db.go b/tsdb/agent/db.go
index a0f7a93b6d..583af6f56b 100644
--- a/tsdb/agent/db.go
+++ b/tsdb/agent/db.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/agent/db_test.go b/tsdb/agent/db_test.go
index 94e84fa2eb..498fba4eb9 100644
--- a/tsdb/agent/db_test.go
+++ b/tsdb/agent/db_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/agent/series.go b/tsdb/agent/series.go
index 76e7342171..4eb691bfd5 100644
--- a/tsdb/agent/series.go
+++ b/tsdb/agent/series.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/agent/series_test.go b/tsdb/agent/series_test.go
index 036a80de4c..4b277b36b7 100644
--- a/tsdb/agent/series_test.go
+++ b/tsdb/agent/series_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/block.go b/tsdb/block.go
index dcbb172e72..3f089b9da7 100644
--- a/tsdb/block.go
+++ b/tsdb/block.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/block_test.go b/tsdb/block_test.go
index d02f83a9e9..855fa5638a 100644
--- a/tsdb/block_test.go
+++ b/tsdb/block_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/blockwriter.go b/tsdb/blockwriter.go
index e038812224..af83a98083 100644
--- a/tsdb/blockwriter.go
+++ b/tsdb/blockwriter.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/blockwriter_test.go b/tsdb/blockwriter_test.go
index becae6aa04..33f0e5a0f3 100644
--- a/tsdb/blockwriter_test.go
+++ b/tsdb/blockwriter_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/bstream.go b/tsdb/chunkenc/bstream.go
index 6e01798f72..abf6e4dbef 100644
--- a/tsdb/chunkenc/bstream.go
+++ b/tsdb/chunkenc/bstream.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/bstream_test.go b/tsdb/chunkenc/bstream_test.go
index 8ac45ef0b6..3098be5945 100644
--- a/tsdb/chunkenc/bstream_test.go
+++ b/tsdb/chunkenc/bstream_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/chunk.go b/tsdb/chunkenc/chunk.go
index 8cccb189fa..fed28c5701 100644
--- a/tsdb/chunkenc/chunk.go
+++ b/tsdb/chunkenc/chunk.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/chunk_test.go b/tsdb/chunkenc/chunk_test.go
index eac9e12b29..d2d0e4c053 100644
--- a/tsdb/chunkenc/chunk_test.go
+++ b/tsdb/chunkenc/chunk_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/float_histogram.go b/tsdb/chunkenc/float_histogram.go
index d960e835f2..797bc596b5 100644
--- a/tsdb/chunkenc/float_histogram.go
+++ b/tsdb/chunkenc/float_histogram.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/float_histogram_test.go b/tsdb/chunkenc/float_histogram_test.go
index d112c81f1c..f27de97516 100644
--- a/tsdb/chunkenc/float_histogram_test.go
+++ b/tsdb/chunkenc/float_histogram_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/histogram.go b/tsdb/chunkenc/histogram.go
index be1c31ae76..e05c49c81d 100644
--- a/tsdb/chunkenc/histogram.go
+++ b/tsdb/chunkenc/histogram.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/histogram_meta.go b/tsdb/chunkenc/histogram_meta.go
index 22bc4a6d3d..874e086812 100644
--- a/tsdb/chunkenc/histogram_meta.go
+++ b/tsdb/chunkenc/histogram_meta.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/histogram_meta_test.go b/tsdb/chunkenc/histogram_meta_test.go
index d3aa979b5e..3eb2a13962 100644
--- a/tsdb/chunkenc/histogram_meta_test.go
+++ b/tsdb/chunkenc/histogram_meta_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/histogram_test.go b/tsdb/chunkenc/histogram_test.go
index c11102b470..38bbd58465 100644
--- a/tsdb/chunkenc/histogram_test.go
+++ b/tsdb/chunkenc/histogram_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/varbit.go b/tsdb/chunkenc/varbit.go
index 00ba027dda..4338555328 100644
--- a/tsdb/chunkenc/varbit.go
+++ b/tsdb/chunkenc/varbit.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/varbit_test.go b/tsdb/chunkenc/varbit_test.go
index 8042b98dc1..dcb43f08df 100644
--- a/tsdb/chunkenc/varbit_test.go
+++ b/tsdb/chunkenc/varbit_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/xor.go b/tsdb/chunkenc/xor.go
index 29e2110705..bbe12a893b 100644
--- a/tsdb/chunkenc/xor.go
+++ b/tsdb/chunkenc/xor.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunkenc/xor_test.go b/tsdb/chunkenc/xor_test.go
index 609a3ac5ea..904e536b49 100644
--- a/tsdb/chunkenc/xor_test.go
+++ b/tsdb/chunkenc/xor_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/chunk_write_queue.go b/tsdb/chunks/chunk_write_queue.go
index bb9f239707..1a046ea00a 100644
--- a/tsdb/chunks/chunk_write_queue.go
+++ b/tsdb/chunks/chunk_write_queue.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/chunk_write_queue_test.go b/tsdb/chunks/chunk_write_queue_test.go
index fd81011091..489ff74210 100644
--- a/tsdb/chunks/chunk_write_queue_test.go
+++ b/tsdb/chunks/chunk_write_queue_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/chunks.go b/tsdb/chunks/chunks.go
index 8b8f5d0f81..681fceb2fb 100644
--- a/tsdb/chunks/chunks.go
+++ b/tsdb/chunks/chunks.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/chunks_test.go b/tsdb/chunks/chunks_test.go
index 6eb00f12ad..f40f996fde 100644
--- a/tsdb/chunks/chunks_test.go
+++ b/tsdb/chunks/chunks_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/head_chunks.go b/tsdb/chunks/head_chunks.go
index 5e143b8b32..ffe7e70fc6 100644
--- a/tsdb/chunks/head_chunks.go
+++ b/tsdb/chunks/head_chunks.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/head_chunks_other.go b/tsdb/chunks/head_chunks_other.go
index f30c5e55e9..42e94fc54d 100644
--- a/tsdb/chunks/head_chunks_other.go
+++ b/tsdb/chunks/head_chunks_other.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/head_chunks_test.go b/tsdb/chunks/head_chunks_test.go
index 2d7744193d..17efd44aa6 100644
--- a/tsdb/chunks/head_chunks_test.go
+++ b/tsdb/chunks/head_chunks_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/head_chunks_windows.go b/tsdb/chunks/head_chunks_windows.go
index 214ee42f59..a16d0ff38e 100644
--- a/tsdb/chunks/head_chunks_windows.go
+++ b/tsdb/chunks/head_chunks_windows.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/queue.go b/tsdb/chunks/queue.go
index 860381a5fe..454d939ce6 100644
--- a/tsdb/chunks/queue.go
+++ b/tsdb/chunks/queue.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/queue_test.go b/tsdb/chunks/queue_test.go
index ab4dd14838..377a8181ff 100644
--- a/tsdb/chunks/queue_test.go
+++ b/tsdb/chunks/queue_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/chunks/samples.go b/tsdb/chunks/samples.go
index a5b16094df..8097bcd72b 100644
--- a/tsdb/chunks/samples.go
+++ b/tsdb/chunks/samples.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/compact.go b/tsdb/compact.go
index 7ad6f8bb24..7c21cbcc13 100644
--- a/tsdb/compact.go
+++ b/tsdb/compact.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/compact_test.go b/tsdb/compact_test.go
index 2b7a52c169..29b90d9bbc 100644
--- a/tsdb/compact_test.go
+++ b/tsdb/compact_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/db.go b/tsdb/db.go
index f765710dd7..3f8bf16209 100644
--- a/tsdb/db.go
+++ b/tsdb/db.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/db_test.go b/tsdb/db_test.go
index 4612eace3b..299ade8826 100644
--- a/tsdb/db_test.go
+++ b/tsdb/db_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/encoding/encoding.go b/tsdb/encoding/encoding.go
index cc7d0990f6..a6d6fe4d44 100644
--- a/tsdb/encoding/encoding.go
+++ b/tsdb/encoding/encoding.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/errors/errors.go b/tsdb/errors/errors.go
index ded4ae3a27..138b38a8d2 100644
--- a/tsdb/errors/errors.go
+++ b/tsdb/errors/errors.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/errors/errors_test.go b/tsdb/errors/errors_test.go
index 146c66bf00..acffdea261 100644
--- a/tsdb/errors/errors_test.go
+++ b/tsdb/errors/errors_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/example_test.go b/tsdb/example_test.go
index 46deae5198..88632b69f9 100644
--- a/tsdb/example_test.go
+++ b/tsdb/example_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/exemplar.go b/tsdb/exemplar.go
index cdbcd5cde6..f0e755839c 100644
--- a/tsdb/exemplar.go
+++ b/tsdb/exemplar.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/exemplar_test.go b/tsdb/exemplar_test.go
index bf6ad2fabb..103332c886 100644
--- a/tsdb/exemplar_test.go
+++ b/tsdb/exemplar_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/dir.go b/tsdb/fileutil/dir.go
index ad039d2231..795c9f221b 100644
--- a/tsdb/fileutil/dir.go
+++ b/tsdb/fileutil/dir.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/dir_unix.go b/tsdb/fileutil/dir_unix.go
index 2afb2aeaba..05c24893cd 100644
--- a/tsdb/fileutil/dir_unix.go
+++ b/tsdb/fileutil/dir_unix.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/dir_windows.go b/tsdb/fileutil/dir_windows.go
index 307077ebc3..cfd55291d5 100644
--- a/tsdb/fileutil/dir_windows.go
+++ b/tsdb/fileutil/dir_windows.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/direct_io.go b/tsdb/fileutil/direct_io.go
index ad306776ca..76815de6b1 100644
--- a/tsdb/fileutil/direct_io.go
+++ b/tsdb/fileutil/direct_io.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/direct_io_force.go b/tsdb/fileutil/direct_io_force.go
index bb65403911..8ae4ef4fd7 100644
--- a/tsdb/fileutil/direct_io_force.go
+++ b/tsdb/fileutil/direct_io_force.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/direct_io_linux.go b/tsdb/fileutil/direct_io_linux.go
index a1d5f9577d..0640b503f6 100644
--- a/tsdb/fileutil/direct_io_linux.go
+++ b/tsdb/fileutil/direct_io_linux.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/direct_io_unsupported.go b/tsdb/fileutil/direct_io_unsupported.go
index a03782fe42..f17c68705f 100644
--- a/tsdb/fileutil/direct_io_unsupported.go
+++ b/tsdb/fileutil/direct_io_unsupported.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/direct_io_writer.go b/tsdb/fileutil/direct_io_writer.go
index 793d081481..3eeb2aa225 100644
--- a/tsdb/fileutil/direct_io_writer.go
+++ b/tsdb/fileutil/direct_io_writer.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/direct_io_writer_test.go b/tsdb/fileutil/direct_io_writer_test.go
index e60df1f3bc..367b7fa6aa 100644
--- a/tsdb/fileutil/direct_io_writer_test.go
+++ b/tsdb/fileutil/direct_io_writer_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/fileutil.go b/tsdb/fileutil/fileutil.go
index 523f99292c..0aa67e113a 100644
--- a/tsdb/fileutil/fileutil.go
+++ b/tsdb/fileutil/fileutil.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock.go b/tsdb/fileutil/flock.go
index e0082e2f2c..345581cc92 100644
--- a/tsdb/fileutil/flock.go
+++ b/tsdb/fileutil/flock.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock_js.go b/tsdb/fileutil/flock_js.go
index 6029cdf4d8..025e678a1d 100644
--- a/tsdb/fileutil/flock_js.go
+++ b/tsdb/fileutil/flock_js.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock_plan9.go b/tsdb/fileutil/flock_plan9.go
index 3b9550e7f2..543195e066 100644
--- a/tsdb/fileutil/flock_plan9.go
+++ b/tsdb/fileutil/flock_plan9.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock_solaris.go b/tsdb/fileutil/flock_solaris.go
index 8ca919f3b0..b7a69d9063 100644
--- a/tsdb/fileutil/flock_solaris.go
+++ b/tsdb/fileutil/flock_solaris.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock_test.go b/tsdb/fileutil/flock_test.go
index 7aff789a26..dec7d4e98d 100644
--- a/tsdb/fileutil/flock_test.go
+++ b/tsdb/fileutil/flock_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock_unix.go b/tsdb/fileutil/flock_unix.go
index 25de0ffb22..eddf427e7e 100644
--- a/tsdb/fileutil/flock_unix.go
+++ b/tsdb/fileutil/flock_unix.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/flock_windows.go b/tsdb/fileutil/flock_windows.go
index 1c17ff4ea3..64ce827324 100644
--- a/tsdb/fileutil/flock_windows.go
+++ b/tsdb/fileutil/flock_windows.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap.go b/tsdb/fileutil/mmap.go
index 782ff27ec9..9893d1014b 100644
--- a/tsdb/fileutil/mmap.go
+++ b/tsdb/fileutil/mmap.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap_386.go b/tsdb/fileutil/mmap_386.go
index 85c0cce096..01e4333a42 100644
--- a/tsdb/fileutil/mmap_386.go
+++ b/tsdb/fileutil/mmap_386.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap_amd64.go b/tsdb/fileutil/mmap_amd64.go
index 71fc568bd5..6d426f1866 100644
--- a/tsdb/fileutil/mmap_amd64.go
+++ b/tsdb/fileutil/mmap_amd64.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap_arm64.go b/tsdb/fileutil/mmap_arm64.go
index 71fc568bd5..6d426f1866 100644
--- a/tsdb/fileutil/mmap_arm64.go
+++ b/tsdb/fileutil/mmap_arm64.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap_js.go b/tsdb/fileutil/mmap_js.go
index f29106fc1e..59e1fcf877 100644
--- a/tsdb/fileutil/mmap_js.go
+++ b/tsdb/fileutil/mmap_js.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap_unix.go b/tsdb/fileutil/mmap_unix.go
index 3d15e1a8c1..b35352fef9 100644
--- a/tsdb/fileutil/mmap_unix.go
+++ b/tsdb/fileutil/mmap_unix.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/mmap_windows.go b/tsdb/fileutil/mmap_windows.go
index 5704b3b96d..8322f68971 100644
--- a/tsdb/fileutil/mmap_windows.go
+++ b/tsdb/fileutil/mmap_windows.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/fileutil/preallocate.go b/tsdb/fileutil/preallocate.go
index c747b7cf81..e9a587b2bd 100644
--- a/tsdb/fileutil/preallocate.go
+++ b/tsdb/fileutil/preallocate.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/preallocate_darwin.go b/tsdb/fileutil/preallocate_darwin.go
index 1d9eb806d1..58f83c5ba5 100644
--- a/tsdb/fileutil/preallocate_darwin.go
+++ b/tsdb/fileutil/preallocate_darwin.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/preallocate_linux.go b/tsdb/fileutil/preallocate_linux.go
index 026c69b354..1271c48928 100644
--- a/tsdb/fileutil/preallocate_linux.go
+++ b/tsdb/fileutil/preallocate_linux.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/preallocate_other.go b/tsdb/fileutil/preallocate_other.go
index e7fd937a43..55a44c7636 100644
--- a/tsdb/fileutil/preallocate_other.go
+++ b/tsdb/fileutil/preallocate_other.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/sync.go b/tsdb/fileutil/sync.go
index e1a4a7fd3d..9390b044a5 100644
--- a/tsdb/fileutil/sync.go
+++ b/tsdb/fileutil/sync.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/sync_darwin.go b/tsdb/fileutil/sync_darwin.go
index d698b896af..3dc42fc57a 100644
--- a/tsdb/fileutil/sync_darwin.go
+++ b/tsdb/fileutil/sync_darwin.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/fileutil/sync_linux.go b/tsdb/fileutil/sync_linux.go
index 2b4c620bb0..138bbee1e5 100644
--- a/tsdb/fileutil/sync_linux.go
+++ b/tsdb/fileutil/sync_linux.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The etcd Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/goversion/goversion.go b/tsdb/goversion/goversion.go
index ec23d25f2e..050ced875d 100644
--- a/tsdb/goversion/goversion.go
+++ b/tsdb/goversion/goversion.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/goversion/goversion_test.go b/tsdb/goversion/goversion_test.go
index 853844fb93..1e52b9655c 100644
--- a/tsdb/goversion/goversion_test.go
+++ b/tsdb/goversion/goversion_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/goversion/init.go b/tsdb/goversion/init.go
index dd15e1f7af..eb97bf7637 100644
--- a/tsdb/goversion/init.go
+++ b/tsdb/goversion/init.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head.go b/tsdb/head.go
index 25a1b88cec..a4df208e6e 100644
--- a/tsdb/head.go
+++ b/tsdb/head.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_append.go b/tsdb/head_append.go
index 356d1c453f..fceb80bd34 100644
--- a/tsdb/head_append.go
+++ b/tsdb/head_append.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_bench_test.go b/tsdb/head_bench_test.go
index a63b0ced50..dc0be0823a 100644
--- a/tsdb/head_bench_test.go
+++ b/tsdb/head_bench_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_dedupelabels.go b/tsdb/head_dedupelabels.go
index a75f337224..f8bcec2e78 100644
--- a/tsdb/head_dedupelabels.go
+++ b/tsdb/head_dedupelabels.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_other.go b/tsdb/head_other.go
index 7e1eea8b05..d6d5795e20 100644
--- a/tsdb/head_other.go
+++ b/tsdb/head_other.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_read.go b/tsdb/head_read.go
index f2681accc0..924b04bf0a 100644
--- a/tsdb/head_read.go
+++ b/tsdb/head_read.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_read_test.go b/tsdb/head_read_test.go
index b9f1700706..cf55973a01 100644
--- a/tsdb/head_read_test.go
+++ b/tsdb/head_read_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_test.go b/tsdb/head_test.go
index f36dc75f07..acdf0ee000 100644
--- a/tsdb/head_test.go
+++ b/tsdb/head_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/head_wal.go b/tsdb/head_wal.go
index 3c9aa7980e..bbcad9d855 100644
--- a/tsdb/head_wal.go
+++ b/tsdb/head_wal.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/index/index.go b/tsdb/index/index.go
index 253a515815..1ddcac9501 100644
--- a/tsdb/index/index.go
+++ b/tsdb/index/index.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/index/index_test.go b/tsdb/index/index_test.go
index 9013a1d5cd..20399dcdcf 100644
--- a/tsdb/index/index_test.go
+++ b/tsdb/index/index_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/index/postings.go b/tsdb/index/postings.go
index 0185f58819..31b93f850d 100644
--- a/tsdb/index/postings.go
+++ b/tsdb/index/postings.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/index/postings_test.go b/tsdb/index/postings_test.go
index 0fbe7a58a2..77b43f76ab 100644
--- a/tsdb/index/postings_test.go
+++ b/tsdb/index/postings_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/index/postingsstats.go b/tsdb/index/postingsstats.go
index f9ee640ff5..ebbe835207 100644
--- a/tsdb/index/postingsstats.go
+++ b/tsdb/index/postingsstats.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/index/postingsstats_test.go b/tsdb/index/postingsstats_test.go
index b218dd9fc7..766c5055c1 100644
--- a/tsdb/index/postingsstats_test.go
+++ b/tsdb/index/postingsstats_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/isolation.go b/tsdb/isolation.go
index 95d3cfa5eb..029efaf181 100644
--- a/tsdb/isolation.go
+++ b/tsdb/isolation.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/isolation_test.go b/tsdb/isolation_test.go
index 1e41b9c753..f2671024e8 100644
--- a/tsdb/isolation_test.go
+++ b/tsdb/isolation_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/mocks_test.go b/tsdb/mocks_test.go
index 986048d3d2..b3d2208bc1 100644
--- a/tsdb/mocks_test.go
+++ b/tsdb/mocks_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/ooo_head.go b/tsdb/ooo_head.go
index b3f5e2b675..c6ae924372 100644
--- a/tsdb/ooo_head.go
+++ b/tsdb/ooo_head.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/ooo_head_read.go b/tsdb/ooo_head_read.go
index 4cecb9fd6c..5d2347c2d7 100644
--- a/tsdb/ooo_head_read.go
+++ b/tsdb/ooo_head_read.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/ooo_head_read_test.go b/tsdb/ooo_head_read_test.go
index 5e754b59b8..4ecaa51fec 100644
--- a/tsdb/ooo_head_read_test.go
+++ b/tsdb/ooo_head_read_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/ooo_head_test.go b/tsdb/ooo_head_test.go
index 8f773b6ef9..99cd357a30 100644
--- a/tsdb/ooo_head_test.go
+++ b/tsdb/ooo_head_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/ooo_isolation.go b/tsdb/ooo_isolation.go
index 3e3e165a0a..3aeee693a9 100644
--- a/tsdb/ooo_isolation.go
+++ b/tsdb/ooo_isolation.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/ooo_isolation_test.go b/tsdb/ooo_isolation_test.go
index 4ff0488ab1..054823b30c 100644
--- a/tsdb/ooo_isolation_test.go
+++ b/tsdb/ooo_isolation_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/querier.go b/tsdb/querier.go
index 788991235f..4a487aa568 100644
--- a/tsdb/querier.go
+++ b/tsdb/querier.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/querier_bench_test.go b/tsdb/querier_bench_test.go
index 514fa05a17..ca9ee119f7 100644
--- a/tsdb/querier_bench_test.go
+++ b/tsdb/querier_bench_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go
index 4fe21c31ff..6933aa617a 100644
--- a/tsdb/querier_test.go
+++ b/tsdb/querier_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/record/record.go b/tsdb/record/record.go
index 5791f60df4..106b8e51bc 100644
--- a/tsdb/record/record.go
+++ b/tsdb/record/record.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/record/record_test.go b/tsdb/record/record_test.go
index bbbea04940..8ebd805d4d 100644
--- a/tsdb/record/record_test.go
+++ b/tsdb/record/record_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/repair.go b/tsdb/repair.go
index 8bdc645b5e..0d9d449a40 100644
--- a/tsdb/repair.go
+++ b/tsdb/repair.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/repair_test.go b/tsdb/repair_test.go
index 8a192c4f78..34fe85f422 100644
--- a/tsdb/repair_test.go
+++ b/tsdb/repair_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/testutil.go b/tsdb/testutil.go
index d41591750b..feb921447d 100644
--- a/tsdb/testutil.go
+++ b/tsdb/testutil.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tombstones/tombstones.go b/tsdb/tombstones/tombstones.go
index bda565eae4..25218782cd 100644
--- a/tsdb/tombstones/tombstones.go
+++ b/tsdb/tombstones/tombstones.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tombstones/tombstones_test.go b/tsdb/tombstones/tombstones_test.go
index de036e22d0..17802672c6 100644
--- a/tsdb/tombstones/tombstones_test.go
+++ b/tsdb/tombstones/tombstones_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tsdbblockutil.go b/tsdb/tsdbblockutil.go
index af2348019a..1c6882b085 100644
--- a/tsdb/tsdbblockutil.go
+++ b/tsdb/tsdbblockutil.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tsdbutil/dir_locker.go b/tsdb/tsdbutil/dir_locker.go
index 4b69e1f9d6..45cabdd3d7 100644
--- a/tsdb/tsdbutil/dir_locker.go
+++ b/tsdb/tsdbutil/dir_locker.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tsdbutil/dir_locker_test.go b/tsdb/tsdbutil/dir_locker_test.go
index 8c027415d3..e3f323932a 100644
--- a/tsdb/tsdbutil/dir_locker_test.go
+++ b/tsdb/tsdbutil/dir_locker_test.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tsdbutil/dir_locker_testutil.go b/tsdb/tsdbutil/dir_locker_testutil.go
index 5a335989c7..ffbf039339 100644
--- a/tsdb/tsdbutil/dir_locker_testutil.go
+++ b/tsdb/tsdbutil/dir_locker_testutil.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/tsdbutil/histogram.go b/tsdb/tsdbutil/histogram.go
index 64311a8c3b..e6a67c8212 100644
--- a/tsdb/tsdbutil/histogram.go
+++ b/tsdb/tsdbutil/histogram.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/wlog/checkpoint.go b/tsdb/wlog/checkpoint.go
index c26f3f1052..57c2faf23e 100644
--- a/tsdb/wlog/checkpoint.go
+++ b/tsdb/wlog/checkpoint.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/wlog/checkpoint_test.go b/tsdb/wlog/checkpoint_test.go
index b83724ea2e..97ca2e768d 100644
--- a/tsdb/wlog/checkpoint_test.go
+++ b/tsdb/wlog/checkpoint_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/wlog/live_reader.go b/tsdb/wlog/live_reader.go
index 004c397270..359f29274b 100644
--- a/tsdb/wlog/live_reader.go
+++ b/tsdb/wlog/live_reader.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/wlog/reader.go b/tsdb/wlog/reader.go
index c559d85b89..54b1baf4c4 100644
--- a/tsdb/wlog/reader.go
+++ b/tsdb/wlog/reader.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/wlog/reader_test.go b/tsdb/wlog/reader_test.go
index 1ddc33e2c8..788a2edfb9 100644
--- a/tsdb/wlog/reader_test.go
+++ b/tsdb/wlog/reader_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/wlog/watcher.go b/tsdb/wlog/watcher.go
index abb5ef9731..a841a44fc8 100644
--- a/tsdb/wlog/watcher.go
+++ b/tsdb/wlog/watcher.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/wlog/watcher_test.go b/tsdb/wlog/watcher_test.go
index 9e6ea65a7f..b9a6504298 100644
--- a/tsdb/wlog/watcher_test.go
+++ b/tsdb/wlog/watcher_test.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/tsdb/wlog/wlog.go b/tsdb/wlog/wlog.go
index 176531c478..5a80d58abf 100644
--- a/tsdb/wlog/wlog.go
+++ b/tsdb/wlog/wlog.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/tsdb/wlog/wlog_test.go b/tsdb/wlog/wlog_test.go
index 1ade42d3ff..79955d499c 100644
--- a/tsdb/wlog/wlog_test.go
+++ b/tsdb/wlog/wlog_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
diff --git a/util/almost/almost.go b/util/almost/almost.go
index 5f866b89b3..b89f968db6 100644
--- a/util/almost/almost.go
+++ b/util/almost/almost.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/almost/almost_test.go b/util/almost/almost_test.go
index fba37f13f6..4e225bf862 100644
--- a/util/almost/almost_test.go
+++ b/util/almost/almost_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/annotations/annotations.go b/util/annotations/annotations.go
index 817f670b5e..a68b2ba4fc 100644
--- a/util/annotations/annotations.go
+++ b/util/annotations/annotations.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/compression/buffers.go b/util/compression/buffers.go
index f510efc042..30f002970b 100644
--- a/util/compression/buffers.go
+++ b/util/compression/buffers.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/compression/compression.go b/util/compression/compression.go
index a1e9b7e530..26cff6a22e 100644
--- a/util/compression/compression.go
+++ b/util/compression/compression.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/compression/compression_test.go b/util/compression/compression_test.go
index 736bb934e3..4c52b8f42e 100644
--- a/util/compression/compression_test.go
+++ b/util/compression/compression_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/convertnhcb/convertnhcb.go b/util/convertnhcb/convertnhcb.go
index 21ae62b3cb..64ec9054a3 100644
--- a/util/convertnhcb/convertnhcb.go
+++ b/util/convertnhcb/convertnhcb.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/convertnhcb/convertnhcb_test.go b/util/convertnhcb/convertnhcb_test.go
index 7486ac18bb..710d47385a 100644
--- a/util/convertnhcb/convertnhcb_test.go
+++ b/util/convertnhcb/convertnhcb_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/documentcli/documentcli.go b/util/documentcli/documentcli.go
index 14382663ee..ebd7d91a5d 100644
--- a/util/documentcli/documentcli.go
+++ b/util/documentcli/documentcli.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/fmtutil/format.go b/util/fmtutil/format.go
index 377f4ece05..a4ac7d43ca 100644
--- a/util/fmtutil/format.go
+++ b/util/fmtutil/format.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/fmtutil/format_test.go b/util/fmtutil/format_test.go
index f1d025806e..73dbe39f45 100644
--- a/util/fmtutil/format_test.go
+++ b/util/fmtutil/format_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/gate/gate.go b/util/gate/gate.go
index 6cb9d583c6..a1066fd74f 100644
--- a/util/gate/gate.go
+++ b/util/gate/gate.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/httputil/compression.go b/util/httputil/compression.go
index e67f9ffd9f..ca9f3c17da 100644
--- a/util/httputil/compression.go
+++ b/util/httputil/compression.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/httputil/compression_test.go b/util/httputil/compression_test.go
index 11df0a7c4c..6bdde914ce 100644
--- a/util/httputil/compression_test.go
+++ b/util/httputil/compression_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/httputil/context.go b/util/httputil/context.go
index 9b16428892..7aaeebdb3e 100644
--- a/util/httputil/context.go
+++ b/util/httputil/context.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/httputil/cors.go b/util/httputil/cors.go
index 2d4cc91ccb..e319762b5f 100644
--- a/util/httputil/cors.go
+++ b/util/httputil/cors.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/httputil/cors_test.go b/util/httputil/cors_test.go
index 30567947a9..d637932267 100644
--- a/util/httputil/cors_test.go
+++ b/util/httputil/cors_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/jsonutil/marshal.go b/util/jsonutil/marshal.go
index d715eabe68..61ce4234eb 100644
--- a/util/jsonutil/marshal.go
+++ b/util/jsonutil/marshal.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/junitxml/junitxml.go b/util/junitxml/junitxml.go
index 14e4b6dbae..8249290830 100644
--- a/util/junitxml/junitxml.go
+++ b/util/junitxml/junitxml.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/junitxml/junitxml_test.go b/util/junitxml/junitxml_test.go
index ad4d0293d0..92a32f2ddf 100644
--- a/util/junitxml/junitxml_test.go
+++ b/util/junitxml/junitxml_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/logging/dedupe.go b/util/logging/dedupe.go
index 8137f4f22b..244cd6495c 100644
--- a/util/logging/dedupe.go
+++ b/util/logging/dedupe.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/logging/dedupe_test.go b/util/logging/dedupe_test.go
index 918c5d60bd..b584f12572 100644
--- a/util/logging/dedupe_test.go
+++ b/util/logging/dedupe_test.go
@@ -1,4 +1,4 @@
-// Copyright 2019 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/logging/file.go b/util/logging/file.go
index 5e379442a2..bce9be9ae6 100644
--- a/util/logging/file.go
+++ b/util/logging/file.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/logging/file_test.go b/util/logging/file_test.go
index bd34bc2a3a..58a55697d9 100644
--- a/util/logging/file_test.go
+++ b/util/logging/file_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/namevalidationutil/namevalidationutil.go b/util/namevalidationutil/namevalidationutil.go
index 2e656b6a19..14796b48f4 100644
--- a/util/namevalidationutil/namevalidationutil.go
+++ b/util/namevalidationutil/namevalidationutil.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/namevalidationutil/namevalidationutil_test.go b/util/namevalidationutil/namevalidationutil_test.go
index 660b6100b0..692bc2692b 100644
--- a/util/namevalidationutil/namevalidationutil_test.go
+++ b/util/namevalidationutil/namevalidationutil_test.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/netconnlimit/netconnlimit.go b/util/netconnlimit/netconnlimit.go
index 3bdd805b83..5f54d0616a 100644
--- a/util/netconnlimit/netconnlimit.go
+++ b/util/netconnlimit/netconnlimit.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Based on golang.org/x/net/netutil:
 //   Copyright 2013 The Go Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
diff --git a/util/netconnlimit/netconnlimit_test.go b/util/netconnlimit/netconnlimit_test.go
index e4d4904209..c33c7b342f 100644
--- a/util/netconnlimit/netconnlimit_test.go
+++ b/util/netconnlimit/netconnlimit_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/notifications/notifications.go b/util/notifications/notifications.go
index 4888a0b664..0e3882ce36 100644
--- a/util/notifications/notifications.go
+++ b/util/notifications/notifications.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/notifications/notifications_test.go b/util/notifications/notifications_test.go
index 3d9ba6bb12..84db90c6e3 100644
--- a/util/notifications/notifications_test.go
+++ b/util/notifications/notifications_test.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/osutil/hostname.go b/util/osutil/hostname.go
index c44cb391b6..f0444114f7 100644
--- a/util/osutil/hostname.go
+++ b/util/osutil/hostname.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/pool/pool.go b/util/pool/pool.go
index 7d5a8e3abf..a7f1bbb54e 100644
--- a/util/pool/pool.go
+++ b/util/pool/pool.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/pool/pool_test.go b/util/pool/pool_test.go
index e1ac13fb90..a14da6be8b 100644
--- a/util/pool/pool_test.go
+++ b/util/pool/pool_test.go
@@ -1,4 +1,4 @@
-// Copyright 2020 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/limits_default.go b/util/runtime/limits_default.go
index 156747d450..51a78423d3 100644
--- a/util/runtime/limits_default.go
+++ b/util/runtime/limits_default.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/limits_windows.go b/util/runtime/limits_windows.go
index ce82d31e6d..1cb7ea33a7 100644
--- a/util/runtime/limits_windows.go
+++ b/util/runtime/limits_windows.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/statfs.go b/util/runtime/statfs.go
index 66bedb5ea1..98dd822e4a 100644
--- a/util/runtime/statfs.go
+++ b/util/runtime/statfs.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/statfs_default.go b/util/runtime/statfs_default.go
index 78cfb1fe41..0cf5c2e616 100644
--- a/util/runtime/statfs_default.go
+++ b/util/runtime/statfs_default.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/statfs_linux_386.go b/util/runtime/statfs_linux_386.go
index a003b2effe..33dbc4c3e9 100644
--- a/util/runtime/statfs_linux_386.go
+++ b/util/runtime/statfs_linux_386.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/statfs_uint32.go b/util/runtime/statfs_uint32.go
index fbf994ea63..2fb4d70849 100644
--- a/util/runtime/statfs_uint32.go
+++ b/util/runtime/statfs_uint32.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/uname_default.go b/util/runtime/uname_default.go
index 0052dbab47..1bdc2e6696 100644
--- a/util/runtime/uname_default.go
+++ b/util/runtime/uname_default.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/uname_linux.go b/util/runtime/uname_linux.go
index ce3bc42a25..f2798cda4b 100644
--- a/util/runtime/uname_linux.go
+++ b/util/runtime/uname_linux.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/vmlimits_default.go b/util/runtime/vmlimits_default.go
index aef4341061..0e3bc0ead5 100644
--- a/util/runtime/vmlimits_default.go
+++ b/util/runtime/vmlimits_default.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runtime/vmlimits_openbsd.go b/util/runtime/vmlimits_openbsd.go
index b40f065883..ce9aa181e6 100644
--- a/util/runtime/vmlimits_openbsd.go
+++ b/util/runtime/vmlimits_openbsd.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/runutil/runutil.go b/util/runutil/runutil.go
index 5a77c332ba..14752ed796 100644
--- a/util/runutil/runutil.go
+++ b/util/runutil/runutil.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/stats/query_stats.go b/util/stats/query_stats.go
index d8ec186f4c..9801d658a7 100644
--- a/util/stats/query_stats.go
+++ b/util/stats/query_stats.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/stats/stats_test.go b/util/stats/stats_test.go
index 28753b95fc..245f7cbc16 100644
--- a/util/stats/stats_test.go
+++ b/util/stats/stats_test.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/stats/timer.go b/util/stats/timer.go
index eca0fcccb0..1b9e430a09 100644
--- a/util/stats/timer.go
+++ b/util/stats/timer.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/strutil/quote.go b/util/strutil/quote.go
index 0a78421fd4..d7e65395f4 100644
--- a/util/strutil/quote.go
+++ b/util/strutil/quote.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/strutil/quote_test.go b/util/strutil/quote_test.go
index de33230551..c077a5ed49 100644
--- a/util/strutil/quote_test.go
+++ b/util/strutil/quote_test.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/strutil/strconv.go b/util/strutil/strconv.go
index 88d2a3b610..77f1acc94d 100644
--- a/util/strutil/strconv.go
+++ b/util/strutil/strconv.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/strutil/strconv_test.go b/util/strutil/strconv_test.go
index f09e7ffb3f..b4b87ee816 100644
--- a/util/strutil/strconv_test.go
+++ b/util/strutil/strconv_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/teststorage/storage.go b/util/teststorage/storage.go
index e0a6f39be2..30a63327ab 100644
--- a/util/teststorage/storage.go
+++ b/util/teststorage/storage.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/cmp.go b/util/testutil/cmp.go
index 3ea1f40168..9be01a5b4b 100644
--- a/util/testutil/cmp.go
+++ b/util/testutil/cmp.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/context.go b/util/testutil/context.go
index 3d2a09d637..15f50fbff5 100644
--- a/util/testutil/context.go
+++ b/util/testutil/context.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/directory.go b/util/testutil/directory.go
index 176acb5dc1..706007d322 100644
--- a/util/testutil/directory.go
+++ b/util/testutil/directory.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/port.go b/util/testutil/port.go
index 91c1291749..3a9be3f1a3 100644
--- a/util/testutil/port.go
+++ b/util/testutil/port.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/roundtrip.go b/util/testutil/roundtrip.go
index 364e0c2642..0bd003ca68 100644
--- a/util/testutil/roundtrip.go
+++ b/util/testutil/roundtrip.go
@@ -1,4 +1,4 @@
-// Copyright 2017 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/synctest/disabled.go b/util/testutil/synctest/disabled.go
index e87454afcf..595b93c650 100644
--- a/util/testutil/synctest/disabled.go
+++ b/util/testutil/synctest/disabled.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/synctest/enabled.go b/util/testutil/synctest/enabled.go
index 61aa85dcf7..d219903809 100644
--- a/util/testutil/synctest/enabled.go
+++ b/util/testutil/synctest/enabled.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/testutil/synctest/synctest.go b/util/testutil/synctest/synctest.go
index 6780798a9b..41750f9892 100644
--- a/util/testutil/synctest/synctest.go
+++ b/util/testutil/synctest/synctest.go
@@ -1,4 +1,4 @@
-// Copyright 2025 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/treecache/treecache.go b/util/treecache/treecache.go
index 86fd207074..32912c5a94 100644
--- a/util/treecache/treecache.go
+++ b/util/treecache/treecache.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/zeropool/pool.go b/util/zeropool/pool.go
index 946ce02091..6eab9f3365 100644
--- a/util/zeropool/pool.go
+++ b/util/zeropool/pool.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/util/zeropool/pool_test.go b/util/zeropool/pool_test.go
index 24598cbfa3..f93e75d539 100644
--- a/util/zeropool/pool_test.go
+++ b/util/zeropool/pool_test.go
@@ -1,4 +1,4 @@
-// Copyright 2023 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/api.go b/web/api/v1/api.go
index 2a6036ba0b..f32fee19f8 100644
--- a/web/api/v1/api.go
+++ b/web/api/v1/api.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/api_test.go b/web/api/v1/api_test.go
index 83e8618630..39c1fa6080 100644
--- a/web/api/v1/api_test.go
+++ b/web/api/v1/api_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/codec.go b/web/api/v1/codec.go
index 492e00a74a..e7e53b466c 100644
--- a/web/api/v1/codec.go
+++ b/web/api/v1/codec.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/codec_test.go b/web/api/v1/codec_test.go
index 911bf206e3..10038b605a 100644
--- a/web/api/v1/codec_test.go
+++ b/web/api/v1/codec_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/errors_test.go b/web/api/v1/errors_test.go
index 5bd943ba98..6e55089e16 100644
--- a/web/api/v1/errors_test.go
+++ b/web/api/v1/errors_test.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/json_codec.go b/web/api/v1/json_codec.go
index 4f3a23e976..adcf0e34bc 100644
--- a/web/api/v1/json_codec.go
+++ b/web/api/v1/json_codec.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/json_codec_test.go b/web/api/v1/json_codec_test.go
index f0a671d6d1..8d17a1759f 100644
--- a/web/api/v1/json_codec_test.go
+++ b/web/api/v1/json_codec_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/api/v1/translate_ast.go b/web/api/v1/translate_ast.go
index dc2e7e2901..3cce0583f9 100644
--- a/web/api/v1/translate_ast.go
+++ b/web/api/v1/translate_ast.go
@@ -1,4 +1,4 @@
-// Copyright 2024 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/federate.go b/web/federate.go
index 443fd73568..584b8d7c4a 100644
--- a/web/federate.go
+++ b/web/federate.go
@@ -1,4 +1,4 @@
-// Copyright 2015 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/federate_test.go b/web/federate_test.go
index 55e20c6b2f..932639e2e6 100644
--- a/web/federate_test.go
+++ b/web/federate_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/ui/assets_embed.go b/web/ui/assets_embed.go
index a5f8f5ddfa..48e4a2c6f1 100644
--- a/web/ui/assets_embed.go
+++ b/web/ui/assets_embed.go
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go b/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go
index 1b58362393..74e8ac0354 100644
--- a/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go
+++ b/web/ui/mantine-ui/src/promql/tools/gen_functions_docs/main.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go b/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go
index 8713772dfe..6b77f368c8 100644
--- a/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go
+++ b/web/ui/mantine-ui/src/promql/tools/gen_functions_list/main.go
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/ui/module/lezer-promql/src/highlight.js b/web/ui/module/lezer-promql/src/highlight.js
index 9c1b5601a3..b452373345 100644
--- a/web/ui/module/lezer-promql/src/highlight.js
+++ b/web/ui/module/lezer-promql/src/highlight.js
@@ -1,4 +1,4 @@
-// Copyright 2022 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/ui/module/lezer-promql/src/tokens.js b/web/ui/module/lezer-promql/src/tokens.js
index 1695ae1d87..523c306ae9 100644
--- a/web/ui/module/lezer-promql/src/tokens.js
+++ b/web/ui/module/lezer-promql/src/tokens.js
@@ -1,4 +1,4 @@
-// Copyright 2021 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/ui/ui.go b/web/ui/ui.go
index 2585951d4d..c427dcf119 100644
--- a/web/ui/ui.go
+++ b/web/ui/ui.go
@@ -1,4 +1,4 @@
-// Copyright 2018 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/web.go b/web/web.go
index e787cbb4ac..afe78e4255 100644
--- a/web/web.go
+++ b/web/web.go
@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
diff --git a/web/web_test.go b/web/web_test.go
index b07e26cfa8..ae7d532f1f 100644
--- a/web/web_test.go
+++ b/web/web_test.go
@@ -1,4 +1,4 @@
-// Copyright 2016 The Prometheus Authors
+// Copyright The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at

From dbb3fc65b65f40d7d4a9c6f81295178487f48abf Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Mon, 5 Jan 2026 15:14:34 +0100
Subject: [PATCH 208/439] Replace entire identifier when autocompleting inside
 of it

When accepting an autocompletion result within an Identifier node (could be a
metric name, function name, keyword, etc.), the inserted completion should
replace the entire Identifier node all the way to its last character, not only
to the current cursor position.

A limitation is that the correct replacement-until-end-of-identifier only works
when e.g. a function name is currently incomplete (which is likely anyway when
trying to replace it with a different one). This is because otherwise the
Identifier node gets replaced with a more specific function node type (like
`Rate`, `SumOverTime`, etc.), and handling all those adds more complexity.

https://github.com/prometheus/prometheus/issues/15839

Signed-off-by: Julius Volz 
---
 .../src/complete/hybrid.test.ts               | 91 ++++++++++++++++++-
 .../codemirror-promql/src/complete/hybrid.ts  | 22 ++++-
 2 files changed, 111 insertions(+), 2 deletions(-)

diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
index 1f3985af63..8250319681 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
@@ -11,7 +11,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-import { analyzeCompletion, computeStartCompletePosition, ContextKind, durationWithUnitRegexp } from './hybrid';
+import { analyzeCompletion, computeStartCompletePosition, computeEndCompletePosition, ContextKind, durationWithUnitRegexp } from './hybrid';
 import { createEditorState, mockedMetricsTerms, mockPrometheusServer } from '../test/utils-test';
 import { Completion, CompletionContext } from '@codemirror/autocomplete';
 import {
@@ -866,6 +866,73 @@ describe('computeStartCompletePosition test', () => {
   });
 });
 
+describe('computeEndCompletePosition test', () => {
+  const testCases = [
+    {
+      title: 'cursor at end of metric name',
+      expr: 'metric_name',
+      pos: 11, // cursor is at the end
+      expectedEnd: 11,
+    },
+    {
+      title: 'cursor in middle of metric name - should extend to end',
+      expr: 'coredns_cache_hits_total',
+      pos: 14, // cursor is after 'coredns_cache_' (before 'hits')
+      expectedEnd: 24, // should extend to end of 'coredns_cache_hits_total'
+    },
+    {
+      title: 'cursor in middle of metric name inside rate() - should extend to end',
+      expr: 'rate(coredns_cache_hits_total[2m])',
+      pos: 19, // cursor is after 'coredns_cache_' (before 'hits')
+      expectedEnd: 29, // should extend to end of 'coredns_cache_hits_total'
+    },
+    {
+      title: 'cursor in middle of metric name inside sum(rate()) - should extend to end',
+      expr: 'sum(rate(coredns_cache_hits_total[2m]))',
+      pos: 24, // cursor is after 'coredns_cache_' (before 'hits')
+      expectedEnd: 33, // should extend to end of 'coredns_cache_hits_total'
+    },
+    {
+      title: 'cursor at beginning of metric name - should extend to end',
+      expr: 'metric_name',
+      pos: 1, // cursor after 'm'
+      expectedEnd: 11,
+    },
+    {
+      title: 'cursor in middle of incomplete function name - should extend to end',
+      expr: 'sum_ov',
+      pos: 4, // cursor after 'sum_' (before 'ov')
+      expectedEnd: 6, // should extend to end of 'sum_ov'
+    },
+    {
+      title: 'cursor in middle of incomplete function name within aggregator - should extend to end',
+      expr: 'sum(sum_ov(foo[5m]))',
+      pos: 8, // cursor after 'sum_' (before 'ov')
+      expectedEnd: 10, // should extend to end of 'sum_ov'
+    },
+    {
+      title: 'empty bracket - returns pos',
+      expr: '{}',
+      pos: 1,
+      expectedEnd: 1,
+    },
+    {
+      title: 'cursor in label matchers - returns pos',
+      expr: 'metric_name{label="value"}',
+      pos: 12, // cursor after '{'
+      expectedEnd: 12,
+    },
+  ];
+  testCases.forEach((value) => {
+    it(value.title, () => {
+      const state = createEditorState(value.expr);
+      const node = syntaxTree(state).resolve(value.pos, -1);
+      const result = computeEndCompletePosition(state, node, value.pos);
+      expect(result).toEqual(value.expectedEnd);
+    });
+  });
+});
+
 describe('autocomplete promQL test', () => {
   beforeEach(() => {
     mockPrometheusServer();
@@ -915,6 +982,28 @@ describe('autocomplete promQL test', () => {
         validFor: /^[a-zA-Z0-9_:]+$/,
       },
     },
+    {
+      title: 'cursor in middle of metric name - to should extend to end (issue #15839)',
+      expr: 'sum(coredns_cache_hits_total)',
+      pos: 18, // cursor is after 'coredns_cache_' (before 'hits')
+      expectedResult: {
+        options: ([] as Completion[]).concat(functionIdentifierTerms, aggregateOpTerms, snippets),
+        from: 4,
+        to: 28, // should extend to end of 'coredns_cache_hits_total'
+        validFor: /^[a-zA-Z0-9_:]+$/,
+      },
+    },
+    {
+      title: 'cursor in middle of metric name inside rate() - to should extend to end (issue #15839)',
+      expr: 'rate(coredns_cache_hits_total[2m])',
+      pos: 19, // cursor is after 'coredns_cache_' (before 'hits')
+      expectedResult: {
+        options: ([] as Completion[]).concat(functionIdentifierTerms, aggregateOpTerms, snippets),
+        from: 5,
+        to: 29, // should extend to end of 'coredns_cache_hits_total'
+        validFor: /^[a-zA-Z0-9_:]+$/,
+      },
+    },
     {
       title: 'offline function/aggregation autocompletion in aggregation 3',
       expr: 'sum(rate())',
diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
index fc79b6fcd6..d89907699a 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
@@ -166,6 +166,20 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i
   } as CompletionResult;
 }
 
+// computeEndCompletePosition calculates the end position for autocompletion replacement.
+// When the cursor is in the middle of an identifier (e.g., metric name), this ensures the entire
+// identifier is replaced, not just the portion before the cursor. This fixes issue #15839.
+// Note: this method is exported only for testing purpose.
+export function computeEndCompletePosition(state: EditorState, node: SyntaxNode, pos: number): number {
+  // For Identifier nodes (metric names), extend the end position to include
+  // the entire identifier, even if the cursor is in the middle.
+  if (node.type.id === Identifier) {
+    return node.to;
+  }
+  // Default: use the cursor position as the end position
+  return pos;
+}
+
 // Matches complete PromQL durations, including compound units (e.g., 5m, 1d2h, 1h30m, etc.).
 // Duration units are a fixed, safe set (no regex metacharacters), so no escaping is needed.
 export const durationWithUnitRegexp = new RegExp(`^(\\d+(${durationTerms.map((term) => term.label).join('|')}))+$`);
@@ -667,7 +681,13 @@ export class HybridComplete implements CompleteStrategy {
       }
     }
     return asyncResult.then((result) => {
-      return arrayToCompletionResult(result, computeStartCompletePosition(state, tree, pos), pos, completeSnippet, span);
+      return arrayToCompletionResult(
+        result,
+        computeStartCompletePosition(state, tree, pos),
+        computeEndCompletePosition(state, tree, pos),
+        completeSnippet,
+        span
+      );
     });
   }
 

From 3980134b43ad6c458a08651bf9599296759b9e7b Mon Sep 17 00:00:00 2001
From: Arve Knudsen 
Date: Mon, 5 Jan 2026 17:59:37 +0100
Subject: [PATCH 209/439] chore: align Renovate configuration with Dependabot
 (#17777)

* chore: align Renovate configuration with Dependabot

---------

Signed-off-by: Arve Knudsen 
Co-authored-by: Ben Kochie 
---
 renovate.json | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/renovate.json b/renovate.json
index 175e1d6464..e9e383337b 100644
--- a/renovate.json
+++ b/renovate.json
@@ -9,8 +9,11 @@
       "gomodTidy",
       "gomodUpdateImportPaths"
     ],
-    "schedule": ["* 0-8 * * 1"],
+    "schedule": ["57 11 21 * *"],
     "timezone": "UTC",
+    "github-actions": {
+      "managerFilePatterns": ["scripts/**"]
+    },
     "packageRules": [
       {
         "description": "Don't update replace directives",
@@ -24,6 +27,30 @@
         "matchPackageNames": ["@prometheus-io/**"],
         "enabled": false
       },
+      {
+        "description": "Group AWS Go dependencies",
+        "matchManagers": ["gomod"],
+        "matchPackageNames": ["github.com/aws/**"],
+        "groupName": "AWS Go dependencies"
+      },
+      {
+        "description": "Group Azure Go dependencies",
+        "matchManagers": ["gomod"],
+        "matchPackageNames": ["github.com/Azure/**"],
+        "groupName": "Azure Go dependencies"
+      },
+      {
+        "description": "Group Kubernetes Go dependencies",
+        "matchManagers": ["gomod"],
+        "matchPackageNames": ["k8s.io/**"],
+        "groupName": "Kubernetes Go dependencies"
+      },
+      {
+        "description": "Group OpenTelemetry Go dependencies",
+        "matchManagers": ["gomod"],
+        "matchPackageNames": ["go.opentelemetry.io/**"],
+        "groupName": "OpenTelemetry Go dependencies"
+      },
       {
         "description": "Group Mantine UI dependencies",
         "matchFileNames": [

From 57961fbeddf5d26d1b8d5aab1dcd253b7ae9a431 Mon Sep 17 00:00:00 2001
From: Arve Knudsen 
Date: Mon, 5 Jan 2026 18:00:08 +0100
Subject: [PATCH 210/439] chore: remove dependabot configuration (#17776)

Remove Dependabot configuration, as we are now using Renovate.

Signed-off-by: Arve Knudsen 
---
 .github/dependabot.yml | 33 ---------------------------------
 1 file changed, 33 deletions(-)
 delete mode 100644 .github/dependabot.yml

diff --git a/.github/dependabot.yml b/.github/dependabot.yml
deleted file mode 100644
index 99a9ce05a4..0000000000
--- a/.github/dependabot.yml
+++ /dev/null
@@ -1,33 +0,0 @@
-version: 2
-updates:
-  - package-ecosystem: "docker"
-    directory: "/"
-    schedule:
-      interval: "monthly"
-  - package-ecosystem: "github-actions"
-    directories:
-      - "/"
-      - "/scripts"
-    schedule:
-      interval: "monthly"
-  - package-ecosystem: "gomod"
-    directories:
-      - "/"
-      - "/documentation/examples/remote_storage"
-      - "/internal/tools"
-    schedule:
-      interval: "monthly"
-    groups:
-      aws:
-        patterns:
-          - "github.com/aws/*"
-      azure:
-        patterns:
-          - "github.com/Azure/*"
-      k8s.io:
-        patterns:
-          - "k8s.io/*"
-      go.opentelemetry.io:
-        patterns:
-          - "go.opentelemetry.io/*"
-    open-pull-requests-limit: 20

From b532eacae88af136f6bb1d67ae6746e7344caa08 Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Mon, 5 Jan 2026 20:09:12 +0100
Subject: [PATCH 211/439] Review fixups - also make it work for label names

Signed-off-by: Julius Volz 
---
 .../src/complete/hybrid.test.ts               | 24 +++++++++++++++++++
 .../codemirror-promql/src/complete/hybrid.ts  | 11 +++++----
 2 files changed, 30 insertions(+), 5 deletions(-)

diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
index 8250319681..5906a692de 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
@@ -922,6 +922,30 @@ describe('computeEndCompletePosition test', () => {
       pos: 12, // cursor after '{'
       expectedEnd: 12,
     },
+    {
+      title: 'cursor in middle of label name in grouping clause - should extend to end',
+      expr: 'sum by (instance_name)',
+      pos: 12, // cursor after 'inst' (before 'ance')
+      expectedEnd: 21, // should extend to end of 'instance_name'
+    },
+    {
+      title: 'cursor in middle of label name in label matcher - should extend to end',
+      expr: 'metric{instance_name="value"}',
+      pos: 11, // cursor after 'inst' (before 'ance')
+      expectedEnd: 20, // should extend to end of 'instance_name'
+    },
+    {
+      title: 'cursor in middle of label name in on() modifier - should extend to end',
+      expr: 'a / on(instance_name) b',
+      pos: 11, // cursor after 'inst' (before 'ance')
+      expectedEnd: 20, // should extend to end of 'instance_name'
+    },
+    {
+      title: 'cursor in middle of label name in ignoring() modifier - should extend to end',
+      expr: 'a / ignoring(instance_name) b',
+      pos: 17, // cursor after 'inst' (before 'ance')
+      expectedEnd: 26, // should extend to end of 'instance_name'
+    },
   ];
   testCases.forEach((value) => {
     it(value.title, () => {
diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
index d89907699a..23e47ce649 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
@@ -167,13 +167,14 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i
 }
 
 // computeEndCompletePosition calculates the end position for autocompletion replacement.
-// When the cursor is in the middle of an identifier (e.g., metric name), this ensures the entire
-// identifier is replaced, not just the portion before the cursor. This fixes issue #15839.
+// When the cursor is in the middle of an identifier (e.g., metric name) or label name, this ensures
+// the entire token is replaced, not just the portion before the cursor. This fixes issue #15839.
 // Note: this method is exported only for testing purpose.
 export function computeEndCompletePosition(state: EditorState, node: SyntaxNode, pos: number): number {
-  // For Identifier nodes (metric names), extend the end position to include
-  // the entire identifier, even if the cursor is in the middle.
-  if (node.type.id === Identifier) {
+  // For Identifier nodes (metric names) and LabelName nodes (label names in matchers,
+  // grouping clauses, etc.), extend the end position to include the entire token,
+  // even if the cursor is in the middle.
+  if (node.type.id === Identifier || node.type.id === LabelName) {
     return node.to;
   }
   // Default: use the cursor position as the end position

From 0a2be8161625ea9d0810ba52dd917df1c5440947 Mon Sep 17 00:00:00 2001
From: VictorFilatov 
Date: Tue, 6 Jan 2026 11:55:03 +0300
Subject: [PATCH 212/439] Fix link in discovery README.md (#17753)

Signed-off-by: VictorFilatov 
---
 discovery/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/discovery/README.md b/discovery/README.md
index d5418e7fb1..5d1adcf145 100644
--- a/discovery/README.md
+++ b/discovery/README.md
@@ -50,7 +50,7 @@ file for use with `file_sd`.
 The general principle with SD is to extract all the potentially useful
 information we can out of the SD, and let the user choose what they need of it
 using
-[relabelling](https://prometheus.io/docs/operating/configuration/#).
+[relabelling](https://prometheus.io/docs/operating/configuration/#relabel_config).
 This information is generally termed metadata.
 
 Metadata is exposed as a set of key/value pairs (labels) per target. The keys

From 3fc800410aed73e2fd2143524f3de6a064c246f7 Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Tue, 6 Jan 2026 10:57:55 +0100
Subject: [PATCH 213/439] Handle autocomplete replacement better for more node
 types

Signed-off-by: Julius Volz 
---
 .../src/complete/hybrid.test.ts               | 64 ++++++++++++++++---
 .../codemirror-promql/src/complete/hybrid.ts  | 33 +++++++---
 2 files changed, 80 insertions(+), 17 deletions(-)

diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
index 5906a692de..40c4356cc6 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
@@ -638,7 +638,7 @@ describe('analyzeCompletion test', () => {
       const state = createEditorState(value.expr);
       const node = syntaxTree(state).resolve(value.pos, -1);
       const result = analyzeCompletion(state, node, value.pos);
-      expect(value.expectedContext).toEqual(result);
+      expect(result).toEqual(value.expectedContext);
     });
   });
 });
@@ -861,7 +861,7 @@ describe('computeStartCompletePosition test', () => {
       const state = createEditorState(value.expr);
       const node = syntaxTree(state).resolve(value.pos, -1);
       const result = computeStartCompletePosition(state, node, value.pos);
-      expect(value.expectedStart).toEqual(result);
+      expect(result).toEqual(value.expectedStart);
     });
   });
 });
@@ -911,16 +911,16 @@ describe('computeEndCompletePosition test', () => {
       expectedEnd: 10, // should extend to end of 'sum_ov'
     },
     {
-      title: 'empty bracket - returns pos',
+      title: 'empty bracket - ends before the closing bracket',
       expr: '{}',
       pos: 1,
       expectedEnd: 1,
     },
     {
-      title: 'cursor in label matchers - returns pos',
+      title: 'cursor in label matchers - ends before the closing bracket',
       expr: 'metric_name{label="value"}',
       pos: 12, // cursor after '{'
-      expectedEnd: 12,
+      expectedEnd: 25,
     },
     {
       title: 'cursor in middle of label name in grouping clause - should extend to end',
@@ -946,6 +946,54 @@ describe('computeEndCompletePosition test', () => {
       pos: 17, // cursor after 'inst' (before 'ance')
       expectedEnd: 26, // should extend to end of 'instance_name'
     },
+    {
+      title: 'cursor in middle of function name rate - should extend to end',
+      expr: 'rate(foo[5m])',
+      pos: 2, // cursor after 'ra' (before 'te')
+      expectedEnd: 4, // should extend to end of 'rate'
+    },
+    {
+      title: 'cursor in middle of function name histogram_quantile - should extend to end',
+      expr: 'histogram_quantile(0.9, rate(foo[5m]))',
+      pos: 10, // cursor after 'histogram_' (before 'quantile')
+      expectedEnd: 18, // should extend to end of 'histogram_quantile'
+    },
+    {
+      title: 'cursor in middle of aggregator sum - should extend to end',
+      expr: 'sum(rate(foo[5m]))',
+      pos: 2, // cursor after 'su' (before 'm')
+      expectedEnd: 3, // should extend to end of 'sum'
+    },
+    {
+      title: 'cursor in middle of aggregator count_values - should extend to end',
+      expr: 'count_values("label", foo)',
+      pos: 6, // cursor after 'count_' (before 'values')
+      expectedEnd: 12, // should extend to end of 'count_values'
+    },
+    {
+      title: 'cursor in middle of nested function - should extend to end',
+      expr: 'sum(rate(foo[5m]))',
+      pos: 6, // cursor after 'ra' inside rate (before 'te')
+      expectedEnd: 8, // should extend to end of 'rate'
+    },
+    {
+      title: 'cursor at beginning of aggregator - should extend to end',
+      expr: 'avg by (instance) (rate(foo[5m]))',
+      pos: 1, // cursor after 'a' (before 'vg')
+      expectedEnd: 3, // should extend to end of 'avg'
+    },
+    {
+      title: 'cursor in middle of function name with binary op - should extend to end',
+      expr: 'rate(foo[5m]) / irate(bar[5m])',
+      pos: 17, // cursor after 'ir' inside irate (before 'ate')
+      expectedEnd: 21, // should extend to end of 'irate'
+    },
+    {
+      title: 'error node - returns pos (cursor position)',
+      expr: 'metric_name !',
+      pos: 13, // cursor at '!' (error node)
+      expectedEnd: 13, // error node returns pos
+    },
   ];
   testCases.forEach((value) => {
     it(value.title, () => {
@@ -1398,7 +1446,7 @@ describe('autocomplete promQL test', () => {
       expectedResult: {
         options: [],
         from: 10,
-        to: 10,
+        to: 11,
         validFor: /^[a-zA-Z0-9_:]+$/,
       },
     },
@@ -1409,7 +1457,7 @@ describe('autocomplete promQL test', () => {
       expectedResult: {
         options: [],
         from: 10,
-        to: 10,
+        to: 12,
         validFor: /^[a-zA-Z0-9_:]+$/,
       },
     },
@@ -1564,7 +1612,7 @@ describe('autocomplete promQL test', () => {
       const context = new CompletionContext(state, value.pos, true);
       const completion = newCompleteStrategy(value.conf);
       const result = await completion.promQL(context);
-      expect(value.expectedResult).toEqual(result);
+      expect(result).toEqual(value.expectedResult);
     });
   });
 
diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
index 23e47ce649..2dd342d305 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
@@ -167,18 +167,33 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i
 }
 
 // computeEndCompletePosition calculates the end position for autocompletion replacement.
-// When the cursor is in the middle of an identifier (e.g., metric name) or label name, this ensures
-// the entire token is replaced, not just the portion before the cursor. This fixes issue #15839.
+// When the cursor is in the middle of a token, this ensures the entire token is replaced,
+// not just the portion before the cursor. This fixes issue #15839.
 // Note: this method is exported only for testing purpose.
 export function computeEndCompletePosition(state: EditorState, node: SyntaxNode, pos: number): number {
-  // For Identifier nodes (metric names) and LabelName nodes (label names in matchers,
-  // grouping clauses, etc.), extend the end position to include the entire token,
-  // even if the cursor is in the middle.
-  if (node.type.id === Identifier || node.type.id === LabelName) {
-    return node.to;
+  // For error nodes, use the cursor position as the end position
+  if (node.type.id === 0) {
+    return pos;
   }
-  // Default: use the cursor position as the end position
-  return pos;
+
+  if (
+    node.type.id === LabelMatchers ||
+    node.type.id === GroupingLabels ||
+    node.type.id === FunctionCallBody ||
+    node.type.id === MatrixSelector ||
+    node.type.id === SubqueryExpr
+  ) {
+    // When we're inside empty brackets, we want to replace up to just before the closing bracket.
+    return node.to - 1;
+  }
+
+  if (node.type.id === StringLiteral && (node.parent?.type.id === UnquotedLabelMatcher || node.parent?.type.id === QuotedLabelMatcher)) {
+    // For label values, we want to replace all content inside the quotes.
+    return node.parent.to - 1;
+  }
+
+  // For all other nodes, extend the end position to include the entire token.
+  return node.to;
 }
 
 // Matches complete PromQL durations, including compound units (e.g., 5m, 1d2h, 1h30m, etc.).

From fe76e6c297c58f21d969782ac3ade29cb8306949 Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Tue, 6 Jan 2026 11:00:01 +0100
Subject: [PATCH 214/439] Remove unneeded state parameter

Signed-off-by: Julius Volz 
---
 web/ui/module/codemirror-promql/src/complete/hybrid.test.ts | 2 +-
 web/ui/module/codemirror-promql/src/complete/hybrid.ts      | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
index 40c4356cc6..facda35ac8 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.test.ts
@@ -999,7 +999,7 @@ describe('computeEndCompletePosition test', () => {
     it(value.title, () => {
       const state = createEditorState(value.expr);
       const node = syntaxTree(state).resolve(value.pos, -1);
-      const result = computeEndCompletePosition(state, node, value.pos);
+      const result = computeEndCompletePosition(node, value.pos);
       expect(result).toEqual(value.expectedEnd);
     });
   });
diff --git a/web/ui/module/codemirror-promql/src/complete/hybrid.ts b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
index 2dd342d305..84c101b43c 100644
--- a/web/ui/module/codemirror-promql/src/complete/hybrid.ts
+++ b/web/ui/module/codemirror-promql/src/complete/hybrid.ts
@@ -170,7 +170,7 @@ function arrayToCompletionResult(data: Completion[], from: number, to: number, i
 // When the cursor is in the middle of a token, this ensures the entire token is replaced,
 // not just the portion before the cursor. This fixes issue #15839.
 // Note: this method is exported only for testing purpose.
-export function computeEndCompletePosition(state: EditorState, node: SyntaxNode, pos: number): number {
+export function computeEndCompletePosition(node: SyntaxNode, pos: number): number {
   // For error nodes, use the cursor position as the end position
   if (node.type.id === 0) {
     return pos;
@@ -700,7 +700,7 @@ export class HybridComplete implements CompleteStrategy {
       return arrayToCompletionResult(
         result,
         computeStartCompletePosition(state, tree, pos),
-        computeEndCompletePosition(state, tree, pos),
+        computeEndCompletePosition(tree, pos),
         completeSnippet,
         span
       );

From 1e317d00987888807ba8998f0d0c452ac5eda463 Mon Sep 17 00:00:00 2001
From: Arthur Silva Sens 
Date: Tue, 6 Jan 2026 09:00:49 -0300
Subject: [PATCH 215/439] Add configuration option to control
 `extra-scrape-metrics` (#17606)

---
 cmd/prometheus/main.go                        |   6 +-
 cmd/prometheus/testdata/features.json         |   2 +-
 config/config.go                              |  31 +++-
 config/config_test.go                         | 154 +++++++++++++++++-
 ...obal_disable_extra_scrape_metrics.good.yml |   6 +
 ...lobal_enable_extra_scrape_metrics.good.yml |   6 +
 ...ocal_disable_extra_scrape_metrics.good.yml |   7 +
 ...local_enable_extra_scrape_metrics.good.yml |   7 +
 docs/configuration/configuration.md           |  12 ++
 docs/feature_flags.md                         |   2 +
 scrape/manager.go                             |   4 +-
 scrape/scrape.go                              |   2 +-
 12 files changed, 226 insertions(+), 13 deletions(-)
 create mode 100644 config/testdata/global_disable_extra_scrape_metrics.good.yml
 create mode 100644 config/testdata/global_enable_extra_scrape_metrics.good.yml
 create mode 100644 config/testdata/local_disable_extra_scrape_metrics.good.yml
 create mode 100644 config/testdata/local_enable_extra_scrape_metrics.good.yml

diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go
index c330671b1e..ee60e58b2e 100644
--- a/cmd/prometheus/main.go
+++ b/cmd/prometheus/main.go
@@ -233,8 +233,10 @@ func (c *flagConfig) setFeatureListOptions(logger *slog.Logger) error {
 				c.tsdb.EnableMemorySnapshotOnShutdown = true
 				logger.Info("Experimental memory snapshot on shutdown enabled")
 			case "extra-scrape-metrics":
-				c.scrape.ExtraMetrics = true
-				logger.Info("Experimental additional scrape metrics enabled")
+				t := true
+				config.DefaultConfig.GlobalConfig.ExtraScrapeMetrics = &t
+				config.DefaultGlobalConfig.ExtraScrapeMetrics = &t
+				logger.Warn("This option for --enable-feature is being phased out. It currently changes the default for the extra_scrape_metrics config setting to true, but will become a no-op in a future version. Stop using this option and set extra_scrape_metrics in the config instead.", "option", o)
 			case "metadata-wal-records":
 				c.scrape.AppendMetadata = true
 				c.web.AppendMetadata = true
diff --git a/cmd/prometheus/testdata/features.json b/cmd/prometheus/testdata/features.json
index fbffd941fd..145bb04d77 100644
--- a/cmd/prometheus/testdata/features.json
+++ b/cmd/prometheus/testdata/features.json
@@ -166,7 +166,7 @@
     "query_offset": true
   },
   "scrape": {
-    "extra_scrape_metrics": false,
+    "extra_scrape_metrics": true,
     "start_timestamp_zero_ingestion": false,
     "type_and_unit_labels": false
   },
diff --git a/config/config.go b/config/config.go
index cce8fc4168..0b9b059ab2 100644
--- a/config/config.go
+++ b/config/config.go
@@ -149,6 +149,10 @@ func LoadFile(filename string, agentMode bool, logger *slog.Logger) (*Config, er
 	return cfg, nil
 }
 
+func boolPtr(b bool) *bool {
+	return &b
+}
+
 // The defaults applied before parsing the respective config sections.
 var (
 	// DefaultConfig is the default top-level configuration.
@@ -158,7 +162,6 @@ var (
 		OTLPConfig:   DefaultOTLPConfig,
 	}
 
-	f bool
 	// DefaultGlobalConfig is the default global configuration.
 	DefaultGlobalConfig = GlobalConfig{
 		ScrapeInterval:     model.Duration(1 * time.Minute),
@@ -173,9 +176,10 @@ var (
 		ScrapeProtocols: nil,
 		// When the native histogram feature flag is enabled,
 		// ScrapeNativeHistograms default changes to true.
-		ScrapeNativeHistograms:         &f,
+		ScrapeNativeHistograms:         boolPtr(false),
 		ConvertClassicHistogramsToNHCB: false,
 		AlwaysScrapeClassicHistograms:  false,
+		ExtraScrapeMetrics:             boolPtr(false),
 		MetricNameValidationScheme:     model.UTF8Validation,
 		MetricNameEscapingScheme:       model.AllowUTF8,
 	}
@@ -513,6 +517,10 @@ type GlobalConfig struct {
 	ConvertClassicHistogramsToNHCB bool `yaml:"convert_classic_histograms_to_nhcb,omitempty"`
 	// Whether to scrape a classic histogram, even if it is also exposed as a native histogram.
 	AlwaysScrapeClassicHistograms bool `yaml:"always_scrape_classic_histograms,omitempty"`
+	// Whether to enable additional scrape metrics.
+	// When enabled, Prometheus stores samples for scrape_timeout_seconds,
+	// scrape_sample_limit, and scrape_body_size_bytes.
+	ExtraScrapeMetrics *bool `yaml:"extra_scrape_metrics,omitempty"`
 }
 
 // ScrapeProtocol represents supported protocol for scraping metrics.
@@ -652,6 +660,9 @@ func (c *GlobalConfig) UnmarshalYAML(unmarshal func(any) error) error {
 	if gc.ScrapeNativeHistograms == nil {
 		gc.ScrapeNativeHistograms = DefaultGlobalConfig.ScrapeNativeHistograms
 	}
+	if gc.ExtraScrapeMetrics == nil {
+		gc.ExtraScrapeMetrics = DefaultGlobalConfig.ExtraScrapeMetrics
+	}
 	if gc.ScrapeProtocols == nil {
 		if DefaultGlobalConfig.ScrapeProtocols != nil {
 			// This is the case where the defaults are set due to a feature flag.
@@ -696,7 +707,8 @@ func (c *GlobalConfig) isZero() bool {
 		c.LabelValueLengthLimit == 0 &&
 		c.KeepDroppedTargets == 0 &&
 		c.MetricNameValidationScheme == model.UnsetValidation &&
-		c.MetricNameEscapingScheme == ""
+		c.MetricNameEscapingScheme == "" &&
+		c.ExtraScrapeMetrics == nil
 }
 
 const DefaultGoGCPercentage = 75
@@ -805,6 +817,11 @@ type ScrapeConfig struct {
 	// blank in config files but must have a value if a ScrapeConfig is created
 	// programmatically.
 	MetricNameEscapingScheme string `yaml:"metric_name_escaping_scheme,omitempty"`
+	// Whether to enable additional scrape metrics.
+	// When enabled, Prometheus stores samples for scrape_timeout_seconds,
+	// scrape_sample_limit, and scrape_body_size_bytes.
+	// If not set (nil), inherits the value from the global configuration.
+	ExtraScrapeMetrics *bool `yaml:"extra_scrape_metrics,omitempty"`
 
 	// We cannot do proper Go type embedding below as the parser will then parse
 	// values arbitrarily into the overflow maps of further-down types.
@@ -906,6 +923,9 @@ func (c *ScrapeConfig) Validate(globalConfig GlobalConfig) error {
 	if c.ScrapeNativeHistograms == nil {
 		c.ScrapeNativeHistograms = globalConfig.ScrapeNativeHistograms
 	}
+	if c.ExtraScrapeMetrics == nil {
+		c.ExtraScrapeMetrics = globalConfig.ExtraScrapeMetrics
+	}
 
 	if c.ScrapeProtocols == nil {
 		switch {
@@ -1054,6 +1074,11 @@ func (c *ScrapeConfig) AlwaysScrapeClassicHistogramsEnabled() bool {
 	return c.AlwaysScrapeClassicHistograms != nil && *c.AlwaysScrapeClassicHistograms
 }
 
+// ExtraScrapeMetricsEnabled returns whether to enable extra scrape metrics.
+func (c *ScrapeConfig) ExtraScrapeMetricsEnabled() bool {
+	return c.ExtraScrapeMetrics != nil && *c.ExtraScrapeMetrics
+}
+
 // StorageConfig configures runtime reloadable configuration options.
 type StorageConfig struct {
 	TSDBConfig      *TSDBConfig      `yaml:"tsdb,omitempty"`
diff --git a/config/config_test.go b/config/config_test.go
index aefdd5248c..08aa0b4f06 100644
--- a/config/config_test.go
+++ b/config/config_test.go
@@ -74,10 +74,6 @@ func mustParseURL(u string) *config.URL {
 	return &config.URL{URL: parsed}
 }
 
-func boolPtr(b bool) *bool {
-	return &b
-}
-
 const (
 	globBodySizeLimit         = 15 * units.MiB
 	globSampleLimit           = 1500
@@ -109,6 +105,7 @@ var expectedConf = &Config{
 		ScrapeNativeHistograms:         boolPtr(false),
 		AlwaysScrapeClassicHistograms:  false,
 		ConvertClassicHistogramsToNHCB: false,
+		ExtraScrapeMetrics:             boolPtr(false),
 		MetricNameValidationScheme:     model.UTF8Validation,
 	},
 
@@ -236,6 +233,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath: DefaultScrapeConfig.MetricsPath,
 			Scheme:      DefaultScrapeConfig.Scheme,
@@ -360,6 +358,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			HTTPClientConfig: config.HTTPClientConfig{
 				BasicAuth: &config.BasicAuth{
@@ -470,6 +469,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -532,6 +532,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath: "/metrics",
 			Scheme:      "http",
@@ -571,6 +572,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -616,6 +618,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath: DefaultScrapeConfig.MetricsPath,
 			Scheme:      DefaultScrapeConfig.Scheme,
@@ -661,6 +664,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -696,6 +700,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -739,6 +744,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -779,6 +785,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -826,6 +833,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -863,6 +871,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -903,6 +912,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -936,6 +946,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -972,6 +983,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      "/federate",
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1008,6 +1020,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1044,6 +1057,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1077,6 +1091,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1118,6 +1133,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1158,6 +1174,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1195,6 +1212,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1231,6 +1249,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1271,6 +1290,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1314,6 +1334,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(true),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1377,6 +1398,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1410,6 +1432,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			HTTPClientConfig: config.DefaultHTTPClientConfig,
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
@@ -1454,6 +1477,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			HTTPClientConfig: config.DefaultHTTPClientConfig,
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
@@ -1504,6 +1528,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1544,6 +1569,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1585,6 +1611,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			HTTPClientConfig: config.DefaultHTTPClientConfig,
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
@@ -1621,6 +1648,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -1659,6 +1687,7 @@ var expectedConf = &Config{
 			ScrapeNativeHistograms:         boolPtr(false),
 			AlwaysScrapeClassicHistograms:  boolPtr(false),
 			ConvertClassicHistogramsToNHCB: boolPtr(false),
+			ExtraScrapeMetrics:             boolPtr(false),
 
 			MetricsPath:      DefaultScrapeConfig.MetricsPath,
 			Scheme:           DefaultScrapeConfig.Scheme,
@@ -2755,6 +2784,7 @@ type ScrapeConfigOptions struct {
 	ScrapeNativeHistograms        bool
 	AlwaysScrapeClassicHistograms bool
 	ConvertClassicHistToNHCB      bool
+	ExtraScrapeMetrics            bool
 }
 
 func TestGetScrapeConfigs(t *testing.T) {
@@ -2788,6 +2818,7 @@ func TestGetScrapeConfigs(t *testing.T) {
 			ScrapeNativeHistograms:         boolPtr(opts.ScrapeNativeHistograms),
 			AlwaysScrapeClassicHistograms:  boolPtr(opts.AlwaysScrapeClassicHistograms),
 			ConvertClassicHistogramsToNHCB: boolPtr(opts.ConvertClassicHistToNHCB),
+			ExtraScrapeMetrics:             boolPtr(opts.ExtraScrapeMetrics),
 		}
 		if opts.ScrapeProtocols == nil {
 			sc.ScrapeProtocols = DefaultScrapeProtocols
@@ -2871,6 +2902,7 @@ func TestGetScrapeConfigs(t *testing.T) {
 					ScrapeNativeHistograms:         boolPtr(false),
 					AlwaysScrapeClassicHistograms:  boolPtr(false),
 					ConvertClassicHistogramsToNHCB: boolPtr(false),
+					ExtraScrapeMetrics:             boolPtr(false),
 
 					MetricsPath: DefaultScrapeConfig.MetricsPath,
 					Scheme:      DefaultScrapeConfig.Scheme,
@@ -2909,6 +2941,7 @@ func TestGetScrapeConfigs(t *testing.T) {
 					ScrapeNativeHistograms:         boolPtr(false),
 					AlwaysScrapeClassicHistograms:  boolPtr(false),
 					ConvertClassicHistogramsToNHCB: boolPtr(false),
+					ExtraScrapeMetrics:             boolPtr(false),
 
 					HTTPClientConfig: config.HTTPClientConfig{
 						TLSConfig: config.TLSConfig{
@@ -3021,6 +3054,26 @@ func TestGetScrapeConfigs(t *testing.T) {
 			configFile:     "testdata/global_scrape_protocols_and_local_disable_scrape_native_hist.good.yml",
 			expectedResult: []*ScrapeConfig{sc(ScrapeConfigOptions{JobName: "prometheus", ScrapeInterval: model.Duration(60 * time.Second), ScrapeTimeout: model.Duration(10 * time.Second), ScrapeNativeHistograms: false, ScrapeProtocols: []ScrapeProtocol{PrometheusText0_0_4}})},
 		},
+		{
+			name:           "A global config that enables extra scrape metrics",
+			configFile:     "testdata/global_enable_extra_scrape_metrics.good.yml",
+			expectedResult: []*ScrapeConfig{sc(ScrapeConfigOptions{JobName: "prometheus", ScrapeInterval: model.Duration(60 * time.Second), ScrapeTimeout: model.Duration(10 * time.Second), ExtraScrapeMetrics: true})},
+		},
+		{
+			name:           "A global config that disables extra scrape metrics",
+			configFile:     "testdata/global_disable_extra_scrape_metrics.good.yml",
+			expectedResult: []*ScrapeConfig{sc(ScrapeConfigOptions{JobName: "prometheus", ScrapeInterval: model.Duration(60 * time.Second), ScrapeTimeout: model.Duration(10 * time.Second), ExtraScrapeMetrics: false})},
+		},
+		{
+			name:           "A global config that disables extra scrape metrics and scrape config that enables it",
+			configFile:     "testdata/local_enable_extra_scrape_metrics.good.yml",
+			expectedResult: []*ScrapeConfig{sc(ScrapeConfigOptions{JobName: "prometheus", ScrapeInterval: model.Duration(60 * time.Second), ScrapeTimeout: model.Duration(10 * time.Second), ExtraScrapeMetrics: true})},
+		},
+		{
+			name:           "A global config that enables extra scrape metrics and scrape config that disables it",
+			configFile:     "testdata/local_disable_extra_scrape_metrics.good.yml",
+			expectedResult: []*ScrapeConfig{sc(ScrapeConfigOptions{JobName: "prometheus", ScrapeInterval: model.Duration(60 * time.Second), ScrapeTimeout: model.Duration(10 * time.Second), ExtraScrapeMetrics: false})},
+		},
 	}
 
 	for _, tc := range testCases {
@@ -3037,6 +3090,99 @@ func TestGetScrapeConfigs(t *testing.T) {
 	}
 }
 
+func TestExtraScrapeMetrics(t *testing.T) {
+	tests := []struct {
+		name          string
+		config        string
+		expectGlobal  *bool
+		expectEnabled bool
+	}{
+		{
+			name: "default values (not set)",
+			config: `
+scrape_configs:
+  - job_name: test
+    static_configs:
+      - targets: ['localhost:9090']
+`,
+			expectGlobal:  boolPtr(false), // inherits from DefaultGlobalConfig
+			expectEnabled: false,
+		},
+		{
+			name: "global enabled",
+			config: `
+global:
+  extra_scrape_metrics: true
+scrape_configs:
+  - job_name: test
+    static_configs:
+      - targets: ['localhost:9090']
+`,
+			expectGlobal:  boolPtr(true),
+			expectEnabled: true,
+		},
+		{
+			name: "global disabled",
+			config: `
+global:
+  extra_scrape_metrics: false
+scrape_configs:
+  - job_name: test
+    static_configs:
+      - targets: ['localhost:9090']
+`,
+			expectGlobal:  boolPtr(false),
+			expectEnabled: false,
+		},
+		{
+			name: "scrape override enabled",
+			config: `
+global:
+  extra_scrape_metrics: false
+scrape_configs:
+  - job_name: test
+    extra_scrape_metrics: true
+    static_configs:
+      - targets: ['localhost:9090']
+`,
+			expectGlobal:  boolPtr(false),
+			expectEnabled: true,
+		},
+		{
+			name: "scrape override disabled",
+			config: `
+global:
+  extra_scrape_metrics: true
+scrape_configs:
+  - job_name: test
+    extra_scrape_metrics: false
+    static_configs:
+      - targets: ['localhost:9090']
+`,
+			expectGlobal:  boolPtr(true),
+			expectEnabled: false,
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			cfg, err := Load(tc.config, promslog.NewNopLogger())
+			require.NoError(t, err)
+
+			// Check global config
+			require.Equal(t, tc.expectGlobal, cfg.GlobalConfig.ExtraScrapeMetrics)
+
+			// Check scrape config
+			scfgs, err := cfg.GetScrapeConfigs()
+			require.NoError(t, err)
+			require.Len(t, scfgs, 1)
+
+			// Check the effective value via the helper method
+			require.Equal(t, tc.expectEnabled, scfgs[0].ExtraScrapeMetricsEnabled())
+		})
+	}
+}
+
 func kubernetesSDHostURL() config.URL {
 	tURL, _ := url.Parse("https://localhost:1234")
 	return config.URL{URL: tURL}
diff --git a/config/testdata/global_disable_extra_scrape_metrics.good.yml b/config/testdata/global_disable_extra_scrape_metrics.good.yml
new file mode 100644
index 0000000000..26c6e4b8b5
--- /dev/null
+++ b/config/testdata/global_disable_extra_scrape_metrics.good.yml
@@ -0,0 +1,6 @@
+global:
+  extra_scrape_metrics: false
+scrape_configs:
+  - job_name: prometheus
+    static_configs:
+      - targets: ['localhost:8080']
diff --git a/config/testdata/global_enable_extra_scrape_metrics.good.yml b/config/testdata/global_enable_extra_scrape_metrics.good.yml
new file mode 100644
index 0000000000..1d7ea2db1c
--- /dev/null
+++ b/config/testdata/global_enable_extra_scrape_metrics.good.yml
@@ -0,0 +1,6 @@
+global:
+  extra_scrape_metrics: true
+scrape_configs:
+  - job_name: prometheus
+    static_configs:
+      - targets: ['localhost:8080']
diff --git a/config/testdata/local_disable_extra_scrape_metrics.good.yml b/config/testdata/local_disable_extra_scrape_metrics.good.yml
new file mode 100644
index 0000000000..a1b7c646fa
--- /dev/null
+++ b/config/testdata/local_disable_extra_scrape_metrics.good.yml
@@ -0,0 +1,7 @@
+global:
+  extra_scrape_metrics: true
+scrape_configs:
+  - job_name: prometheus
+    static_configs:
+      - targets: ['localhost:8080']
+    extra_scrape_metrics: false
diff --git a/config/testdata/local_enable_extra_scrape_metrics.good.yml b/config/testdata/local_enable_extra_scrape_metrics.good.yml
new file mode 100644
index 0000000000..a1c8b2808e
--- /dev/null
+++ b/config/testdata/local_enable_extra_scrape_metrics.good.yml
@@ -0,0 +1,7 @@
+global:
+  extra_scrape_metrics: false
+scrape_configs:
+  - job_name: prometheus
+    static_configs:
+      - targets: ['localhost:8080']
+    extra_scrape_metrics: true
diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md
index 3b71f26fc2..4079daae02 100644
--- a/docs/configuration/configuration.md
+++ b/docs/configuration/configuration.md
@@ -159,6 +159,12 @@ global:
   # native histogram with custom buckets.
   [ always_scrape_classic_histograms:  | default = false ]
 
+  # When enabled, Prometheus stores additional time series for each scrape:
+  # scrape_timeout_seconds, scrape_sample_limit, and scrape_body_size_bytes.
+  # These metrics help monitor how close targets are to their configured limits.
+  # This option can be overridden per scrape config.
+  [ extra_scrape_metrics:  | default = false ]
+
   # The following explains the various combinations of the last three options
   # in various exposition cases.
   #
@@ -647,6 +653,12 @@ metric_relabel_configs:
 # native histogram with custom buckets.
 [ always_scrape_classic_histograms:  | default =  ]
 
+# When enabled, Prometheus stores additional time series for this scrape job:
+# scrape_timeout_seconds, scrape_sample_limit, and scrape_body_size_bytes.
+# These metrics help monitor how close targets are to their configured limits.
+# If not set, inherits the value from the global configuration.
+[ extra_scrape_metrics:  | default =  ]
+
 # See global configuration above for further explanations of how the last three
 # options combine their effects.
 
diff --git a/docs/feature_flags.md b/docs/feature_flags.md
index 74daa11c13..af08eebb45 100644
--- a/docs/feature_flags.md
+++ b/docs/feature_flags.md
@@ -28,6 +28,8 @@ and m-mapped chunks, while a WAL replay from disk is only needed for the parts o
 
 `--enable-feature=extra-scrape-metrics`
 
+> **Note:** This feature flag is deprecated. Please use the `extra_scrape_metrics` configuration option instead (available at both global and scrape-config level). The feature flag will be removed in a future major version. See the [configuration documentation](configuration/configuration.md) for more details.
+
 When enabled, for each instance scrape, Prometheus stores a sample in the following additional time series:
 
 - `scrape_timeout_seconds`. The configured `scrape_timeout` for a target. This allows you to measure each target to find out how close they are to timing out with `scrape_duration_seconds / scrape_timeout_seconds`.
diff --git a/scrape/manager.go b/scrape/manager.go
index bd68c186c0..a2297aa824 100644
--- a/scrape/manager.go
+++ b/scrape/manager.go
@@ -70,7 +70,8 @@ func NewManager(o *Options, logger *slog.Logger, newScrapeFailureLogger func(str
 
 	// Register scrape features.
 	if r := o.FeatureRegistry; r != nil {
-		r.Set(features.Scrape, "extra_scrape_metrics", o.ExtraMetrics)
+		// "Extra scrape metrics" is always enabled because it moved from feature flag to config file.
+		r.Enable(features.Scrape, "extra_scrape_metrics")
 		r.Set(features.Scrape, "start_timestamp_zero_ingestion", o.EnableStartTimestampZeroIngestion)
 		r.Set(features.Scrape, "type_and_unit_labels", o.EnableTypeAndUnitLabels)
 	}
@@ -80,7 +81,6 @@ func NewManager(o *Options, logger *slog.Logger, newScrapeFailureLogger func(str
 
 // Options are the configuration parameters to the scrape manager.
 type Options struct {
-	ExtraMetrics bool
 	// Option used by downstream scraper users like OpenTelemetry Collector
 	// to help lookup metric metadata. Should be false for Prometheus.
 	PassMetadataInContext bool
diff --git a/scrape/scrape.go b/scrape/scrape.go
index 33683b4caf..70ca8ad42a 100644
--- a/scrape/scrape.go
+++ b/scrape/scrape.go
@@ -1212,12 +1212,12 @@ func newScrapeLoop(opts scrapeLoopOptions) *scrapeLoop {
 		fallbackScrapeProtocol:        opts.sp.config.ScrapeFallbackProtocol.HeaderMediaType(),
 		enableCompression:             opts.sp.config.EnableCompression,
 		mrc:                           opts.sp.config.MetricRelabelConfigs,
+		reportExtraMetrics:            opts.sp.config.ExtraScrapeMetricsEnabled(),
 		validationScheme:              opts.sp.config.MetricNameValidationScheme,
 
 		// scrape.Options.
 		enableSTZeroIngestion:   opts.sp.options.EnableStartTimestampZeroIngestion,
 		enableTypeAndUnitLabels: opts.sp.options.EnableTypeAndUnitLabels,
-		reportExtraMetrics:      opts.sp.options.ExtraMetrics,
 		appendMetadataToWAL:     opts.sp.options.AppendMetadata,
 		passMetadataInContext:   opts.sp.options.PassMetadataInContext,
 		skipOffsetting:          opts.sp.options.skipOffsetting,

From f13283a5be5c60c672cebffea0cccf101425bda1 Mon Sep 17 00:00:00 2001
From: Chuanye Gao 
Date: Tue, 6 Jan 2026 19:58:23 +0800
Subject: [PATCH 216/439] web: fix ready endpoint stopping header and add test
 coverage

Signed-off-by: Chuanye Gao 
---
 web/web.go      |  2 +-
 web/web_test.go | 21 +++++++++++++++++++++
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/web/web.go b/web/web.go
index afe78e4255..4df447be64 100644
--- a/web/web.go
+++ b/web/web.go
@@ -634,8 +634,8 @@ func (h *Handler) testReady(f http.HandlerFunc) http.HandlerFunc {
 		case Ready:
 			f(w, r)
 		case NotReady:
-			w.WriteHeader(http.StatusServiceUnavailable)
 			w.Header().Set("X-Prometheus-Stopping", "false")
+			w.WriteHeader(http.StatusServiceUnavailable)
 			fmt.Fprintf(w, "Service Unavailable")
 		case Stopping:
 			w.Header().Set("X-Prometheus-Stopping", "true")
diff --git a/web/web_test.go b/web/web_test.go
index ae7d532f1f..ce682912a9 100644
--- a/web/web_test.go
+++ b/web/web_test.go
@@ -140,11 +140,32 @@ func TestReadyAndHealthy(t *testing.T) {
 		resp, err = http.Get(u)
 		require.NoError(t, err)
 		require.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)
+		require.Equal(t, "false", resp.Header.Get("X-Prometheus-Stopping"))
 		cleanupTestResponse(t, resp)
 
 		resp, err = http.Head(u)
 		require.NoError(t, err)
 		require.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)
+		require.Equal(t, "false", resp.Header.Get("X-Prometheus-Stopping"))
+		cleanupTestResponse(t, resp)
+	}
+
+	// Set to stopping
+	webHandler.SetReady(Stopping)
+
+	for _, u := range []string{
+		baseURL + "/-/ready",
+	} {
+		resp, err = http.Get(u)
+		require.NoError(t, err)
+		require.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)
+		require.Equal(t, "true", resp.Header.Get("X-Prometheus-Stopping"))
+		cleanupTestResponse(t, resp)
+
+		resp, err = http.Head(u)
+		require.NoError(t, err)
+		require.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)
+		require.Equal(t, "true", resp.Header.Get("X-Prometheus-Stopping"))
 		cleanupTestResponse(t, resp)
 	}
 

From dcda4840a70ac6f3f29c82b0e7947c4586537a41 Mon Sep 17 00:00:00 2001
From: Patryk Prus 
Date: Tue, 6 Jan 2026 08:07:23 -0500
Subject: [PATCH 217/439] tsdb/index: export sentinel errors for size limit
 failures (#17773)

* tsdb/index: export sentinel errors for size limit failures

---------

Signed-off-by: Patryk Prus 
Co-authored-by: Arve Knudsen 
---
 tsdb/index/index.go | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/tsdb/index/index.go b/tsdb/index/index.go
index 1ddcac9501..8a76770821 100644
--- a/tsdb/index/index.go
+++ b/tsdb/index/index.go
@@ -17,6 +17,7 @@ import (
 	"bufio"
 	"context"
 	"encoding/binary"
+	"errors"
 	"fmt"
 	"hash"
 	"hash/crc32"
@@ -94,6 +95,13 @@ func (s indexWriterStage) String() string {
 	return ""
 }
 
+// ErrPostingsOffsetTableTooLarge is returned when the postings offset table length
+// would exceed 4 bytes (table would exceed the 4GB limit).
+var ErrPostingsOffsetTableTooLarge = errors.New("length size exceeds 4 bytes")
+
+// ErrIndexExceeds64GiB is returned when the index file would exceed the 64GiB limit.
+var ErrIndexExceeds64GiB = errors.New("exceeding max size of 64GiB")
+
 // The table gets initialized with sync.Once but may still cause a race
 // with any other use of the crc32 package anywhere. Thus we initialize it
 // before.
@@ -303,7 +311,7 @@ func (fw *FileWriter) Write(bufs ...[]byte) error {
 		// Once we move to compressed/varint representations in those areas, this limitation
 		// can be lifted.
 		if fw.pos > 16*math.MaxUint32 {
-			return fmt.Errorf("%q exceeding max size of 64GiB", fw.name)
+			return fmt.Errorf("%q %w", fw.name, ErrIndexExceeds64GiB)
 		}
 	}
 	return nil
@@ -660,7 +668,7 @@ func (w *Writer) writeLengthAndHash(startPos uint64) error {
 	w.buf1.Reset()
 	l := w.f.pos - startPos - 4
 	if l > math.MaxUint32 {
-		return fmt.Errorf("length size exceeds 4 bytes: %d", l)
+		return fmt.Errorf("%w: %d", ErrPostingsOffsetTableTooLarge, l)
 	}
 	w.buf1.PutBE32int(int(l))
 	if err := w.writeAt(w.buf1.Get(), startPos); err != nil {

From 167418a5ad1b72ce4904e8f1c3cdde05735abbb5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jan-Otto=20Kr=C3=B6pke?= 
Date: Tue, 6 Jan 2026 14:34:29 +0100
Subject: [PATCH 218/439] fix: renovate configuration (#17793)

Co-authored-by: Arve Knudsen 
---
 renovate.json | 170 +++++++++++++++++++++++++-------------------------
 1 file changed, 85 insertions(+), 85 deletions(-)

diff --git a/renovate.json b/renovate.json
index e9e383337b..350cfe2a0d 100644
--- a/renovate.json
+++ b/renovate.json
@@ -1,89 +1,89 @@
 {
-    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
-    "extends": [
-      "config:recommended"
-    ],
-    "separateMultipleMajor": true,
-    "baseBranches": ["main"],
-    "postUpdateOptions": [
-      "gomodTidy",
-      "gomodUpdateImportPaths"
-    ],
-    "schedule": ["57 11 21 * *"],
-    "timezone": "UTC",
-    "github-actions": {
-      "managerFilePatterns": ["scripts/**"]
+  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
+  "extends": [
+    "config:recommended"
+  ],
+  "separateMultipleMajor": true,
+  "baseBranches": ["main"],
+  "postUpdateOptions": [
+    "gomodTidy",
+    "gomodUpdateImportPaths"
+  ],
+  "schedule": ["* 11 21 * *"],
+  "timezone": "UTC",
+  "github-actions": {
+    "managerFilePatterns": ["scripts/**"]
+  },
+  "packageRules": [
+    {
+      "description": "Don't update replace directives",
+      "matchPackageNames": [
+        "github.com/fsnotify/fsnotify"
+      ],
+      "enabled": false
     },
-    "packageRules": [
-      {
-        "description": "Don't update replace directives",
-        "matchPackageNames": [
-          "github.com/fsnotify/fsnotify"
-        ],
-        "enabled": false
-      },
-      {
-        "description": "Don't update prometheus-io namespace packages",
-        "matchPackageNames": ["@prometheus-io/**"],
-        "enabled": false
-      },
-      {
-        "description": "Group AWS Go dependencies",
-        "matchManagers": ["gomod"],
-        "matchPackageNames": ["github.com/aws/**"],
-        "groupName": "AWS Go dependencies"
-      },
-      {
-        "description": "Group Azure Go dependencies",
-        "matchManagers": ["gomod"],
-        "matchPackageNames": ["github.com/Azure/**"],
-        "groupName": "Azure Go dependencies"
-      },
-      {
-        "description": "Group Kubernetes Go dependencies",
-        "matchManagers": ["gomod"],
-        "matchPackageNames": ["k8s.io/**"],
-        "groupName": "Kubernetes Go dependencies"
-      },
-      {
-        "description": "Group OpenTelemetry Go dependencies",
-        "matchManagers": ["gomod"],
-        "matchPackageNames": ["go.opentelemetry.io/**"],
-        "groupName": "OpenTelemetry Go dependencies"
-      },
-      {
-        "description": "Group Mantine UI dependencies",
-        "matchFileNames": [
-          "web/ui/mantine-ui/package.json"
-        ],
-        "groupName": "Mantine UI",
-        "matchUpdateTypes": ["minor", "patch"],
-        "enabled": true
-      },
-      {
-        "description": "Group React App dependencies",
-        "matchFileNames": [
-          "web/ui/react-app/package.json"
-        ],
-        "groupName": "React App",
-        "matchUpdateTypes": ["minor", "patch"],
-        "enabled": true
-      },
-      {
-        "description": "Group module dependencies",
-        "matchFileNames": [
-          "web/ui/module/**/package.json"
-        ],
-        "groupName": "Modules",
-        "matchUpdateTypes": ["minor", "patch"],
-        "enabled": true
-      }
-    ],
-    "branchPrefix": "deps-update/",
-    "vulnerabilityAlerts": {
-      "enabled": true,
-      "labels": ["security-update"]
+    {
+      "description": "Don't update prometheus-io namespace packages",
+      "matchPackageNames": ["@prometheus-io/**"],
+      "enabled": false
     },
-    "osvVulnerabilityAlerts": true,
-    "dependencyDashboardApproval": false
+    {
+      "description": "Group AWS Go dependencies",
+      "matchManagers": ["gomod"],
+      "matchPackageNames": ["github.com/aws/**"],
+      "groupName": "AWS Go dependencies"
+    },
+    {
+      "description": "Group Azure Go dependencies",
+      "matchManagers": ["gomod"],
+      "matchPackageNames": ["github.com/Azure/**"],
+      "groupName": "Azure Go dependencies"
+    },
+    {
+      "description": "Group Kubernetes Go dependencies",
+      "matchManagers": ["gomod"],
+      "matchPackageNames": ["k8s.io/**"],
+      "groupName": "Kubernetes Go dependencies"
+    },
+    {
+      "description": "Group OpenTelemetry Go dependencies",
+      "matchManagers": ["gomod"],
+      "matchPackageNames": ["go.opentelemetry.io/**"],
+      "groupName": "OpenTelemetry Go dependencies"
+    },
+    {
+      "description": "Group Mantine UI dependencies",
+      "matchFileNames": [
+        "web/ui/mantine-ui/package.json"
+      ],
+      "groupName": "Mantine UI",
+      "matchUpdateTypes": ["minor", "patch"],
+      "enabled": true
+    },
+    {
+      "description": "Group React App dependencies",
+      "matchFileNames": [
+        "web/ui/react-app/package.json"
+      ],
+      "groupName": "React App",
+      "matchUpdateTypes": ["minor", "patch"],
+      "enabled": true
+    },
+    {
+      "description": "Group module dependencies",
+      "matchFileNames": [
+        "web/ui/module/**/package.json"
+      ],
+      "groupName": "Modules",
+      "matchUpdateTypes": ["minor", "patch"],
+      "enabled": true
+    }
+  ],
+  "branchPrefix": "deps-update/",
+  "vulnerabilityAlerts": {
+    "enabled": true,
+    "labels": ["security-update"]
+  },
+  "osvVulnerabilityAlerts": true,
+  "dependencyDashboardApproval": false
 }

From 5b257abc5264a67f649a839e03a3225f94886386 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 6 Jan 2026 14:44:23 +0100
Subject: [PATCH 219/439] chore(deps): update dependency prettier to v3.7.4
 (#17782)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 web/ui/package-lock.json | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json
index 5df415da49..32dc306240 100644
--- a/web/ui/package-lock.json
+++ b/web/ui/package-lock.json
@@ -7551,9 +7551,9 @@
       }
     },
     "node_modules/prettier": {
-      "version": "3.6.2",
-      "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.6.2.tgz",
-      "integrity": "sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ==",
+      "version": "3.7.4",
+      "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.7.4.tgz",
+      "integrity": "sha512-v6UNi1+3hSlVvv8fSaoUbggEM5VErKmmpGA7Pl3HF8V6uKY7rvClBOJlH6yNwQtfTueNkGVpOv/mtWL9L4bgRA==",
       "dev": true,
       "license": "MIT",
       "bin": {

From a588145bc14b26b1597254b0bce36577a5819820 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 6 Jan 2026 14:46:47 +0100
Subject: [PATCH 220/439] fix(deps): update
 github.com/prometheus/client_golang/exp digest to 2cd067e (#17781)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/go.mod b/go.mod
index 808b391c45..0cda3321e8 100644
--- a/go.mod
+++ b/go.mod
@@ -56,7 +56,7 @@ require (
 	github.com/ovh/go-ovh v1.9.0
 	github.com/prometheus/alertmanager v0.30.0
 	github.com/prometheus/client_golang v1.23.2
-	github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca
+	github.com/prometheus/client_golang/exp v0.0.0-20260101091701-2cd067eb23c9
 	github.com/prometheus/client_model v0.6.2
 	github.com/prometheus/common v0.67.4
 	github.com/prometheus/common/assets v0.2.0
diff --git a/go.sum b/go.sum
index bbe0ea9129..4f470060b8 100644
--- a/go.sum
+++ b/go.sum
@@ -462,8 +462,8 @@ github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5Fsn
 github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
 github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
 github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
-github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca h1:BOxmsLoL2ymn8lXJtorca7N/m+2vDQUDoEtPjf0iAxA=
-github.com/prometheus/client_golang/exp v0.0.0-20251212205219-7ba246a648ca/go.mod h1:gndBHh3ZdjBozGcGrjUYjN3UJLRS3l2drALtu4lUt+k=
+github.com/prometheus/client_golang/exp v0.0.0-20260101091701-2cd067eb23c9 h1:al1B/YzHmaXhacIFkrZSDSUpnPHV4ZPMfENQpvk3PZQ=
+github.com/prometheus/client_golang/exp v0.0.0-20260101091701-2cd067eb23c9/go.mod h1:PmAYDB13uBFBG9qE1qxZZgZWhg7Rg6SfKM5DMK7hjyI=
 github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
 github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=

From ce8bb9ee9b0c2e60d8a53d64938eaa53d3a0943c Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 6 Jan 2026 13:52:10 +0000
Subject: [PATCH 221/439] chore(deps): update quay.io/prometheus/golang-builder
 docker tag to v1.25 (#17783)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 .github/workflows/ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 22b8b55a26..8d25176252 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -57,7 +57,7 @@ jobs:
       GOEXPERIMENT: synctest
     container:
       # The go version in this image should be N-1 wrt test_go.
-      image: quay.io/prometheus/golang-builder:1.24-base
+      image: quay.io/prometheus/golang-builder:1.25-base
     steps:
       - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:

From 6286e3fb55ff3fe2e9158b09b29a3dc040502933 Mon Sep 17 00:00:00 2001
From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com>
Date: Tue, 6 Jan 2026 15:53:09 +0100
Subject: [PATCH 222/439] fix(deps): update github.com/hashicorp/nomad/api
 digest to e8f2200 (#17780)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
---
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/go.mod b/go.mod
index 0cda3321e8..f0207b5e06 100644
--- a/go.mod
+++ b/go.mod
@@ -39,7 +39,7 @@ require (
 	github.com/gophercloud/gophercloud/v2 v2.9.0
 	github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853
 	github.com/hashicorp/consul/api v1.32.1
-	github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671
+	github.com/hashicorp/nomad/api v0.0.0-20260106084653-e8f2200c7039
 	github.com/hetznercloud/hcloud-go/v2 v2.32.0
 	github.com/ionos-cloud/sdk-go/v6 v6.3.6
 	github.com/json-iterator/go v1.1.12
diff --git a/go.sum b/go.sum
index 4f470060b8..ca7936bdcc 100644
--- a/go.sum
+++ b/go.sum
@@ -305,8 +305,8 @@ github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/
 github.com/hashicorp/memberlist v0.5.0/go.mod h1:yvyXLpo0QaGE59Y7hDTsTzDD25JYBZ4mHgHUZ8lrOI0=
 github.com/hashicorp/memberlist v0.5.3 h1:tQ1jOCypD0WvMemw/ZhhtH+PWpzcftQvgCorLu0hndk=
 github.com/hashicorp/memberlist v0.5.3/go.mod h1:h60o12SZn/ua/j0B6iKAZezA4eDaGsIuPO70eOaJ6WE=
-github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671 h1:4NbynIRljuOUvAQNLLJA1yuWcoL5EC3Qn4c7HCngUds=
-github.com/hashicorp/nomad/api v0.0.0-20251222083347-1355d4cb1671/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE=
+github.com/hashicorp/nomad/api v0.0.0-20260106084653-e8f2200c7039 h1:77URO0yPjlPjRc00KbjoBTG2dqHXFKA7Fv3s98w16kM=
+github.com/hashicorp/nomad/api v0.0.0-20260106084653-e8f2200c7039/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE=
 github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY=
 github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4=
 github.com/hetznercloud/hcloud-go/v2 v2.32.0 h1:BRe+k7ESdYv3xQLBGdKUfk+XBFRJNGKzq70nJI24ciM=

From cd875bd8c9211d7606981223d59ab3adf73432f2 Mon Sep 17 00:00:00 2001
From: Bryan Boreham 
Date: Tue, 6 Jan 2026 16:30:06 +0000
Subject: [PATCH 223/439] Cut release 3.9.0 (#17796)

Signed-off-by: Bryan Boreham 
---
 CHANGELOG.md                                 |  2 +-
 VERSION                                      |  2 +-
 web/ui/mantine-ui/package.json               |  4 ++--
 web/ui/module/codemirror-promql/package.json |  4 ++--
 web/ui/module/lezer-promql/package.json      |  2 +-
 web/ui/package-lock.json                     | 14 +++++++-------
 web/ui/package.json                          |  2 +-
 7 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 05c9b71b0f..6113dd0156 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,6 +1,6 @@
 # Changelog
 
-## 3.9.0-rc.0 / 2025-12-18
+## 3.9.0 / 2026-01-06
 
 - [CHANGE] Native Histograms are no longer experimental! Make the `native-histogram` feature flag a no-op. Use `scrape_native_histograms` config option instead. #17528
 - [CHANGE] API: Add maximum limit of 10,000 sets of statistics to TSDB status endpoint. #17647
diff --git a/VERSION b/VERSION
index 44fc2364a9..a5c4c76339 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-3.9.0-rc.0
+3.9.0
diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json
index 7958d5db91..3ee4c6c48c 100644
--- a/web/ui/mantine-ui/package.json
+++ b/web/ui/mantine-ui/package.json
@@ -1,7 +1,7 @@
 {
   "name": "@prometheus-io/mantine-ui",
   "private": true,
-  "version": "0.309.0-rc.0",
+  "version": "0.309.0",
   "type": "module",
   "scripts": {
     "start": "vite",
@@ -28,7 +28,7 @@
     "@microsoft/fetch-event-source": "^2.0.1",
     "@nexucis/fuzzy": "^0.5.1",
     "@nexucis/kvsearch": "^0.9.1",
-    "@prometheus-io/codemirror-promql": "0.309.0-rc.0",
+    "@prometheus-io/codemirror-promql": "0.309.0",
     "@reduxjs/toolkit": "^2.10.1",
     "@tabler/icons-react": "^3.35.0",
     "@tanstack/react-query": "^5.90.7",
diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json
index 6ad2116497..227dc67ed6 100644
--- a/web/ui/module/codemirror-promql/package.json
+++ b/web/ui/module/codemirror-promql/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@prometheus-io/codemirror-promql",
-  "version": "0.309.0-rc.0",
+  "version": "0.309.0",
   "description": "a CodeMirror mode for the PromQL language",
   "types": "dist/esm/index.d.ts",
   "module": "dist/esm/index.js",
@@ -29,7 +29,7 @@
   },
   "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md",
   "dependencies": {
-    "@prometheus-io/lezer-promql": "0.309.0-rc.0",
+    "@prometheus-io/lezer-promql": "0.309.0",
     "lru-cache": "^11.2.2"
   },
   "devDependencies": {
diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json
index d83e1a6488..e1cf0ad67b 100644
--- a/web/ui/module/lezer-promql/package.json
+++ b/web/ui/module/lezer-promql/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@prometheus-io/lezer-promql",
-  "version": "0.309.0-rc.0",
+  "version": "0.309.0",
   "description": "lezer-based PromQL grammar",
   "main": "dist/index.cjs",
   "type": "module",
diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json
index 23ae580c20..c52491732d 100644
--- a/web/ui/package-lock.json
+++ b/web/ui/package-lock.json
@@ -1,12 +1,12 @@
 {
   "name": "prometheus-io",
-  "version": "0.309.0-rc.0",
+  "version": "0.309.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "prometheus-io",
-      "version": "0.309.0-rc.0",
+      "version": "0.309.0",
       "workspaces": [
         "mantine-ui",
         "module/*"
@@ -24,7 +24,7 @@
     },
     "mantine-ui": {
       "name": "@prometheus-io/mantine-ui",
-      "version": "0.309.0-rc.0",
+      "version": "0.309.0",
       "dependencies": {
         "@codemirror/autocomplete": "^6.19.1",
         "@codemirror/language": "^6.11.3",
@@ -42,7 +42,7 @@
         "@microsoft/fetch-event-source": "^2.0.1",
         "@nexucis/fuzzy": "^0.5.1",
         "@nexucis/kvsearch": "^0.9.1",
-        "@prometheus-io/codemirror-promql": "0.309.0-rc.0",
+        "@prometheus-io/codemirror-promql": "0.309.0",
         "@reduxjs/toolkit": "^2.10.1",
         "@tabler/icons-react": "^3.35.0",
         "@tanstack/react-query": "^5.90.7",
@@ -88,10 +88,10 @@
     },
     "module/codemirror-promql": {
       "name": "@prometheus-io/codemirror-promql",
-      "version": "0.309.0-rc.0",
+      "version": "0.309.0",
       "license": "Apache-2.0",
       "dependencies": {
-        "@prometheus-io/lezer-promql": "0.309.0-rc.0",
+        "@prometheus-io/lezer-promql": "0.309.0",
         "lru-cache": "^11.2.2"
       },
       "devDependencies": {
@@ -121,7 +121,7 @@
     },
     "module/lezer-promql": {
       "name": "@prometheus-io/lezer-promql",
-      "version": "0.309.0-rc.0",
+      "version": "0.309.0",
       "license": "Apache-2.0",
       "devDependencies": {
         "@lezer/generator": "^1.8.0",
diff --git a/web/ui/package.json b/web/ui/package.json
index dd7d25628a..0f054c34a7 100644
--- a/web/ui/package.json
+++ b/web/ui/package.json
@@ -1,7 +1,7 @@
 {
   "name": "prometheus-io",
   "description": "Monorepo for the Prometheus UI",
-  "version": "0.309.0-rc.0",
+  "version": "0.309.0",
   "private": true,
   "scripts": {
     "build": "bash build_ui.sh --all",

From fc330642e497a5bdcb3dc3ab982a81b4e67c904c Mon Sep 17 00:00:00 2001
From: Rushabh Mehta 
Date: Fri, 26 Dec 2025 23:54:17 +0530
Subject: [PATCH 224/439] promql: Preallocate slice in extendFloats

Signed-off-by: Rushabh Mehta 
---
 promql/engine.go | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/promql/engine.go b/promql/engine.go
index a9f0dd2952..112fe17426 100644
--- a/promql/engine.go
+++ b/promql/engine.go
@@ -4418,9 +4418,9 @@ func extendFloats(floats []FPoint, mint, maxt int64, smoothed bool) []FPoint {
 		lastSampleIndex--
 	}
 
-	// TODO: Preallocate the length of the new list.
-	out := make([]FPoint, 0)
-	// Create the new floats list with the boundary samples and the inner samples.
+	count := max(lastSampleIndex-firstSampleIndex+1, 0)
+	out := make([]FPoint, 0, count+2)
+
 	out = append(out, FPoint{T: mint, F: left})
 	out = append(out, floats[firstSampleIndex:lastSampleIndex+1]...)
 	out = append(out, FPoint{T: maxt, F: right})

From 66bdc88013e6c6098da7026ce828d3b33235d527 Mon Sep 17 00:00:00 2001
From: George Krajcsovits 
Date: Wed, 7 Jan 2026 08:44:57 +0100
Subject: [PATCH 225/439] fix(remote_read): NHCB not returned over remote read
 samples (#17794)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

NHCB is native histograms with custom buckets.

prompb is used for both remote write 1.0 and remote read. We do not
support NHCB over remote write 1.0 , however we should absolutely
support it for remote read.

Prometheus remote write 1.0 client already refuses to send NHCB.
Prometheus remote write 1.0 server accepts NHCB, but doesn't store
custom values, corrupting the result. I'm now handling NHCB correctly,
instead of refusing or corrupting.

Signed-off-by: György Krajcsovits 
---
 prompb/codec.go                     |  5 ++-
 prompb/rwcommon/codec_test.go       | 21 +++++--------
 storage/remote/read_handler_test.go | 48 ++++++++++++++++++++++++++---
 3 files changed, 56 insertions(+), 18 deletions(-)

diff --git a/prompb/codec.go b/prompb/codec.go
index 9eb668a8e7..36490984a0 100644
--- a/prompb/codec.go
+++ b/prompb/codec.go
@@ -110,7 +110,7 @@ func (h Histogram) ToFloatHistogram() *histogram.FloatHistogram {
 			PositiveBuckets:  h.GetPositiveCounts(),
 			NegativeSpans:    spansProtoToSpans(h.GetNegativeSpans()),
 			NegativeBuckets:  h.GetNegativeCounts(),
-			CustomValues:     h.CustomValues,
+			CustomValues:     h.CustomValues, // CustomValues are immutable.
 		}
 	}
 	// Conversion from integer histogram.
@@ -125,6 +125,7 @@ func (h Histogram) ToFloatHistogram() *histogram.FloatHistogram {
 		PositiveBuckets:  deltasToCounts(h.GetPositiveDeltas()),
 		NegativeSpans:    spansProtoToSpans(h.GetNegativeSpans()),
 		NegativeBuckets:  deltasToCounts(h.GetNegativeDeltas()),
+		CustomValues:     h.CustomValues, // CustomValues are immutable.
 	}
 }
 
@@ -161,6 +162,7 @@ func FromIntHistogram(timestamp int64, h *histogram.Histogram) Histogram {
 		PositiveDeltas: h.PositiveBuckets,
 		ResetHint:      Histogram_ResetHint(h.CounterResetHint),
 		Timestamp:      timestamp,
+		CustomValues:   h.CustomValues, // CustomValues are immutable.
 	}
 }
 
@@ -178,6 +180,7 @@ func FromFloatHistogram(timestamp int64, fh *histogram.FloatHistogram) Histogram
 		PositiveCounts: fh.PositiveBuckets,
 		ResetHint:      Histogram_ResetHint(fh.CounterResetHint),
 		Timestamp:      timestamp,
+		CustomValues:   fh.CustomValues, // CustomValues are immutable.
 	}
 }
 
diff --git a/prompb/rwcommon/codec_test.go b/prompb/rwcommon/codec_test.go
index 2e0a72eff9..ee92581f59 100644
--- a/prompb/rwcommon/codec_test.go
+++ b/prompb/rwcommon/codec_test.go
@@ -198,17 +198,14 @@ func testFloatHistogram() histogram.FloatHistogram {
 
 func TestFromIntToFloatOrIntHistogram(t *testing.T) {
 	t.Run("v1", func(t *testing.T) {
-		// v1 does not support nhcb.
-		testIntHistWithoutNHCB := testIntHistogram()
-		testIntHistWithoutNHCB.CustomValues = nil
-		testFloatHistWithoutNHCB := testFloatHistogram()
-		testFloatHistWithoutNHCB.CustomValues = nil
+		testIntHist := testIntHistogram()
+		testFloatHist := testFloatHistogram()
 
-		h := prompb.FromIntHistogram(123, &testIntHistWithoutNHCB)
+		h := prompb.FromIntHistogram(123, &testIntHist)
 		require.False(t, h.IsFloatHistogram())
 		require.Equal(t, int64(123), h.Timestamp)
-		require.Equal(t, testIntHistWithoutNHCB, *h.ToIntHistogram())
-		require.Equal(t, testFloatHistWithoutNHCB, *h.ToFloatHistogram())
+		require.Equal(t, testIntHist, *h.ToIntHistogram())
+		require.Equal(t, testFloatHist, *h.ToFloatHistogram())
 	})
 	t.Run("v2", func(t *testing.T) {
 		testIntHist := testIntHistogram()
@@ -224,15 +221,13 @@ func TestFromIntToFloatOrIntHistogram(t *testing.T) {
 
 func TestFromFloatToFloatHistogram(t *testing.T) {
 	t.Run("v1", func(t *testing.T) {
-		// v1 does not support nhcb.
-		testFloatHistWithoutNHCB := testFloatHistogram()
-		testFloatHistWithoutNHCB.CustomValues = nil
+		testFloatHist := testFloatHistogram()
 
-		h := prompb.FromFloatHistogram(123, &testFloatHistWithoutNHCB)
+		h := prompb.FromFloatHistogram(123, &testFloatHist)
 		require.True(t, h.IsFloatHistogram())
 		require.Equal(t, int64(123), h.Timestamp)
 		require.Nil(t, h.ToIntHistogram())
-		require.Equal(t, testFloatHistWithoutNHCB, *h.ToFloatHistogram())
+		require.Equal(t, testFloatHist, *h.ToFloatHistogram())
 	})
 	t.Run("v2", func(t *testing.T) {
 		testFloatHist := testFloatHistogram()
diff --git a/storage/remote/read_handler_test.go b/storage/remote/read_handler_test.go
index 255a037d1e..a59c940f30 100644
--- a/storage/remote/read_handler_test.go
+++ b/storage/remote/read_handler_test.go
@@ -15,7 +15,6 @@ package remote
 
 import (
 	"bytes"
-	"context"
 	"errors"
 	"io"
 	"net/http"
@@ -28,6 +27,7 @@ import (
 	"github.com/stretchr/testify/require"
 
 	"github.com/prometheus/prometheus/config"
+	"github.com/prometheus/prometheus/model/histogram"
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/prompb"
 	"github.com/prometheus/prometheus/promql/promqltest"
@@ -64,13 +64,19 @@ func TestSampledReadEndpoint(t *testing.T) {
 	matcher3, err := labels.NewMatcher(labels.MatchEqual, "__name__", "test_histogram_metric1")
 	require.NoError(t, err)
 
+	matcher4, err := labels.NewMatcher(labels.MatchEqual, "__name__", "test_nhcb_metric1")
+	require.NoError(t, err)
+
 	query1, err := ToQuery(0, 1, []*labels.Matcher{matcher1, matcher2}, &storage.SelectHints{Step: 0, Func: "avg"})
 	require.NoError(t, err)
 
 	query2, err := ToQuery(0, 1, []*labels.Matcher{matcher3, matcher2}, &storage.SelectHints{Step: 0, Func: "avg"})
 	require.NoError(t, err)
 
-	req := &prompb.ReadRequest{Queries: []*prompb.Query{query1, query2}}
+	query3, err := ToQuery(0, 1, []*labels.Matcher{matcher4, matcher2}, &storage.SelectHints{Step: 0, Func: "avg"})
+	require.NoError(t, err)
+
+	req := &prompb.ReadRequest{Queries: []*prompb.Query{query1, query2, query3}}
 	data, err := proto.Marshal(req)
 	require.NoError(t, err)
 
@@ -97,7 +103,7 @@ func TestSampledReadEndpoint(t *testing.T) {
 	err = proto.Unmarshal(uncompressed, &resp)
 	require.NoError(t, err)
 
-	require.Len(t, resp.Results, 2, "Expected 2 results.")
+	require.Len(t, resp.Results, 3, "Expected 3 results.")
 
 	require.Equal(t, &prompb.QueryResult{
 		Timeseries: []*prompb.TimeSeries{
@@ -129,6 +135,33 @@ func TestSampledReadEndpoint(t *testing.T) {
 			},
 		},
 	}, resp.Results[1])
+
+	require.Equal(t, &prompb.QueryResult{
+		Timeseries: []*prompb.TimeSeries{
+			{
+				Labels: []prompb.Label{
+					{Name: "__name__", Value: "test_nhcb_metric1"},
+					{Name: "b", Value: "c"},
+					{Name: "baz", Value: "qux"},
+					{Name: "d", Value: "e"},
+				},
+				Histograms: []prompb.Histogram{{
+					// We cannot use prompb.FromFloatHistogram as that's one
+					// of the things we are testing here.
+					Schema:    histogram.CustomBucketsSchema,
+					Count:     &prompb.Histogram_CountFloat{CountFloat: 5},
+					Sum:       18.4,
+					ZeroCount: &prompb.Histogram_ZeroCountFloat{},
+					PositiveSpans: []prompb.BucketSpan{
+						{Offset: 0, Length: 2},
+						{Offset: 1, Length: 2},
+					},
+					PositiveCounts: []float64{1, 2, 1, 1},
+					CustomValues:   []float64{0, 1, 2, 3, 4},
+				}},
+			},
+		},
+	}, resp.Results[2])
 }
 
 func BenchmarkStreamReadEndpoint(b *testing.B) {
@@ -433,10 +466,17 @@ func TestStreamReadEndpoint(t *testing.T) {
 func addNativeHistogramsToTestSuite(t *testing.T, storage *teststorage.TestStorage, n int) {
 	lbls := labels.FromStrings("__name__", "test_histogram_metric1", "baz", "qux")
 
-	app := storage.Appender(context.TODO())
+	app := storage.Appender(t.Context())
 	for i, fh := range tsdbutil.GenerateTestFloatHistograms(n) {
 		_, err := app.AppendHistogram(0, lbls, int64(i)*int64(60*time.Second/time.Millisecond), nil, fh)
 		require.NoError(t, err)
 	}
+
+	lbls = labels.FromStrings("__name__", "test_nhcb_metric1", "baz", "qux")
+	for i, fh := range tsdbutil.GenerateTestCustomBucketsFloatHistograms(n) {
+		_, err := app.AppendHistogram(0, lbls, int64(i)*int64(60*time.Second/time.Millisecond), nil, fh)
+		require.NoError(t, err)
+	}
+
 	require.NoError(t, app.Commit())
 }

From 99c8351d0ef189a2d074efa59ac6c2321614f7c7 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Wed, 7 Jan 2026 13:21:56 +0100
Subject: [PATCH 226/439] chore(deps): bump
 github.com/hetznercloud/hcloud-go/v2 from 2.32.0 to 2.33.0 (#17762)

* chore(deps): bump github.com/hetznercloud/hcloud-go/v2

Bumps [github.com/hetznercloud/hcloud-go/v2](https://github.com/hetznercloud/hcloud-go) from 2.32.0 to 2.33.0.
- [Release notes](https://github.com/hetznercloud/hcloud-go/releases)
- [Changelog](https://github.com/hetznercloud/hcloud-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hetznercloud/hcloud-go/compare/v2.32.0...v2.33.0)

---
updated-dependencies:
- dependency-name: github.com/hetznercloud/hcloud-go/v2
  dependency-version: 2.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

* Use `server.Datacenter` until next minor release - disable linting of it in the meantime

---------

Signed-off-by: dependabot[bot] 
Signed-off-by: Arve Knudsen 
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arve Knudsen 
---
 discovery/hetzner/hcloud.go | 6 +++---
 go.mod                      | 2 +-
 go.sum                      | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/discovery/hetzner/hcloud.go b/discovery/hetzner/hcloud.go
index 61869459a3..7fe55ffded 100644
--- a/discovery/hetzner/hcloud.go
+++ b/discovery/hetzner/hcloud.go
@@ -98,13 +98,13 @@ func (d *hcloudDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, er
 			hetznerLabelRole:              model.LabelValue(HetznerRoleHcloud),
 			hetznerLabelServerID:          model.LabelValue(strconv.FormatInt(server.ID, 10)),
 			hetznerLabelServerName:        model.LabelValue(server.Name),
-			hetznerLabelDatacenter:        model.LabelValue(server.Datacenter.Name),
+			hetznerLabelDatacenter:        model.LabelValue(server.Datacenter.Name), //nolint:staticcheck // server.Datacenter is deprecated but kept for backwards compatibility until the next minor release
 			hetznerLabelPublicIPv4:        model.LabelValue(server.PublicNet.IPv4.IP.String()),
 			hetznerLabelPublicIPv6Network: model.LabelValue(server.PublicNet.IPv6.Network.String()),
 			hetznerLabelServerStatus:      model.LabelValue(server.Status),
 
-			hetznerLabelHcloudDatacenterLocation:            model.LabelValue(server.Datacenter.Location.Name),
-			hetznerLabelHcloudDatacenterLocationNetworkZone: model.LabelValue(server.Datacenter.Location.NetworkZone),
+			hetznerLabelHcloudDatacenterLocation:            model.LabelValue(server.Datacenter.Location.Name),        //nolint:staticcheck // server.Datacenter is deprecated but kept for backwards compatibility until the next minor release
+			hetznerLabelHcloudDatacenterLocationNetworkZone: model.LabelValue(server.Datacenter.Location.NetworkZone), //nolint:staticcheck // server.Datacenter is deprecated but kept for backwards compatibility until the next minor release
 			hetznerLabelHcloudType:                          model.LabelValue(server.ServerType.Name),
 			hetznerLabelHcloudCPUCores:                      model.LabelValue(strconv.Itoa(server.ServerType.Cores)),
 			hetznerLabelHcloudCPUType:                       model.LabelValue(server.ServerType.CPUType),
diff --git a/go.mod b/go.mod
index f0207b5e06..61c555abc2 100644
--- a/go.mod
+++ b/go.mod
@@ -40,7 +40,7 @@ require (
 	github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853
 	github.com/hashicorp/consul/api v1.32.1
 	github.com/hashicorp/nomad/api v0.0.0-20260106084653-e8f2200c7039
-	github.com/hetznercloud/hcloud-go/v2 v2.32.0
+	github.com/hetznercloud/hcloud-go/v2 v2.33.0
 	github.com/ionos-cloud/sdk-go/v6 v6.3.6
 	github.com/json-iterator/go v1.1.12
 	github.com/klauspost/compress v1.18.2
diff --git a/go.sum b/go.sum
index ca7936bdcc..b3333208dd 100644
--- a/go.sum
+++ b/go.sum
@@ -309,8 +309,8 @@ github.com/hashicorp/nomad/api v0.0.0-20260106084653-e8f2200c7039 h1:77URO0yPjlP
 github.com/hashicorp/nomad/api v0.0.0-20260106084653-e8f2200c7039/go.mod h1:sldFTIgs+FsUeKU3LwVjviAIuksxD8TzDOn02MYwslE=
 github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY=
 github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4=
-github.com/hetznercloud/hcloud-go/v2 v2.32.0 h1:BRe+k7ESdYv3xQLBGdKUfk+XBFRJNGKzq70nJI24ciM=
-github.com/hetznercloud/hcloud-go/v2 v2.32.0/go.mod h1:hAanyyfn9M0cMmZ68CXzPCF54KRb9EXd8eiE2FHKGIE=
+github.com/hetznercloud/hcloud-go/v2 v2.33.0 h1:g9hwuo60IXbupXJCYMlO4xDXgxxMPuFk31iOpLXDCV4=
+github.com/hetznercloud/hcloud-go/v2 v2.33.0/go.mod h1:GzYEl7slIGKc6Ttt08hjiJvGj8/PbWzcQf6IUi02dIs=
 github.com/ionos-cloud/sdk-go/v6 v6.3.6 h1:l/TtKgdQ1wUH3DDe2SfFD78AW+TJWdEbDpQhHkWd6CM=
 github.com/ionos-cloud/sdk-go/v6 v6.3.6/go.mod h1:nUGHP4kZHAZngCVr4v6C8nuargFrtvt7GrzH/hqn7c4=
 github.com/jarcoal/httpmock v1.4.1 h1:0Ju+VCFuARfFlhVXFc2HxlcQkfB+Xq12/EotHko+x2A=

From 22463b1e9fb2e15bf13093033fd1d08a651ae096 Mon Sep 17 00:00:00 2001
From: Julius Hinze 
Date: Wed, 7 Jan 2026 13:25:50 +0100
Subject: [PATCH 227/439] tsdb: add support for OOO exemplars in
 CircularExemplarStorage (#17469)

* tsdb: add support for OOO exemplars in CircularExemplarStorage

Doubly linked exemplar storage resize.
Split exemplar buffer resize into shrink and grow functions.
Skip duplicate OOO exemplars, re-initialize emptied index after deleting its last exemplar.

Signed-off-by: Julius Hinze 
---
 tsdb/exemplar.go            | 372 ++++++++++++-----
 tsdb/exemplar_test.go       | 776 ++++++++++++++++++++++++++++++++++--
 tsdb/head.go                |   4 +-
 util/teststorage/storage.go |   2 +-
 4 files changed, 1018 insertions(+), 136 deletions(-)

diff --git a/tsdb/exemplar.go b/tsdb/exemplar.go
index f0e755839c..b58976c911 100644
--- a/tsdb/exemplar.go
+++ b/tsdb/exemplar.go
@@ -36,10 +36,11 @@ const (
 )
 
 type CircularExemplarStorage struct {
-	lock      sync.RWMutex
-	exemplars []circularBufferEntry
-	nextIndex int
-	metrics   *ExemplarMetrics
+	lock                sync.RWMutex
+	exemplars           []circularBufferEntry
+	nextIndex           int
+	metrics             *ExemplarMetrics
+	oooTimeWindowMillis int64
 
 	// Map of series labels as a string to index entry, which points to the first
 	// and last exemplar for the series in the exemplars circular buffer.
@@ -55,6 +56,7 @@ type indexEntry struct {
 type circularBufferEntry struct {
 	exemplar exemplar.Exemplar
 	next     int
+	prev     int
 	ref      *indexEntry
 }
 
@@ -115,15 +117,19 @@ func NewExemplarMetrics(reg prometheus.Registerer) *ExemplarMetrics {
 // If we assume the average case 95 bytes per exemplar we can fit 5651272 exemplars in
 // 1GB of extra memory, accounting for the fact that this is heap allocated space.
 // If len <= 0, then the exemplar storage is essentially a noop storage but can later be
-// resized to store exemplars.
-func NewCircularExemplarStorage(length int64, m *ExemplarMetrics) (ExemplarStorage, error) {
+// resized to store exemplars. If oooTimeWindowMillis <= 0, out-of-order exemplars are disabled.
+func NewCircularExemplarStorage(length int64, m *ExemplarMetrics, oooTimeWindowMillis int64) (ExemplarStorage, error) {
 	if length < 0 {
 		length = 0
 	}
+	if oooTimeWindowMillis < 0 {
+		oooTimeWindowMillis = 0
+	}
 	c := &CircularExemplarStorage{
-		exemplars: make([]circularBufferEntry, length),
-		index:     make(map[string]*indexEntry, length/estimatedExemplarsPerSeries),
-		metrics:   m,
+		exemplars:           make([]circularBufferEntry, length),
+		index:               make(map[string]*indexEntry, length/estimatedExemplarsPerSeries),
+		metrics:             m,
+		oooTimeWindowMillis: oooTimeWindowMillis,
 	}
 
 	c.metrics.maxExemplars.Set(float64(length))
@@ -171,6 +177,9 @@ func (ce *CircularExemplarStorage) Select(start, end int64, matchers ...[]*label
 		}
 		se.SeriesLabels = idx.seriesLabels
 
+		// TODO: Since we maintain a doubly-linked-list, we can also iterate from head to tail
+		//  which might be more performant if the selected interval is skewed to the head.
+
 		// Loop through all exemplars in the circular buffer for the current series.
 		for e.exemplar.Ts <= end {
 			if e.exemplar.Ts >= start {
@@ -253,16 +262,12 @@ func (ce *CircularExemplarStorage) validateExemplar(idx *indexEntry, e exemplar.
 		return storage.ErrDuplicateExemplar
 	}
 
-	// Since during the scrape the exemplars are sorted first by timestamp, then value, then labels,
-	// if any of these conditions are true, we know that the exemplar is either a duplicate
-	// of a previous one (but not the most recent one as that is checked above) or out of order.
-	// We now allow exemplars with duplicate timestamps as long as they have different values and/or labels
-	// since that can happen for different buckets of a native histogram.
-	// We do not distinguish between duplicates and out of order as iterating through the exemplars
-	// to check for that would be expensive (versus just comparing with the most recent one) especially
-	// since this is run under a lock, and not worth it as we just need to return an error so we do not
-	// append the exemplar.
-	if e.Ts < newestExemplar.Ts ||
+	// Reject exemplars older than the OOO time window relative to the newest exemplar.
+	// Exemplars with the same timestamp are ordered by value then label hash to detect
+	// duplicates without iterating through all stored exemplars, which would be too
+	// expensive under lock. Exemplars with equal timestamps but different values or
+	// labels are allowed to support multiple buckets of native histograms.
+	if (e.Ts < newestExemplar.Ts && e.Ts <= newestExemplar.Ts-ce.oooTimeWindowMillis) ||
 		(e.Ts == newestExemplar.Ts && e.Value < newestExemplar.Value) ||
 		(e.Ts == newestExemplar.Ts && e.Value == newestExemplar.Value && e.Labels.Hash() < newestExemplar.Labels.Hash()) {
 		if appended {
@@ -273,8 +278,19 @@ func (ce *CircularExemplarStorage) validateExemplar(idx *indexEntry, e exemplar.
 	return nil
 }
 
-// Resize changes the size of exemplar buffer by allocating a new buffer and migrating data to it.
-// Exemplars are kept when possible. Shrinking will discard oldest data (in order of ingest) as needed.
+// SetOutOfOrderTimeWindow sets the out-of-order time window for exemplars in
+// milliseconds. Exemplars older than it are not added to the circular exemplar
+// buffer.
+func (ce *CircularExemplarStorage) SetOutOfOrderTimeWindow(d int64) {
+	ce.lock.Lock()
+	defer ce.lock.Unlock()
+	ce.oooTimeWindowMillis = d
+}
+
+// Resize changes the size of exemplar buffer by allocating a new buffer and
+// migrating data to it. Exemplars are kept when possible. Shrinking will discard
+// old data (in order of ingestion) as needed. Returns the number of migrated
+// exemplars.
 func (ce *CircularExemplarStorage) Resize(l int64) int {
 	// Accept negative values as just 0 size.
 	if l <= 0 {
@@ -284,65 +300,83 @@ func (ce *CircularExemplarStorage) Resize(l int64) int {
 	ce.lock.Lock()
 	defer ce.lock.Unlock()
 
-	if l == int64(len(ce.exemplars)) {
-		return 0
-	}
-
-	oldBuffer := ce.exemplars
-	oldNextIndex := int64(ce.nextIndex)
-
-	ce.exemplars = make([]circularBufferEntry, l)
-	ce.index = make(map[string]*indexEntry, l/estimatedExemplarsPerSeries)
-	ce.nextIndex = 0
-
-	// Replay as many entries as needed, starting with oldest first.
-	count := min(l, int64(len(oldBuffer)))
-
+	oldSize := int64(len(ce.exemplars))
 	migrated := 0
-
-	if l > 0 && len(oldBuffer) > 0 {
-		// Rewind previous next index by count with wrap-around.
-		// This math is essentially looking at nextIndex, where we would write the next exemplar to,
-		// and find the index in the old exemplar buffer that we should start migrating exemplars from.
-		// This way we don't migrate exemplars that would just be overwritten when migrating later exemplars.
-		startIndex := (oldNextIndex - count + int64(len(oldBuffer))) % int64(len(oldBuffer))
-
-		var buf [1024]byte
-		for i := range count {
-			idx := (startIndex + i) % int64(len(oldBuffer))
-			if oldBuffer[idx].ref != nil {
-				ce.migrate(&oldBuffer[idx], buf[:])
-				migrated++
-			}
-		}
+	switch {
+	case l == oldSize:
+		// NOOP.
+		return migrated
+	case l > oldSize:
+		migrated = ce.grow(l)
+	case l < oldSize:
+		migrated = ce.shrink(l)
 	}
 
 	ce.computeMetrics()
 	ce.metrics.maxExemplars.Set(float64(l))
-
 	return migrated
 }
 
-// migrate is like AddExemplar but reuses existing structs. Expected to be called in batch and requires
-// external lock and does not compute metrics.
-func (ce *CircularExemplarStorage) migrate(entry *circularBufferEntry, buf []byte) {
-	seriesLabels := entry.ref.seriesLabels.Bytes(buf[:0])
-
-	idx, ok := ce.index[string(seriesLabels)]
-	if !ok {
-		idx = entry.ref
-		idx.oldest = ce.nextIndex
-		ce.index[string(seriesLabels)] = idx
-	} else {
-		entry.ref = idx
-		ce.exemplars[idx.newest].next = ce.nextIndex
+// grow the circular buffer to have size l by allocating a new slice and copying
+// the old data to it. After growing, ce.nextIndex points to the next free entry
+// in the buffer. This function must be called with the lock acquired.
+func (ce *CircularExemplarStorage) grow(l int64) int {
+	oldSize := len(ce.exemplars)
+	newSlice := make([]circularBufferEntry, l)
+	ranges := []intRange{
+		{from: ce.nextIndex, to: oldSize},
+		{from: 0, to: ce.nextIndex},
 	}
-	idx.newest = ce.nextIndex
+	ce.nextIndex = copyExemplarRanges(ce.index, newSlice, ce.exemplars, ranges)
+	ce.exemplars = newSlice
+	return oldSize
+}
 
-	entry.next = noExemplar
-	ce.exemplars[ce.nextIndex] = *entry
+// shrink the circular buffer by either trimming from the right or deleting the
+// oldest samples to accommodate the new size l. This function must be called
+// with the lock acquired.
+func (ce *CircularExemplarStorage) shrink(l int64) (migrated int) {
+	oldSize := len(ce.exemplars)
+	diff := int(int64(oldSize) - l)
+	deleteStart := ce.nextIndex
+	deleteEnd := (deleteStart + diff) % oldSize
 
-	ce.nextIndex = (ce.nextIndex + 1) % len(ce.exemplars)
+	// Remove items from the buffer starting from c.nextIndex. This drops older
+	// entries first in the order of ingestion.
+	for i := range diff {
+		idx := (deleteStart + i) % oldSize
+		ref := ce.exemplars[idx].ref
+		if ce.removeExemplar(&ce.exemplars[idx]) {
+			ce.removeIndex(ref)
+		}
+	}
+
+	newSlice := make([]circularBufferEntry, int(l))
+
+	switch {
+	case deleteStart == deleteEnd:
+		// The entire buffer was cleared (shrink to zero). Note that we don't have to
+		// delete the index since removeExemplar already did. Simply remove all elements
+		// and reset tracking pointers.
+		ce.exemplars = newSlice
+		ce.nextIndex = 0
+		return 0
+	case deleteStart < deleteEnd:
+		// We delete an "inner" section of the circular buffer.
+		migrated = copyExemplarRanges(ce.index, newSlice, ce.exemplars, []intRange{
+			{from: deleteEnd, to: oldSize},
+			{from: 0, to: deleteStart},
+		})
+	case deleteStart > deleteEnd:
+		// We keep an "inner" section of the circular buffer.
+		migrated = copyExemplarRanges(ce.index, newSlice, ce.exemplars, []intRange{
+			{from: deleteEnd, to: deleteStart},
+		})
+	}
+
+	ce.nextIndex = migrated % int(l)
+	ce.exemplars = newSlice
+	return migrated
 }
 
 func (ce *CircularExemplarStorage) AddExemplar(l labels.Labels, e exemplar.Exemplar) error {
@@ -358,7 +392,7 @@ func (ce *CircularExemplarStorage) AddExemplar(l labels.Labels, e exemplar.Exemp
 	var buf [1024]byte
 	seriesLabels := l.Bytes(buf[:])
 
-	idx, ok := ce.index[string(seriesLabels)]
+	idx, indexExists := ce.index[string(seriesLabels)]
 	err := ce.validateExemplar(idx, e, true)
 	if err != nil {
 		if errors.Is(err, storage.ErrDuplicateExemplar) {
@@ -368,32 +402,77 @@ func (ce *CircularExemplarStorage) AddExemplar(l labels.Labels, e exemplar.Exemp
 		return err
 	}
 
-	if !ok {
-		idx = &indexEntry{oldest: ce.nextIndex, seriesLabels: l}
-		ce.index[string(seriesLabels)] = idx
-	} else {
-		ce.exemplars[idx.newest].next = ce.nextIndex
-	}
-
-	if prev := &ce.exemplars[ce.nextIndex]; prev.ref != nil {
-		// There exists an exemplar already on this ce.nextIndex entry,
-		// drop it, to make place for others.
-		if prev.next == noExemplar {
-			// Last item for this series, remove index entry.
-			var buf [1024]byte
-			prevLabels := prev.ref.seriesLabels.Bytes(buf[:])
-			delete(ce.index, string(prevLabels))
-		} else {
-			prev.ref.oldest = prev.next
+	// If we insert an out-of-order exemplar, we preemptively find the insertion
+	// index to check for duplicates.
+	var insertionIndex int
+	if indexExists {
+		outOfOrder := e.Ts >= ce.exemplars[idx.oldest].exemplar.Ts && e.Ts < ce.exemplars[idx.newest].exemplar.Ts
+		if outOfOrder {
+			insertionIndex = ce.findInsertionIndex(e, idx)
+			if ce.exemplars[insertionIndex].exemplar.Ts == e.Ts {
+				// Assume duplicate exemplar, noop.
+				// Native histograms will exercise this code path a lot due to
+				// having multiple exemplars per series so checking the
+				// value and labels would be too expensive.
+				return nil
+			}
 		}
 	}
 
-	// Default the next value to -1 (which we use to detect that we've iterated through all exemplars for a series in Select)
-	// since this is the first exemplar stored for this series.
-	ce.exemplars[ce.nextIndex].next = noExemplar
+	// If the index didn't exist (new series), create one.
+	if !indexExists {
+		idx = &indexEntry{seriesLabels: l}
+		ce.index[string(seriesLabels)] = idx
+	}
+
+	// Remove entries if the buffer is full. Note that this doesn't invalidate the
+	// insertion index since out-of-order exemplars cannot be the oldest exemplar.
+	if prev := &ce.exemplars[ce.nextIndex]; prev.ref != nil {
+		prevRef := prev.ref
+		if ce.removeExemplar(prev) {
+			if prevRef == idx {
+				// Do not delete the indexEntry we're inserting to.
+				indexExists = false
+			} else {
+				ce.removeIndex(prevRef)
+			}
+		}
+	}
+
+	// We create a new entry in the linked list.
 	ce.exemplars[ce.nextIndex].exemplar = e
 	ce.exemplars[ce.nextIndex].ref = idx
-	idx.newest = ce.nextIndex
+
+	switch {
+	case !indexExists:
+		// Add the first and only exemplar to the list.
+		idx.oldest = ce.nextIndex
+		idx.newest = ce.nextIndex
+		ce.exemplars[ce.nextIndex].prev = noExemplar
+		ce.exemplars[ce.nextIndex].next = noExemplar
+	case e.Ts >= ce.exemplars[idx.newest].exemplar.Ts:
+		// Add the exemplar at the tip (after newest).
+		ce.exemplars[idx.newest].next = ce.nextIndex
+		ce.exemplars[ce.nextIndex].prev = idx.newest
+		ce.exemplars[ce.nextIndex].next = noExemplar
+		idx.newest = ce.nextIndex
+	case e.Ts < ce.exemplars[idx.oldest].exemplar.Ts:
+		// Add the exemplar at the tail (before oldest).
+		ce.exemplars[idx.oldest].prev = ce.nextIndex
+		ce.exemplars[ce.nextIndex].prev = noExemplar
+		ce.exemplars[ce.nextIndex].next = idx.oldest
+		idx.oldest = ce.nextIndex
+	default:
+		// Insert the exemplar into the list by finding the most recent
+		// in-order exemplar that precedes it, and placing it after.
+		nextExemplar := ce.exemplars[insertionIndex].next
+		ce.exemplars[ce.nextIndex].prev = insertionIndex
+		ce.exemplars[ce.nextIndex].next = nextExemplar
+		ce.exemplars[insertionIndex].next = ce.nextIndex
+		if nextExemplar != noExemplar {
+			ce.exemplars[nextExemplar].prev = ce.nextIndex
+		}
+	}
 
 	ce.nextIndex = (ce.nextIndex + 1) % len(ce.exemplars)
 
@@ -402,6 +481,56 @@ func (ce *CircularExemplarStorage) AddExemplar(l labels.Labels, e exemplar.Exemp
 	return nil
 }
 
+// removeExemplar removes the given entry from the circular buffer. Returns true
+// iff the deleted entry was the last entry (and the index is now empty).
+// This function must be called with the lock acquired.
+func (ce *CircularExemplarStorage) removeExemplar(entry *circularBufferEntry) bool {
+	ref := entry.ref
+	if ref == nil {
+		return false
+	}
+
+	if entry.prev != noExemplar {
+		ce.exemplars[entry.prev].next = entry.next
+	} else {
+		ref.oldest = entry.next
+	}
+
+	if entry.next != noExemplar {
+		ce.exemplars[entry.next].prev = entry.prev
+	} else {
+		ref.newest = entry.prev
+	}
+
+	// Mark this item as deleted.
+	entry.ref = nil
+
+	return ref.oldest == noExemplar && ref.newest == noExemplar
+}
+
+// removeIndex removes an indexEntry from the circular exemplar storage.
+// This function must be called with the lock acquired.
+func (ce *CircularExemplarStorage) removeIndex(ref *indexEntry) {
+	var buf [1024]byte
+	entryLabels := ref.seriesLabels.Bytes(buf[:])
+	delete(ce.index, string(entryLabels))
+}
+
+// findInsertionIndex finds the position at which e should be placed in the
+// doubly-linked list by traversing the linked list from idx.newest to idx.oldest
+// and following back links. Since out-of-order exemplars commonly lie close to
+// the newest entry, traversing from newest to oldest is usually faster.
+func (ce *CircularExemplarStorage) findInsertionIndex(e exemplar.Exemplar, idx *indexEntry) int {
+	for i := idx.newest; i != noExemplar; {
+		current := ce.exemplars[i]
+		if current.exemplar.Ts <= e.Ts {
+			return i
+		}
+		i = current.prev
+	}
+	return idx.oldest
+}
+
 func (ce *CircularExemplarStorage) computeMetrics() {
 	ce.metrics.seriesWithExemplarsInStorage.Set(float64(len(ce.index)))
 
@@ -443,3 +572,64 @@ func (ce *CircularExemplarStorage) IterateExemplars(f func(seriesLabels labels.L
 	}
 	return nil
 }
+
+type intRange struct {
+	from, to int
+}
+
+func (e intRange) contains(i int) bool {
+	return i >= e.from && i < e.to
+}
+
+// copyExemplarRanges copies non-overlapping ranges from src into dest and
+// adjusts list pointers in dest and index accordingly. Returns the number of
+// copied items.
+func copyExemplarRanges(
+	index map[string]*indexEntry,
+	dest, src []circularBufferEntry,
+	ranges []intRange,
+) int {
+	offsets := make([]int, len(ranges))
+	n := 0
+	for i, rng := range ranges {
+		offsets[i] = n - rng.from
+		n += copy(dest[n:], src[rng.from:rng.to])
+	}
+	migratedEntries := n
+	for di := range n {
+		e := &dest[di]
+		if e.ref == nil {
+			// We potentially copied empty entries. Subtract them now to correctly show the
+			// number of "migrated" items.
+			migratedEntries--
+			continue
+		}
+		for i, rng := range ranges {
+			if rng.contains(e.prev) {
+				e.prev += offsets[i]
+				break
+			}
+		}
+		for i, rng := range ranges {
+			if rng.contains(e.next) {
+				e.next += offsets[i]
+				break
+			}
+		}
+	}
+	for _, idx := range index {
+		for i, rng := range ranges {
+			if rng.contains(idx.oldest) {
+				idx.oldest += offsets[i]
+				break
+			}
+		}
+		for i, rng := range ranges {
+			if rng.contains(idx.newest) {
+				idx.newest += offsets[i]
+				break
+			}
+		}
+	}
+	return migratedEntries
+}
diff --git a/tsdb/exemplar_test.go b/tsdb/exemplar_test.go
index 103332c886..01ffeb9541 100644
--- a/tsdb/exemplar_test.go
+++ b/tsdb/exemplar_test.go
@@ -18,6 +18,7 @@ import (
 	"fmt"
 	"math"
 	"reflect"
+	"sort"
 	"strconv"
 	"strings"
 	"sync"
@@ -35,7 +36,7 @@ var eMetrics = NewExemplarMetrics(prometheus.DefaultRegisterer)
 
 // Tests the same exemplar cases as AddExemplar, but specifically the ValidateExemplar function so it can be relied on externally.
 func TestValidateExemplar(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(2, eMetrics)
+	exs, err := NewCircularExemplarStorage(2, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -76,54 +77,624 @@ func TestValidateExemplar(t *testing.T) {
 	require.Equal(t, storage.ErrExemplarLabelLength, es.ValidateExemplar(l, e4))
 }
 
-func TestAddExemplar(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(2, eMetrics)
-	require.NoError(t, err)
-	es := exs.(*CircularExemplarStorage)
+func TestCircularExemplarStorage_AddExemplar(t *testing.T) {
+	series1 := labels.FromStrings("trace_id", "foo")
+	series2 := labels.FromStrings("trace_id", "bar")
 
-	l := labels.FromStrings("service", "asdf")
-	e := exemplar.Exemplar{
-		Labels: labels.FromStrings("trace_id", "qwerty"),
-		Value:  0.1,
-		Ts:     1,
+	series1Matcher := []*labels.Matcher{{
+		Type:  labels.MatchEqual,
+		Name:  "trace_id",
+		Value: series1.Get("trace_id"),
+	}}
+
+	series2Matcher := []*labels.Matcher{{
+		Type:  labels.MatchEqual,
+		Name:  "trace_id",
+		Value: series2.Get("trace_id"),
+	}}
+
+	testCases := []struct {
+		name          string
+		size          int64
+		exemplars     []exemplar.Exemplar
+		wantExemplars []exemplar.Exemplar
+		matcher       []*labels.Matcher
+		wantError     error
+	}{
+		{
+			name: "insert after newest",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+		},
+		{
+			name: "insert before oldest",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 2},
+				{Labels: series1, Value: 0.2, Ts: 1},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 1},
+				{Labels: series1, Value: 0.1, Ts: 2},
+			},
+		},
+		{
+			name: "insert in between",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 3},
+				{Labels: series1, Value: 0.3, Ts: 2},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.3, Ts: 2},
+				{Labels: series1, Value: 0.2, Ts: 3},
+			},
+		},
+		{
+			name: "insert after newest with overflow",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+		},
+		{
+			name: "insert before oldest with overflow",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 0},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.4, Ts: 0},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+		},
+		{
+			name: "insert between with overflow",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 3},
+				{Labels: series1, Value: 0.3, Ts: 4},
+				{Labels: series1, Value: 0.4, Ts: 2},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.4, Ts: 2},
+				{Labels: series1, Value: 0.2, Ts: 3},
+				{Labels: series1, Value: 0.3, Ts: 4},
+			},
+		},
+		{
+			name: "insert out of the OOO window",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 200},
+				{Labels: series1, Value: 0.2, Ts: 1},
+			},
+			wantError: storage.ErrOutOfOrderExemplar,
+		},
+		{
+			name: "insert multiple series",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 3},
+				{Labels: series2, Value: 0.3, Ts: 4},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 3},
+			},
+		},
+		{
+			name: "insert multiple series with overflow",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series2, Value: 0.1, Ts: 1},
+				{Labels: series2, Value: 0.2, Ts: 2},
+				{Labels: series2, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			matcher: series2Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series2, Value: 0.2, Ts: 2},
+				{Labels: series2, Value: 0.3, Ts: 3},
+			},
+		},
+		{
+			name: "series1 overflows series2 out-of-order",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series2, Value: 0.1, Ts: 3},
+				{Labels: series2, Value: 0.2, Ts: 2},
+				{Labels: series2, Value: 0.3, Ts: 4},
+				{Labels: series1, Value: 0.4, Ts: 4},
+				{Labels: series1, Value: 0.5, Ts: 1},
+			},
+			matcher: series2Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series2, Value: 0.3, Ts: 4},
+			},
+		},
+		{
+			name: "ignore duplicate exemplars",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 3},
+				{Labels: series1, Value: 0.1, Ts: 3},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 3},
+			},
+		},
+		{
+			name: "ignore duplicate exemplars when buffer is full",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 3},
+				{Labels: series1, Value: 0.2, Ts: 4},
+				{Labels: series1, Value: 0.3, Ts: 5},
+				{Labels: series1, Value: 0.3, Ts: 5},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 3},
+				{Labels: series1, Value: 0.2, Ts: 4},
+				{Labels: series1, Value: 0.3, Ts: 5},
+			},
+		},
+		{
+			name: "empty timestamps are valid",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 0},
+				{Labels: series1, Value: 0.2, Ts: 0},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 0},
+				{Labels: series1, Value: 0.2, Ts: 0},
+			},
+		},
+		{
+			name: "exemplar label length exceeds maximum",
+			size: 3,
+			exemplars: []exemplar.Exemplar{
+				{Labels: labels.FromStrings("a", strings.Repeat("b", exemplar.ExemplarMaxLabelSetLength)), Value: 0.1, Ts: 2},
+			},
+			wantError: storage.ErrExemplarLabelLength,
+		},
+		{
+			name: "native histograms",
+			size: 6,
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+		},
+		{
+			name: "evict only exemplar for series then re-add",
+			size: 2,
+			exemplars: []exemplar.Exemplar{
+				// series1 at index 0, series2 at index 1, then series1 evicts its own only exemplar
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series2, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			matcher: series1Matcher,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+		},
+	}
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			exs, err := NewCircularExemplarStorage(tc.size, eMetrics, 100)
+			require.NoError(t, err)
+			es := exs.(*CircularExemplarStorage)
+
+			// Add exemplars and compare tc.wantErr against the first exemplar failing.
+			var addError error
+			for i, ex := range tc.exemplars {
+				addError = es.AddExemplar(ex.Labels, ex)
+				if addError != nil {
+					break
+				}
+				if testing.Verbose() {
+					t.Logf("Buffer[%d]:\n%s", i, debugCircularBuffer(es))
+				}
+			}
+			if tc.wantError == nil {
+				require.NoError(t, addError)
+			} else {
+				require.ErrorIs(t, addError, tc.wantError)
+			}
+			if addError != nil {
+				return
+			}
+
+			// Ensure exemplars are returned correctly and in-order.
+			gotExemplars, err := es.Select(0, 1000, tc.matcher)
+			require.NoError(t, err)
+			if len(tc.wantExemplars) == 0 {
+				require.Empty(t, gotExemplars)
+			} else {
+				require.Len(t, gotExemplars, 1)
+				require.Equal(t, tc.wantExemplars, gotExemplars[0].Exemplars)
+			}
+		})
+	}
+}
+
+func TestCircularExemplarStorage_Resize(t *testing.T) {
+	series1 := labels.FromStrings("trace_id", "foo")
+	series2 := labels.FromStrings("trace_id", "bar")
+	matcher1 := []*labels.Matcher{
+		labels.MustNewMatcher(labels.MatchRegexp, "trace_id", "(foo|bar)"),
 	}
 
-	require.NoError(t, es.AddExemplar(l, e))
-	require.Equal(t, 0, es.index[string(l.Bytes(nil))].newest, "exemplar was not stored correctly")
-
-	e2 := exemplar.Exemplar{
-		Labels: labels.FromStrings("trace_id", "zxcvb"),
-		Value:  0.1,
-		Ts:     2,
+	testCases := []struct {
+		name          string
+		exemplars     []exemplar.Exemplar
+		resize        int64
+		wantExemplars []exemplar.Exemplar
+		wantNextIndex int
+		wantError     error
+	}{
+		{
+			name: "in-order, grow",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			resize: 10,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			wantNextIndex: 2,
+		},
+		{
+			name: "in-order, shrink",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			resize: 2,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			wantNextIndex: 0,
+		},
+		{
+			name: "out-of-order, shrink",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.1, Ts: 1},
+			},
+			resize: 2,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			wantNextIndex: 0,
+		},
+		{
+			name: "out-of-order, grow",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			resize: 5,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			wantNextIndex: 2,
+		},
+		{
+			name: "duplicate timestamps",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 1},
+				{Labels: series1, Value: 0.3, Ts: 2},
+			},
+			resize: 3,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 1},
+				{Labels: series1, Value: 0.3, Ts: 2},
+			},
+		},
+		{
+			name:          "empty input, grow",
+			exemplars:     []exemplar.Exemplar{},
+			resize:        10,
+			wantExemplars: []exemplar.Exemplar{},
+			wantNextIndex: 0,
+		},
+		{
+			name:          "empty input, shrink",
+			exemplars:     []exemplar.Exemplar{},
+			resize:        1,
+			wantExemplars: []exemplar.Exemplar{},
+			wantNextIndex: 0,
+		},
+		{
+			name: "shrink to zero",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			resize:        0,
+			wantExemplars: []exemplar.Exemplar{},
+			wantNextIndex: 0,
+		},
+		{
+			name: "multiple series, shrink",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series2, Value: 1.1, Ts: 2},
+				{Labels: series1, Value: 0.2, Ts: 3},
+				{Labels: series2, Value: 1.2, Ts: 4},
+			},
+			resize: 2,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 3},
+				{Labels: series2, Value: 1.2, Ts: 4},
+			},
+			wantNextIndex: 0,
+		},
+		{
+			name: "shrink to one",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			resize: 1,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			wantNextIndex: 0,
+		},
+		{
+			name: "shrink to two",
+			exemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+			},
+			resize: 2,
+			wantExemplars: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+			},
+			wantNextIndex: 1,
+		},
 	}
 
-	require.NoError(t, es.AddExemplar(l, e2))
-	require.Equal(t, 1, es.index[string(l.Bytes(nil))].newest, "exemplar was not stored correctly, location of newest exemplar for series in index did not update")
-	require.True(t, es.exemplars[es.index[string(l.Bytes(nil))].newest].exemplar.Equals(e2), "exemplar was not stored correctly, expected %+v got: %+v", e2, es.exemplars[es.index[string(l.Bytes(nil))].newest].exemplar)
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			exs, err := NewCircularExemplarStorage(3, eMetrics, 100)
+			require.NoError(t, err)
+			es := exs.(*CircularExemplarStorage)
 
-	require.NoError(t, es.AddExemplar(l, e2), "no error is expected attempting to add duplicate exemplar")
+			for _, ex := range tc.exemplars {
+				require.NoError(t, es.AddExemplar(ex.Labels, ex))
+			}
 
-	e3 := e2
-	e3.Ts = 3
-	require.NoError(t, es.AddExemplar(l, e3), "no error is expected when attempting to add duplicate exemplar, even with different timestamp")
+			// Resize the circular buffer.
+			if testing.Verbose() {
+				t.Logf("Buffer[before-resize]:\n%s", debugCircularBuffer(es))
+			}
+			es.Resize(tc.resize)
+			if testing.Verbose() {
+				t.Logf("Buffer[after-resize]:\n%s", debugCircularBuffer(es))
+			}
 
-	e3.Ts = 1
-	e3.Value = 0.3
-	require.Equal(t, storage.ErrOutOfOrderExemplar, es.AddExemplar(l, e3))
-
-	e4 := exemplar.Exemplar{
-		Labels: labels.FromStrings("a", strings.Repeat("b", exemplar.ExemplarMaxLabelSetLength)),
-		Value:  0.1,
-		Ts:     2,
+			// Ensure exemplars are returned correctly and in-order.
+			gotExemplars, err := es.Select(0, 1000, matcher1)
+			require.NoError(t, err)
+			flat := make([]exemplar.Exemplar, 0)
+			for _, group := range gotExemplars {
+				flat = append(flat, group.Exemplars...)
+			}
+			sort.Slice(flat, func(i, j int) bool {
+				return flat[i].Ts < flat[j].Ts
+			})
+			require.Equal(t, tc.wantExemplars, flat, "exemplar mismatch")
+			require.Equal(t, tc.wantNextIndex, es.nextIndex, "next index mismatch")
+		})
+	}
+
+	resizeTwiceCases := []struct {
+		name           string
+		addExemplars1  []exemplar.Exemplar
+		resize1        int64
+		wantExemplars1 []exemplar.Exemplar
+		resize2        int64
+		addExemplars2  []exemplar.Exemplar
+		wantExemplars2 []exemplar.Exemplar
+	}{
+		{
+			name: "shrink then grow ordered",
+			addExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			resize1: 2,
+			wantExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			resize2: 5,
+			addExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.5, Ts: 5},
+				{Labels: series1, Value: 0.6, Ts: 6},
+				{Labels: series1, Value: 0.7, Ts: 7},
+			},
+			wantExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+				{Labels: series1, Value: 0.5, Ts: 5},
+				{Labels: series1, Value: 0.6, Ts: 6},
+				{Labels: series1, Value: 0.7, Ts: 7},
+			},
+		},
+		{
+			name: "shrink then grow out-of-order",
+			addExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.4, Ts: 4},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			resize1: 2,
+			wantExemplars1: []exemplar.Exemplar{
+				// We delete in the order of ingestion, not temporally.
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			resize2: 5,
+			addExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.7, Ts: 7},
+				{Labels: series1, Value: 0.6, Ts: 6},
+				{Labels: series1, Value: 0.5, Ts: 5},
+			},
+			wantExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.5, Ts: 5},
+				{Labels: series1, Value: 0.6, Ts: 6},
+				{Labels: series1, Value: 0.7, Ts: 7},
+			},
+		},
+		{
+			name: "grow then shrink ordered",
+			addExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			resize1: 5,
+			wantExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			resize2: 2,
+			addExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.5, Ts: 5},
+				{Labels: series1, Value: 0.6, Ts: 6},
+				{Labels: series1, Value: 0.7, Ts: 7},
+			},
+			wantExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.6, Ts: 6},
+				{Labels: series1, Value: 0.7, Ts: 7},
+			},
+		},
+		{
+			name: "grow then shrink out-of-order",
+			addExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.4, Ts: 4},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+			},
+			resize1: 5,
+			wantExemplars1: []exemplar.Exemplar{
+				// We delete in the order of ingestion, not temporally.
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			resize2: 2,
+			addExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.7, Ts: 7},
+				{Labels: series1, Value: 0.5, Ts: 5},
+				{Labels: series1, Value: 0.6, Ts: 6},
+			},
+			wantExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.5, Ts: 5},
+				{Labels: series1, Value: 0.6, Ts: 6},
+			},
+		},
+	}
+
+	for _, tc := range resizeTwiceCases {
+		t.Run(tc.name, func(t *testing.T) {
+			exs, err := NewCircularExemplarStorage(3, eMetrics, 100)
+			require.NoError(t, err)
+			es := exs.(*CircularExemplarStorage)
+			for _, ex := range tc.addExemplars1 {
+				require.NoError(t, es.AddExemplar(ex.Labels, ex))
+			}
+			es.Resize(tc.resize1)
+			gotExemplars, err := es.Select(0, 1000, matcher1)
+			require.NoError(t, err)
+			require.Len(t, gotExemplars, 1)
+			require.Equal(t, tc.wantExemplars1, gotExemplars[0].Exemplars)
+			es.Resize(tc.resize2)
+			for _, ex := range tc.addExemplars2 {
+				require.NoError(t, es.AddExemplar(ex.Labels, ex))
+			}
+			if testing.Verbose() {
+				t.Logf("Buffer[after-resize2]:\n%s", debugCircularBuffer(es))
+			}
+			gotExemplars, err = es.Select(0, 1000, matcher1)
+			require.NoError(t, err)
+			require.Len(t, gotExemplars, 1)
+			require.Equal(t, tc.wantExemplars2, gotExemplars[0].Exemplars)
+		})
 	}
-	require.Equal(t, storage.ErrExemplarLabelLength, es.AddExemplar(l, e4))
 }
 
 func TestStorageOverflow(t *testing.T) {
 	// Test that circular buffer index and assignment
 	// works properly, adding more exemplars than can
 	// be stored and then querying for them.
-	exs, err := NewCircularExemplarStorage(5, eMetrics)
+	exs, err := NewCircularExemplarStorage(5, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -152,7 +723,7 @@ func TestStorageOverflow(t *testing.T) {
 }
 
 func TestSelectExemplar(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(5, eMetrics)
+	exs, err := NewCircularExemplarStorage(5, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -179,7 +750,7 @@ func TestSelectExemplar(t *testing.T) {
 }
 
 func TestSelectExemplar_MultiSeries(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(5, eMetrics)
+	exs, err := NewCircularExemplarStorage(5, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -223,7 +794,7 @@ func TestSelectExemplar_MultiSeries(t *testing.T) {
 
 func TestSelectExemplar_TimeRange(t *testing.T) {
 	var lenEs int64 = 5
-	exs, err := NewCircularExemplarStorage(lenEs, eMetrics)
+	exs, err := NewCircularExemplarStorage(lenEs, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -251,7 +822,7 @@ func TestSelectExemplar_TimeRange(t *testing.T) {
 // Test to ensure that even though a series matches more than one matcher from the
 // query that it's exemplars are only included in the result a single time.
 func TestSelectExemplar_DuplicateSeries(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(4, eMetrics)
+	exs, err := NewCircularExemplarStorage(4, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -286,7 +857,7 @@ func TestSelectExemplar_DuplicateSeries(t *testing.T) {
 }
 
 func TestIndexOverwrite(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(2, eMetrics)
+	exs, err := NewCircularExemplarStorage(2, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -374,7 +945,7 @@ func TestResize(t *testing.T) {
 
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
-			exs, err := NewCircularExemplarStorage(tc.startSize, eMetrics)
+			exs, err := NewCircularExemplarStorage(tc.startSize, eMetrics, 0)
 			require.NoError(t, err)
 			es := exs.(*CircularExemplarStorage)
 
@@ -386,7 +957,14 @@ func TestResize(t *testing.T) {
 				require.NoError(t, err)
 			}
 
+			if testing.Verbose() {
+				t.Logf("Buffer[before-resize]:\n%s", debugCircularBuffer(es))
+			}
 			resized := es.Resize(tc.newCount)
+			if testing.Verbose() {
+				t.Logf("Buffer[after-resize]:\n%s", debugCircularBuffer(es))
+			}
+
 			require.Equal(t, tc.expectedMigrated, resized)
 
 			q, err := es.Querier(context.TODO())
@@ -421,7 +999,7 @@ func BenchmarkAddExemplar(b *testing.B) {
 			b.Run(fmt.Sprintf("%d/%d", n, capacity), func(b *testing.B) {
 				for b.Loop() {
 					b.StopTimer()
-					exs, err := NewCircularExemplarStorage(int64(capacity), eMetrics)
+					exs, err := NewCircularExemplarStorage(int64(capacity), eMetrics, 0)
 					require.NoError(b, err)
 					es := exs.(*CircularExemplarStorage)
 					var l labels.Labels
@@ -442,6 +1020,91 @@ func BenchmarkAddExemplar(b *testing.B) {
 	}
 }
 
+func BenchmarkAddExemplar_OutOfOrder(b *testing.B) {
+	// We need to include these labels since we do length calculation
+	// before adding.
+	exLabels := labels.FromStrings("trace_id", "89620921")
+
+	const (
+		capacity = 5000
+	)
+
+	fillOneSeries := func(es *CircularExemplarStorage) {
+		for i := range capacity {
+			e := exemplar.Exemplar{Value: float64(i), Ts: int64(i), Labels: exLabels}
+			if err := es.AddExemplar(exLabels, e); err != nil {
+				panic(err)
+			}
+		}
+	}
+
+	fillMultipleSeries := func(es *CircularExemplarStorage) {
+		for i := range capacity {
+			l := labels.FromStrings("service", strconv.Itoa(i))
+			e := exemplar.Exemplar{Value: float64(i), Ts: int64(i), Labels: l}
+			if err := es.AddExemplar(l, e); err != nil {
+				panic(err)
+			}
+		}
+	}
+
+	outOfOrder := func(ts *int64, _ *labels.Labels) {
+		switch *ts % 3 {
+		case 0:
+			return
+		case 1:
+			*ts = capacity - *ts
+		case 2:
+			*ts = (capacity - *ts) + 100
+		}
+	}
+
+	reverseOrder := func(ts *int64, _ *labels.Labels) {
+		*ts = capacity - *ts
+	}
+
+	multipleSeries := func(f func(*int64, *labels.Labels)) func(*int64, *labels.Labels) {
+		return func(ts *int64, l *labels.Labels) {
+			f(ts, l)
+			*l = labels.FromStrings("service", strconv.Itoa(int(*ts)))
+		}
+	}
+
+	for fillName, setup := range map[string]func(es *CircularExemplarStorage){
+		"empty":         func(*CircularExemplarStorage) {},
+		"full-one":      fillOneSeries,
+		"full-multiple": fillMultipleSeries,
+	} {
+		for orderName, forEach := range map[string]func(ts *int64, l *labels.Labels){
+			"in-order":           func(*int64, *labels.Labels) {},
+			"reverse":            reverseOrder,
+			"out-of-order":       outOfOrder,
+			"multi-in-order":     multipleSeries(func(*int64, *labels.Labels) {}),
+			"multi-reverse":      multipleSeries(reverseOrder),
+			"multi-out-of-order": multipleSeries(outOfOrder),
+		} {
+			b.Run(fmt.Sprintf("%s/%s", fillName, orderName), func(b *testing.B) {
+				exs, err := NewCircularExemplarStorage(int64(capacity), eMetrics, 100000)
+				require.NoError(b, err)
+				es := exs.(*CircularExemplarStorage)
+				l := labels.FromStrings("service", "0")
+				setup(es)
+				b.ResetTimer()
+				for b.Loop() {
+					for i := range capacity {
+						ts := int64(i)
+						forEach(&ts, &l)
+						err = es.AddExemplar(l, exemplar.Exemplar{Value: float64(i), Ts: ts, Labels: l})
+						if err != nil {
+							b.Fatalf("Failed to insert item %d %s: %v", i, l, err)
+						}
+					}
+				}
+			})
+		}
+	}
+}
+
 func BenchmarkResizeExemplars(b *testing.B) {
 	testCases := []struct {
 		name         string
@@ -479,7 +1142,7 @@ func BenchmarkResizeExemplars(b *testing.B) {
 		b.Run(fmt.Sprintf("%s-%d-to-%d", tc.name, tc.startSize, tc.endSize), func(b *testing.B) {
 			for b.Loop() {
 				b.StopTimer()
-				exs, err := NewCircularExemplarStorage(tc.startSize, eMetrics)
+				exs, err := NewCircularExemplarStorage(tc.startSize, eMetrics, 0)
 				require.NoError(b, err)
 				es := exs.(*CircularExemplarStorage)
 
@@ -504,7 +1167,7 @@ func BenchmarkResizeExemplars(b *testing.B) {
 // TestCircularExemplarStorage_Concurrent_AddExemplar_Resize tries to provoke a data race between AddExemplar and Resize.
 // Run with race detection enabled.
 func TestCircularExemplarStorage_Concurrent_AddExemplar_Resize(t *testing.T) {
-	exs, err := NewCircularExemplarStorage(0, eMetrics)
+	exs, err := NewCircularExemplarStorage(0, eMetrics, 0)
 	require.NoError(t, err)
 	es := exs.(*CircularExemplarStorage)
 
@@ -537,3 +1200,30 @@ func TestCircularExemplarStorage_Concurrent_AddExemplar_Resize(t *testing.T) {
 		}
 	}
 }
+
+// debugCircularBuffer iterates all exemplars in the circular exemplar storage
+// and returns them as a string. The textual representation contains index
+// pointers and helps debugging exemplar storage.
+func debugCircularBuffer(ce *CircularExemplarStorage) string {
+	var sb strings.Builder
+	for i, e := range ce.exemplars {
+		if e.ref == nil {
+			continue
+		}
+		sb.WriteString(fmt.Sprintf(
+			"i: %d, ts: %d, next: %d, prev: %d",
+			i, e.exemplar.Ts, e.next, e.prev,
+		))
+		for _, idx := range ce.index {
+			if i == idx.newest {
+				sb.WriteString(" <- newest " + idx.seriesLabels.String())
+			}
+			if i == idx.oldest {
+				sb.WriteString(" <- oldest " + idx.seriesLabels.String())
+			}
+		}
+		sb.WriteString("\n")
+	}
+	sb.WriteString(fmt.Sprintf("Next index: %d\n", ce.nextIndex))
+	return sb.String()
+}
diff --git a/tsdb/head.go b/tsdb/head.go
index a4df208e6e..955c0ae5a7 100644
--- a/tsdb/head.go
+++ b/tsdb/head.go
@@ -327,7 +327,7 @@ func (h *Head) resetInMemoryState() error {
 	if em == nil {
 		em = NewExemplarMetrics(h.reg)
 	}
-	es, err := NewCircularExemplarStorage(h.opts.MaxExemplars.Load(), em)
+	es, err := NewCircularExemplarStorage(h.opts.MaxExemplars.Load(), em, h.opts.OutOfOrderTimeWindow.Load())
 	if err != nil {
 		return err
 	}
@@ -1037,6 +1037,8 @@ func (h *Head) ApplyConfig(cfg *config.Config, wbl *wlog.WL) {
 		return
 	}
 
+	h.exemplars.(*CircularExemplarStorage).SetOutOfOrderTimeWindow(oooTimeWindow)
+
 	// Head uses opts.MaxExemplars in combination with opts.EnableExemplarStorage
 	// to decide if it should pass exemplars along to its exemplar storage, so we
 	// need to update opts.MaxExemplars here.
diff --git a/util/teststorage/storage.go b/util/teststorage/storage.go
index 30a63327ab..17efdda77d 100644
--- a/util/teststorage/storage.go
+++ b/util/teststorage/storage.go
@@ -65,7 +65,7 @@ func NewWithError(outOfOrderTimeWindow ...int64) (*TestStorage, error) {
 	reg := prometheus.NewRegistry()
 	eMetrics := tsdb.NewExemplarMetrics(reg)
 
-	es, err := tsdb.NewCircularExemplarStorage(10, eMetrics)
+	es, err := tsdb.NewCircularExemplarStorage(10, eMetrics, opts.OutOfOrderTimeWindow)
 	if err != nil {
 		return nil, fmt.Errorf("opening test exemplar storage: %w", err)
 	}

From f1719fa1d4e56303f608031fb809c6e8b7b945b8 Mon Sep 17 00:00:00 2001
From: Bryan Boreham 
Date: Wed, 7 Jan 2026 14:01:02 +0000
Subject: [PATCH 228/439] [BUGFIX] Agent: fix crash from invalid type in pool
 (#17802)

We have separate pools for Appender and AppenderV2 objects, and must not
put another kind of object into them.

Signed-off-by: Bryan Boreham 
---
 tsdb/agent/db.go                | 16 ++++++++++++----
 tsdb/agent/db_append_v2.go      | 10 ++++++++++
 tsdb/agent/db_append_v2_test.go |  4 ++++
 tsdb/agent/db_test.go           |  3 +++
 4 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/tsdb/agent/db.go b/tsdb/agent/db.go
index a0f7a93b6d..3e9d7ed714 100644
--- a/tsdb/agent/db.go
+++ b/tsdb/agent/db.go
@@ -1124,13 +1124,22 @@ func (a *appender) AppendSTZeroSample(ref storage.SeriesRef, l labels.Labels, t,
 }
 
 // Commit submits the collected samples and purges the batch.
-func (a *appenderBase) Commit() error {
+func (a *appender) Commit() error {
+	defer a.appenderPool.Put(a)
+	return a.commit()
+}
+
+func (a *appender) Rollback() error {
+	defer a.appenderPool.Put(a)
+	return a.rollback()
+}
+
+func (a *appenderBase) commit() error {
 	if err := a.log(); err != nil {
 		return err
 	}
 
 	a.clearData()
-	a.appenderPool.Put(a)
 
 	if a.writeNotified != nil {
 		a.writeNotified.Notify()
@@ -1244,7 +1253,7 @@ func (a *appenderBase) clearData() {
 	a.floatHistogramSeries = a.floatHistogramSeries[:0]
 }
 
-func (a *appenderBase) Rollback() error {
+func (a *appenderBase) rollback() error {
 	// Series are created in-memory regardless of rollback. This means we must
 	// log them to the WAL, otherwise subsequent commits may reference a series
 	// which was never written to the WAL.
@@ -1253,7 +1262,6 @@ func (a *appenderBase) Rollback() error {
 	}
 
 	a.clearData()
-	a.appenderPool.Put(a)
 	return nil
 }
 
diff --git a/tsdb/agent/db_append_v2.go b/tsdb/agent/db_append_v2.go
index f356a4feae..bb2601e1e3 100644
--- a/tsdb/agent/db_append_v2.go
+++ b/tsdb/agent/db_append_v2.go
@@ -127,6 +127,16 @@ func (a *appenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t int64
 	return storage.SeriesRef(s.ref), partialErr
 }
 
+func (a *appenderV2) Commit() error {
+	defer a.appenderV2Pool.Put(a)
+	return a.commit()
+}
+
+func (a *appenderV2) Rollback() error {
+	defer a.appenderV2Pool.Put(a)
+	return a.rollback()
+}
+
 func (a *appenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemplar) error {
 	var errs []error
 	for _, e := range exemplar {
diff --git a/tsdb/agent/db_append_v2_test.go b/tsdb/agent/db_append_v2_test.go
index 6a85e93c35..3e10a1163b 100644
--- a/tsdb/agent/db_append_v2_test.go
+++ b/tsdb/agent/db_append_v2_test.go
@@ -224,6 +224,10 @@ func TestCommit_AppendV2(t *testing.T) {
 	require.Equal(t, numSeries*numDatapoints, walExemplarsCount, "unexpected number of exemplars")
 	require.Equal(t, numSeries*numHistograms*2, walHistogramCount, "unexpected number of histograms")
 	require.Equal(t, numSeries*numHistograms*2, walFloatHistogramCount, "unexpected number of float histograms")
+
+	// Check that we can still create both kinds of Appender - see https://github.com/prometheus/prometheus/issues/17800.
+	_ = s.Appender(context.TODO())
+	_ = s.AppenderV2(context.TODO())
 }
 
 func TestRollback_AppendV2(t *testing.T) {
diff --git a/tsdb/agent/db_test.go b/tsdb/agent/db_test.go
index 94e84fa2eb..d2e005c175 100644
--- a/tsdb/agent/db_test.go
+++ b/tsdb/agent/db_test.go
@@ -259,6 +259,9 @@ func TestCommit(t *testing.T) {
 	require.Equal(t, numSeries*numDatapoints, walExemplarsCount, "unexpected number of exemplars")
 	require.Equal(t, numSeries*numHistograms*2, walHistogramCount, "unexpected number of histograms")
 	require.Equal(t, numSeries*numHistograms*2, walFloatHistogramCount, "unexpected number of float histograms")
+
+	// Check that we can get another appender after this - see https://github.com/prometheus/prometheus/issues/17800.
+	_ = s.Appender(context.TODO())
 }
 
 func TestRollback(t *testing.T) {

From fff29d330d633f30c36e9fe1a39dd6e15d775904 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marvin=20R=C3=B6sch?= 
Date: Wed, 7 Jan 2026 15:27:01 +0100
Subject: [PATCH 229/439] [BUGFIX] Scraping: drop sample if relabeling config
 says so
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marvin Rösch 
---
 scrape/scrape.go      |  4 +++-
 scrape/scrape_test.go | 44 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+), 1 deletion(-)

diff --git a/scrape/scrape.go b/scrape/scrape.go
index 70ca8ad42a..1a99155d09 100644
--- a/scrape/scrape.go
+++ b/scrape/scrape.go
@@ -613,7 +613,9 @@ func mutateSampleLabels(lset labels.Labels, target *Target, honor bool, rc []*re
 		}
 	}
 
-	relabel.ProcessBuilder(lb, rc...)
+	if keep := relabel.ProcessBuilder(lb, rc...); !keep {
+		return labels.EmptyLabels()
+	}
 
 	return lb.Labels()
 }
diff --git a/scrape/scrape_test.go b/scrape/scrape_test.go
index 7aa633d387..c2b2ae132c 100644
--- a/scrape/scrape_test.go
+++ b/scrape/scrape_test.go
@@ -5932,3 +5932,47 @@ func TestNewScrapeLoopHonorLabelsWiring(t *testing.T) {
 		})
 	}
 }
+
+func TestDropsSeriesFromMetricRelabeling(t *testing.T) {
+	target := &Target{}
+	relabelConfig := []*relabel.Config{
+		{
+			SourceLabels:         model.LabelNames{"__name__"},
+			Regex:                relabel.MustNewRegexp("test_metric.*$"),
+			Action:               relabel.Keep,
+			NameValidationScheme: model.UTF8Validation,
+		},
+		{
+			SourceLabels:         model.LabelNames{"__name__"},
+			Regex:                relabel.MustNewRegexp("test_metric_2$"),
+			Action:               relabel.Drop,
+			NameValidationScheme: model.UTF8Validation,
+		},
+	}
+	sl, _ := newTestScrapeLoop(t, func(sl *scrapeLoop) {
+		sl.sampleMutator = func(l labels.Labels) labels.Labels {
+			return mutateSampleLabels(l, target, true, relabelConfig)
+		}
+	})
+
+	app := sl.appender()
+	total, added, seriesAdded, err := app.append([]byte("test_metric_1 1\n"), "text/plain", time.Time{})
+	require.NoError(t, err)
+	require.Equal(t, 1, total)
+	require.Equal(t, 1, added)
+	require.Equal(t, 1, seriesAdded)
+
+	total, added, seriesAdded, err = app.append([]byte("test_metric_2 1\n"), "text/plain", time.Time{})
+	require.NoError(t, err)
+	require.Equal(t, 1, total)
+	require.Equal(t, 0, added)
+	require.Equal(t, 0, seriesAdded)
+
+	total, added, seriesAdded, err = app.append([]byte("unwanted_metric 1\n"), "text/plain", time.Time{})
+	require.NoError(t, err)
+	require.Equal(t, 1, total)
+	require.Equal(t, 0, added)
+	require.Equal(t, 0, seriesAdded)
+
+	require.NoError(t, app.Commit())
+}

From ae711852559722dbad0797195a89fd39ee0d2324 Mon Sep 17 00:00:00 2001
From: Bryan Boreham 
Date: Wed, 7 Jan 2026 14:56:10 +0000
Subject: [PATCH 230/439] Scraping: add a test for relabel with keep and drop
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Co-authored-by: Marvin Rösch 
Signed-off-by: Bryan Boreham 
---
 scrape/scrape_test.go | 44 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/scrape/scrape_test.go b/scrape/scrape_test.go
index eab1499158..270d1909dd 100644
--- a/scrape/scrape_test.go
+++ b/scrape/scrape_test.go
@@ -5982,3 +5982,47 @@ func TestScrapeLoopDisableStalenessMarkerInjection(t *testing.T) {
 		}
 	}
 }
+
+func TestDropsSeriesFromRelabeling(t *testing.T) {
+	s := teststorage.New(t)
+	defer s.Close()
+	ctx := t.Context()
+
+	target := &Target{}
+	relabelConfig := []*relabel.Config{
+		{
+			SourceLabels: model.LabelNames{"__name__"},
+			Regex:        relabel.MustNewRegexp(".*_total$"),
+			Action:       relabel.Keep,
+		},
+		{
+			SourceLabels: model.LabelNames{"__name__"},
+			Regex:        relabel.MustNewRegexp("test_metric_2_total$"),
+			Action:       relabel.Drop,
+		},
+	}
+	metricsText := []byte(`
+# HELP test_metric_1_total This is a counter
+# TYPE test_metric_1_total counter
+test_metric_1_total 123
+# HELP test_metric_2_total This is a counter
+# TYPE test_metric_2_total counter
+test_metric_2_total 234
+# HELP disk_usage_bytes This is a gauge
+# TYPE disk_usage_bytes gauge
+disk_usage_bytes 456
+`)
+
+	sl := newBasicScrapeLoop(t, ctx, &testScraper{}, s.Appender, 0)
+	sl.sampleMutator = func(l labels.Labels) labels.Labels {
+		return mutateSampleLabels(l, target, true, relabelConfig)
+	}
+
+	slApp := sl.appender(ctx)
+	total, added, seriesAdded, err := sl.append(slApp, metricsText, "text/plain", time.Time{})
+	require.NoError(t, err)
+	require.NoError(t, slApp.Rollback())
+	require.Equal(t, 3, total)
+	require.Equal(t, 1, added)
+	require.Equal(t, 1, seriesAdded)
+}

From 66c8e31956777630949b1be5009d1a7e6de83921 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Marvin=20R=C3=B6sch?= 
Date: Wed, 7 Jan 2026 15:27:01 +0100
Subject: [PATCH 231/439] [BUGFIX] Scraping: drop sample if relabeling config
 says so
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Marvin Rösch 
---
 scrape/scrape.go | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scrape/scrape.go b/scrape/scrape.go
index b653873bad..a8cd15d30c 100644
--- a/scrape/scrape.go
+++ b/scrape/scrape.go
@@ -716,7 +716,9 @@ func mutateSampleLabels(lset labels.Labels, target *Target, honor bool, rc []*re
 		}
 	}
 
-	relabel.ProcessBuilder(lb, rc...)
+	if keep := relabel.ProcessBuilder(lb, rc...); !keep {
+		return labels.EmptyLabels()
+	}
 
 	return lb.Labels()
 }

From 9ec59baffb547e24f1468a53eb82901e58feabd8 Mon Sep 17 00:00:00 2001
From: Bryan Boreham 
Date: Wed, 7 Jan 2026 16:05:27 +0000
Subject: [PATCH 232/439] Cut v3.9.1 (#17804)

Signed-off-by: Bryan Boreham 
---
 CHANGELOG.md                                 |  5 +++++
 VERSION                                      |  2 +-
 web/ui/mantine-ui/package.json               |  4 ++--
 web/ui/module/codemirror-promql/package.json |  4 ++--
 web/ui/module/lezer-promql/package.json      |  2 +-
 web/ui/package-lock.json                     | 14 +++++++-------
 web/ui/package.json                          |  2 +-
 7 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6113dd0156..05f886024d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,10 @@
 # Changelog
 
+## 3.9.1 / 2026-01-07
+
+ - [BUGFIX] Agent: fix crash shortly after startup from invalid type of object. #17802
+ - [BUGFIX] Scraping: fix relabel keep/drop not working. #17807
+
 ## 3.9.0 / 2026-01-06
 
 - [CHANGE] Native Histograms are no longer experimental! Make the `native-histogram` feature flag a no-op. Use `scrape_native_histograms` config option instead. #17528
diff --git a/VERSION b/VERSION
index a5c4c76339..6bd10744ae 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-3.9.0
+3.9.1
diff --git a/web/ui/mantine-ui/package.json b/web/ui/mantine-ui/package.json
index 3ee4c6c48c..f38a2d965f 100644
--- a/web/ui/mantine-ui/package.json
+++ b/web/ui/mantine-ui/package.json
@@ -1,7 +1,7 @@
 {
   "name": "@prometheus-io/mantine-ui",
   "private": true,
-  "version": "0.309.0",
+  "version": "0.309.1",
   "type": "module",
   "scripts": {
     "start": "vite",
@@ -28,7 +28,7 @@
     "@microsoft/fetch-event-source": "^2.0.1",
     "@nexucis/fuzzy": "^0.5.1",
     "@nexucis/kvsearch": "^0.9.1",
-    "@prometheus-io/codemirror-promql": "0.309.0",
+    "@prometheus-io/codemirror-promql": "0.309.1",
     "@reduxjs/toolkit": "^2.10.1",
     "@tabler/icons-react": "^3.35.0",
     "@tanstack/react-query": "^5.90.7",
diff --git a/web/ui/module/codemirror-promql/package.json b/web/ui/module/codemirror-promql/package.json
index 227dc67ed6..06b75f735c 100644
--- a/web/ui/module/codemirror-promql/package.json
+++ b/web/ui/module/codemirror-promql/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@prometheus-io/codemirror-promql",
-  "version": "0.309.0",
+  "version": "0.309.1",
   "description": "a CodeMirror mode for the PromQL language",
   "types": "dist/esm/index.d.ts",
   "module": "dist/esm/index.js",
@@ -29,7 +29,7 @@
   },
   "homepage": "https://github.com/prometheus/prometheus/blob/main/web/ui/module/codemirror-promql/README.md",
   "dependencies": {
-    "@prometheus-io/lezer-promql": "0.309.0",
+    "@prometheus-io/lezer-promql": "0.309.1",
     "lru-cache": "^11.2.2"
   },
   "devDependencies": {
diff --git a/web/ui/module/lezer-promql/package.json b/web/ui/module/lezer-promql/package.json
index e1cf0ad67b..eccae9a163 100644
--- a/web/ui/module/lezer-promql/package.json
+++ b/web/ui/module/lezer-promql/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@prometheus-io/lezer-promql",
-  "version": "0.309.0",
+  "version": "0.309.1",
   "description": "lezer-based PromQL grammar",
   "main": "dist/index.cjs",
   "type": "module",
diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json
index c52491732d..d7dd079718 100644
--- a/web/ui/package-lock.json
+++ b/web/ui/package-lock.json
@@ -1,12 +1,12 @@
 {
   "name": "prometheus-io",
-  "version": "0.309.0",
+  "version": "0.309.1",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "prometheus-io",
-      "version": "0.309.0",
+      "version": "0.309.1",
       "workspaces": [
         "mantine-ui",
         "module/*"
@@ -24,7 +24,7 @@
     },
     "mantine-ui": {
       "name": "@prometheus-io/mantine-ui",
-      "version": "0.309.0",
+      "version": "0.309.1",
       "dependencies": {
         "@codemirror/autocomplete": "^6.19.1",
         "@codemirror/language": "^6.11.3",
@@ -42,7 +42,7 @@
         "@microsoft/fetch-event-source": "^2.0.1",
         "@nexucis/fuzzy": "^0.5.1",
         "@nexucis/kvsearch": "^0.9.1",
-        "@prometheus-io/codemirror-promql": "0.309.0",
+        "@prometheus-io/codemirror-promql": "0.309.1",
         "@reduxjs/toolkit": "^2.10.1",
         "@tabler/icons-react": "^3.35.0",
         "@tanstack/react-query": "^5.90.7",
@@ -88,10 +88,10 @@
     },
     "module/codemirror-promql": {
       "name": "@prometheus-io/codemirror-promql",
-      "version": "0.309.0",
+      "version": "0.309.1",
       "license": "Apache-2.0",
       "dependencies": {
-        "@prometheus-io/lezer-promql": "0.309.0",
+        "@prometheus-io/lezer-promql": "0.309.1",
         "lru-cache": "^11.2.2"
       },
       "devDependencies": {
@@ -121,7 +121,7 @@
     },
     "module/lezer-promql": {
       "name": "@prometheus-io/lezer-promql",
-      "version": "0.309.0",
+      "version": "0.309.1",
       "license": "Apache-2.0",
       "devDependencies": {
         "@lezer/generator": "^1.8.0",
diff --git a/web/ui/package.json b/web/ui/package.json
index 0f054c34a7..e634652b41 100644
--- a/web/ui/package.json
+++ b/web/ui/package.json
@@ -1,7 +1,7 @@
 {
   "name": "prometheus-io",
   "description": "Monorepo for the Prometheus UI",
-  "version": "0.309.0",
+  "version": "0.309.1",
   "private": true,
   "scripts": {
     "build": "bash build_ui.sh --all",

From a919e6d5ef9f058cb7a2b1ed83f13f85c8cb147f Mon Sep 17 00:00:00 2001
From: Charles Korn 
Date: Thu, 8 Jan 2026 21:20:23 +1100
Subject: [PATCH 233/439] model/labels: improve performance of regex matchers
 like `.*-.*-.*` (#17707)

#14173 introduced an optimisation to better handle regex patterns like .*-.*-.*. It identifies strings the pattern cannot possibly match (because they do not contain all of the literal values) and returns false from MatchString early.

However, if the string does contain all literal values, then the Go regex engine is used to confirm that the string does match the pattern. But this is not necessary in the case where the start and end of the pattern is .* and everything in between is either a literal or .*: if the string contains all of the literals in order, then it matches the pattern, and invoking Go's regex engine to confirm this is unnecessary and quite slow.

* Add some more test cases
* Add benchmark, since existing benchmark doesn't show much impact given most of the random test strings will not match the patterns.

Signed-off-by: Charles Korn 
---
 model/labels/regexp.go      | 47 ++++++++++++++++++++++++++-
 model/labels/regexp_test.go | 65 ++++++++++++++++++++++++++++++++-----
 2 files changed, 102 insertions(+), 10 deletions(-)

diff --git a/model/labels/regexp.go b/model/labels/regexp.go
index 5123bbc7dd..a4bdf885ee 100644
--- a/model/labels/regexp.go
+++ b/model/labels/regexp.go
@@ -77,7 +77,18 @@ func NewFastRegexMatcher(v string) (*FastRegexMatcher, error) {
 		if matches, caseSensitive := findSetMatches(parsed); caseSensitive {
 			m.setMatches = matches
 		}
-		m.stringMatcher = stringMatcherFromRegexp(parsed)
+
+		// Check if we have a pattern like .*-.*-.*.
+		// If so, then we can rely on the containsInOrder check in compileMatchStringFunction,
+		// so no further inspection of the string is required.
+		// We can't do this in stringMatcherFromRegexpInternal as we only want to apply this
+		// if the top-level pattern satisfies this requirement.
+		if isSimpleConcatenationPattern(parsed) {
+			m.stringMatcher = trueMatcher{}
+		} else {
+			m.stringMatcher = stringMatcherFromRegexp(parsed)
+		}
+
 		m.matchString = m.compileMatchStringFunction()
 	}
 
@@ -566,6 +577,40 @@ func stringMatcherFromRegexpInternal(re *syntax.Regexp) StringMatcher {
 	return nil
 }
 
+// isSimpleConcatenationPattern returns true if re contains only literals or wildcard matchers,
+// and starts and ends with a wildcard matcher (eg. .*-.*-.*).
+func isSimpleConcatenationPattern(re *syntax.Regexp) bool {
+	if re.Op != syntax.OpConcat {
+		return false
+	}
+
+	if len(re.Sub) < 2 {
+		return false
+	}
+
+	first := re.Sub[0]
+	last := re.Sub[len(re.Sub)-1]
+	if !isMatchAny(first) || !isMatchAny(last) {
+		return false
+	}
+
+	for _, re := range re.Sub[1 : len(re.Sub)-1] {
+		if !isMatchAny(re) && !isCaseSensitiveLiteral(re) {
+			return false
+		}
+	}
+
+	return true
+}
+
+func isMatchAny(re *syntax.Regexp) bool {
+	return re.Op == syntax.OpStar && re.Sub[0].Op == syntax.OpAnyChar
+}
+
+func isCaseSensitiveLiteral(re *syntax.Regexp) bool {
+	return re.Op == syntax.OpLiteral && isCaseSensitive(re)
+}
+
 // containsStringMatcher matches a string if it contains any of the substrings.
 // If left and right are not nil, it's a contains operation where left and right must match.
 // If left is nil, it's a hasPrefix operation and right must match.
diff --git a/model/labels/regexp_test.go b/model/labels/regexp_test.go
index 2fb5e806f0..85cbe02a1f 100644
--- a/model/labels/regexp_test.go
+++ b/model/labels/regexp_test.go
@@ -87,6 +87,9 @@ var (
 		"ſſs",
 		// Concat of literals and wildcards.
 		".*-.*-.*-.*-.*",
+		".+-.*-.*-.*-.+",
+		"-.*-.*-.*-.*",
+		".*-.*-.*-.*-",
 		"(.+)-(.+)-(.+)-(.+)-(.+)",
 		"((.*))(?i:f)((.*))o((.*))o((.*))",
 		"((.*))f((.*))(?i:o)((.*))o((.*))",
@@ -96,6 +99,11 @@ var (
 		"FOO", "Foo", "fOo", "foO", "OO", "Oo", "\nfoo\n", strings.Repeat("f", 20), "prometheus", "prometheus_api_v1", "prometheus_api_v1_foo",
 		"10.0.1.20", "10.0.2.10", "10.0.3.30", "10.0.4.40",
 		"foofoo0", "foofoo", "😀foo0", "ſſs", "ſſS", "AAAAAAAAAAAAAAAAAAAAAAAA", "BBBBBBBBBBBBBBBBBBBBBBBB", "cccccccccccccccccccccccC", "ſſſſſſſſſſſſſſſſſſſſſſſſS", "SSSSSSSSSSSSSSSSSSSSSSSSſ",
+		"a-b-c-d-e",
+		"aaaaaa-bbbbbb-cccccc-dddddd-eeeeee",
+		"aaaaaa----eeeeee",
+		"----",
+		"-a-a-a-",
 
 		// Values matching / not matching the test regexps on long alternations.
 		"zQPbMkNO", "zQPbMkNo", "jyyfj00j0061", "jyyfj00j006", "jyyfj00j00612", "NNSPdvMi", "NNSPdvMiXXX", "NNSPdvMixxx", "nnSPdvMi", "nnSPdvMiXXX",
@@ -162,6 +170,7 @@ func TestOptimizeConcatRegex(t *testing.T) {
 		{regex: "^5..$", prefix: "5", suffix: "", contains: nil},
 		{regex: "^release.*", prefix: "release", suffix: "", contains: nil},
 		{regex: "^env-[0-9]+laio[1]?[^0-9].*", prefix: "env-", suffix: "", contains: []string{"laio"}},
+		{regex: ".*-.*-.*-.*-.*", prefix: "", suffix: "", contains: []string{"-", "-", "-", "-"}},
 	}
 
 	for _, c := range cases {
@@ -341,7 +350,7 @@ func BenchmarkToNormalizedLower(b *testing.B) {
 	}
 }
 
-func TestStringMatcherFromRegexp(t *testing.T) {
+func TestNewFastRegexMatcher(t *testing.T) {
 	for _, c := range []struct {
 		pattern string
 		exp     StringMatcher
@@ -364,12 +373,12 @@ func TestStringMatcherFromRegexp(t *testing.T) {
 		{`(?i:((foo1|foo2|bar)))`, orStringMatcher([]StringMatcher{orStringMatcher([]StringMatcher{&equalStringMatcher{s: "FOO1", caseSensitive: false}, &equalStringMatcher{s: "FOO2", caseSensitive: false}}), &equalStringMatcher{s: "BAR", caseSensitive: false}})},
 		{"^((?i:foo|oo)|(bar))$", orStringMatcher([]StringMatcher{&equalStringMatcher{s: "FOO", caseSensitive: false}, &equalStringMatcher{s: "OO", caseSensitive: false}, &equalStringMatcher{s: "bar", caseSensitive: true}})},
 		{"(?i:(foo1|foo2|bar))", orStringMatcher([]StringMatcher{orStringMatcher([]StringMatcher{&equalStringMatcher{s: "FOO1", caseSensitive: false}, &equalStringMatcher{s: "FOO2", caseSensitive: false}}), &equalStringMatcher{s: "BAR", caseSensitive: false}})},
-		{".*foo.*", &containsStringMatcher{substrings: []string{"foo"}, left: trueMatcher{}, right: trueMatcher{}}},
-		{"(.*)foo.*", &containsStringMatcher{substrings: []string{"foo"}, left: trueMatcher{}, right: trueMatcher{}}},
-		{"(.*)foo(.*)", &containsStringMatcher{substrings: []string{"foo"}, left: trueMatcher{}, right: trueMatcher{}}},
+		{".*foo.*", trueMatcher{}},     // The containsInOrder check done in the function returned by compileMatchStringFunction is sufficient.
+		{"(.*)foo.*", trueMatcher{}},   // The containsInOrder check done in the function returned by compileMatchStringFunction is sufficient.
+		{"(.*)foo(.*)", trueMatcher{}}, // The containsInOrder check done in the function returned by compileMatchStringFunction is sufficient.
 		{"(.+)foo(.*)", &containsStringMatcher{substrings: []string{"foo"}, left: &anyNonEmptyStringMatcher{matchNL: true}, right: trueMatcher{}}},
 		{"^.+foo.+", &containsStringMatcher{substrings: []string{"foo"}, left: &anyNonEmptyStringMatcher{matchNL: true}, right: &anyNonEmptyStringMatcher{matchNL: true}}},
-		{"^(.*)(foo)(.*)$", &containsStringMatcher{substrings: []string{"foo"}, left: trueMatcher{}, right: trueMatcher{}}},
+		{"^(.*)(foo)(.*)$", trueMatcher{}}, // The containsInOrder check done in the function returned by compileMatchStringFunction is sufficient.
 		{"^(.*)(foo|foobar)(.*)$", &containsStringMatcher{substrings: []string{"foo", "foobar"}, left: trueMatcher{}, right: trueMatcher{}}},
 		{"^(.*)(foo|foobar)(.+)$", &containsStringMatcher{substrings: []string{"foo", "foobar"}, left: trueMatcher{}, right: &anyNonEmptyStringMatcher{matchNL: true}}},
 		{"^(.*)(bar|b|buzz)(.+)$", &containsStringMatcher{substrings: []string{"bar", "b", "buzz"}, left: trueMatcher{}, right: &anyNonEmptyStringMatcher{matchNL: true}}},
@@ -388,7 +397,7 @@ func TestStringMatcherFromRegexp(t *testing.T) {
 		{"(api|rpc)_(v1|prom)_((?i)push|query)", nil},
 		{"[a-z][a-z]", nil},
 		{"[1^3]", nil},
-		{".*foo.*bar.*", nil},
+		{".*foo.*bar.*", trueMatcher{}}, // The containsInOrder check done in the function returned by compileMatchStringFunction is sufficient.
 		{`\d*`, nil},
 		{".", nil},
 		{"/|/bar.*", &literalPrefixSensitiveStringMatcher{prefix: "/", right: orStringMatcher{emptyStringMatcher{}, &literalPrefixSensitiveStringMatcher{prefix: "bar", right: trueMatcher{}}}}},
@@ -415,10 +424,9 @@ func TestStringMatcherFromRegexp(t *testing.T) {
 	} {
 		t.Run(c.pattern, func(t *testing.T) {
 			t.Parallel()
-			parsed, err := syntax.Parse(c.pattern, syntax.Perl|syntax.DotNL)
+			matcher, err := NewFastRegexMatcher(c.pattern)
 			require.NoError(t, err)
-			matches := stringMatcherFromRegexp(parsed)
-			require.Equal(t, c.exp, matches)
+			require.Equal(t, c.exp, matcher.stringMatcher)
 		})
 	}
 }
@@ -1389,3 +1397,42 @@ func TestToNormalisedLower(t *testing.T) {
 		require.Equal(t, expectedOutput, toNormalisedLower(input, nil))
 	}
 }
+
+func TestIsSimpleConcatenationPattern(t *testing.T) {
+	testCases := map[string]bool{
+		".*-.*-.*-.*-.*": true,
+		".+-.*-.*-.*-.+": false,
+		"-.*-.*-.*-.*":   false,
+		".*-.*-.*-.*-":   false,
+		"-":              false,
+		".*":             false,
+	}
+
+	for testCase, expected := range testCases {
+		t.Run(testCase, func(t *testing.T) {
+			re, err := syntax.Parse(testCase, syntax.Perl|syntax.DotNL)
+			require.NoError(t, err)
+			require.Equal(t, expected, isSimpleConcatenationPattern(re))
+		})
+	}
+}
+
+func BenchmarkFastRegexMatcher_ConcatenatedPattern(b *testing.B) {
+	pattern, err := NewFastRegexMatcher(".*-.*-.*-.*-.*")
+	require.NoError(b, err)
+
+	testCases := []string{
+		"a-b-c-d-e",
+		"aaaaaa-bbbbbb-cccccc-dddddd-eeeeee",
+		"aaaaaa----eeeeee",
+		"----",
+		"-a-a-a-",
+		"abcd",
+	}
+
+	for b.Loop() {
+		for _, s := range testCases {
+			pattern.MatchString(s)
+		}
+	}
+}

From 6a81e4441e725baf55e97f58299ceb15e7cf4491 Mon Sep 17 00:00:00 2001
From: Vilius Pranckaitis 
Date: Thu, 8 Jan 2026 13:58:05 +0200
Subject: [PATCH 234/439] promql: avoid unnecessary `Metric.Get()` calls in
 `functions.go` (#17676)

Moved some Metric.Get() calls in PromQL functions to avoid unnecessary label extraction.
In many cases, this work was done to extract metric name, and was only used if annotations were emitted.

In the same go I also replaced labels.MetricName with model.MetricNameLabel, since the former was deprecated.

Signed-off-by: Vilius Pranckaitis 
---
 promql/functions.go               | 89 ++++++++++++++-----------------
 promql/functions_internal_test.go |  7 ++-
 2 files changed, 45 insertions(+), 51 deletions(-)

diff --git a/promql/functions.go b/promql/functions.go
index 3f2079aba0..9c04392232 100644
--- a/promql/functions.go
+++ b/promql/functions.go
@@ -200,9 +200,8 @@ func extrapolatedRate(vals Matrix, args parser.Expressions, enh *EvalNodeHelper,
 	// We need either at least two Histograms and no Floats, or at least two
 	// Floats and no Histograms to calculate a rate. Otherwise, drop this
 	// Vector element.
-	metricName := samples.Metric.Get(labels.MetricName)
 	if len(samples.Histograms) > 0 && len(samples.Floats) > 0 {
-		return enh.Out, annos.Add(annotations.NewMixedFloatsHistogramsWarning(metricName, args[0].PositionRange()))
+		return enh.Out, annos.Add(annotations.NewMixedFloatsHistogramsWarning(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 
 	switch {
@@ -211,7 +210,7 @@ func extrapolatedRate(vals Matrix, args parser.Expressions, enh *EvalNodeHelper,
 		firstT = samples.Histograms[0].T
 		lastT = samples.Histograms[numSamplesMinusOne].T
 		var newAnnos annotations.Annotations
-		resultHistogram, newAnnos = histogramRate(samples.Histograms, isCounter, metricName, args[0].PositionRange())
+		resultHistogram, newAnnos = histogramRate(samples.Histograms, isCounter, samples.Metric, args[0].PositionRange())
 		annos.Merge(newAnnos)
 		if resultHistogram == nil {
 			// The histograms are not compatible with each other.
@@ -305,7 +304,7 @@ func extrapolatedRate(vals Matrix, args parser.Expressions, enh *EvalNodeHelper,
 // points[0] to be a histogram. It returns nil if any other Point in points is
 // not a histogram, and a warning wrapped in an annotation in that case.
 // Otherwise, it returns the calculated histogram and an empty annotation.
-func histogramRate(points []HPoint, isCounter bool, metricName string, pos posrange.PositionRange) (*histogram.FloatHistogram, annotations.Annotations) {
+func histogramRate(points []HPoint, isCounter bool, labels labels.Labels, pos posrange.PositionRange) (*histogram.FloatHistogram, annotations.Annotations) {
 	var (
 		prev               = points[0].H
 		usingCustomBuckets = prev.UsesCustomBuckets()
@@ -314,14 +313,14 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra
 	)
 
 	if last == nil {
-		return nil, annos.Add(annotations.NewMixedFloatsHistogramsWarning(metricName, pos))
+		return nil, annos.Add(annotations.NewMixedFloatsHistogramsWarning(getMetricName(labels), pos))
 	}
 
 	// We check for gauge type histograms in the loop below, but the loop
 	// below does not run on the first and last point, so check the first
 	// and last point now.
 	if isCounter && (prev.CounterResetHint == histogram.GaugeType || last.CounterResetHint == histogram.GaugeType) {
-		annos.Add(annotations.NewNativeHistogramNotCounterWarning(metricName, pos))
+		annos.Add(annotations.NewNativeHistogramNotCounterWarning(getMetricName(labels), pos))
 	}
 
 	// Null out the 1st sample if there is a counter reset between the 1st
@@ -338,7 +337,7 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra
 	}
 
 	if last.UsesCustomBuckets() != usingCustomBuckets {
-		return nil, annos.Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, pos))
+		return nil, annos.Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(labels), pos))
 	}
 
 	// First iteration to find out two things:
@@ -348,19 +347,19 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra
 	for _, currPoint := range points[1 : len(points)-1] {
 		curr := currPoint.H
 		if curr == nil {
-			return nil, annotations.New().Add(annotations.NewMixedFloatsHistogramsWarning(metricName, pos))
+			return nil, annotations.New().Add(annotations.NewMixedFloatsHistogramsWarning(getMetricName(labels), pos))
 		}
 		if !isCounter {
 			continue
 		}
 		if curr.CounterResetHint == histogram.GaugeType {
-			annos.Add(annotations.NewNativeHistogramNotCounterWarning(metricName, pos))
+			annos.Add(annotations.NewNativeHistogramNotCounterWarning(getMetricName(labels), pos))
 		}
 		if curr.Schema < minSchema {
 			minSchema = curr.Schema
 		}
 		if curr.UsesCustomBuckets() != usingCustomBuckets {
-			return nil, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, pos))
+			return nil, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(labels), pos))
 		}
 	}
 
@@ -371,7 +370,7 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra
 	_, _, nhcbBoundsReconciled, err := h.Sub(prev)
 	if err != nil {
 		if errors.Is(err, histogram.ErrHistogramsIncompatibleSchema) {
-			return nil, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, pos))
+			return nil, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(labels), pos))
 		}
 	}
 	if nhcbBoundsReconciled {
@@ -387,7 +386,7 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra
 				_, _, nhcbBoundsReconciled, err := h.Add(prev)
 				if err != nil {
 					if errors.Is(err, histogram.ErrHistogramsIncompatibleSchema) {
-						return nil, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, pos))
+						return nil, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(labels), pos))
 					}
 				}
 				if nhcbBoundsReconciled {
@@ -397,7 +396,7 @@ func histogramRate(points []HPoint, isCounter bool, metricName string, pos posra
 			prev = curr
 		}
 	} else if points[0].H.CounterResetHint != histogram.GaugeType || points[len(points)-1].H.CounterResetHint != histogram.GaugeType {
-		annos.Add(annotations.NewNativeHistogramNotGaugeWarning(metricName, pos))
+		annos.Add(annotations.NewNativeHistogramNotGaugeWarning(getMetricName(labels), pos))
 	}
 
 	h.CounterResetHint = histogram.GaugeType
@@ -431,10 +430,9 @@ func funcIdelta(_ []Vector, matrixVals Matrix, args parser.Expressions, enh *Eva
 
 func instantValue(vals Matrix, args parser.Expressions, out Vector, isRate bool) (Vector, annotations.Annotations) {
 	var (
-		samples    = vals[0]
-		metricName = samples.Metric.Get(labels.MetricName)
-		ss         = make([]Sample, 0, 2)
-		annos      annotations.Annotations
+		samples = vals[0]
+		ss      = make([]Sample, 0, 2)
+		annos   annotations.Annotations
 	)
 
 	// No sense in trying to compute a rate without at least two points. Drop
@@ -500,11 +498,11 @@ func instantValue(vals Matrix, args parser.Expressions, out Vector, isRate bool)
 		resultSample.H = ss[1].H.Copy()
 		// irate should only be applied to counters.
 		if isRate && (ss[1].H.CounterResetHint == histogram.GaugeType || ss[0].H.CounterResetHint == histogram.GaugeType) {
-			annos.Add(annotations.NewNativeHistogramNotCounterWarning(metricName, args.PositionRange()))
+			annos.Add(annotations.NewNativeHistogramNotCounterWarning(getMetricName(samples.Metric), args.PositionRange()))
 		}
 		// idelta should only be applied to gauges.
 		if !isRate && (ss[1].H.CounterResetHint != histogram.GaugeType || ss[0].H.CounterResetHint != histogram.GaugeType) {
-			annos.Add(annotations.NewNativeHistogramNotGaugeWarning(metricName, args.PositionRange()))
+			annos.Add(annotations.NewNativeHistogramNotGaugeWarning(getMetricName(samples.Metric), args.PositionRange()))
 		}
 		if !isRate || !ss[1].H.DetectReset(ss[0].H) {
 			// This subtraction may deliberately include conflicting
@@ -513,7 +511,7 @@ func instantValue(vals Matrix, args parser.Expressions, out Vector, isRate bool)
 			// conflicting counter resets is ignored here.
 			_, _, nhcbBoundsReconciled, err := resultSample.H.Sub(ss[0].H)
 			if errors.Is(err, histogram.ErrHistogramsIncompatibleSchema) {
-				return out, annos.Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, args.PositionRange()))
+				return out, annos.Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(samples.Metric), args.PositionRange()))
 			}
 			if nhcbBoundsReconciled {
 				annos.Add(annotations.NewMismatchedCustomBucketsHistogramsInfo(args.PositionRange(), annotations.HistogramSub))
@@ -523,7 +521,7 @@ func instantValue(vals Matrix, args parser.Expressions, out Vector, isRate bool)
 		resultSample.H.Compact(0)
 	default:
 		// Mix of a float and a histogram.
-		return out, annos.Add(annotations.NewMixedFloatsHistogramsWarning(metricName, args.PositionRange()))
+		return out, annos.Add(annotations.NewMixedFloatsHistogramsWarning(getMetricName(samples.Metric), args.PositionRange()))
 	}
 
 	if isRate {
@@ -565,7 +563,6 @@ func calcTrendValue(i int, tf, s0, s1, b float64) float64 {
 // https://en.wikipedia.org/wiki/Exponential_smoothing .
 func funcDoubleExponentialSmoothing(vectorVals []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalNodeHelper) (Vector, annotations.Annotations) {
 	samples := matrixVal[0]
-	metricName := samples.Metric.Get(labels.MetricName)
 	// The smoothing factor argument.
 	sf := vectorVals[0][0].F
 
@@ -586,7 +583,7 @@ func funcDoubleExponentialSmoothing(vectorVals []Vector, matrixVal Matrix, args
 	if l < 2 {
 		// Annotate mix of float and histogram.
 		if l == 1 && len(samples.Histograms) > 0 {
-			return enh.Out, annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+			return enh.Out, annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 		}
 		return enh.Out, nil
 	}
@@ -609,7 +606,7 @@ func funcDoubleExponentialSmoothing(vectorVals []Vector, matrixVal Matrix, args
 		s0, s1 = s1, x+y
 	}
 	if len(samples.Histograms) > 0 {
-		return append(enh.Out, Sample{F: s1}), annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		return append(enh.Out, Sample{F: s1}), annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 	return append(enh.Out, Sample{F: s1}), nil
 }
@@ -795,8 +792,7 @@ func aggrHistOverTime(matrixVal Matrix, enh *EvalNodeHelper, aggrFn func(Series)
 func funcAvgOverTime(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalNodeHelper) (Vector, annotations.Annotations) {
 	firstSeries := matrixVal[0]
 	if len(firstSeries.Floats) > 0 && len(firstSeries.Histograms) > 0 {
-		metricName := firstSeries.Metric.Get(labels.MetricName)
-		return enh.Out, annotations.New().Add(annotations.NewMixedFloatsHistogramsWarning(metricName, args[0].PositionRange()))
+		return enh.Out, annotations.New().Add(annotations.NewMixedFloatsHistogramsWarning(getMetricName(firstSeries.Metric), args[0].PositionRange()))
 	}
 	// For the average calculation of histograms, we use incremental mean
 	// calculation without the help of Kahan summation (but this should
@@ -871,9 +867,8 @@ func funcAvgOverTime(_ []Vector, matrixVal Matrix, args parser.Expressions, enh
 			return mean, nil
 		})
 		if err != nil {
-			metricName := firstSeries.Metric.Get(labels.MetricName)
 			if errors.Is(err, histogram.ErrHistogramsIncompatibleSchema) {
-				return enh.Out, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, args[0].PositionRange()))
+				return enh.Out, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(firstSeries.Metric), args[0].PositionRange()))
 			}
 		}
 		return vec, annos
@@ -980,8 +975,7 @@ func funcMadOverTime(_ []Vector, matrixVal Matrix, args parser.Expressions, enh
 		return enh.Out, nil
 	}
 	if len(samples.Histograms) > 0 {
-		metricName := samples.Metric.Get(labels.MetricName)
-		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 	return aggrOverTime(matrixVal, enh, func(s Series) float64 {
 		values := make(vectorByValueHeap, 0, len(s.Floats))
@@ -1059,8 +1053,7 @@ func compareOverTime(matrixVal Matrix, args parser.Expressions, enh *EvalNodeHel
 		return enh.Out, nil
 	}
 	if len(samples.Histograms) > 0 {
-		metricName := samples.Metric.Get(labels.MetricName)
-		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 	return aggrOverTime(matrixVal, enh, func(s Series) float64 {
 		maxVal := s.Floats[0].F
@@ -1096,8 +1089,7 @@ func funcMinOverTime(_ []Vector, matrixVals Matrix, args parser.Expressions, enh
 func funcSumOverTime(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalNodeHelper) (Vector, annotations.Annotations) {
 	firstSeries := matrixVal[0]
 	if len(firstSeries.Floats) > 0 && len(firstSeries.Histograms) > 0 {
-		metricName := firstSeries.Metric.Get(labels.MetricName)
-		return enh.Out, annotations.New().Add(annotations.NewMixedFloatsHistogramsWarning(metricName, args[0].PositionRange()))
+		return enh.Out, annotations.New().Add(annotations.NewMixedFloatsHistogramsWarning(getMetricName(firstSeries.Metric), args[0].PositionRange()))
 	}
 	if len(firstSeries.Floats) == 0 {
 		// The passed values only contain histograms.
@@ -1138,9 +1130,8 @@ func funcSumOverTime(_ []Vector, matrixVal Matrix, args parser.Expressions, enh
 			return sum, nil
 		})
 		if err != nil {
-			metricName := firstSeries.Metric.Get(labels.MetricName)
 			if errors.Is(err, histogram.ErrHistogramsIncompatibleSchema) {
-				return enh.Out, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(metricName, args[0].PositionRange()))
+				return enh.Out, annotations.New().Add(annotations.NewMixedExponentialCustomHistogramsWarning(getMetricName(firstSeries.Metric), args[0].PositionRange()))
 			}
 		}
 		return vec, annos
@@ -1170,8 +1161,7 @@ func funcQuantileOverTime(vectorVals []Vector, matrixVal Matrix, args parser.Exp
 		annos.Add(annotations.NewInvalidQuantileWarning(q, args[0].PositionRange()))
 	}
 	if len(el.Histograms) > 0 {
-		metricName := el.Metric.Get(labels.MetricName)
-		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(el.Metric), args[0].PositionRange()))
 	}
 	values := make(vectorByValueHeap, 0, len(el.Floats))
 	for _, f := range el.Floats {
@@ -1187,8 +1177,7 @@ func varianceOverTime(matrixVal Matrix, args parser.Expressions, enh *EvalNodeHe
 		return enh.Out, nil
 	}
 	if len(samples.Histograms) > 0 {
-		metricName := samples.Metric.Get(labels.MetricName)
-		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		annos.Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 	return aggrOverTime(matrixVal, enh, func(s Series) float64 {
 		var count float64
@@ -1478,14 +1467,13 @@ func linearRegression(samples []FPoint, interceptTime int64) (slope, intercept f
 // === deriv(node parser.ValueTypeMatrix) (Vector, Annotations) ===
 func funcDeriv(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalNodeHelper) (Vector, annotations.Annotations) {
 	samples := matrixVal[0]
-	metricName := samples.Metric.Get(labels.MetricName)
 
 	// No sense in trying to compute a derivative without at least two float points.
 	// Drop this Vector element.
 	if len(samples.Floats) < 2 {
 		// Annotate mix of float and histogram.
 		if len(samples.Floats) == 1 && len(samples.Histograms) > 0 {
-			return enh.Out, annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+			return enh.Out, annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 		}
 		return enh.Out, nil
 	}
@@ -1495,7 +1483,7 @@ func funcDeriv(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalN
 	// https://github.com/prometheus/prometheus/issues/2674
 	slope, _ := linearRegression(samples.Floats, samples.Floats[0].T)
 	if len(samples.Histograms) > 0 {
-		return append(enh.Out, Sample{F: slope}), annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		return append(enh.Out, Sample{F: slope}), annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 	return append(enh.Out, Sample{F: slope}), nil
 }
@@ -1504,21 +1492,20 @@ func funcDeriv(_ []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalN
 func funcPredictLinear(vectorVals []Vector, matrixVal Matrix, args parser.Expressions, enh *EvalNodeHelper) (Vector, annotations.Annotations) {
 	samples := matrixVal[0]
 	duration := vectorVals[0][0].F
-	metricName := samples.Metric.Get(labels.MetricName)
 
 	// No sense in trying to predict anything without at least two float points.
 	// Drop this Vector element.
 	if len(samples.Floats) < 2 {
 		// Annotate mix of float and histogram.
 		if len(samples.Floats) == 1 && len(samples.Histograms) > 0 {
-			return enh.Out, annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+			return enh.Out, annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 		}
 		return enh.Out, nil
 	}
 
 	slope, intercept := linearRegression(samples.Floats, enh.Ts)
 	if len(samples.Histograms) > 0 {
-		return append(enh.Out, Sample{F: slope*duration + intercept}), annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(metricName, args[0].PositionRange()))
+		return append(enh.Out, Sample{F: slope*duration + intercept}), annotations.New().Add(annotations.NewHistogramIgnoredInMixedRangeInfo(getMetricName(samples.Metric), args[0].PositionRange()))
 	}
 	return append(enh.Out, Sample{F: slope*duration + intercept}), nil
 }
@@ -1624,7 +1611,7 @@ func funcHistogramFraction(vectorVals []Vector, _ Matrix, args parser.Expression
 		if !enh.enableDelayedNameRemoval {
 			sample.Metric = sample.Metric.DropReserved(schema.IsMetadataLabel)
 		}
-		hf, hfAnnos := HistogramFraction(lower, upper, sample.H, sample.Metric.Get(model.MetricNameLabel), args[0].PositionRange())
+		hf, hfAnnos := HistogramFraction(lower, upper, sample.H, getMetricName(sample.Metric), args[0].PositionRange())
 		annos.Merge(hfAnnos)
 		enh.Out = append(enh.Out, Sample{
 			Metric:   sample.Metric,
@@ -1672,7 +1659,7 @@ func funcHistogramQuantile(vectorVals []Vector, _ Matrix, args parser.Expression
 		if !enh.enableDelayedNameRemoval {
 			sample.Metric = sample.Metric.DropReserved(schema.IsMetadataLabel)
 		}
-		hq, hqAnnos := HistogramQuantile(q, sample.H, sample.Metric.Get(model.MetricNameLabel), args[0].PositionRange())
+		hq, hqAnnos := HistogramQuantile(q, sample.H, getMetricName(sample.Metric), args[0].PositionRange())
 		annos.Merge(hqAnnos)
 		enh.Out = append(enh.Out, Sample{
 			Metric:   sample.Metric,
@@ -1687,7 +1674,7 @@ func funcHistogramQuantile(vectorVals []Vector, _ Matrix, args parser.Expression
 			res, forcedMonotonicity, _ := BucketQuantile(q, mb.buckets)
 			if forcedMonotonicity {
 				if enh.enableDelayedNameRemoval {
-					annos.Add(annotations.NewHistogramQuantileForcedMonotonicityInfo(mb.metric.Get(labels.MetricName), args[1].PositionRange()))
+					annos.Add(annotations.NewHistogramQuantileForcedMonotonicityInfo(getMetricName(mb.metric), args[1].PositionRange()))
 				} else {
 					annos.Add(annotations.NewHistogramQuantileForcedMonotonicityInfo("", args[1].PositionRange()))
 				}
@@ -2224,3 +2211,7 @@ func stringSliceFromArgs(args parser.Expressions) []string {
 	}
 	return tmp
 }
+
+func getMetricName(metric labels.Labels) string {
+	return metric.Get(model.MetricNameLabel)
+}
diff --git a/promql/functions_internal_test.go b/promql/functions_internal_test.go
index e5cd839459..bb52e4976b 100644
--- a/promql/functions_internal_test.go
+++ b/promql/functions_internal_test.go
@@ -18,9 +18,11 @@ import (
 	"math"
 	"testing"
 
+	"github.com/prometheus/common/model"
 	"github.com/stretchr/testify/require"
 
 	"github.com/prometheus/prometheus/model/histogram"
+	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/promql/parser/posrange"
 )
 
@@ -29,10 +31,11 @@ func TestHistogramRateCounterResetHint(t *testing.T) {
 		{T: 0, H: &histogram.FloatHistogram{CounterResetHint: histogram.CounterReset, Count: 5, Sum: 5}},
 		{T: 1, H: &histogram.FloatHistogram{CounterResetHint: histogram.UnknownCounterReset, Count: 10, Sum: 10}},
 	}
-	fh, _ := histogramRate(points, false, "foo", posrange.PositionRange{})
+	labels := labels.FromMap(map[string]string{model.MetricNameLabel: "foo"})
+	fh, _ := histogramRate(points, false, labels, posrange.PositionRange{})
 	require.Equal(t, histogram.GaugeType, fh.CounterResetHint)
 
-	fh, _ = histogramRate(points, true, "foo", posrange.PositionRange{})
+	fh, _ = histogramRate(points, true, labels, posrange.PositionRange{})
 	require.Equal(t, histogram.GaugeType, fh.CounterResetHint)
 }
 

From 14de1eb043f2b264056a9d1426d5db8c068c3b32 Mon Sep 17 00:00:00 2001
From: Arthur Silva Sens 
Date: Thu, 8 Jan 2026 10:06:33 -0300
Subject: [PATCH 235/439] Make service discoveries removable through build tags
 (#17736)

* Make service discoveries removable through build tags

Signed-off-by: Arthur Silva Sens 

* Fix cross-platform build issues

Signed-off-by: Arthur Silva Sens 

* Change build tags used

Signed-off-by: Arthur Silva Sens 

* Remove year from License header

Signed-off-by: Arthur Silva Sens 

* Remove plugins automation

Signed-off-by: Arthur Silva Sens 

* Update README

Signed-off-by: Arthur Silva Sens 

* Update README.md

Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: Arthur Silva Sens 

---------

Signed-off-by: Arthur Silva Sens 
Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com>
---
 Makefile                       |  9 +---
 README.md                      | 31 +++++++++---
 plugins.yml                    | 24 ---------
 plugins/generate.go            | 93 ----------------------------------
 plugins/plugin_aws.go          | 20 ++++++++
 plugins/plugin_azure.go        | 20 ++++++++
 plugins/plugin_consul.go       | 20 ++++++++
 plugins/plugin_digitalocean.go | 20 ++++++++
 plugins/plugin_dns.go          | 20 ++++++++
 plugins/plugin_eureka.go       | 20 ++++++++
 plugins/plugin_gce.go          | 20 ++++++++
 plugins/plugin_hetzner.go      | 20 ++++++++
 plugins/plugin_ionos.go        | 20 ++++++++
 plugins/plugin_kubernetes.go   | 20 ++++++++
 plugins/plugin_linode.go       | 20 ++++++++
 plugins/plugin_marathon.go     | 20 ++++++++
 plugins/plugin_moby.go         | 20 ++++++++
 plugins/plugin_nomad.go        | 20 ++++++++
 plugins/plugin_openstack.go    | 20 ++++++++
 plugins/plugin_ovhcloud.go     | 20 ++++++++
 plugins/plugin_puppetdb.go     | 20 ++++++++
 plugins/plugin_scaleway.go     | 20 ++++++++
 plugins/plugin_stackit.go      | 20 ++++++++
 plugins/plugin_triton.go       | 20 ++++++++
 plugins/plugin_uyuni.go        | 20 ++++++++
 plugins/plugin_vultr.go        | 20 ++++++++
 plugins/plugin_xds.go          | 20 ++++++++
 plugins/plugin_zookeeper.go    | 20 ++++++++
 plugins/plugins.go             | 67 ------------------------
 29 files changed, 504 insertions(+), 200 deletions(-)
 delete mode 100644 plugins.yml
 delete mode 100644 plugins/generate.go
 create mode 100644 plugins/plugin_aws.go
 create mode 100644 plugins/plugin_azure.go
 create mode 100644 plugins/plugin_consul.go
 create mode 100644 plugins/plugin_digitalocean.go
 create mode 100644 plugins/plugin_dns.go
 create mode 100644 plugins/plugin_eureka.go
 create mode 100644 plugins/plugin_gce.go
 create mode 100644 plugins/plugin_hetzner.go
 create mode 100644 plugins/plugin_ionos.go
 create mode 100644 plugins/plugin_kubernetes.go
 create mode 100644 plugins/plugin_linode.go
 create mode 100644 plugins/plugin_marathon.go
 create mode 100644 plugins/plugin_moby.go
 create mode 100644 plugins/plugin_nomad.go
 create mode 100644 plugins/plugin_openstack.go
 create mode 100644 plugins/plugin_ovhcloud.go
 create mode 100644 plugins/plugin_puppetdb.go
 create mode 100644 plugins/plugin_scaleway.go
 create mode 100644 plugins/plugin_stackit.go
 create mode 100644 plugins/plugin_triton.go
 create mode 100644 plugins/plugin_uyuni.go
 create mode 100644 plugins/plugin_vultr.go
 create mode 100644 plugins/plugin_xds.go
 create mode 100644 plugins/plugin_zookeeper.go
 delete mode 100644 plugins/plugins.go

diff --git a/Makefile b/Makefile
index bc5d67da6b..8c15ceb2e9 100644
--- a/Makefile
+++ b/Makefile
@@ -166,15 +166,8 @@ tarball: npm_licenses common-tarball
 .PHONY: docker
 docker: npm_licenses common-docker
 
-plugins/plugins.go: plugins.yml plugins/generate.go
-	@echo ">> creating plugins list"
-	$(GO) generate -tags plugins ./plugins
-
-.PHONY: plugins
-plugins: plugins/plugins.go
-
 .PHONY: build
-build: assets npm_licenses assets-compress plugins common-build
+build: assets npm_licenses assets-compress common-build
 
 .PHONY: bench_tsdb
 bench_tsdb: $(PROMU)
diff --git a/README.md b/README.md
index ae4ae50431..7b04a51cee 100644
--- a/README.md
+++ b/README.md
@@ -113,16 +113,31 @@ The Makefile provides several targets:
 
 ### Service discovery plugins
 
-Prometheus is bundled with many service discovery plugins.
-When building Prometheus from source, you can edit the [plugins.yml](./plugins.yml)
-file to disable some service discoveries. The file is a yaml-formatted list of go
-import path that will be built into the Prometheus binary.
+Prometheus is bundled with many service discovery plugins. You can customize
+which service discoveries are included in your build using Go build tags.
 
-After you have changed the file, you
-need to run `make build` again.
+To exclude service discoveries when building with `make build`, add the desired
+tags to the `.promu.yml` file under `build.tags.all`:
 
-If you are using another method to compile Prometheus, `make plugins` will
-generate the plugins file accordingly.
+```yaml
+build:
+    tags:
+        all:
+            - netgo
+            - builtinassets
+            - remove_all_sd           # Exclude all optional SDs
+            - enable_kubernetes_sd    # Re-enable only kubernetes
+```
+
+Then run `make build` as usual. Alternatively, when using `go build` directly:
+
+```bash
+go build -tags "remove_all_sd,enable_kubernetes_sd" ./cmd/prometheus
+```
+
+Available build tags:
+* `remove_all_sd` - Exclude all optional service discoveries (keeps file_sd, static_sd, and http_sd)
+* `enable__sd` - Re-enable a specific SD when using `remove_all_sd`
 
 If you add out-of-tree plugins, which we do not endorse at the moment,
 additional steps might be needed to adjust the `go.mod` and `go.sum` files. As
diff --git a/plugins.yml b/plugins.yml
deleted file mode 100644
index 0541fe4852..0000000000
--- a/plugins.yml
+++ /dev/null
@@ -1,24 +0,0 @@
-- github.com/prometheus/prometheus/discovery/aws
-- github.com/prometheus/prometheus/discovery/azure
-- github.com/prometheus/prometheus/discovery/consul
-- github.com/prometheus/prometheus/discovery/digitalocean
-- github.com/prometheus/prometheus/discovery/dns
-- github.com/prometheus/prometheus/discovery/eureka
-- github.com/prometheus/prometheus/discovery/gce
-- github.com/prometheus/prometheus/discovery/hetzner
-- github.com/prometheus/prometheus/discovery/ionos
-- github.com/prometheus/prometheus/discovery/kubernetes
-- github.com/prometheus/prometheus/discovery/linode
-- github.com/prometheus/prometheus/discovery/marathon
-- github.com/prometheus/prometheus/discovery/moby
-- github.com/prometheus/prometheus/discovery/nomad
-- github.com/prometheus/prometheus/discovery/openstack
-- github.com/prometheus/prometheus/discovery/ovhcloud
-- github.com/prometheus/prometheus/discovery/puppetdb
-- github.com/prometheus/prometheus/discovery/scaleway
-- github.com/prometheus/prometheus/discovery/stackit
-- github.com/prometheus/prometheus/discovery/triton
-- github.com/prometheus/prometheus/discovery/uyuni
-- github.com/prometheus/prometheus/discovery/vultr
-- github.com/prometheus/prometheus/discovery/xds
-- github.com/prometheus/prometheus/discovery/zookeeper
diff --git a/plugins/generate.go b/plugins/generate.go
deleted file mode 100644
index c0e58ec83b..0000000000
--- a/plugins/generate.go
+++ /dev/null
@@ -1,93 +0,0 @@
-// Copyright The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-//go:build plugins
-
-package main
-
-import (
-	"fmt"
-	"log"
-	"os"
-	"path"
-	"path/filepath"
-
-	"go.yaml.in/yaml/v2"
-)
-
-//go:generate go run generate.go
-
-func main() {
-	data, err := os.ReadFile(filepath.Join("..", "plugins.yml"))
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	var plugins []string
-	err = yaml.Unmarshal(data, &plugins)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	f, err := os.Create("plugins.go")
-	if err != nil {
-		log.Fatal(err)
-	}
-	defer f.Close()
-	_, err = f.WriteString(`// Copyright The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-// Code generated by "make plugins". DO NOT EDIT.
-
-package plugins
-
-`)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	if len(plugins) == 0 {
-		return
-	}
-
-	_, err = f.WriteString("import (\n")
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	for _, plugin := range plugins {
-		_, err = f.WriteString(fmt.Sprintf("\t// Register %s plugin.\n", path.Base(plugin)))
-		if err != nil {
-			log.Fatal(err)
-		}
-		_, err = f.WriteString(fmt.Sprintf("\t_ \"%s\"\n", plugin))
-		if err != nil {
-			log.Fatal(err)
-		}
-	}
-
-	_, err = f.WriteString(")\n")
-	if err != nil {
-		log.Fatal(err)
-	}
-}
diff --git a/plugins/plugin_aws.go b/plugins/plugin_aws.go
new file mode 100644
index 0000000000..711ef38c3e
--- /dev/null
+++ b/plugins/plugin_aws.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_aws_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/aws" // Register aws plugin.
+)
diff --git a/plugins/plugin_azure.go b/plugins/plugin_azure.go
new file mode 100644
index 0000000000..1f72812b8a
--- /dev/null
+++ b/plugins/plugin_azure.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_azure_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/azure" // Register azure plugin.
+)
diff --git a/plugins/plugin_consul.go b/plugins/plugin_consul.go
new file mode 100644
index 0000000000..6ff5003041
--- /dev/null
+++ b/plugins/plugin_consul.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_consul_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/consul" // Register consul plugin.
+)
diff --git a/plugins/plugin_digitalocean.go b/plugins/plugin_digitalocean.go
new file mode 100644
index 0000000000..927180e90b
--- /dev/null
+++ b/plugins/plugin_digitalocean.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_digitalocean_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/digitalocean" // Register digitalocean plugin.
+)
diff --git a/plugins/plugin_dns.go b/plugins/plugin_dns.go
new file mode 100644
index 0000000000..7bec66371e
--- /dev/null
+++ b/plugins/plugin_dns.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_dns_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/dns" // Register dns plugin.
+)
diff --git a/plugins/plugin_eureka.go b/plugins/plugin_eureka.go
new file mode 100644
index 0000000000..e4011da02a
--- /dev/null
+++ b/plugins/plugin_eureka.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_eureka_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/eureka" // Register eureka plugin.
+)
diff --git a/plugins/plugin_gce.go b/plugins/plugin_gce.go
new file mode 100644
index 0000000000..1c67657260
--- /dev/null
+++ b/plugins/plugin_gce.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_gce_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/gce" // Register gce plugin.
+)
diff --git a/plugins/plugin_hetzner.go b/plugins/plugin_hetzner.go
new file mode 100644
index 0000000000..f6b7db4563
--- /dev/null
+++ b/plugins/plugin_hetzner.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_hetzner_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/hetzner" // Register hetzner plugin.
+)
diff --git a/plugins/plugin_ionos.go b/plugins/plugin_ionos.go
new file mode 100644
index 0000000000..bf53b73053
--- /dev/null
+++ b/plugins/plugin_ionos.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_ionos_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/ionos" // Register ionos plugin.
+)
diff --git a/plugins/plugin_kubernetes.go b/plugins/plugin_kubernetes.go
new file mode 100644
index 0000000000..7145cedb2e
--- /dev/null
+++ b/plugins/plugin_kubernetes.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_kubernetes_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/kubernetes" // Register kubernetes plugin.
+)
diff --git a/plugins/plugin_linode.go b/plugins/plugin_linode.go
new file mode 100644
index 0000000000..4eb24b409c
--- /dev/null
+++ b/plugins/plugin_linode.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_linode_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/linode" // Register linode plugin.
+)
diff --git a/plugins/plugin_marathon.go b/plugins/plugin_marathon.go
new file mode 100644
index 0000000000..c26219a37a
--- /dev/null
+++ b/plugins/plugin_marathon.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_marathon_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/marathon" // Register marathon plugin.
+)
diff --git a/plugins/plugin_moby.go b/plugins/plugin_moby.go
new file mode 100644
index 0000000000..2c7c8e158b
--- /dev/null
+++ b/plugins/plugin_moby.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_moby_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/moby" // Register moby plugin.
+)
diff --git a/plugins/plugin_nomad.go b/plugins/plugin_nomad.go
new file mode 100644
index 0000000000..7251e507a2
--- /dev/null
+++ b/plugins/plugin_nomad.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_nomad_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/nomad" // Register nomad plugin.
+)
diff --git a/plugins/plugin_openstack.go b/plugins/plugin_openstack.go
new file mode 100644
index 0000000000..0dd227e8ac
--- /dev/null
+++ b/plugins/plugin_openstack.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_openstack_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/openstack" // Register openstack plugin.
+)
diff --git a/plugins/plugin_ovhcloud.go b/plugins/plugin_ovhcloud.go
new file mode 100644
index 0000000000..e3c372db8c
--- /dev/null
+++ b/plugins/plugin_ovhcloud.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_ovhcloud_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/ovhcloud" // Register ovhcloud plugin.
+)
diff --git a/plugins/plugin_puppetdb.go b/plugins/plugin_puppetdb.go
new file mode 100644
index 0000000000..33e82b6eac
--- /dev/null
+++ b/plugins/plugin_puppetdb.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_puppetdb_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/puppetdb" // Register puppetdb plugin.
+)
diff --git a/plugins/plugin_scaleway.go b/plugins/plugin_scaleway.go
new file mode 100644
index 0000000000..88e58ac646
--- /dev/null
+++ b/plugins/plugin_scaleway.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_scaleway_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/scaleway" // Register scaleway plugin.
+)
diff --git a/plugins/plugin_stackit.go b/plugins/plugin_stackit.go
new file mode 100644
index 0000000000..ac19419c27
--- /dev/null
+++ b/plugins/plugin_stackit.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_stackit_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/stackit" // Register stackit plugin.
+)
diff --git a/plugins/plugin_triton.go b/plugins/plugin_triton.go
new file mode 100644
index 0000000000..48989df8dd
--- /dev/null
+++ b/plugins/plugin_triton.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_triton_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/triton" // Register triton plugin.
+)
diff --git a/plugins/plugin_uyuni.go b/plugins/plugin_uyuni.go
new file mode 100644
index 0000000000..09f9ff033d
--- /dev/null
+++ b/plugins/plugin_uyuni.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_uyuni_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/uyuni" // Register uyuni plugin.
+)
diff --git a/plugins/plugin_vultr.go b/plugins/plugin_vultr.go
new file mode 100644
index 0000000000..5de4747cc7
--- /dev/null
+++ b/plugins/plugin_vultr.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_vultr_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/vultr" // Register vultr plugin.
+)
diff --git a/plugins/plugin_xds.go b/plugins/plugin_xds.go
new file mode 100644
index 0000000000..e0b0f048d2
--- /dev/null
+++ b/plugins/plugin_xds.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_xds_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/xds" // Register xds plugin.
+)
diff --git a/plugins/plugin_zookeeper.go b/plugins/plugin_zookeeper.go
new file mode 100644
index 0000000000..0852432920
--- /dev/null
+++ b/plugins/plugin_zookeeper.go
@@ -0,0 +1,20 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build !remove_all_sd || enable_zookeeper_sd
+
+package plugins
+
+import (
+	_ "github.com/prometheus/prometheus/discovery/zookeeper" // Register zookeeper plugin.
+)
diff --git a/plugins/plugins.go b/plugins/plugins.go
deleted file mode 100644
index 686fdfb325..0000000000
--- a/plugins/plugins.go
+++ /dev/null
@@ -1,67 +0,0 @@
-// Copyright The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-// Code generated by "make plugins". DO NOT EDIT.
-
-package plugins
-
-import (
-	// Register aws plugin.
-	_ "github.com/prometheus/prometheus/discovery/aws"
-	// Register azure plugin.
-	_ "github.com/prometheus/prometheus/discovery/azure"
-	// Register consul plugin.
-	_ "github.com/prometheus/prometheus/discovery/consul"
-	// Register digitalocean plugin.
-	_ "github.com/prometheus/prometheus/discovery/digitalocean"
-	// Register dns plugin.
-	_ "github.com/prometheus/prometheus/discovery/dns"
-	// Register eureka plugin.
-	_ "github.com/prometheus/prometheus/discovery/eureka"
-	// Register gce plugin.
-	_ "github.com/prometheus/prometheus/discovery/gce"
-	// Register hetzner plugin.
-	_ "github.com/prometheus/prometheus/discovery/hetzner"
-	// Register ionos plugin.
-	_ "github.com/prometheus/prometheus/discovery/ionos"
-	// Register kubernetes plugin.
-	_ "github.com/prometheus/prometheus/discovery/kubernetes"
-	// Register linode plugin.
-	_ "github.com/prometheus/prometheus/discovery/linode"
-	// Register marathon plugin.
-	_ "github.com/prometheus/prometheus/discovery/marathon"
-	// Register moby plugin.
-	_ "github.com/prometheus/prometheus/discovery/moby"
-	// Register nomad plugin.
-	_ "github.com/prometheus/prometheus/discovery/nomad"
-	// Register openstack plugin.
-	_ "github.com/prometheus/prometheus/discovery/openstack"
-	// Register ovhcloud plugin.
-	_ "github.com/prometheus/prometheus/discovery/ovhcloud"
-	// Register puppetdb plugin.
-	_ "github.com/prometheus/prometheus/discovery/puppetdb"
-	// Register scaleway plugin.
-	_ "github.com/prometheus/prometheus/discovery/scaleway"
-	// Register stackit plugin.
-	_ "github.com/prometheus/prometheus/discovery/stackit"
-	// Register triton plugin.
-	_ "github.com/prometheus/prometheus/discovery/triton"
-	// Register uyuni plugin.
-	_ "github.com/prometheus/prometheus/discovery/uyuni"
-	// Register vultr plugin.
-	_ "github.com/prometheus/prometheus/discovery/vultr"
-	// Register xds plugin.
-	_ "github.com/prometheus/prometheus/discovery/xds"
-	// Register zookeeper plugin.
-	_ "github.com/prometheus/prometheus/discovery/zookeeper"
-)

From 16703766f4f42dcce4edd3ec32187efd9f10dcbf Mon Sep 17 00:00:00 2001
From: Arve Knudsen 
Date: Fri, 9 Jan 2026 10:04:37 +0100
Subject: [PATCH 236/439] promql: fix info() returning empty when filtering by
 overlapping labels (#17817)

When filtering by a label that exists on both the input metric and
target_info (e.g., info(metric, {host_name="orbstack"}) where host_name
exists on both), the function incorrectly returned empty results.

The bug was in combineWithInfoVector: when no new labels were added
(because they all overlapped with base metric labels), the code entered
the "no match" filtering block even though an info series WAS matched.

The fix checks len(seenInfoMetrics) == 0 to correctly identify when no
info series matched. If an info series matched (seenInfoMetrics is
non-empty), the series is kept even if no new labels were added.

Fixes #17813

Signed-off-by: Arve Knudsen 
---
 promql/info.go                       |  7 ++++---
 promql/promqltest/testdata/info.test | 16 ++++++++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/promql/info.go b/promql/info.go
index ab4250104d..204ac44b40 100644
--- a/promql/info.go
+++ b/promql/info.go
@@ -424,9 +424,10 @@ func (ev *evaluator) combineWithInfoVector(base, info Vector, ignoreSeries map[u
 		}
 
 		infoLbls := enh.lb.Labels()
-		if infoLbls.Len() == 0 {
-			// If there's at least one data label matcher not matching the empty string,
-			// we have to ignore this series as there are no matching info series.
+		if len(seenInfoMetrics) == 0 {
+			// No info series matched this base series. If there's at least one data
+			// label matcher not matching the empty string, we have to ignore this
+			// series as there are no matching info series.
 			allMatchersMatchEmpty := true
 			for _, ms := range dataLabelMatchers {
 				for _, m := range ms {
diff --git a/promql/promqltest/testdata/info.test b/promql/promqltest/testdata/info.test
index 891e0eaa53..e15a429675 100644
--- a/promql/promqltest/testdata/info.test
+++ b/promql/promqltest/testdata/info.test
@@ -34,6 +34,22 @@ eval range from 0m to 10m step 5m info(metric, {data=~".+", non_existent=~".*"})
 eval range from 0m to 10m step 5m info(metric_with_overlapping_label)
     metric_with_overlapping_label{data="base", instance="a", job="1", label="value", another_data="another info"} 0 1 2
 
+# Filtering by a label that exists on both base metric and target_info should work.
+# This is a regression test for https://github.com/prometheus/prometheus/issues/17813.
+# Note: data="base" on base metric, data="info" on target_info - the filter matches target_info.
+eval range from 0m to 10m step 5m info(metric_with_overlapping_label, {data="info"})
+    metric_with_overlapping_label{data="base", instance="a", job="1", label="value"} 0 1 2
+
+# Filtering by a label that exists on both base metric and target_info with regex should work.
+eval range from 0m to 10m step 5m info(metric_with_overlapping_label, {data=~".+"})
+    metric_with_overlapping_label{data="base", instance="a", job="1", label="value"} 0 1 2
+
+# Filtering by a label that exists on both base metric and target_info with same value.
+# The selector matches the target_info, and the join succeeds via identifying labels.
+# Note: Only the instance label is considered for inclusion, but it already exists on base.
+eval range from 0m to 10m step 5m info(metric_with_overlapping_label, {instance="a"})
+    metric_with_overlapping_label{data="base", instance="a", job="1", label="value"} 0 1 2
+
 # Include data labels from target_info specifically.
 eval range from 0m to 10m step 5m info(metric, {__name__="target_info"})
     metric{data="info", instance="a", job="1", label="value", another_data="another info"} 0 1 2

From 9cb3641ccd9141d70555b965d1bc277b541e36a8 Mon Sep 17 00:00:00 2001
From: Ganesh Vernekar 
Date: Fri, 9 Jan 2026 07:00:53 -0800
Subject: [PATCH 237/439] Volunteer to shepherd the release v3.10 (#17822)

Signed-off-by: Ganesh Vernekar 
---
 RELEASE.md | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/RELEASE.md b/RELEASE.md
index c7375b35aa..5a8f8601ab 100644
--- a/RELEASE.md
+++ b/RELEASE.md
@@ -7,19 +7,20 @@ This page describes the release process and the currently planned schedule for u
 Release cadence of first pre-releases being cut is 6 weeks.
 Please see [the v2.55 RELEASE.md](https://github.com/prometheus/prometheus/blob/release-2.55/RELEASE.md) for the v2 release series schedule.
 
-| release series | date of first pre-release (year-month-day) | release shepherd                   |
-|----------------|--------------------------------------------|------------------------------------|
-| v3.0           | 2024-11-14                                 | Jan Fajerski (GitHub: @jan--f)     |
-| v3.1           | 2024-12-17                                 | Bryan Boreham (GitHub: @bboreham)  |
-| v3.2           | 2025-01-28                                 | Jan Fajerski (GitHub: @jan--f)     |
-| v3.3           | 2025-03-11                                 | Ayoub Mrini (Github: @machine424)  |
-| v3.4           | 2025-04-29                                 | Jan-Otto Kröpke (Github: @jkroepke)|
-| v3.5 LTS       | 2025-06-03                                 | Bryan Boreham (GitHub: @bboreham)  |
-| v3.6           | 2025-08-01                                 | Ayoub Mrini (Github: @machine424)  |
-| v3.7           | 2025-09-25                                 | Arthur Sens and George Krajcsovits (Github: @ArthurSens and @krajorama)|
-| v3.8           | 2025-11-06                                 | Jan Fajerski (GitHub: @jan--f)     |
-| v3.9           | 2025-12-18                                 | Bryan Boreham (GitHub: @bboreham)  |
-| v3.10          | 2026-02-05                                 | **volunteer welcome**              |
+| release series | date of first pre-release (year-month-day) | release shepherd                                                        |
+|----------------|--------------------------------------------|-------------------------------------------------------------------------|
+| v3.0           | 2024-11-14                                 | Jan Fajerski (GitHub: @jan--f)                                          |
+| v3.1           | 2024-12-17                                 | Bryan Boreham (GitHub: @bboreham)                                       |
+| v3.2           | 2025-01-28                                 | Jan Fajerski (GitHub: @jan--f)                                          |
+| v3.3           | 2025-03-11                                 | Ayoub Mrini (Github: @machine424)                                       |
+| v3.4           | 2025-04-29                                 | Jan-Otto Kröpke (Github: @jkroepke)                                     |
+| v3.5 LTS       | 2025-06-03                                 | Bryan Boreham (GitHub: @bboreham)                                       |
+| v3.6           | 2025-08-01                                 | Ayoub Mrini (Github: @machine424)                                       |
+| v3.7           | 2025-09-25                                 | Arthur Sens and George Krajcsovits (Github: @ArthurSens and @krajorama) |
+| v3.8           | 2025-11-06                                 | Jan Fajerski (GitHub: @jan--f)                                          |
+| v3.9           | 2025-12-18                                 | Bryan Boreham (GitHub: @bboreham)                                       |
+| v3.10          | 2026-02-05                                 | Ganesh Vernekar (Github: @codesome)                                     |
+| v3.11          | 2026-03-19                                 | **volunteer welcome**                                                   |
 
 If you are interested in volunteering please create a pull request against the [prometheus/prometheus](https://github.com/prometheus/prometheus) repository and propose yourself for the release series of your choice.
 

From 9a56fecb753c8b9e43ac8d9776c3209bf22c5569 Mon Sep 17 00:00:00 2001
From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Date: Fri, 9 Jan 2026 17:52:38 +0100
Subject: [PATCH 238/439] scripts: use git ls-files and check go.work files in
 version check

Update check-go-mod-version.sh to use git ls-files instead of find for
better performance and to respect .gitignore. Also include go.work files
in the version check to ensure consistency across workspace files and
modules.

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---
 scripts/check-go-mod-version.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/check-go-mod-version.sh b/scripts/check-go-mod-version.sh
index d651a62036..96317de2e6 100755
--- a/scripts/check-go-mod-version.sh
+++ b/scripts/check-go-mod-version.sh
@@ -1,12 +1,12 @@
 #!/usr/bin/env bash
 
-readarray -t mod_files < <(find . -type f -name go.mod)
+readarray -t mod_files < <(git ls-files go.mod go.work '*/go.mod' || find . -type f -name go.mod -or -name go.work)
 
 echo "Checking files ${mod_files[@]}"
 
 matches=$(awk '$1 == "go" {print $2}' "${mod_files[@]}" | sort -u | wc -l)
 
 if [[ "${matches}" -ne 1 ]]; then
-  echo 'Not all go.mod files have matching go versions'
+  echo 'Not all go.mod/go.work files have matching go versions'
   exit 1
 fi

From f9242d470726dca15e326bd1093f0ae916a4bf9f Mon Sep 17 00:00:00 2001
From: Ritik Shukla 
Date: Sat, 10 Jan 2026 16:05:21 +0530
Subject: [PATCH 239/439] util: enhance test coverage for strutil package

- Added comprehensive edge case tests for SanitizeLabelName (10 cases)
- Added comprehensive edge case tests for SanitizeFullLabelName (15 cases)
- Added more test cases for link generation functions (4 additional cases)
- Fixed unicode test case: corrected expected underscores from 7 to 5
- Fixed digits test case: corrected expected output from '_____' to '_2345'
- Converted tests to table-driven format with named subtests
- Achieved 100% code coverage for the package

Signed-off-by: Ritik Shukla 
---
 util/strutil/strconv_test.go | 189 +++++++++++++++++++++++++++++++----
 1 file changed, 169 insertions(+), 20 deletions(-)

diff --git a/util/strutil/strconv_test.go b/util/strutil/strconv_test.go
index b4b87ee816..362fa79a6a 100644
--- a/util/strutil/strconv_test.go
+++ b/util/strutil/strconv_test.go
@@ -36,6 +36,26 @@ var linkTests = []linkTest{
 		"/graph?g0.expr=sum%28incoming_http_requests_total%7Bsystem%3D%22trackmetadata%22%7D%29&g0.tab=0",
 		"/graph?g0.expr=sum%28incoming_http_requests_total%7Bsystem%3D%22trackmetadata%22%7D%29&g0.tab=1",
 	},
+	{
+		"up",
+		"/graph?g0.expr=up&g0.tab=0",
+		"/graph?g0.expr=up&g0.tab=1",
+	},
+	{
+		"rate(http_requests_total[5m])",
+		"/graph?g0.expr=rate%28http_requests_total%5B5m%5D%29&g0.tab=0",
+		"/graph?g0.expr=rate%28http_requests_total%5B5m%5D%29&g0.tab=1",
+	},
+	{
+		"",
+		"/graph?g0.expr=&g0.tab=0",
+		"/graph?g0.expr=&g0.tab=1",
+	},
+	{
+		"metric_name{label=\"value with spaces\"}",
+		"/graph?g0.expr=metric_name%7Blabel%3D%22value+with+spaces%22%7D&g0.tab=0",
+		"/graph?g0.expr=metric_name%7Blabel%3D%22value+with+spaces%22%7D&g0.tab=1",
+	},
 }
 
 func TestLink(t *testing.T) {
@@ -51,29 +71,158 @@ func TestLink(t *testing.T) {
 }
 
 func TestSanitizeLabelName(t *testing.T) {
-	actual := SanitizeLabelName("fooClientLABEL")
-	expected := "fooClientLABEL"
-	require.Equal(t, expected, actual, "SanitizeLabelName failed for label (%s)", expected)
+	tests := []struct {
+		name     string
+		input    string
+		expected string
+	}{
+		{
+			name:     "valid label name",
+			input:    "fooClientLABEL",
+			expected: "fooClientLABEL",
+		},
+		{
+			name:     "label with special characters",
+			input:    "barClient.LABEL$$##",
+			expected: "barClient_LABEL____",
+		},
+		{
+			name:     "label starting with digit",
+			input:    "123label",
+			expected: "123label",
+		},
+		{
+			name:     "label with dashes",
+			input:    "my-label-name",
+			expected: "my_label_name",
+		},
+		{
+			name:     "label with spaces",
+			input:    "my label name",
+			expected: "my_label_name",
+		},
+		{
+			name:     "label with mixed case and numbers",
+			input:    "Test123Label456",
+			expected: "Test123Label456",
+		},
+		{
+			name:     "label with unicode characters",
+			input:    "test-ñ-ü-label",
+			expected: "test_____label",
+		},
+		{
+			name:     "empty string",
+			input:    "",
+			expected: "",
+		},
+		{
+			name:     "only underscores",
+			input:    "___",
+			expected: "___",
+		},
+		{
+			name:     "label with colons",
+			input:    "namespace:metric_name",
+			expected: "namespace_metric_name",
+		},
+	}
 
-	actual = SanitizeLabelName("barClient.LABEL$$##")
-	expected = "barClient_LABEL____"
-	require.Equal(t, expected, actual, "SanitizeLabelName failed for label (%s)", expected)
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			actual := SanitizeLabelName(tt.input)
+			require.Equal(t, tt.expected, actual, "SanitizeLabelName(%q) = %q, want %q", tt.input, actual, tt.expected)
+		})
+	}
 }
 
 func TestSanitizeFullLabelName(t *testing.T) {
-	actual := SanitizeFullLabelName("fooClientLABEL")
-	expected := "fooClientLABEL"
-	require.Equal(t, expected, actual, "SanitizeFullLabelName failed for label (%s)", expected)
+	tests := []struct {
+		name     string
+		input    string
+		expected string
+	}{
+		{
+			name:     "valid label name",
+			input:    "fooClientLABEL",
+			expected: "fooClientLABEL",
+		},
+		{
+			name:     "label with special characters",
+			input:    "barClient.LABEL$$##",
+			expected: "barClient_LABEL____",
+		},
+		{
+			name:     "label starting with digit",
+			input:    "0zerothClient1LABEL",
+			expected: "_zerothClient1LABEL",
+		},
+		{
+			name:     "empty string",
+			input:    "",
+			expected: "_",
+		},
+		{
+			name:     "label starting with multiple digits",
+			input:    "123abc",
+			expected: "_23abc",
+		},
+		{
+			name:     "label with dashes",
+			input:    "my-label-name",
+			expected: "my_label_name",
+		},
+		{
+			name:     "label with spaces",
+			input:    "my label name",
+			expected: "my_label_name",
+		},
+		{
+			name:     "label with numbers in middle",
+			input:    "Test123Label456",
+			expected: "Test123Label456",
+		},
+		{
+			name:     "single underscore",
+			input:    "_",
+			expected: "_",
+		},
+		{
+			name:     "label starting with underscore",
+			input:    "_validLabel",
+			expected: "_validLabel",
+		},
+		{
+			name:     "label with colons",
+			input:    "namespace:metric_name",
+			expected: "namespace_metric_name",
+		},
+		{
+			name:     "label with unicode characters",
+			input:    "test-ñ-ü-label",
+			expected: "test_____label",
+		},
+		{
+			name:     "only digits",
+			input:    "12345",
+			expected: "_2345",
+		},
+		{
+			name:     "label with mixed invalid characters at start",
+			input:    "!@#test",
+			expected: "___test",
+		},
+		{
+			name:     "label with consecutive digits at start",
+			input:    "0123test",
+			expected: "_123test",
+		},
+	}
 
-	actual = SanitizeFullLabelName("barClient.LABEL$$##")
-	expected = "barClient_LABEL____"
-	require.Equal(t, expected, actual, "SanitizeFullLabelName failed for label (%s)", expected)
-
-	actual = SanitizeFullLabelName("0zerothClient1LABEL")
-	expected = "_zerothClient1LABEL"
-	require.Equal(t, expected, actual, "SanitizeFullLabelName failed for label (%s)", expected)
-
-	actual = SanitizeFullLabelName("")
-	expected = "_"
-	require.Equal(t, expected, actual, "SanitizeFullLabelName failed for the empty label")
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			actual := SanitizeFullLabelName(tt.input)
+			require.Equal(t, tt.expected, actual, "SanitizeFullLabelName(%q) = %q, want %q", tt.input, actual, tt.expected)
+		})
+	}
 }

From 035952bc8b34661b17a4889afe9437dc8cf97887 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=EA=B9=80=EB=AF=BC=EC=98=81?= 
Date: Sun, 11 Jan 2026 00:29:23 +0900
Subject: [PATCH 240/439] refactor(ui): Remove explicit any from globals
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: 김민영 
---
 web/ui/react-app/src/globals.ts       | 7 +++----
 web/ui/react-app/src/types/index.d.ts | 5 +++++
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/web/ui/react-app/src/globals.ts b/web/ui/react-app/src/globals.ts
index d2a5f1d50a..7a59bdbffd 100644
--- a/web/ui/react-app/src/globals.ts
+++ b/web/ui/react-app/src/globals.ts
@@ -1,6 +1,5 @@
 import jquery from 'jquery';
+import moment from 'moment';
 
-// eslint-disable-next-line @typescript-eslint/no-explicit-any
-(window as any).jQuery = jquery;
-// eslint-disable-next-line @typescript-eslint/no-explicit-any
-(window as any).moment = require('moment');
+window.jQuery = jquery;
+window.moment = moment;
diff --git a/web/ui/react-app/src/types/index.d.ts b/web/ui/react-app/src/types/index.d.ts
index addf1cc702..9cf8fbd7cc 100644
--- a/web/ui/react-app/src/types/index.d.ts
+++ b/web/ui/react-app/src/types/index.d.ts
@@ -68,3 +68,8 @@ interface JQueryStatic {
     scale: () => Color;
   };
 }
+
+interface Window {
+  jQuery: JQueryStatic;
+  moment: typeof import('moment');
+}

From de0a864b5c4fab5d0d3ccfe89fdb80110396e521 Mon Sep 17 00:00:00 2001
From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Date: Thu, 23 Oct 2025 17:52:03 +0200
Subject: [PATCH 241/439] Fuzzing: Move to go fuzzing

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---
 .github/workflows/fuzzing.yml   |  40 +++++----
 Makefile                        |   5 ++
 promql/promqltest/test.go       |  33 +++++++
 util/fuzzing/.gitignore         |   1 +
 util/fuzzing/corpus.go          | 122 ++++++++++++++++++++++++++
 util/fuzzing/corpus_gen/main.go | 116 ++++++++++++++++++++++++
 util/fuzzing/fuzz_test.go       | 150 ++++++++++++++++++++++++++++++++
 7 files changed, 448 insertions(+), 19 deletions(-)
 create mode 100644 util/fuzzing/.gitignore
 create mode 100644 util/fuzzing/corpus.go
 create mode 100644 util/fuzzing/corpus_gen/main.go
 create mode 100644 util/fuzzing/fuzz_test.go

diff --git a/.github/workflows/fuzzing.yml b/.github/workflows/fuzzing.yml
index f9f7abafd6..64c50c3db3 100644
--- a/.github/workflows/fuzzing.yml
+++ b/.github/workflows/fuzzing.yml
@@ -1,30 +1,32 @@
-name: CIFuzz
+name: Fuzzing
 on:
   workflow_call:
 permissions:
   contents: read
 
 jobs:
-  Fuzzing:
+  fuzzing:
+    name: Run Go Fuzz Tests
     runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        fuzz_test: [FuzzParseMetricText, FuzzParseOpenMetric, FuzzParseMetricSelector, FuzzParseExpr]
     steps:
-      - name: Build Fuzzers
-        id: build
-        uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@4bf20ff8dfda18ad651583ebca9fb17a7ce1940a # master
+      - name: Checkout repository
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
-          oss-fuzz-project-name: "prometheus"
-          dry-run: false
-      - name: Run Fuzzers
-        uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@4bf20ff8dfda18ad651583ebca9fb17a7ce1940a # master
-        # Note: Regularly check for updates to the pinned commit hash at:
-        # https://github.com/google/oss-fuzz/tree/master/infra/cifuzz/actions/run_fuzzers
+          persist-credentials: false
+      - name: Install Go
+        uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
-          oss-fuzz-project-name: "prometheus"
-          fuzz-seconds: 600
-          dry-run: false
-      - name: Upload Crash
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
-        if: failure() && steps.build.outcome == 'success'
+          go-version: 1.25.x
+      - name: Run Fuzzing
+        run: go test -fuzz=${{ matrix.fuzz_test }}$ -fuzztime=5m ./util/fuzzing
+        continue-on-error: true
+        id: fuzz
+      - name: Upload Crash Artifacts
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        if: failure()
         with:
-          name: artifacts
-          path: ./out/artifacts
+          name: fuzz-artifacts-${{ matrix.fuzz_test }}
+          path: promql/testdata/fuzz/${{ matrix.fuzz_test }}
diff --git a/Makefile b/Makefile
index 8c15ceb2e9..8bc4a3dcaa 100644
--- a/Makefile
+++ b/Makefile
@@ -220,3 +220,8 @@ check-node-version:
 bump-go-version:
 	@echo ">> bumping Go minor version"
 	@./scripts/bump_go_version.sh
+
+.PHONY: generate-fuzzing-seed-corpus
+generate-fuzzing-seed-corpus:
+	@echo ">> Generating fuzzing seed corpus"
+	@$(GO) generate -tags fuzzing ./util/fuzzing/corpus_gen
diff --git a/promql/promqltest/test.go b/promql/promqltest/test.go
index 1c4226b461..fc3872d197 100644
--- a/promql/promqltest/test.go
+++ b/promql/promqltest/test.go
@@ -113,6 +113,39 @@ func NewTestEngineWithOpts(tb testing.TB, opts promql.EngineOpts) *promql.Engine
 	return ng
 }
 
+// GetBuiltInExprs returns all the eval statement expressions from the built-in test files.
+func GetBuiltInExprs() ([]string, error) {
+	files, err := fs.Glob(testsFs, "*/*.test")
+	if err != nil {
+		return nil, err
+	}
+
+	var exprs []string
+	for _, fn := range files {
+		content, err := fs.ReadFile(testsFs, fn)
+		if err != nil {
+			return nil, err
+		}
+
+		// Create a minimal test struct just for parsing
+		testInstance := &test{
+			cmds: []testCommand{},
+		}
+		if err := testInstance.parse(string(content)); err != nil {
+			return nil, err
+		}
+
+		// Extract expressions from eval commands
+		for _, cmd := range testInstance.cmds {
+			if evalCmd, ok := cmd.(*evalCmd); ok {
+				exprs = append(exprs, evalCmd.expr)
+			}
+		}
+	}
+
+	return exprs, nil
+}
+
 // RunBuiltinTests runs an acceptance test suite against the provided engine.
 func RunBuiltinTests(t TBRun, engine promql.QueryEngine) {
 	RunBuiltinTestsWithStorage(t, engine, newTestStorage)
diff --git a/util/fuzzing/.gitignore b/util/fuzzing/.gitignore
new file mode 100644
index 0000000000..539a5ec32d
--- /dev/null
+++ b/util/fuzzing/.gitignore
@@ -0,0 +1 @@
+Fuzz*_seed_corpus.zip
diff --git a/util/fuzzing/corpus.go b/util/fuzzing/corpus.go
new file mode 100644
index 0000000000..52930b2669
--- /dev/null
+++ b/util/fuzzing/corpus.go
@@ -0,0 +1,122 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fuzzing
+
+import (
+	"github.com/prometheus/prometheus/promql/parser"
+	"github.com/prometheus/prometheus/promql/promqltest"
+)
+
+// GetCorpusForFuzzParseMetricText returns the seed corpus for FuzzParseMetricText.
+func GetCorpusForFuzzParseMetricText() [][]byte {
+	return [][]byte{
+		[]byte(""),
+		[]byte("metric_name 1.0"),
+		[]byte("# HELP metric_name help text\n# TYPE metric_name counter\nmetric_name 1.0"),
+		[]byte("o { quantile = \"1.0\", a = \"b\" } 8.3835e-05"),
+		[]byte("# HELP api_http_request_count The total number of HTTP requests.\n# TYPE api_http_request_count counter\nhttp_request_count{method=\"post\",code=\"200\"} 1027 1395066363000"),
+		[]byte("msdos_file_access_time_ms{path=\"C:\\\\DIR\\\\FILE.TXT\",error=\"Cannot find file:\\n\\\"FILE.TXT\\\"\"} 1.234e3"),
+		[]byte("metric_without_timestamp_and_labels 12.47"),
+		[]byte("something_weird{problem=\"division by zero\"} +Inf -3982045"),
+		[]byte("http_request_duration_seconds_bucket{le=\"+Inf\"} 144320"),
+		[]byte("go_gc_duration_seconds{ quantile=\"0.9\", a=\"b\"} 8.3835e-05"),
+		[]byte("go_gc_duration_seconds{ quantile=\"1.0\", a=\"b\" } 8.3835e-05"),
+		[]byte("go_gc_duration_seconds{ quantile = \"1.0\", a = \"b\" } 8.3835e-05"),
+	}
+}
+
+// GetCorpusForFuzzParseOpenMetric returns the seed corpus for FuzzParseOpenMetric.
+func GetCorpusForFuzzParseOpenMetric() [][]byte {
+	return [][]byte{
+		[]byte(""),
+		[]byte("# TYPE metric_name counter\nmetric_name_total 1.0"),
+		[]byte("# HELP metric_name help text\n# TYPE metric_name counter\nmetric_name_total 1.0\n# EOF"),
+	}
+}
+
+// GetCorpusForFuzzParseMetricSelector returns the seed corpus for FuzzParseMetricSelector.
+func GetCorpusForFuzzParseMetricSelector() []string {
+	return []string{
+		"",
+		"metric_name",
+		`metric_name{label="value"}`,
+		`{label="value"}`,
+		`metric_name{label=~"val.*"}`,
+	}
+}
+
+// GetCorpusForFuzzParseExpr returns the seed corpus for FuzzParseExpr.
+func GetCorpusForFuzzParseExpr() ([]string, error) {
+	// Enable experimental features to parse all test expressions.
+	parser.EnableExperimentalFunctions = true
+	parser.ExperimentalDurationExpr = true
+	parser.EnableExtendedRangeSelectors = true
+	defer func() {
+		parser.EnableExperimentalFunctions = false
+		parser.ExperimentalDurationExpr = false
+		parser.EnableExtendedRangeSelectors = false
+	}()
+
+	// Get built-in test expressions.
+	builtInExprs, err := promqltest.GetBuiltInExprs()
+	if err != nil {
+		return nil, err
+	}
+
+	// Add additional seed corpus.
+	additionalExprs := []string{
+		"",
+		"1",
+		"metric_name",
+		`"str"`,
+		// Numeric literals
+		".5",
+		"5.",
+		"123.4567",
+		"5e3",
+		"5e-3",
+		"+5.5e-3",
+		"0xc",
+		"0755",
+		"-0755",
+		"+Inf",
+		"-Inf",
+		// Basic binary operations
+		"1 + 1",
+		"1 - 1",
+		"1 * 1",
+		"1 / 1",
+		"1 % 1",
+		// Comparison operators
+		"1 == 1",
+		"1 != 1",
+		"1 > 1",
+		"1 >= 1",
+		"1 < 1",
+		"1 <= 1",
+		// Operations with identifiers
+		"foo == 1",
+		"foo * bar",
+		"2.5 / bar",
+		"foo and bar",
+		"foo or bar",
+		// Complex expressions
+		"+1 + -2 * 1",
+		"1 + 2/(3*1)",
+		// Comment
+		"#comment",
+	}
+
+	return append(builtInExprs, additionalExprs...), nil
+}
diff --git a/util/fuzzing/corpus_gen/main.go b/util/fuzzing/corpus_gen/main.go
new file mode 100644
index 0000000000..aa38a79a48
--- /dev/null
+++ b/util/fuzzing/corpus_gen/main.go
@@ -0,0 +1,116 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//go:build fuzzing
+
+//go:generate go run -tags fuzzing .
+
+package main
+
+import (
+	"archive/zip"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+
+	"github.com/prometheus/prometheus/util/fuzzing"
+)
+
+func main() {
+	if err := run(); err != nil {
+		fmt.Fprintf(os.Stderr, "Error: %v\n", err)
+		os.Exit(1)
+	}
+	fmt.Println("Successfully generated all seed corpus ZIP files.")
+}
+
+func run() error {
+	// Generate FuzzParseExpr seed corpus.
+	exprs, err := fuzzing.GetCorpusForFuzzParseExpr()
+	if err != nil {
+		return fmt.Errorf("failed to get corpus for FuzzParseExpr: %w", err)
+	}
+	if err := generateZipFromStrings("fuzzParseExpr", exprs); err != nil {
+		return fmt.Errorf("failed to generate FuzzParseExpr_seed_corpus.zip: %w", err)
+	}
+	fmt.Printf("Generated fuzzParseExpr_seed_corpus.zip with %d entries.\n", len(exprs))
+
+	// Generate FuzzParseMetricSelector seed corpus.
+	selectors := fuzzing.GetCorpusForFuzzParseMetricSelector()
+	if err := generateZipFromStrings("fuzzParseMetricSelector", selectors); err != nil {
+		return fmt.Errorf("failed to generate FuzzParseMetricSelector_seed_corpus.zip: %w", err)
+	}
+	fmt.Printf("Generated fuzzParseMetricSelector_seed_corpus.zip with %d entries.\n", len(selectors))
+
+	// Generate FuzzParseMetricText seed corpus.
+	metrics := fuzzing.GetCorpusForFuzzParseMetricText()
+	if err := generateZipFromBytes("fuzzParseMetricText", metrics); err != nil {
+		return fmt.Errorf("failed to generate FuzzParseMetricText_seed_corpus.zip: %w", err)
+	}
+	fmt.Printf("Generated fuzzParseMetricText_seed_corpus.zip with %d entries.\n", len(metrics))
+
+	// Generate FuzzParseOpenMetric seed corpus.
+	openMetrics := fuzzing.GetCorpusForFuzzParseOpenMetric()
+	if err := generateZipFromBytes("fuzzParseOpenMetric", openMetrics); err != nil {
+		return fmt.Errorf("failed to generate FuzzParseOpenMetric_seed_corpus.zip: %w", err)
+	}
+	fmt.Printf("Generated fuzzParseOpenMetric_seed_corpus.zip with %d entries.\n", len(openMetrics))
+
+	return nil
+}
+
+// generateZipFromBytes creates a seed corpus ZIP file from a slice of byte slices.
+func generateZipFromBytes(fuzzName string, corpus [][]byte) error {
+	// Sort corpus deterministically.
+	sorted := make([][]byte, len(corpus))
+	copy(sorted, corpus)
+	sort.Slice(sorted, func(i, j int) bool {
+		return string(sorted[i]) < string(sorted[j])
+	})
+
+	// Create ZIP file in parent directory.
+	zipPath := filepath.Join("..", fuzzName+"_seed_corpus.zip")
+	zipFile, err := os.Create(zipPath)
+	if err != nil {
+		return fmt.Errorf("failed to create zip file: %w", err)
+	}
+	defer zipFile.Close()
+
+	zipWriter := zip.NewWriter(zipFile)
+	defer zipWriter.Close()
+
+	// Add each corpus entry as a file.
+	for i, entry := range sorted {
+		fileName := fmt.Sprintf("expr%d", i)
+		writer, err := zipWriter.Create(fileName)
+		if err != nil {
+			return fmt.Errorf("failed to create zip entry %s: %w", fileName, err)
+		}
+		if _, err := writer.Write(entry); err != nil {
+			return fmt.Errorf("failed to write zip entry %s: %w", fileName, err)
+		}
+	}
+
+	return nil
+}
+
+// generateZipFromStrings creates a seed corpus ZIP file from a slice of strings.
+func generateZipFromStrings(fuzzName string, corpus []string) error {
+	// Convert []string to [][]byte and delegate to generateZipFromBytes
+	byteCorpus := make([][]byte, len(corpus))
+	for i, s := range corpus {
+		byteCorpus[i] = []byte(s)
+	}
+	return generateZipFromBytes(fuzzName, byteCorpus)
+}
diff --git a/util/fuzzing/fuzz_test.go b/util/fuzzing/fuzz_test.go
new file mode 100644
index 0000000000..8356fdad71
--- /dev/null
+++ b/util/fuzzing/fuzz_test.go
@@ -0,0 +1,150 @@
+// Copyright The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fuzzing
+
+import (
+	"errors"
+	"io"
+	"testing"
+
+	"github.com/prometheus/prometheus/model/labels"
+	"github.com/prometheus/prometheus/model/textparse"
+	"github.com/prometheus/prometheus/promql/parser"
+)
+
+const (
+	// Input size above which we know that Prometheus would consume too much
+	// memory. The recommended way to deal with it is check input size.
+	// https://google.github.io/oss-fuzz/getting-started/new-project-guide/#input-size
+	maxInputSize = 10240
+)
+
+// Use package-scope symbol table to avoid memory allocation on every fuzzing operation.
+var symbolTable = labels.NewSymbolTable()
+
+// FuzzParseMetricText fuzzes the metric parser with "text/plain" content type.
+//
+// Note that this is not the parser for the text-based exposition-format; that
+// lives in github.com/prometheus/client_golang/text.
+func FuzzParseMetricText(f *testing.F) {
+	// Add seed corpus
+	for _, corpus := range GetCorpusForFuzzParseMetricText() {
+		f.Add(corpus)
+	}
+
+	f.Fuzz(func(t *testing.T, in []byte) {
+		p, warning := textparse.New(in, "text/plain", symbolTable, textparse.ParserOptions{})
+		if p == nil || warning != nil {
+			// An invalid content type is being passed, which should not happen
+			// in this context.
+			t.Skip()
+		}
+
+		var err error
+		for {
+			_, err = p.Next()
+			if err != nil {
+				break
+			}
+		}
+		if errors.Is(err, io.EOF) {
+			err = nil
+		}
+
+		// We don't care about errors, just that we don't panic.
+		_ = err
+	})
+}
+
+// FuzzParseOpenMetric fuzzes the metric parser with "application/openmetrics-text" content type.
+func FuzzParseOpenMetric(f *testing.F) {
+	// Add seed corpus
+	for _, corpus := range GetCorpusForFuzzParseOpenMetric() {
+		f.Add(corpus)
+	}
+
+	f.Fuzz(func(t *testing.T, in []byte) {
+		p, warning := textparse.New(in, "application/openmetrics-text", symbolTable, textparse.ParserOptions{})
+		if p == nil || warning != nil {
+			// An invalid content type is being passed, which should not happen
+			// in this context.
+			t.Skip()
+		}
+
+		var err error
+		for {
+			_, err = p.Next()
+			if err != nil {
+				break
+			}
+		}
+		if errors.Is(err, io.EOF) {
+			err = nil
+		}
+
+		// We don't care about errors, just that we don't panic.
+		_ = err
+	})
+}
+
+// FuzzParseMetricSelector fuzzes the metric selector parser.
+func FuzzParseMetricSelector(f *testing.F) {
+	// Add seed corpus
+	for _, corpus := range GetCorpusForFuzzParseMetricSelector() {
+		f.Add(corpus)
+	}
+
+	f.Fuzz(func(t *testing.T, in string) {
+		if len(in) > maxInputSize {
+			t.Skip()
+		}
+		_, err := parser.ParseMetricSelector(in)
+		// We don't care about errors, just that we don't panic.
+		_ = err
+	})
+}
+
+// FuzzParseExpr fuzzes the expression parser.
+func FuzzParseExpr(f *testing.F) {
+	parser.EnableExperimentalFunctions = true
+	parser.ExperimentalDurationExpr = true
+	parser.EnableExtendedRangeSelectors = true
+	f.Cleanup(func() {
+		parser.EnableExperimentalFunctions = false
+		parser.ExperimentalDurationExpr = false
+		parser.EnableExtendedRangeSelectors = false
+	})
+
+	// Add seed corpus from built-in test expressions
+	corpus, err := GetCorpusForFuzzParseExpr()
+	if err != nil {
+		f.Fatal(err)
+	}
+	if len(corpus) < 1000 {
+		f.Fatalf("loading exprs is likely broken: got %d expressions, expected at least 1000", len(corpus))
+	}
+
+	for _, expr := range corpus {
+		f.Add(expr)
+	}
+
+	f.Fuzz(func(t *testing.T, in string) {
+		if len(in) > maxInputSize {
+			t.Skip()
+		}
+		_, err := parser.ParseExpr(in)
+		// We don't care about errors, just that we don't panic.
+		_ = err
+	})
+}

From a769a7eeb78e786cc0663823a7bfa514d7a07149 Mon Sep 17 00:00:00 2001
From: Nick Pillitteri <56quarters@users.noreply.github.com>
Date: Mon, 12 Jan 2026 10:51:45 -0500
Subject: [PATCH 242/439] model/labels: fix regex with capture, wildcards,
 literal  (#17828)

This change fixes an issue introduced in #17707. When a regex
with a wildcard, literal, and final wildcard surounded by a
capture group was parsed - the capture group was not removed
first preventing `optimizeConcatRegex` from running.

Found via fuzz testing.

Signed-off-by: Nick Pillitteri 
---
 model/labels/regexp.go      | 4 ++++
 model/labels/regexp_test.go | 1 +
 2 files changed, 5 insertions(+)

diff --git a/model/labels/regexp.go b/model/labels/regexp.go
index a4bdf885ee..5f4f753419 100644
--- a/model/labels/regexp.go
+++ b/model/labels/regexp.go
@@ -71,6 +71,10 @@ func NewFastRegexMatcher(v string) (*FastRegexMatcher, error) {
 		if err != nil {
 			return nil, err
 		}
+
+		// Remove any capture operations before trying to optimize the remaining operations.
+		clearCapture(parsed)
+
 		if parsed.Op == syntax.OpConcat {
 			m.prefix, m.suffix, m.contains = optimizeConcatRegex(parsed)
 		}
diff --git a/model/labels/regexp_test.go b/model/labels/regexp_test.go
index 85cbe02a1f..d4385c7481 100644
--- a/model/labels/regexp_test.go
+++ b/model/labels/regexp_test.go
@@ -93,6 +93,7 @@ var (
 		"(.+)-(.+)-(.+)-(.+)-(.+)",
 		"((.*))(?i:f)((.*))o((.*))o((.*))",
 		"((.*))f((.*))(?i:o)((.*))o((.*))",
+		"(.*0.*)",
 	}
 	values = []string{
 		"foo", " foo bar", "bar", "buzz\nbar", "bar foo", "bfoo", "\n", "\nfoo", "foo\n", "hello foo world", "hello foo\n world", "",

From c980c74f51e9933846b12859f8470671b1b6e5fb Mon Sep 17 00:00:00 2001
From: Arthur Silva Sens 
Date: Tue, 13 Jan 2026 03:32:12 -0300
Subject: [PATCH 243/439] Update google/pprof to allow go 1.24.0 (#17843)

Signed-off-by: Arthur Silva Sens 
---
 documentation/examples/remote_storage/go.mod | 2 +-
 go.mod                                       | 4 ++--
 go.sum                                       | 4 ++--
 internal/tools/go.mod                        | 2 +-
 web/ui/mantine-ui/src/promql/tools/go.mod    | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/documentation/examples/remote_storage/go.mod b/documentation/examples/remote_storage/go.mod
index b77f248bf5..17076faddd 100644
--- a/documentation/examples/remote_storage/go.mod
+++ b/documentation/examples/remote_storage/go.mod
@@ -1,6 +1,6 @@
 module github.com/prometheus/prometheus/documentation/examples/remote_storage
 
-go 1.24.9
+go 1.24.0
 
 require (
 	github.com/alecthomas/kingpin/v2 v2.4.0
diff --git a/go.mod b/go.mod
index 61c555abc2..24619581d1 100644
--- a/go.mod
+++ b/go.mod
@@ -1,6 +1,6 @@
 module github.com/prometheus/prometheus
 
-go 1.24.9
+go 1.24.0
 
 require (
 	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0
@@ -34,7 +34,7 @@ require (
 	github.com/gogo/protobuf v1.3.2
 	github.com/golang/snappy v1.0.0
 	github.com/google/go-cmp v0.7.0
-	github.com/google/pprof v0.0.0-20251213031049-b05bdaca462f
+	github.com/google/pprof v0.0.0-20260111202518-71be6bfdd440
 	github.com/google/uuid v1.6.0
 	github.com/gophercloud/gophercloud/v2 v2.9.0
 	github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853
diff --git a/go.sum b/go.sum
index b3333208dd..216cc63a7c 100644
--- a/go.sum
+++ b/go.sum
@@ -236,8 +236,8 @@ github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX
 github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
 github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
-github.com/google/pprof v0.0.0-20251213031049-b05bdaca462f h1:HU1RgM6NALf/KW9HEY6zry3ADbDKcmpQ+hJedoNGQYQ=
-github.com/google/pprof v0.0.0-20251213031049-b05bdaca462f/go.mod h1:67FPmZWbr+KDT/VlpWtw6sO9XSjpJmLuHpoLmWiTGgY=
+github.com/google/pprof v0.0.0-20260111202518-71be6bfdd440 h1:oKBqR+eQXiIM7X8K1JEg9aoTEePLq/c6Awe484abOuA=
+github.com/google/pprof v0.0.0-20260111202518-71be6bfdd440/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
 github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
 github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
diff --git a/internal/tools/go.mod b/internal/tools/go.mod
index a7a1ebec54..c8b62b5ca7 100644
--- a/internal/tools/go.mod
+++ b/internal/tools/go.mod
@@ -1,6 +1,6 @@
 module github.com/prometheus/prometheus/internal/tools
 
-go 1.24.9
+go 1.24.0
 
 require (
 	github.com/bufbuild/buf v1.62.1
diff --git a/web/ui/mantine-ui/src/promql/tools/go.mod b/web/ui/mantine-ui/src/promql/tools/go.mod
index 32b64019e9..a3abc881e2 100644
--- a/web/ui/mantine-ui/src/promql/tools/go.mod
+++ b/web/ui/mantine-ui/src/promql/tools/go.mod
@@ -1,6 +1,6 @@
 module github.com/prometheus/prometheus/web/ui/mantine-ui/src/promql/tools
 
-go 1.24.9
+go 1.24.0
 
 require (
 	github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853

From d9ed02665843e66ab50d4a95e11a0a56cc21ab2f Mon Sep 17 00:00:00 2001
From: Ben Kochie 
Date: Tue, 13 Jan 2026 07:35:11 +0100
Subject: [PATCH 244/439] Refractor promtool errors (#17842)

Replace use of `tsdb/errors` with standard library `errors`.

Signed-off-by: SuperQ 
---
 cmd/promtool/backfill.go |  5 ++---
 cmd/promtool/rules.go    |  6 +++---
 cmd/promtool/tsdb.go     | 11 +++++------
 3 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/cmd/promtool/backfill.go b/cmd/promtool/backfill.go
index f04a76b0a5..e7a9a7f18a 100644
--- a/cmd/promtool/backfill.go
+++ b/cmd/promtool/backfill.go
@@ -27,7 +27,6 @@ import (
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/model/textparse"
 	"github.com/prometheus/prometheus/tsdb"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 )
 
 func getMinAndMaxTimestamps(p textparse.Parser) (int64, int64, error) {
@@ -94,7 +93,7 @@ func createBlocks(input []byte, mint, maxt, maxBlockDuration int64, maxSamplesIn
 		return err
 	}
 	defer func() {
-		returnErr = tsdb_errors.NewMulti(returnErr, db.Close()).Err()
+		returnErr = errors.Join(returnErr, db.Close())
 	}()
 
 	var (
@@ -125,7 +124,7 @@ func createBlocks(input []byte, mint, maxt, maxBlockDuration int64, maxSamplesIn
 				return fmt.Errorf("block writer: %w", err)
 			}
 			defer func() {
-				err = tsdb_errors.NewMulti(err, w.Close()).Err()
+				err = errors.Join(err, w.Close())
 			}()
 
 			ctx := context.Background()
diff --git a/cmd/promtool/rules.go b/cmd/promtool/rules.go
index 3960206f6b..bb45178e9c 100644
--- a/cmd/promtool/rules.go
+++ b/cmd/promtool/rules.go
@@ -15,6 +15,7 @@ package main
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"log/slog"
 	"time"
@@ -28,7 +29,6 @@ import (
 	"github.com/prometheus/prometheus/rules"
 	"github.com/prometheus/prometheus/storage"
 	"github.com/prometheus/prometheus/tsdb"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 )
 
 const maxSamplesInMemory = 5000
@@ -143,7 +143,7 @@ func (importer *ruleImporter) importRule(ctx context.Context, ruleExpr, ruleName
 		var closed bool
 		defer func() {
 			if !closed {
-				err = tsdb_errors.NewMulti(err, w.Close()).Err()
+				err = errors.Join(err, w.Close())
 			}
 		}()
 		app := newMultipleAppender(ctx, w)
@@ -181,7 +181,7 @@ func (importer *ruleImporter) importRule(ctx context.Context, ruleExpr, ruleName
 		if err := app.flushAndCommit(ctx); err != nil {
 			return fmt.Errorf("flush and commit: %w", err)
 		}
-		err = tsdb_errors.NewMulti(err, w.Close()).Err()
+		err = errors.Join(err, w.Close())
 		closed = true
 	}
 
diff --git a/cmd/promtool/tsdb.go b/cmd/promtool/tsdb.go
index 9ccd1da714..d0016ec0aa 100644
--- a/cmd/promtool/tsdb.go
+++ b/cmd/promtool/tsdb.go
@@ -43,7 +43,6 @@ import (
 	"github.com/prometheus/prometheus/tsdb"
 	"github.com/prometheus/prometheus/tsdb/chunkenc"
 	"github.com/prometheus/prometheus/tsdb/chunks"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 	"github.com/prometheus/prometheus/tsdb/index"
 )
@@ -339,7 +338,7 @@ func listBlocks(path string, humanReadable bool) error {
 		return err
 	}
 	defer func() {
-		err = tsdb_errors.NewMulti(err, db.Close()).Err()
+		err = errors.Join(err, db.Close())
 	}()
 	blocks, err := db.Blocks()
 	if err != nil {
@@ -425,7 +424,7 @@ func analyzeBlock(ctx context.Context, path, blockID string, limit int, runExten
 		return err
 	}
 	defer func() {
-		err = tsdb_errors.NewMulti(err, db.Close()).Err()
+		err = errors.Join(err, db.Close())
 	}()
 
 	meta := block.Meta()
@@ -625,7 +624,7 @@ func analyzeCompaction(ctx context.Context, block tsdb.BlockReader, indexr tsdb.
 		return err
 	}
 	defer func() {
-		err = tsdb_errors.NewMulti(err, chunkr.Close()).Err()
+		err = errors.Join(err, chunkr.Close())
 	}()
 
 	totalChunks := 0
@@ -713,7 +712,7 @@ func dumpTSDBData(ctx context.Context, dbDir, sandboxDirRoot string, mint, maxt
 		return err
 	}
 	defer func() {
-		err = tsdb_errors.NewMulti(err, db.Close()).Err()
+		err = errors.Join(err, db.Close())
 	}()
 	q, err := db.Querier(mint, maxt)
 	if err != nil {
@@ -743,7 +742,7 @@ func dumpTSDBData(ctx context.Context, dbDir, sandboxDirRoot string, mint, maxt
 	}
 
 	if ws := ss.Warnings(); len(ws) > 0 {
-		return tsdb_errors.NewMulti(ws.AsErrors()...).Err()
+		return errors.Join(ws.AsErrors()...)
 	}
 
 	if ss.Err() != nil {

From 03a1d7a3501412c509989c2aedfeefc291c73e08 Mon Sep 17 00:00:00 2001
From: Ben Kochie 
Date: Tue, 13 Jan 2026 09:53:49 +0100
Subject: [PATCH 245/439] Refactor storage package errors (#17844)

Replace use of `tsdb/errors` with standard library `errors`.

Signed-off-by: SuperQ 
---
 storage/merge.go | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/storage/merge.go b/storage/merge.go
index a86a26891f..12d6d3ac0d 100644
--- a/storage/merge.go
+++ b/storage/merge.go
@@ -17,6 +17,7 @@ import (
 	"bytes"
 	"container/heap"
 	"context"
+	"errors"
 	"fmt"
 	"math"
 	"sync"
@@ -25,7 +26,6 @@ import (
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/tsdb/chunkenc"
 	"github.com/prometheus/prometheus/tsdb/chunks"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/util/annotations"
 )
 
@@ -269,13 +269,13 @@ func (q *mergeGenericQuerier) LabelNames(ctx context.Context, hints *LabelHints,
 
 // Close releases the resources of the generic querier.
 func (q *mergeGenericQuerier) Close() error {
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	for _, querier := range q.queriers {
 		if err := querier.Close(); err != nil {
-			errs.Add(err)
+			errs = append(errs, err)
 		}
 	}
-	return errs.Err()
+	return errors.Join(errs...)
 }
 
 func truncateToLimit(s []string, hints *LabelHints) []string {
@@ -679,11 +679,11 @@ func (c *chainSampleIterator) Next() chunkenc.ValueType {
 }
 
 func (c *chainSampleIterator) Err() error {
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	for _, iter := range c.iterators {
-		errs.Add(iter.Err())
+		errs = append(errs, iter.Err())
 	}
-	return errs.Err()
+	return errors.Join(errs...)
 }
 
 type samplesIteratorHeap []chunkenc.Iterator
@@ -821,12 +821,12 @@ func (c *compactChunkIterator) Next() bool {
 }
 
 func (c *compactChunkIterator) Err() error {
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	for _, iter := range c.iterators {
-		errs.Add(iter.Err())
+		errs = append(errs, iter.Err())
 	}
-	errs.Add(c.err)
-	return errs.Err()
+	errs = append(errs, c.err)
+	return errors.Join(errs...)
 }
 
 type chunkIteratorHeap []chunks.Iterator
@@ -904,9 +904,9 @@ func (c *concatenatingChunkIterator) Next() bool {
 }
 
 func (c *concatenatingChunkIterator) Err() error {
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	for _, iter := range c.iterators {
-		errs.Add(iter.Err())
+		errs = append(errs, iter.Err())
 	}
-	return errs.Err()
+	return errors.Join(errs...)
 }

From 802e959ec2bbce436adaa01df14d75b2ce2c5554 Mon Sep 17 00:00:00 2001
From: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
Date: Tue, 13 Jan 2026 10:02:58 +0100
Subject: [PATCH 246/439] chore(fuzzing): Meet required check expectation

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
---
 .github/workflows/fuzzing.yml | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/fuzzing.yml b/.github/workflows/fuzzing.yml
index 64c50c3db3..776e0a67c5 100644
--- a/.github/workflows/fuzzing.yml
+++ b/.github/workflows/fuzzing.yml
@@ -1,4 +1,4 @@
-name: Fuzzing
+name: fuzzing
 on:
   workflow_call:
 permissions:
@@ -30,3 +30,18 @@ jobs:
         with:
           name: fuzz-artifacts-${{ matrix.fuzz_test }}
           path: promql/testdata/fuzz/${{ matrix.fuzz_test }}
+  fuzzing_status:
+    # This status check aggregates the individual matrix jobs of the fuzzing
+    # step into a final status. Fails if a single matrix job fails, succeeds if
+    # all matrix jobs succeed.
+    name: Fuzzing
+    runs-on: ubuntu-latest
+    needs: [fuzzing]
+    if: always()
+    steps:
+      - name: Successful fuzzing
+        if: ${{ !(contains(needs.*.result, 'failure')) && !(contains(needs.*.result, 'cancelled')) }}
+        run: exit 0
+      - name: Failing or cancelled fuzzing
+        if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
+        run: exit 1

From 836caf7b162467210d1c1f2449034ddf9a40a86d Mon Sep 17 00:00:00 2001
From: Ben Kochie 
Date: Tue, 13 Jan 2026 10:21:33 +0100
Subject: [PATCH 247/439] Refactor storage package errors (#17846)

Replace use of `tsdb/errors` with standard library `errors`.

Signed-off-by: SuperQ 
---
 storage/fanout.go | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/storage/fanout.go b/storage/fanout.go
index 246a955b73..afcf993b3f 100644
--- a/storage/fanout.go
+++ b/storage/fanout.go
@@ -15,6 +15,7 @@ package storage
 
 import (
 	"context"
+	"errors"
 	"log/slog"
 
 	"github.com/prometheus/common/model"
@@ -23,7 +24,6 @@ import (
 	"github.com/prometheus/prometheus/model/histogram"
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/model/metadata"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 )
 
 type fanout struct {
@@ -82,11 +82,14 @@ func (f *fanout) Querier(mint, maxt int64) (Querier, error) {
 		querier, err := storage.Querier(mint, maxt)
 		if err != nil {
 			// Close already open Queriers, append potential errors to returned error.
-			errs := tsdb_errors.NewMulti(err, primary.Close())
-			for _, q := range secondaries {
-				errs.Add(q.Close())
+			errs := []error{
+				err,
+				primary.Close(),
 			}
-			return nil, errs.Err()
+			for _, q := range secondaries {
+				errs = append(errs, q.Close())
+			}
+			return nil, errors.Join(errs...)
 		}
 		if _, ok := querier.(noopQuerier); !ok {
 			secondaries = append(secondaries, querier)
@@ -106,11 +109,14 @@ func (f *fanout) ChunkQuerier(mint, maxt int64) (ChunkQuerier, error) {
 		querier, err := storage.ChunkQuerier(mint, maxt)
 		if err != nil {
 			// Close already open Queriers, append potential errors to returned error.
-			errs := tsdb_errors.NewMulti(err, primary.Close())
-			for _, q := range secondaries {
-				errs.Add(q.Close())
+			errs := []error{
+				err,
+				primary.Close(),
 			}
-			return nil, errs.Err()
+			for _, q := range secondaries {
+				errs = append(errs, q.Close())
+			}
+			return nil, errors.Join(errs...)
 		}
 		secondaries = append(secondaries, querier)
 	}
@@ -132,11 +138,13 @@ func (f *fanout) Appender(ctx context.Context) Appender {
 
 // Close closes the storage and all its underlying resources.
 func (f *fanout) Close() error {
-	errs := tsdb_errors.NewMulti(f.primary.Close())
-	for _, s := range f.secondaries {
-		errs.Add(s.Close())
+	errs := []error{
+		f.primary.Close(),
 	}
-	return errs.Err()
+	for _, s := range f.secondaries {
+		errs = append(errs, s.Close())
+	}
+	return errors.Join(errs...)
 }
 
 // fanoutAppender implements Appender.

From f331aa6d147374c18348f6504b8ff0bde8404f85 Mon Sep 17 00:00:00 2001
From: Gregor Zeitlinger 
Date: Tue, 13 Jan 2026 11:41:57 +0100
Subject: [PATCH 248/439] fix typo in release notes about
 `scrape_native_histograms`  (#17655)

* fix typo

Signed-off-by: Gregor Zeitlinger 

* add changelog

Signed-off-by: Gregor Zeitlinger 

* Update CHANGELOG.md

Co-authored-by: George Krajcsovits 
Signed-off-by: Gregor Zeitlinger 

* remove migration notes again

Signed-off-by: Gregor Zeitlinger 

---------

Signed-off-by: Gregor Zeitlinger 
Co-authored-by: George Krajcsovits 
---
 CHANGELOG.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d43bb24720..a1afb0af59 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -54,7 +54,7 @@
 * [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #17592
 * [FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. #16483
 * [FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. #17406
-* [FEATURE] Native histograms are now a stable, but optional feature, use the `scrape_native_histogram` config setting. #17232 #17315
+* [FEATURE] Native histograms are now a stable, but optional feature, use the `scrape_native_histograms` config setting. #17232 #17315
 * [FEATURE] UI: Support anchored and smoothed keyword in promql editor. #17239
 * [FEATURE] UI: Show detailed relabeling steps for each discovered target. #17337
 * [FEATURE] Alerting: Add urlQueryEscape to template functions. #17403

From 72a23934ad6fbdb9a545cbe199c820166bb5ba7c Mon Sep 17 00:00:00 2001
From: Ben Kochie 
Date: Tue, 13 Jan 2026 14:38:58 +0100
Subject: [PATCH 249/439] Refactor various tsdb sub-packages (#17847)

Migrate various tsdb related packages from `tsdb/errors` to the standard
library `errors` package.

Signed-off-by: SuperQ 
---
 tsdb/agent/db.go              |  3 +--
 tsdb/chunks/chunks.go         | 34 ++++++++++++++++++++++------------
 tsdb/chunks/head_chunks.go    | 23 +++++++++++------------
 tsdb/index/index.go           |  5 ++---
 tsdb/tombstones/tombstones.go |  3 +--
 tsdb/tsdbutil/dir_locker.go   |  8 +++-----
 tsdb/wlog/checkpoint.go       |  7 +++----
 tsdb/wlog/reader_test.go      | 13 +++++++++++--
 8 files changed, 54 insertions(+), 42 deletions(-)

diff --git a/tsdb/agent/db.go b/tsdb/agent/db.go
index 7de2ed678f..1b29b223d7 100644
--- a/tsdb/agent/db.go
+++ b/tsdb/agent/db.go
@@ -37,7 +37,6 @@ import (
 	"github.com/prometheus/prometheus/storage/remote"
 	"github.com/prometheus/prometheus/tsdb"
 	"github.com/prometheus/prometheus/tsdb/chunks"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/record"
 	"github.com/prometheus/prometheus/tsdb/tsdbutil"
 	"github.com/prometheus/prometheus/tsdb/wlog"
@@ -798,7 +797,7 @@ func (db *DB) Close() error {
 
 	db.metrics.Unregister()
 
-	return tsdb_errors.NewMulti(db.locker.Release(), db.wal.Close()).Err()
+	return errors.Join(db.locker.Release(), db.wal.Close())
 }
 
 type appenderBase struct {
diff --git a/tsdb/chunks/chunks.go b/tsdb/chunks/chunks.go
index 681fceb2fb..f8fc9a2e95 100644
--- a/tsdb/chunks/chunks.go
+++ b/tsdb/chunks/chunks.go
@@ -25,7 +25,6 @@ import (
 	"strconv"
 
 	"github.com/prometheus/prometheus/tsdb/chunkenc"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 )
 
@@ -431,13 +430,15 @@ func cutSegmentFile(dirFile *os.File, magicNumber uint32, chunksFormat byte, all
 	}
 	defer func() {
 		if returnErr != nil {
-			errs := tsdb_errors.NewMulti(returnErr)
+			errs := []error{
+				returnErr,
+			}
 			if f != nil {
-				errs.Add(f.Close())
+				errs = append(errs, f.Close())
 			}
 			// Calling RemoveAll on a non-existent file does not return error.
-			errs.Add(os.RemoveAll(ptmp))
-			returnErr = errs.Err()
+			errs = append(errs, os.RemoveAll(ptmp))
+			returnErr = errors.Join(errs...)
 		}
 	}()
 	if allocSize > 0 {
@@ -665,10 +666,10 @@ func NewDirReader(dir string, pool chunkenc.Pool) (*Reader, error) {
 	for _, fn := range files {
 		f, err := fileutil.OpenMmapFile(fn)
 		if err != nil {
-			return nil, tsdb_errors.NewMulti(
+			return nil, errors.Join(
 				fmt.Errorf("mmap files: %w", err),
-				tsdb_errors.CloseAll(cs),
-			).Err()
+				closeAll(cs),
+			)
 		}
 		cs = append(cs, f)
 		bs = append(bs, realByteSlice(f.Bytes()))
@@ -676,16 +677,16 @@ func NewDirReader(dir string, pool chunkenc.Pool) (*Reader, error) {
 
 	reader, err := newReader(bs, cs, pool)
 	if err != nil {
-		return nil, tsdb_errors.NewMulti(
+		return nil, errors.Join(
 			err,
-			tsdb_errors.CloseAll(cs),
-		).Err()
+			closeAll(cs),
+		)
 	}
 	return reader, nil
 }
 
 func (s *Reader) Close() error {
-	return tsdb_errors.CloseAll(s.cs)
+	return closeAll(s.cs)
 }
 
 // Size returns the size of the chunks.
@@ -774,3 +775,12 @@ func sequenceFiles(dir string) ([]string, error) {
 	}
 	return res, nil
 }
+
+// closeAll closes all given closers while recording error in MultiError.
+func closeAll(cs []io.Closer) error {
+	var errs []error
+	for _, c := range cs {
+		errs = append(errs, c.Close())
+	}
+	return errors.Join(errs...)
+}
diff --git a/tsdb/chunks/head_chunks.go b/tsdb/chunks/head_chunks.go
index ffe7e70fc6..809cd6b889 100644
--- a/tsdb/chunks/head_chunks.go
+++ b/tsdb/chunks/head_chunks.go
@@ -32,7 +32,6 @@ import (
 	"go.uber.org/atomic"
 
 	"github.com/prometheus/prometheus/tsdb/chunkenc"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 )
 
@@ -304,7 +303,7 @@ func (cdm *ChunkDiskMapper) openMMapFiles() (returnErr error) {
 	cdm.closers = map[int]io.Closer{}
 	defer func() {
 		if returnErr != nil {
-			returnErr = tsdb_errors.NewMulti(returnErr, closeAllFromMap(cdm.closers)).Err()
+			returnErr = errors.Join(returnErr, closeAllFromMap(cdm.closers))
 
 			cdm.mmappedChunkFiles = nil
 			cdm.closers = nil
@@ -614,7 +613,7 @@ func (cdm *ChunkDiskMapper) cut() (seq, offset int, returnErr error) {
 		// The file should not be closed if there is no error,
 		// its kept open in the ChunkDiskMapper.
 		if returnErr != nil {
-			returnErr = tsdb_errors.NewMulti(returnErr, newFile.Close()).Err()
+			returnErr = errors.Join(returnErr, newFile.Close())
 		}
 	}()
 
@@ -970,7 +969,7 @@ func (cdm *ChunkDiskMapper) Truncate(fileNo uint32) error {
 	}
 	cdm.readPathMtx.RUnlock()
 
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	// Cut a new file only if the current file has some chunks.
 	if cdm.curFileSize() > HeadChunkFileHeaderSize {
 		// There is a known race condition here because between the check of curFileSize() and the call to CutNewFile()
@@ -979,7 +978,7 @@ func (cdm *ChunkDiskMapper) Truncate(fileNo uint32) error {
 		cdm.CutNewFile()
 	}
 	pendingDeletes, err := cdm.deleteFiles(removedFiles)
-	errs.Add(err)
+	errs = append(errs, err)
 
 	if len(chkFileIndices) == len(removedFiles) {
 		// All files were deleted. Reset the current sequence.
@@ -1003,7 +1002,7 @@ func (cdm *ChunkDiskMapper) Truncate(fileNo uint32) error {
 		cdm.evtlPosMtx.Unlock()
 	}
 
-	return errs.Err()
+	return errors.Join(errs...)
 }
 
 // deleteFiles deletes the given file sequences in order of the sequence.
@@ -1098,23 +1097,23 @@ func (cdm *ChunkDiskMapper) Close() error {
 	}
 	cdm.closed = true
 
-	errs := tsdb_errors.NewMulti(
+	errs := []error{
 		closeAllFromMap(cdm.closers),
 		cdm.finalizeCurFile(),
 		cdm.dir.Close(),
-	)
+	}
 	cdm.mmappedChunkFiles = map[int]*mmappedChunkFile{}
 	cdm.closers = map[int]io.Closer{}
 
-	return errs.Err()
+	return errors.Join(errs...)
 }
 
 func closeAllFromMap(cs map[int]io.Closer) error {
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	for _, c := range cs {
-		errs.Add(c.Close())
+		errs = append(errs, c.Close())
 	}
-	return errs.Err()
+	return errors.Join(errs...)
 }
 
 const inBufferShards = 128 // 128 is a randomly chosen number.
diff --git a/tsdb/index/index.go b/tsdb/index/index.go
index 8a76770821..493264b87f 100644
--- a/tsdb/index/index.go
+++ b/tsdb/index/index.go
@@ -33,7 +33,6 @@ import (
 	"github.com/prometheus/prometheus/storage"
 	"github.com/prometheus/prometheus/tsdb/chunks"
 	"github.com/prometheus/prometheus/tsdb/encoding"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 )
 
@@ -1007,10 +1006,10 @@ func NewFileReader(path string, decoder PostingsDecoder) (*Reader, error) {
 	}
 	r, err := newReader(realByteSlice(f.Bytes()), f, decoder)
 	if err != nil {
-		return nil, tsdb_errors.NewMulti(
+		return nil, errors.Join(
 			err,
 			f.Close(),
-		).Err()
+		)
 	}
 
 	return r, nil
diff --git a/tsdb/tombstones/tombstones.go b/tsdb/tombstones/tombstones.go
index 25218782cd..b7bcd8801b 100644
--- a/tsdb/tombstones/tombstones.go
+++ b/tsdb/tombstones/tombstones.go
@@ -28,7 +28,6 @@ import (
 
 	"github.com/prometheus/prometheus/storage"
 	"github.com/prometheus/prometheus/tsdb/encoding"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 )
 
@@ -128,7 +127,7 @@ func WriteFile(logger *slog.Logger, dir string, tr Reader) (int64, error) {
 	size += n
 
 	if err := f.Sync(); err != nil {
-		return 0, tsdb_errors.NewMulti(err, f.Close()).Err()
+		return 0, errors.Join(err, f.Close())
 	}
 
 	if err = f.Close(); err != nil {
diff --git a/tsdb/tsdbutil/dir_locker.go b/tsdb/tsdbutil/dir_locker.go
index 45cabdd3d7..139e66859a 100644
--- a/tsdb/tsdbutil/dir_locker.go
+++ b/tsdb/tsdbutil/dir_locker.go
@@ -22,7 +22,6 @@ import (
 
 	"github.com/prometheus/client_golang/prometheus"
 
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 )
 
@@ -94,10 +93,9 @@ func (l *DirLocker) Release() error {
 		return nil
 	}
 
-	errs := tsdb_errors.NewMulti()
-	errs.Add(l.releaser.Release())
-	errs.Add(os.Remove(l.path))
+	releaserErr := l.releaser.Release()
+	removeErr := os.Remove(l.path)
 
 	l.releaser = nil
-	return errs.Err()
+	return errors.Join(releaserErr, removeErr)
 }
diff --git a/tsdb/wlog/checkpoint.go b/tsdb/wlog/checkpoint.go
index 57c2faf23e..6742141fbc 100644
--- a/tsdb/wlog/checkpoint.go
+++ b/tsdb/wlog/checkpoint.go
@@ -28,7 +28,6 @@ import (
 
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/tsdb/chunks"
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/tsdb/fileutil"
 	"github.com/prometheus/prometheus/tsdb/record"
 	"github.com/prometheus/prometheus/tsdb/tombstones"
@@ -71,14 +70,14 @@ func DeleteCheckpoints(dir string, maxIndex int) error {
 		return err
 	}
 
-	errs := tsdb_errors.NewMulti()
+	var errs []error
 	for _, checkpoint := range checkpoints {
 		if checkpoint.index >= maxIndex {
 			break
 		}
-		errs.Add(os.RemoveAll(filepath.Join(dir, checkpoint.name)))
+		errs = append(errs, os.RemoveAll(filepath.Join(dir, checkpoint.name)))
 	}
-	return errs.Err()
+	return errors.Join(errs...)
 }
 
 // CheckpointPrefix is the prefix used for checkpoint files.
diff --git a/tsdb/wlog/reader_test.go b/tsdb/wlog/reader_test.go
index 788a2edfb9..971423e5cc 100644
--- a/tsdb/wlog/reader_test.go
+++ b/tsdb/wlog/reader_test.go
@@ -18,6 +18,7 @@ import (
 	"bytes"
 	"crypto/rand"
 	"encoding/binary"
+	"errors"
 	"fmt"
 	"hash/crc32"
 	"io"
@@ -32,7 +33,6 @@ import (
 	"github.com/prometheus/common/promslog"
 	"github.com/stretchr/testify/require"
 
-	tsdb_errors "github.com/prometheus/prometheus/tsdb/errors"
 	"github.com/prometheus/prometheus/util/compression"
 )
 
@@ -287,7 +287,7 @@ func (m *multiReadCloser) Read(p []byte) (n int, err error) {
 }
 
 func (m *multiReadCloser) Close() error {
-	return tsdb_errors.NewMulti(tsdb_errors.CloseAll(m.closers)).Err()
+	return errors.Join(closeAll(m.closers))
 }
 
 func allSegments(dir string) (io.ReadCloser, error) {
@@ -549,3 +549,12 @@ func TestReaderData(t *testing.T) {
 		})
 	}
 }
+
+// closeAll closes all given closers while recording error in MultiError.
+func closeAll(cs []io.Closer) error {
+	var errs []error
+	for _, c := range cs {
+		errs = append(errs, c.Close())
+	}
+	return errors.Join(errs...)
+}

From c7bc56cf6c8f9c92e98beddca26ed9b47f8a5ac9 Mon Sep 17 00:00:00 2001
From: Devarsh 
Date: Tue, 13 Jan 2026 22:37:27 +0530
Subject: [PATCH 250/439] Add scrape commit and total duration metrics (#17665)

* Add scrape commit and total duration metrics

Signed-off-by: Devarsh 

* update metric based on the review

Signed-off-by: Devarsh 

* conditionally record scrape duration

Signed-off-by: Devarsh 

* Fix formatting in scrape.go

Signed-off-by: Devarsh 

---------

Signed-off-by: Devarsh 
---
 scrape/metrics.go | 12 ++++++++++++
 scrape/scrape.go  |  5 +++++
 2 files changed, 17 insertions(+)

diff --git a/scrape/metrics.go b/scrape/metrics.go
index 4662a9fd9e..34f1e28dba 100644
--- a/scrape/metrics.go
+++ b/scrape/metrics.go
@@ -56,6 +56,7 @@ type scrapeMetrics struct {
 	targetScrapeExemplarOutOfOrder         prometheus.Counter
 	targetScrapePoolExceededLabelLimits    prometheus.Counter
 	targetScrapeNativeHistogramBucketLimit prometheus.Counter
+	targetScrapeDuration                   prometheus.Histogram
 }
 
 func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) {
@@ -252,6 +253,15 @@ func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) {
 			Help: "Total number of exemplar rejected due to not being out of the expected order.",
 		},
 	)
+	sm.targetScrapeDuration = prometheus.NewHistogram(
+		prometheus.HistogramOpts{
+			Name:                            "prometheus_target_scrape_duration_seconds",
+			Help:                            "Total duration of the scrape from start to commit completion in seconds.",
+			NativeHistogramBucketFactor:     1.1,
+			NativeHistogramMaxBucketNumber:  100,
+			NativeHistogramMinResetDuration: 1 * time.Hour,
+		},
+	)
 
 	for _, collector := range []prometheus.Collector{
 		// Used by Manager.
@@ -284,6 +294,7 @@ func newScrapeMetrics(reg prometheus.Registerer) (*scrapeMetrics, error) {
 		sm.targetScrapeExemplarOutOfOrder,
 		sm.targetScrapePoolExceededLabelLimits,
 		sm.targetScrapeNativeHistogramBucketLimit,
+		sm.targetScrapeDuration,
 	} {
 		err := reg.Register(collector)
 		if err != nil {
@@ -324,6 +335,7 @@ func (sm *scrapeMetrics) Unregister() {
 	sm.reg.Unregister(sm.targetScrapeExemplarOutOfOrder)
 	sm.reg.Unregister(sm.targetScrapePoolExceededLabelLimits)
 	sm.reg.Unregister(sm.targetScrapeNativeHistogramBucketLimit)
+	sm.reg.Unregister(sm.targetScrapeDuration)
 }
 
 type TargetsGatherer interface {
diff --git a/scrape/scrape.go b/scrape/scrape.go
index 1a99155d09..58df858b3d 100644
--- a/scrape/scrape.go
+++ b/scrape/scrape.go
@@ -1335,6 +1335,11 @@ func (sl *scrapeLoop) scrapeAndReport(last, appendTime time.Time, errc chan<- er
 			return
 		}
 		err = app.Commit()
+		if sl.reportExtraMetrics {
+			totalDuration := time.Since(start)
+			// Record total scrape duration metric.
+			sl.metrics.targetScrapeDuration.Observe(totalDuration.Seconds())
+		}
 		if err != nil {
 			sl.l.Error("Scrape commit failed", "err", err)
 		}

From 5499260964b94e4a396f0ce2327388b174cb38f8 Mon Sep 17 00:00:00 2001
From: Rahulrairai59 
Date: Tue, 13 Jan 2026 21:25:56 -0600
Subject: [PATCH 251/439] Update react-router version to v7.12.0 to fix
 CVE-2026-21884 in package-lock.json

To fix CVE-2026-21884 HIGH severity vulnerability

Signed-off-by: Rahulrairai59 
---
 web/ui/package-lock.json | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/web/ui/package-lock.json b/web/ui/package-lock.json
index 764fd87820..a1f72ff228 100644
--- a/web/ui/package-lock.json
+++ b/web/ui/package-lock.json
@@ -7823,9 +7823,9 @@
       }
     },
     "node_modules/react-router": {
-      "version": "7.9.5",
-      "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.9.5.tgz",
-      "integrity": "sha512-JmxqrnBZ6E9hWmf02jzNn9Jm3UqyeimyiwzD69NjxGySG6lIz/1LVPsoTCwN7NBX2XjCEa1LIX5EMz1j2b6u6A==",
+      "version": "7.12.0",
+      "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.12.0.tgz",
+      "integrity": "sha512-kTPDYPFzDVGIIGNLS5VJykK0HfHLY5MF3b+xj0/tTyNYL1gF1qs7u67Z9jEhQk2sQ98SUaHxlG31g1JtF7IfVw==",
       "dependencies": {
         "cookie": "^1.0.1",
         "set-cookie-parser": "^2.6.0"

From 70bc06718dbc4b0e7f588e9c6e8363c6c41da85f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Mon, 12 Jan 2026 13:14:54 +0100
Subject: [PATCH 252/439] feat(tsdb): new AppenderV2 and AtST interface for
 chunks
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

No implementation yet. Just to test the shape of the interface.
AtST is implemented for trivial cases, anything else is hard coded
to return 0.

Ref: https://github.com/prometheus/prometheus/issues/17791

Signed-off-by: György Krajcsovits 
---
 promql/engine_test.go                   | 10 ++--
 promql/histogram_stats_iterator_test.go |  2 +
 promql/value.go                         |  5 ++
 rules/alerting_test.go                  |  2 +
 storage/buffer.go                       | 15 ++++++
 storage/buffer_test.go                  |  8 +++
 storage/interface.go                    | 23 ++++----
 storage/interface_test.go               | 19 ++++++-
 storage/merge.go                        |  7 +++
 storage/merge_test.go                   |  4 ++
 storage/remote/codec.go                 | 11 ++++
 storage/remote/codec_test.go            |  2 +-
 storage/series.go                       | 22 +++++---
 tsdb/chunkenc/chunk.go                  | 51 ++++++++++++------
 tsdb/chunkenc/chunk_test.go             |  6 +--
 tsdb/chunkenc/float_histogram.go        | 10 ++--
 tsdb/chunkenc/float_histogram_test.go   | 58 ++++++++++----------
 tsdb/chunkenc/histogram.go              | 10 ++--
 tsdb/chunkenc/histogram_test.go         | 70 ++++++++++++-------------
 tsdb/chunkenc/xor.go                    | 10 ++--
 tsdb/chunkenc/xor_test.go               |  2 +-
 tsdb/chunks/chunks.go                   |  7 +--
 tsdb/chunks/head_chunks_test.go         |  2 +-
 tsdb/chunks/samples.go                  | 13 +++--
 tsdb/head.go                            |  5 +-
 tsdb/head_append.go                     |  9 ++--
 tsdb/ooo_head.go                        |  9 ++--
 tsdb/querier.go                         | 34 +++++++++---
 tsdb/querier_test.go                    | 14 +++--
 29 files changed, 295 insertions(+), 145 deletions(-)

diff --git a/promql/engine_test.go b/promql/engine_test.go
index 7b7a67a54b..0eff93af4c 100644
--- a/promql/engine_test.go
+++ b/promql/engine_test.go
@@ -3747,12 +3747,12 @@ func TestHistogramRateWithFloatStaleness(t *testing.T) {
 		recoded bool
 	)
 
-	newc, recoded, app, err = app.AppendHistogram(nil, 0, h1.Copy(), false)
+	newc, recoded, app, err = app.AppendHistogram(nil, 0, 0, h1.Copy(), false)
 	require.NoError(t, err)
 	require.False(t, recoded)
 	require.Nil(t, newc)
 
-	newc, recoded, _, err = app.AppendHistogram(nil, 10, h1.Copy(), false)
+	newc, recoded, _, err = app.AppendHistogram(nil, 0, 10, h1.Copy(), false)
 	require.NoError(t, err)
 	require.False(t, recoded)
 	require.Nil(t, newc)
@@ -3762,7 +3762,7 @@ func TestHistogramRateWithFloatStaleness(t *testing.T) {
 	app, err = c2.Appender()
 	require.NoError(t, err)
 
-	app.Append(20, math.Float64frombits(value.StaleNaN))
+	app.Append(0, 20, math.Float64frombits(value.StaleNaN))
 
 	// Make a chunk with two normal histograms that have zero value.
 	h2 := histogram.Histogram{
@@ -3773,12 +3773,12 @@ func TestHistogramRateWithFloatStaleness(t *testing.T) {
 	app, err = c3.Appender()
 	require.NoError(t, err)
 
-	newc, recoded, app, err = app.AppendHistogram(nil, 30, h2.Copy(), false)
+	newc, recoded, app, err = app.AppendHistogram(nil, 0, 30, h2.Copy(), false)
 	require.NoError(t, err)
 	require.False(t, recoded)
 	require.Nil(t, newc)
 
-	newc, recoded, _, err = app.AppendHistogram(nil, 40, h2.Copy(), false)
+	newc, recoded, _, err = app.AppendHistogram(nil, 0, 40, h2.Copy(), false)
 	require.NoError(t, err)
 	require.False(t, recoded)
 	require.Nil(t, newc)
diff --git a/promql/histogram_stats_iterator_test.go b/promql/histogram_stats_iterator_test.go
index cfea8a568e..d3a76820da 100644
--- a/promql/histogram_stats_iterator_test.go
+++ b/promql/histogram_stats_iterator_test.go
@@ -235,4 +235,6 @@ func (h *histogramIterator) AtFloatHistogram(*histogram.FloatHistogram) (int64,
 
 func (*histogramIterator) AtT() int64 { return 0 }
 
+func (*histogramIterator) AtST() int64 { return 0 }
+
 func (*histogramIterator) Err() error { return nil }
diff --git a/promql/value.go b/promql/value.go
index 02cb021024..17afdfc410 100644
--- a/promql/value.go
+++ b/promql/value.go
@@ -487,6 +487,11 @@ func (ssi *storageSeriesIterator) AtT() int64 {
 	return ssi.currT
 }
 
+// TODO(krajorama): implement AtST.
+func (*storageSeriesIterator) AtST() int64 {
+	return 0
+}
+
 func (ssi *storageSeriesIterator) Next() chunkenc.ValueType {
 	if ssi.currH != nil {
 		ssi.iHistograms++
diff --git a/rules/alerting_test.go b/rules/alerting_test.go
index a2c7abcd56..caf32e6472 100644
--- a/rules/alerting_test.go
+++ b/rules/alerting_test.go
@@ -697,12 +697,14 @@ func TestQueryForStateSeries(t *testing.T) {
 		{
 			selectMockFunction: func(bool, *storage.SelectHints, ...*labels.Matcher) storage.SeriesSet {
 				return storage.TestSeriesSet(storage.MockSeries(
+					nil,
 					[]int64{1, 2, 3},
 					[]float64{1, 2, 3},
 					[]string{"__name__", "ALERTS_FOR_STATE", "alertname", "TestRule", "severity", "critical"},
 				))
 			},
 			expectedSeries: storage.MockSeries(
+				nil,
 				[]int64{1, 2, 3},
 				[]float64{1, 2, 3},
 				[]string{"__name__", "ALERTS_FOR_STATE", "alertname", "TestRule", "severity", "critical"},
diff --git a/storage/buffer.go b/storage/buffer.go
index 223c4fa42b..c6a32821d8 100644
--- a/storage/buffer.go
+++ b/storage/buffer.go
@@ -171,6 +171,11 @@ func (s fSample) T() int64 {
 	return s.t
 }
 
+// TODO(krajorama): implement ST.
+func (fSample) ST() int64 {
+	return 0
+}
+
 func (s fSample) F() float64 {
 	return s.f
 }
@@ -200,6 +205,11 @@ func (s hSample) T() int64 {
 	return s.t
 }
 
+// TODO(krajorama): implement ST.
+func (hSample) ST() int64 {
+	return 0
+}
+
 func (hSample) F() float64 {
 	panic("F() called for hSample")
 }
@@ -229,6 +239,11 @@ func (s fhSample) T() int64 {
 	return s.t
 }
 
+// TODO(krajorama): implement ST.
+func (fhSample) ST() int64 {
+	return 0
+}
+
 func (fhSample) F() float64 {
 	panic("F() called for fhSample")
 }
diff --git a/storage/buffer_test.go b/storage/buffer_test.go
index fc6603d4a5..beb9d8e71c 100644
--- a/storage/buffer_test.go
+++ b/storage/buffer_test.go
@@ -402,6 +402,10 @@ func (*mockSeriesIterator) AtT() int64 {
 	return 0 // Not really mocked.
 }
 
+func (*mockSeriesIterator) AtST() int64 {
+	return 0 // Not really mocked.
+}
+
 type fakeSeriesIterator struct {
 	nsamples int64
 	step     int64
@@ -428,6 +432,10 @@ func (it *fakeSeriesIterator) AtT() int64 {
 	return it.idx * it.step
 }
 
+func (*fakeSeriesIterator) AtST() int64 {
+	return 0 // No start timestamps in this fake iterator.
+}
+
 func (it *fakeSeriesIterator) Next() chunkenc.ValueType {
 	it.idx++
 	if it.idx >= it.nsamples {
diff --git a/storage/interface.go b/storage/interface.go
index 23b8b48a0c..a75ac3f58d 100644
--- a/storage/interface.go
+++ b/storage/interface.go
@@ -473,9 +473,10 @@ type Series interface {
 }
 
 type mockSeries struct {
-	timestamps []int64
-	values     []float64
-	labelSet   []string
+	startTimestamps []int64
+	timestamps      []int64
+	values          []float64
+	labelSet        []string
 }
 
 func (s mockSeries) Labels() labels.Labels {
@@ -483,15 +484,19 @@ func (s mockSeries) Labels() labels.Labels {
 }
 
 func (s mockSeries) Iterator(chunkenc.Iterator) chunkenc.Iterator {
-	return chunkenc.MockSeriesIterator(s.timestamps, s.values)
+	return chunkenc.MockSeriesIterator(s.startTimestamps, s.timestamps, s.values)
 }
 
-// MockSeries returns a series with custom timestamps, values and labelSet.
-func MockSeries(timestamps []int64, values []float64, labelSet []string) Series {
+// MockSeries returns a series with custom start timestamp, timestamps, values,
+// and labelSet.
+// Start timestamps is optional, pass nil or empty slice to indicate no start
+// timestamps.
+func MockSeries(startTimestamps, timestamps []int64, values []float64, labelSet []string) Series {
 	return mockSeries{
-		timestamps: timestamps,
-		values:     values,
-		labelSet:   labelSet,
+		startTimestamps: startTimestamps,
+		timestamps:      timestamps,
+		values:          values,
+		labelSet:        labelSet,
 	}
 }
 
diff --git a/storage/interface_test.go b/storage/interface_test.go
index d28e5177e3..3ea4b757e7 100644
--- a/storage/interface_test.go
+++ b/storage/interface_test.go
@@ -23,7 +23,7 @@ import (
 )
 
 func TestMockSeries(t *testing.T) {
-	s := storage.MockSeries([]int64{1, 2, 3}, []float64{1, 2, 3}, []string{"__name__", "foo"})
+	s := storage.MockSeries(nil, []int64{1, 2, 3}, []float64{1, 2, 3}, []string{"__name__", "foo"})
 	it := s.Iterator(nil)
 	ts := []int64{}
 	vs := []float64{}
@@ -35,3 +35,20 @@ func TestMockSeries(t *testing.T) {
 	require.Equal(t, []int64{1, 2, 3}, ts)
 	require.Equal(t, []float64{1, 2, 3}, vs)
 }
+
+func TestMockSeriesWithST(t *testing.T) {
+	s := storage.MockSeries([]int64{0, 1, 2}, []int64{1, 2, 3}, []float64{1, 2, 3}, []string{"__name__", "foo"})
+	it := s.Iterator(nil)
+	ts := []int64{}
+	vs := []float64{}
+	st := []int64{}
+	for it.Next() == chunkenc.ValFloat {
+		t, v := it.At()
+		ts = append(ts, t)
+		vs = append(vs, v)
+		st = append(st, it.AtST())
+	}
+	require.Equal(t, []int64{1, 2, 3}, ts)
+	require.Equal(t, []float64{1, 2, 3}, vs)
+	require.Equal(t, []int64{0, 1, 2}, st)
+}
diff --git a/storage/merge.go b/storage/merge.go
index 12d6d3ac0d..76bf0994e0 100644
--- a/storage/merge.go
+++ b/storage/merge.go
@@ -599,6 +599,13 @@ func (c *chainSampleIterator) AtT() int64 {
 	return c.curr.AtT()
 }
 
+func (c *chainSampleIterator) AtST() int64 {
+	if c.curr == nil {
+		panic("chainSampleIterator.AtST called before first .Next or after .Next returned false.")
+	}
+	return c.curr.AtST()
+}
+
 func (c *chainSampleIterator) Next() chunkenc.ValueType {
 	var (
 		currT           int64
diff --git a/storage/merge_test.go b/storage/merge_test.go
index 6e2daaeb3a..0060950d6f 100644
--- a/storage/merge_test.go
+++ b/storage/merge_test.go
@@ -1716,6 +1716,10 @@ func (errIterator) AtT() int64 {
 	return 0
 }
 
+func (errIterator) AtST() int64 {
+	return 0
+}
+
 func (e errIterator) Err() error {
 	return e.err
 }
diff --git a/storage/remote/codec.go b/storage/remote/codec.go
index 9f0fb7d92a..c689a51164 100644
--- a/storage/remote/codec.go
+++ b/storage/remote/codec.go
@@ -564,6 +564,12 @@ func (c *concreteSeriesIterator) AtT() int64 {
 	return c.series.floats[c.floatsCur].Timestamp
 }
 
+// TODO(krajorama): implement AtST. Maybe. concreteSeriesIterator is used
+// for turning query results into an iterable, but query results do not have ST.
+func (*concreteSeriesIterator) AtST() int64 {
+	return 0
+}
+
 const noTS = int64(math.MaxInt64)
 
 // Next implements chunkenc.Iterator.
@@ -832,6 +838,11 @@ func (it *chunkedSeriesIterator) AtT() int64 {
 	return it.cur.AtT()
 }
 
+// TODO(krajorama): test AtST once we have a chunk format that provides ST.
+func (it *chunkedSeriesIterator) AtST() int64 {
+	return it.cur.AtST()
+}
+
 func (it *chunkedSeriesIterator) Err() error {
 	return it.err
 }
diff --git a/storage/remote/codec_test.go b/storage/remote/codec_test.go
index e6e7813c7b..5da8c8176c 100644
--- a/storage/remote/codec_test.go
+++ b/storage/remote/codec_test.go
@@ -1146,7 +1146,7 @@ func buildTestChunks(t *testing.T) []prompb.Chunk {
 		minTimeMs := time
 
 		for j := range numSamplesPerTestChunk {
-			a.Append(time, float64(i+j))
+			a.Append(0, time, float64(i+j))
 			time += int64(1000)
 		}
 
diff --git a/storage/series.go b/storage/series.go
index 7e130d494d..d114438078 100644
--- a/storage/series.go
+++ b/storage/series.go
@@ -138,6 +138,11 @@ func (it *listSeriesIterator) AtT() int64 {
 	return s.T()
 }
 
+func (it *listSeriesIterator) AtST() int64 {
+	s := it.samples.Get(it.idx)
+	return s.ST()
+}
+
 func (it *listSeriesIterator) Next() chunkenc.ValueType {
 	it.idx++
 	if it.idx >= it.samples.Len() {
@@ -355,18 +360,20 @@ func (s *seriesToChunkEncoder) Iterator(it chunks.Iterator) chunks.Iterator {
 		lastType = typ
 
 		var (
-			t  int64
-			v  float64
-			h  *histogram.Histogram
-			fh *histogram.FloatHistogram
+			st, t int64
+			v     float64
+			h     *histogram.Histogram
+			fh    *histogram.FloatHistogram
 		)
 		switch typ {
 		case chunkenc.ValFloat:
 			t, v = seriesIter.At()
-			app.Append(t, v)
+			st = seriesIter.AtST()
+			app.Append(st, t, v)
 		case chunkenc.ValHistogram:
 			t, h = seriesIter.AtHistogram(nil)
-			newChk, recoded, app, err = app.AppendHistogram(nil, t, h, false)
+			st = seriesIter.AtST()
+			newChk, recoded, app, err = app.AppendHistogram(nil, st, t, h, false)
 			if err != nil {
 				return errChunksIterator{err: err}
 			}
@@ -381,7 +388,8 @@ func (s *seriesToChunkEncoder) Iterator(it chunks.Iterator) chunks.Iterator {
 			}
 		case chunkenc.ValFloatHistogram:
 			t, fh = seriesIter.AtFloatHistogram(nil)
-			newChk, recoded, app, err = app.AppendFloatHistogram(nil, t, fh, false)
+			st = seriesIter.AtST()
+			newChk, recoded, app, err = app.AppendFloatHistogram(nil, st, t, fh, false)
 			if err != nil {
 				return errChunksIterator{err: err}
 			}
diff --git a/tsdb/chunkenc/chunk.go b/tsdb/chunkenc/chunk.go
index fed28c5701..d5e028e681 100644
--- a/tsdb/chunkenc/chunk.go
+++ b/tsdb/chunkenc/chunk.go
@@ -99,9 +99,9 @@ type Iterable interface {
 	Iterator(Iterator) Iterator
 }
 
-// Appender adds sample pairs to a chunk.
+// Appender adds sample with start timestamp, timestamp, and value to a chunk.
 type Appender interface {
-	Append(int64, float64)
+	Append(st, t int64, v float64)
 
 	// AppendHistogram and AppendFloatHistogram append a histogram sample to a histogram or float histogram chunk.
 	// Appending a histogram may require creating a completely new chunk or recoding (changing) the current chunk.
@@ -114,8 +114,8 @@ type Appender interface {
 	// The returned bool isRecoded can be used to distinguish between the new Chunk c being a completely new Chunk
 	// or the current Chunk recoded to a new Chunk.
 	// The Appender app that can be used for the next append is always returned.
-	AppendHistogram(prev *HistogramAppender, t int64, h *histogram.Histogram, appendOnly bool) (c Chunk, isRecoded bool, app Appender, err error)
-	AppendFloatHistogram(prev *FloatHistogramAppender, t int64, h *histogram.FloatHistogram, appendOnly bool) (c Chunk, isRecoded bool, app Appender, err error)
+	AppendHistogram(prev *HistogramAppender, st, t int64, h *histogram.Histogram, appendOnly bool) (c Chunk, isRecoded bool, app Appender, err error)
+	AppendFloatHistogram(prev *FloatHistogramAppender, st, t int64, h *histogram.FloatHistogram, appendOnly bool) (c Chunk, isRecoded bool, app Appender, err error)
 }
 
 // Iterator is a simple iterator that can only get the next value.
@@ -151,6 +151,10 @@ type Iterator interface {
 	// AtT returns the current timestamp.
 	// Before the iterator has advanced, the behaviour is unspecified.
 	AtT() int64
+	// AtST returns the current start timestamp.
+	// Return 0 if the start timestamp is not implemented or not set.
+	// Before the iterator has advanced, the behaviour is unspecified.
+	AtST() int64
 	// Err returns the current error. It should be used only after the
 	// iterator is exhausted, i.e. `Next` or `Seek` have returned ValNone.
 	Err() error
@@ -208,25 +212,30 @@ func (v ValueType) NewChunk() (Chunk, error) {
 	}
 }
 
-// MockSeriesIterator returns an iterator for a mock series with custom timeStamps and values.
-func MockSeriesIterator(timestamps []int64, values []float64) Iterator {
+// MockSeriesIterator returns an iterator for a mock series with custom
+// start timestamp, timestamps, and values.
+// Start timestamps is optional, pass nil or empty slice to indicate no start
+// timestamps.
+func MockSeriesIterator(startTimestamps, timestamps []int64, values []float64) Iterator {
 	return &mockSeriesIterator{
-		timeStamps: timestamps,
-		values:     values,
-		currIndex:  -1,
+		startTimestamps: startTimestamps,
+		timestamps:      timestamps,
+		values:          values,
+		currIndex:       -1,
 	}
 }
 
 type mockSeriesIterator struct {
-	timeStamps []int64
-	values     []float64
-	currIndex  int
+	timestamps      []int64
+	startTimestamps []int64
+	values          []float64
+	currIndex       int
 }
 
 func (*mockSeriesIterator) Seek(int64) ValueType { return ValNone }
 
 func (it *mockSeriesIterator) At() (int64, float64) {
-	return it.timeStamps[it.currIndex], it.values[it.currIndex]
+	return it.timestamps[it.currIndex], it.values[it.currIndex]
 }
 
 func (*mockSeriesIterator) AtHistogram(*histogram.Histogram) (int64, *histogram.Histogram) {
@@ -238,11 +247,18 @@ func (*mockSeriesIterator) AtFloatHistogram(*histogram.FloatHistogram) (int64, *
 }
 
 func (it *mockSeriesIterator) AtT() int64 {
-	return it.timeStamps[it.currIndex]
+	return it.timestamps[it.currIndex]
+}
+
+func (it *mockSeriesIterator) AtST() int64 {
+	if len(it.startTimestamps) == 0 {
+		return 0
+	}
+	return it.startTimestamps[it.currIndex]
 }
 
 func (it *mockSeriesIterator) Next() ValueType {
-	if it.currIndex < len(it.timeStamps)-1 {
+	if it.currIndex < len(it.timestamps)-1 {
 		it.currIndex++
 		return ValFloat
 	}
@@ -268,8 +284,9 @@ func (nopIterator) AtHistogram(*histogram.Histogram) (int64, *histogram.Histogra
 func (nopIterator) AtFloatHistogram(*histogram.FloatHistogram) (int64, *histogram.FloatHistogram) {
 	return math.MinInt64, nil
 }
-func (nopIterator) AtT() int64 { return math.MinInt64 }
-func (nopIterator) Err() error { return nil }
+func (nopIterator) AtT() int64  { return math.MinInt64 }
+func (nopIterator) AtST() int64 { return 0 }
+func (nopIterator) Err() error  { return nil }
 
 // Pool is used to create and reuse chunk references to avoid allocations.
 type Pool interface {
diff --git a/tsdb/chunkenc/chunk_test.go b/tsdb/chunkenc/chunk_test.go
index d2d0e4c053..41bb23ddd1 100644
--- a/tsdb/chunkenc/chunk_test.go
+++ b/tsdb/chunkenc/chunk_test.go
@@ -65,7 +65,7 @@ func testChunk(t *testing.T, c Chunk) {
 			require.NoError(t, err)
 		}
 
-		app.Append(ts, v)
+		app.Append(0, ts, v)
 		exp = append(exp, pair{t: ts, v: v})
 	}
 
@@ -226,7 +226,7 @@ func benchmarkIterator(b *testing.B, newChunk func() Chunk) {
 			if j > 250 {
 				break
 			}
-			a.Append(p.t, p.v)
+			a.Append(0, p.t, p.v)
 			j++
 		}
 	}
@@ -303,7 +303,7 @@ func benchmarkAppender(b *testing.B, deltas func() (int64, float64), newChunk fu
 			b.Fatalf("get appender: %s", err)
 		}
 		for _, p := range exp {
-			a.Append(p.t, p.v)
+			a.Append(0, p.t, p.v)
 		}
 	}
 }
diff --git a/tsdb/chunkenc/float_histogram.go b/tsdb/chunkenc/float_histogram.go
index 797bc596b5..6af2fa68e2 100644
--- a/tsdb/chunkenc/float_histogram.go
+++ b/tsdb/chunkenc/float_histogram.go
@@ -195,7 +195,7 @@ func (a *FloatHistogramAppender) NumSamples() int {
 
 // Append implements Appender. This implementation panics because normal float
 // samples must never be appended to a histogram chunk.
-func (*FloatHistogramAppender) Append(int64, float64) {
+func (*FloatHistogramAppender) Append(int64, int64, float64) {
 	panic("appended a float sample to a histogram chunk")
 }
 
@@ -682,11 +682,11 @@ func (*FloatHistogramAppender) recodeHistogram(
 	}
 }
 
-func (*FloatHistogramAppender) AppendHistogram(*HistogramAppender, int64, *histogram.Histogram, bool) (Chunk, bool, Appender, error) {
+func (*FloatHistogramAppender) AppendHistogram(*HistogramAppender, int64, int64, *histogram.Histogram, bool) (Chunk, bool, Appender, error) {
 	panic("appended a histogram sample to a float histogram chunk")
 }
 
-func (a *FloatHistogramAppender) AppendFloatHistogram(prev *FloatHistogramAppender, t int64, h *histogram.FloatHistogram, appendOnly bool) (Chunk, bool, Appender, error) {
+func (a *FloatHistogramAppender) AppendFloatHistogram(prev *FloatHistogramAppender, _, t int64, h *histogram.FloatHistogram, appendOnly bool) (Chunk, bool, Appender, error) {
 	if a.NumSamples() == 0 {
 		a.appendFloatHistogram(t, h)
 		if h.CounterResetHint == histogram.GaugeType {
@@ -938,6 +938,10 @@ func (it *floatHistogramIterator) AtT() int64 {
 	return it.t
 }
 
+func (*floatHistogramIterator) AtST() int64 {
+	return 0
+}
+
 func (it *floatHistogramIterator) Err() error {
 	return it.err
 }
diff --git a/tsdb/chunkenc/float_histogram_test.go b/tsdb/chunkenc/float_histogram_test.go
index f27de97516..cbeb3171ce 100644
--- a/tsdb/chunkenc/float_histogram_test.go
+++ b/tsdb/chunkenc/float_histogram_test.go
@@ -63,7 +63,7 @@ func TestFirstFloatHistogramExplicitCounterReset(t *testing.T) {
 			chk := NewFloatHistogramChunk()
 			app, err := chk.Appender()
 			require.NoError(t, err)
-			newChk, recoded, newApp, err := app.AppendFloatHistogram(nil, 0, h, false)
+			newChk, recoded, newApp, err := app.AppendFloatHistogram(nil, 0, 0, h, false)
 			require.NoError(t, err)
 			require.Nil(t, newChk)
 			require.False(t, recoded)
@@ -101,7 +101,7 @@ func TestFloatHistogramChunkSameBuckets(t *testing.T) {
 		},
 		NegativeBuckets: []int64{2, 1, -1, -1}, // counts: 2, 3, 2, 1 (total 8)
 	}
-	chk, _, app, err := app.AppendFloatHistogram(nil, ts, h.ToFloat(nil), false)
+	chk, _, app, err := app.AppendFloatHistogram(nil, 0, ts, h.ToFloat(nil), false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	exp = append(exp, floatResult{t: ts, h: h.ToFloat(nil)})
@@ -115,7 +115,7 @@ func TestFloatHistogramChunkSameBuckets(t *testing.T) {
 	h.Sum = 24.4
 	h.PositiveBuckets = []int64{5, -2, 1, -2} // counts: 5, 3, 4, 2 (total 14)
 	h.NegativeBuckets = []int64{4, -1, 1, -1} // counts: 4, 3, 4, 4 (total 15)
-	chk, _, _, err = app.AppendFloatHistogram(nil, ts, h.ToFloat(nil), false)
+	chk, _, _, err = app.AppendFloatHistogram(nil, 0, ts, h.ToFloat(nil), false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	expH := h.ToFloat(nil)
@@ -134,7 +134,7 @@ func TestFloatHistogramChunkSameBuckets(t *testing.T) {
 	h.Sum = 24.4
 	h.PositiveBuckets = []int64{6, 1, -3, 6} // counts: 6, 7, 4, 10 (total 27)
 	h.NegativeBuckets = []int64{5, 1, -2, 3} // counts: 5, 6, 4, 7 (total 22)
-	chk, _, _, err = app.AppendFloatHistogram(nil, ts, h.ToFloat(nil), false)
+	chk, _, _, err = app.AppendFloatHistogram(nil, 0, ts, h.ToFloat(nil), false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	expH = h.ToFloat(nil)
@@ -224,7 +224,7 @@ func TestFloatHistogramChunkBucketChanges(t *testing.T) {
 		NegativeBuckets: []int64{1},
 	}
 
-	chk, _, app, err := app.AppendFloatHistogram(nil, ts1, h1.ToFloat(nil), false)
+	chk, _, app, err := app.AppendFloatHistogram(nil, 0, ts1, h1.ToFloat(nil), false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	require.Equal(t, 1, c.NumSamples())
@@ -260,7 +260,7 @@ func TestFloatHistogramChunkBucketChanges(t *testing.T) {
 	require.True(t, ok) // Only new buckets came in.
 	require.False(t, cr)
 	c, app = hApp.recode(posInterjections, negInterjections, h2.PositiveSpans, h2.NegativeSpans)
-	chk, _, _, err = app.AppendFloatHistogram(nil, ts2, h2.ToFloat(nil), false)
+	chk, _, _, err = app.AppendFloatHistogram(nil, 0, ts2, h2.ToFloat(nil), false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	require.Equal(t, 2, c.NumSamples())
@@ -330,7 +330,7 @@ func TestFloatHistogramChunkAppendable(t *testing.T) {
 
 		ts := int64(1234567890)
 
-		chk, _, app, err := app.AppendFloatHistogram(nil, ts, h.Copy(), false)
+		chk, _, app, err := app.AppendFloatHistogram(nil, 0, ts, h.Copy(), false)
 		require.NoError(t, err)
 		require.Nil(t, chk)
 		require.Equal(t, 1, c.NumSamples())
@@ -557,7 +557,7 @@ func TestFloatHistogramChunkAppendable(t *testing.T) {
 		nextChunk := NewFloatHistogramChunk()
 		app, err := nextChunk.Appender()
 		require.NoError(t, err)
-		newChunk, recoded, newApp, err := app.AppendFloatHistogram(hApp, ts+1, h2, false)
+		newChunk, recoded, newApp, err := app.AppendFloatHistogram(hApp, 0, ts+1, h2, false)
 		require.NoError(t, err)
 		require.Nil(t, newChunk)
 		require.False(t, recoded)
@@ -575,7 +575,7 @@ func TestFloatHistogramChunkAppendable(t *testing.T) {
 		nextChunk := NewFloatHistogramChunk()
 		app, err := nextChunk.Appender()
 		require.NoError(t, err)
-		newChunk, recoded, newApp, err := app.AppendFloatHistogram(hApp, ts+1, h2, false)
+		newChunk, recoded, newApp, err := app.AppendFloatHistogram(hApp, 0, ts+1, h2, false)
 		require.NoError(t, err)
 		require.Nil(t, newChunk)
 		require.False(t, recoded)
@@ -602,7 +602,7 @@ func TestFloatHistogramChunkAppendable(t *testing.T) {
 		nextChunk := NewFloatHistogramChunk()
 		app, err := nextChunk.Appender()
 		require.NoError(t, err)
-		newChunk, recoded, newApp, err := app.AppendFloatHistogram(hApp, ts+1, h2, false)
+		newChunk, recoded, newApp, err := app.AppendFloatHistogram(hApp, 0, ts+1, h2, false)
 		require.NoError(t, err)
 		require.Nil(t, newChunk)
 		require.False(t, recoded)
@@ -717,7 +717,7 @@ func TestFloatHistogramChunkAppendable(t *testing.T) {
 
 func assertNewFloatHistogramChunkOnAppend(t *testing.T, oldChunk Chunk, hApp *FloatHistogramAppender, ts int64, h *histogram.FloatHistogram, expectHeader CounterResetHeader, expectHint histogram.CounterResetHint) {
 	oldChunkBytes := oldChunk.Bytes()
-	newChunk, recoded, newAppender, err := hApp.AppendFloatHistogram(nil, ts, h, false)
+	newChunk, recoded, newAppender, err := hApp.AppendFloatHistogram(nil, 0, ts, h, false)
 	require.Equal(t, oldChunkBytes, oldChunk.Bytes()) // Sanity check that previous chunk is untouched.
 	require.NoError(t, err)
 	require.NotNil(t, newChunk)
@@ -732,7 +732,7 @@ func assertNewFloatHistogramChunkOnAppend(t *testing.T, oldChunk Chunk, hApp *Fl
 
 func assertNoNewFloatHistogramChunkOnAppend(t *testing.T, oldChunk Chunk, hApp *FloatHistogramAppender, ts int64, h *histogram.FloatHistogram, expectHeader CounterResetHeader) {
 	oldChunkBytes := oldChunk.Bytes()
-	newChunk, recoded, newAppender, err := hApp.AppendFloatHistogram(nil, ts, h, false)
+	newChunk, recoded, newAppender, err := hApp.AppendFloatHistogram(nil, 0, ts, h, false)
 	require.Greater(t, len(oldChunk.Bytes()), len(oldChunkBytes)) // Check that current chunk is bigger than previously.
 	require.NoError(t, err)
 	require.Nil(t, newChunk)
@@ -745,7 +745,7 @@ func assertNoNewFloatHistogramChunkOnAppend(t *testing.T, oldChunk Chunk, hApp *
 
 func assertRecodedFloatHistogramChunkOnAppend(t *testing.T, prevChunk Chunk, hApp *FloatHistogramAppender, ts int64, h *histogram.FloatHistogram, expectHeader CounterResetHeader) {
 	prevChunkBytes := prevChunk.Bytes()
-	newChunk, recoded, newAppender, err := hApp.AppendFloatHistogram(nil, ts, h, false)
+	newChunk, recoded, newAppender, err := hApp.AppendFloatHistogram(nil, 0, ts, h, false)
 	require.Equal(t, prevChunkBytes, prevChunk.Bytes()) // Sanity check that previous chunk is untouched. This may change in the future if we implement in-place recoding.
 	require.NoError(t, err)
 	require.NotNil(t, newChunk)
@@ -959,7 +959,7 @@ func TestFloatHistogramChunkAppendableWithEmptySpan(t *testing.T) {
 			require.NoError(t, err)
 			require.Equal(t, 0, c.NumSamples())
 
-			_, _, _, err = app.AppendFloatHistogram(nil, 1, tc.h1, true)
+			_, _, _, err = app.AppendFloatHistogram(nil, 0, 1, tc.h1, true)
 			require.NoError(t, err)
 			require.Equal(t, 1, c.NumSamples())
 			hApp, _ := app.(*FloatHistogramAppender)
@@ -1019,7 +1019,7 @@ func TestFloatHistogramChunkAppendableGauge(t *testing.T) {
 
 		ts := int64(1234567890)
 
-		chk, _, app, err := app.AppendFloatHistogram(nil, ts, h.Copy(), false)
+		chk, _, app, err := app.AppendFloatHistogram(nil, 0, ts, h.Copy(), false)
 		require.NoError(t, err)
 		require.Nil(t, chk)
 		require.Equal(t, 1, c.NumSamples())
@@ -1259,7 +1259,7 @@ func TestFloatHistogramAppendOnlyErrors(t *testing.T) {
 
 		h := tsdbutil.GenerateTestFloatHistogram(0)
 		var isRecoded bool
-		c, isRecoded, app, err = app.AppendFloatHistogram(nil, 1, h, true)
+		c, isRecoded, app, err = app.AppendFloatHistogram(nil, 0, 1, h, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.NoError(t, err)
@@ -1267,7 +1267,7 @@ func TestFloatHistogramAppendOnlyErrors(t *testing.T) {
 		// Add erroring histogram.
 		h2 := h.Copy()
 		h2.Schema++
-		c, isRecoded, _, err = app.AppendFloatHistogram(nil, 2, h2, true)
+		c, isRecoded, _, err = app.AppendFloatHistogram(nil, 0, 2, h2, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.EqualError(t, err, "float histogram schema change")
@@ -1281,7 +1281,7 @@ func TestFloatHistogramAppendOnlyErrors(t *testing.T) {
 
 		h := tsdbutil.GenerateTestFloatHistogram(0)
 		var isRecoded bool
-		c, isRecoded, app, err = app.AppendFloatHistogram(nil, 1, h, true)
+		c, isRecoded, app, err = app.AppendFloatHistogram(nil, 0, 1, h, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.NoError(t, err)
@@ -1289,7 +1289,7 @@ func TestFloatHistogramAppendOnlyErrors(t *testing.T) {
 		// Add erroring histogram.
 		h2 := h.Copy()
 		h2.CounterResetHint = histogram.CounterReset
-		c, isRecoded, _, err = app.AppendFloatHistogram(nil, 2, h2, true)
+		c, isRecoded, _, err = app.AppendFloatHistogram(nil, 0, 2, h2, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.EqualError(t, err, "float histogram counter reset")
@@ -1303,7 +1303,7 @@ func TestFloatHistogramAppendOnlyErrors(t *testing.T) {
 
 		h := tsdbutil.GenerateTestCustomBucketsFloatHistogram(0)
 		var isRecoded bool
-		c, isRecoded, app, err = app.AppendFloatHistogram(nil, 1, h, true)
+		c, isRecoded, app, err = app.AppendFloatHistogram(nil, 0, 1, h, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.NoError(t, err)
@@ -1311,7 +1311,7 @@ func TestFloatHistogramAppendOnlyErrors(t *testing.T) {
 		// Add erroring histogram.
 		h2 := h.Copy()
 		h2.CustomValues = []float64{0, 1, 2, 3, 4, 5, 6, 7}
-		c, isRecoded, _, err = app.AppendFloatHistogram(nil, 2, h2, true)
+		c, isRecoded, _, err = app.AppendFloatHistogram(nil, 0, 2, h2, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.EqualError(t, err, "float histogram counter reset")
@@ -1344,10 +1344,10 @@ func TestFloatHistogramUniqueSpansAfterNext(t *testing.T) {
 	app, err := c.Appender()
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendFloatHistogram(nil, 0, h1, false)
+	_, _, _, err = app.AppendFloatHistogram(nil, 0, 0, h1, false)
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendFloatHistogram(nil, 1, h2, false)
+	_, _, _, err = app.AppendFloatHistogram(nil, 0, 1, h2, false)
 	require.NoError(t, err)
 
 	// Create an iterator and advance to the first histogram.
@@ -1390,10 +1390,10 @@ func TestFloatHistogramUniqueCustomValuesAfterNext(t *testing.T) {
 	app, err := c.Appender()
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendFloatHistogram(nil, 0, h1, false)
+	_, _, _, err = app.AppendFloatHistogram(nil, 0, 0, h1, false)
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendFloatHistogram(nil, 1, h2, false)
+	_, _, _, err = app.AppendFloatHistogram(nil, 0, 1, h2, false)
 	require.NoError(t, err)
 
 	// Create an iterator and advance to the first histogram.
@@ -1435,7 +1435,7 @@ func TestFloatHistogramEmptyBucketsWithGaps(t *testing.T) {
 	c := NewFloatHistogramChunk()
 	app, err := c.Appender()
 	require.NoError(t, err)
-	_, _, _, err = app.AppendFloatHistogram(nil, 1, h1, false)
+	_, _, _, err = app.AppendFloatHistogram(nil, 0, 1, h1, false)
 	require.NoError(t, err)
 
 	h2 := &histogram.FloatHistogram{
@@ -1448,7 +1448,7 @@ func TestFloatHistogramEmptyBucketsWithGaps(t *testing.T) {
 	}
 	require.NoError(t, h2.Validate())
 
-	newC, recoded, _, err := app.AppendFloatHistogram(nil, 2, h2, false)
+	newC, recoded, _, err := app.AppendFloatHistogram(nil, 0, 2, h2, false)
 	require.NoError(t, err)
 	require.True(t, recoded)
 	require.NotNil(t, newC)
@@ -1483,7 +1483,7 @@ func TestFloatHistogramIteratorFailIfSchemaInValid(t *testing.T) {
 			app, err := c.Appender()
 			require.NoError(t, err)
 
-			_, _, _, err = app.AppendFloatHistogram(nil, 1, h, false)
+			_, _, _, err = app.AppendFloatHistogram(nil, 0, 1, h, false)
 			require.NoError(t, err)
 
 			it := c.Iterator(nil)
@@ -1512,7 +1512,7 @@ func TestFloatHistogramIteratorReduceSchema(t *testing.T) {
 			app, err := c.Appender()
 			require.NoError(t, err)
 
-			_, _, _, err = app.AppendFloatHistogram(nil, 1, h, false)
+			_, _, _, err = app.AppendFloatHistogram(nil, 0, 1, h, false)
 			require.NoError(t, err)
 
 			it := c.Iterator(nil)
diff --git a/tsdb/chunkenc/histogram.go b/tsdb/chunkenc/histogram.go
index e05c49c81d..4e77f387d3 100644
--- a/tsdb/chunkenc/histogram.go
+++ b/tsdb/chunkenc/histogram.go
@@ -219,7 +219,7 @@ func (a *HistogramAppender) NumSamples() int {
 
 // Append implements Appender. This implementation panics because normal float
 // samples must never be appended to a histogram chunk.
-func (*HistogramAppender) Append(int64, float64) {
+func (*HistogramAppender) Append(int64, int64, float64) {
 	panic("appended a float sample to a histogram chunk")
 }
 
@@ -734,11 +734,11 @@ func (a *HistogramAppender) writeSumDelta(v float64) {
 	xorWrite(a.b, v, a.sum, &a.leading, &a.trailing)
 }
 
-func (*HistogramAppender) AppendFloatHistogram(*FloatHistogramAppender, int64, *histogram.FloatHistogram, bool) (Chunk, bool, Appender, error) {
+func (*HistogramAppender) AppendFloatHistogram(*FloatHistogramAppender, int64, int64, *histogram.FloatHistogram, bool) (Chunk, bool, Appender, error) {
 	panic("appended a float histogram sample to a histogram chunk")
 }
 
-func (a *HistogramAppender) AppendHistogram(prev *HistogramAppender, t int64, h *histogram.Histogram, appendOnly bool) (Chunk, bool, Appender, error) {
+func (a *HistogramAppender) AppendHistogram(prev *HistogramAppender, _, t int64, h *histogram.Histogram, appendOnly bool) (Chunk, bool, Appender, error) {
 	if a.NumSamples() == 0 {
 		a.appendHistogram(t, h)
 		if h.CounterResetHint == histogram.GaugeType {
@@ -1075,6 +1075,10 @@ func (it *histogramIterator) AtT() int64 {
 	return it.t
 }
 
+func (*histogramIterator) AtST() int64 {
+	return 0
+}
+
 func (it *histogramIterator) Err() error {
 	return it.err
 }
diff --git a/tsdb/chunkenc/histogram_test.go b/tsdb/chunkenc/histogram_test.go
index 38bbd58465..6ac8500e64 100644
--- a/tsdb/chunkenc/histogram_test.go
+++ b/tsdb/chunkenc/histogram_test.go
@@ -64,7 +64,7 @@ func TestFirstHistogramExplicitCounterReset(t *testing.T) {
 			chk := NewHistogramChunk()
 			app, err := chk.Appender()
 			require.NoError(t, err)
-			newChk, recoded, newApp, err := app.AppendHistogram(nil, 0, h, false)
+			newChk, recoded, newApp, err := app.AppendHistogram(nil, 0, 0, h, false)
 			require.NoError(t, err)
 			require.Nil(t, newChk)
 			require.False(t, recoded)
@@ -102,7 +102,7 @@ func TestHistogramChunkSameBuckets(t *testing.T) {
 		},
 		NegativeBuckets: []int64{2, 1, -1, -1}, // counts: 2, 3, 2, 1 (total 8)
 	}
-	chk, _, app, err := app.AppendHistogram(nil, ts, h, false)
+	chk, _, app, err := app.AppendHistogram(nil, 0, ts, h, false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	exp = append(exp, result{t: ts, h: h, fh: h.ToFloat(nil)})
@@ -116,7 +116,7 @@ func TestHistogramChunkSameBuckets(t *testing.T) {
 	h.Sum = 24.4
 	h.PositiveBuckets = []int64{5, -2, 1, -2} // counts: 5, 3, 4, 2 (total 14)
 	h.NegativeBuckets = []int64{4, -1, 1, -1} // counts: 4, 3, 4, 4 (total 15)
-	chk, _, _, err = app.AppendHistogram(nil, ts, h, false)
+	chk, _, _, err = app.AppendHistogram(nil, 0, ts, h, false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	hExp := h.Copy()
@@ -135,7 +135,7 @@ func TestHistogramChunkSameBuckets(t *testing.T) {
 	h.Sum = 24.4
 	h.PositiveBuckets = []int64{6, 1, -3, 6} // counts: 6, 7, 4, 10 (total 27)
 	h.NegativeBuckets = []int64{5, 1, -2, 3} // counts: 5, 6, 4, 7 (total 22)
-	chk, _, _, err = app.AppendHistogram(nil, ts, h, false)
+	chk, _, _, err = app.AppendHistogram(nil, 0, ts, h, false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	hExp = h.Copy()
@@ -235,7 +235,7 @@ func TestHistogramChunkBucketChanges(t *testing.T) {
 		NegativeBuckets: []int64{1},
 	}
 
-	chk, _, app, err := app.AppendHistogram(nil, ts1, h1, false)
+	chk, _, app, err := app.AppendHistogram(nil, 0, ts1, h1, false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 	require.Equal(t, 1, c.NumSamples())
@@ -271,7 +271,7 @@ func TestHistogramChunkBucketChanges(t *testing.T) {
 	require.True(t, ok) // Only new buckets came in.
 	require.Equal(t, NotCounterReset, cr)
 	c, app = hApp.recode(posInterjections, negInterjections, h2.PositiveSpans, h2.NegativeSpans)
-	chk, _, _, err = app.AppendHistogram(nil, ts2, h2, false)
+	chk, _, _, err = app.AppendHistogram(nil, 0, ts2, h2, false)
 	require.NoError(t, err)
 	require.Nil(t, chk)
 
@@ -344,7 +344,7 @@ func TestHistogramChunkAppendable(t *testing.T) {
 
 		ts := int64(1234567890)
 
-		chk, _, app, err := app.AppendHistogram(nil, ts, h.Copy(), false)
+		chk, _, app, err := app.AppendHistogram(nil, 0, ts, h.Copy(), false)
 		require.NoError(t, err)
 		require.Nil(t, chk)
 		require.Equal(t, 1, c.NumSamples())
@@ -581,7 +581,7 @@ func TestHistogramChunkAppendable(t *testing.T) {
 		nextChunk := NewHistogramChunk()
 		app, err := nextChunk.Appender()
 		require.NoError(t, err)
-		newChunk, recoded, newApp, err := app.AppendHistogram(hApp, ts+1, h2, false)
+		newChunk, recoded, newApp, err := app.AppendHistogram(hApp, 0, ts+1, h2, false)
 		require.NoError(t, err)
 		require.Nil(t, newChunk)
 		require.False(t, recoded)
@@ -599,7 +599,7 @@ func TestHistogramChunkAppendable(t *testing.T) {
 		nextChunk := NewHistogramChunk()
 		app, err := nextChunk.Appender()
 		require.NoError(t, err)
-		newChunk, recoded, newApp, err := app.AppendHistogram(hApp, ts+1, h2, false)
+		newChunk, recoded, newApp, err := app.AppendHistogram(hApp, 0, ts+1, h2, false)
 		require.NoError(t, err)
 		require.Nil(t, newChunk)
 		require.False(t, recoded)
@@ -629,7 +629,7 @@ func TestHistogramChunkAppendable(t *testing.T) {
 		nextChunk := NewHistogramChunk()
 		app, err := nextChunk.Appender()
 		require.NoError(t, err)
-		newChunk, recoded, newApp, err := app.AppendHistogram(hApp, ts+1, h2, false)
+		newChunk, recoded, newApp, err := app.AppendHistogram(hApp, 0, ts+1, h2, false)
 		require.NoError(t, err)
 		require.Nil(t, newChunk)
 		require.False(t, recoded)
@@ -776,7 +776,7 @@ func TestHistogramChunkAppendable(t *testing.T) {
 
 func assertNewHistogramChunkOnAppend(t *testing.T, oldChunk Chunk, hApp *HistogramAppender, ts int64, h *histogram.Histogram, expectHeader CounterResetHeader, expectHint histogram.CounterResetHint) {
 	oldChunkBytes := oldChunk.Bytes()
-	newChunk, recoded, newAppender, err := hApp.AppendHistogram(nil, ts, h, false)
+	newChunk, recoded, newAppender, err := hApp.AppendHistogram(nil, 0, ts, h, false)
 	require.Equal(t, oldChunkBytes, oldChunk.Bytes()) // Sanity check that previous chunk is untouched.
 	require.NoError(t, err)
 	require.NotNil(t, newChunk)
@@ -791,7 +791,7 @@ func assertNewHistogramChunkOnAppend(t *testing.T, oldChunk Chunk, hApp *Histogr
 
 func assertNoNewHistogramChunkOnAppend(t *testing.T, currChunk Chunk, hApp *HistogramAppender, ts int64, h *histogram.Histogram, expectHeader CounterResetHeader) {
 	prevChunkBytes := currChunk.Bytes()
-	newChunk, recoded, newAppender, err := hApp.AppendHistogram(nil, ts, h, false)
+	newChunk, recoded, newAppender, err := hApp.AppendHistogram(nil, 0, ts, h, false)
 	require.Greater(t, len(currChunk.Bytes()), len(prevChunkBytes)) // Check that current chunk is bigger than previously.
 	require.NoError(t, err)
 	require.Nil(t, newChunk)
@@ -804,7 +804,7 @@ func assertNoNewHistogramChunkOnAppend(t *testing.T, currChunk Chunk, hApp *Hist
 
 func assertRecodedHistogramChunkOnAppend(t *testing.T, prevChunk Chunk, hApp *HistogramAppender, ts int64, h *histogram.Histogram, expectHeader CounterResetHeader) {
 	prevChunkBytes := prevChunk.Bytes()
-	newChunk, recoded, newAppender, err := hApp.AppendHistogram(nil, ts, h, false)
+	newChunk, recoded, newAppender, err := hApp.AppendHistogram(nil, 0, ts, h, false)
 	require.Equal(t, prevChunkBytes, prevChunk.Bytes()) // Sanity check that previous chunk is untouched. This may change in the future if we implement in-place recoding.
 	require.NoError(t, err)
 	require.NotNil(t, newChunk)
@@ -1029,7 +1029,7 @@ func TestHistogramChunkAppendableWithEmptySpan(t *testing.T) {
 			require.NoError(t, err)
 			require.Equal(t, 0, c.NumSamples())
 
-			_, _, _, err = app.AppendHistogram(nil, 1, tc.h1, true)
+			_, _, _, err = app.AppendHistogram(nil, 1, 0, tc.h1, true)
 			require.NoError(t, err)
 			require.Equal(t, 1, c.NumSamples())
 			hApp, _ := app.(*HistogramAppender)
@@ -1172,7 +1172,7 @@ func TestAtFloatHistogram(t *testing.T) {
 	app, err := chk.Appender()
 	require.NoError(t, err)
 	for i := range input {
-		newc, _, _, err := app.AppendHistogram(nil, int64(i), &input[i], false)
+		newc, _, _, err := app.AppendHistogram(nil, 0, int64(i), &input[i], false)
 		require.NoError(t, err)
 		require.Nil(t, newc)
 	}
@@ -1230,7 +1230,7 @@ func TestHistogramChunkAppendableGauge(t *testing.T) {
 
 		ts := int64(1234567890)
 
-		chk, _, app, err := app.AppendHistogram(nil, ts, h.Copy(), false)
+		chk, _, app, err := app.AppendHistogram(nil, 0, ts, h.Copy(), false)
 		require.NoError(t, err)
 		require.Nil(t, chk)
 		require.Equal(t, 1, c.NumSamples())
@@ -1471,7 +1471,7 @@ func TestHistogramAppendOnlyErrors(t *testing.T) {
 
 		h := tsdbutil.GenerateTestHistogram(0)
 		var isRecoded bool
-		c, isRecoded, app, err = app.AppendHistogram(nil, 1, h, true)
+		c, isRecoded, app, err = app.AppendHistogram(nil, 0, 1, h, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.NoError(t, err)
@@ -1479,7 +1479,7 @@ func TestHistogramAppendOnlyErrors(t *testing.T) {
 		// Add erroring histogram.
 		h2 := h.Copy()
 		h2.Schema++
-		c, isRecoded, _, err = app.AppendHistogram(nil, 2, h2, true)
+		c, isRecoded, _, err = app.AppendHistogram(nil, 0, 2, h2, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.EqualError(t, err, "histogram schema change")
@@ -1493,7 +1493,7 @@ func TestHistogramAppendOnlyErrors(t *testing.T) {
 
 		h := tsdbutil.GenerateTestHistogram(0)
 		var isRecoded bool
-		c, isRecoded, app, err = app.AppendHistogram(nil, 1, h, true)
+		c, isRecoded, app, err = app.AppendHistogram(nil, 0, 1, h, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.NoError(t, err)
@@ -1501,7 +1501,7 @@ func TestHistogramAppendOnlyErrors(t *testing.T) {
 		// Add erroring histogram.
 		h2 := h.Copy()
 		h2.CounterResetHint = histogram.CounterReset
-		c, isRecoded, _, err = app.AppendHistogram(nil, 2, h2, true)
+		c, isRecoded, _, err = app.AppendHistogram(nil, 0, 2, h2, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.EqualError(t, err, "histogram counter reset")
@@ -1515,7 +1515,7 @@ func TestHistogramAppendOnlyErrors(t *testing.T) {
 
 		h := tsdbutil.GenerateTestCustomBucketsHistogram(0)
 		var isRecoded bool
-		c, isRecoded, app, err = app.AppendHistogram(nil, 1, h, true)
+		c, isRecoded, app, err = app.AppendHistogram(nil, 0, 1, h, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.NoError(t, err)
@@ -1523,7 +1523,7 @@ func TestHistogramAppendOnlyErrors(t *testing.T) {
 		// Add erroring histogram.
 		h2 := h.Copy()
 		h2.CustomValues = []float64{0, 1, 2, 3, 4, 5, 6, 7}
-		c, isRecoded, _, err = app.AppendHistogram(nil, 2, h2, true)
+		c, isRecoded, _, err = app.AppendHistogram(nil, 0, 2, h2, true)
 		require.Nil(t, c)
 		require.False(t, isRecoded)
 		require.EqualError(t, err, "histogram counter reset")
@@ -1556,10 +1556,10 @@ func TestHistogramUniqueSpansAfterNextWithAtHistogram(t *testing.T) {
 	app, err := c.Appender()
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 0, h1, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 0, h1, false)
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 1, h2, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 1, h2, false)
 	require.NoError(t, err)
 
 	// Create an iterator and advance to the first histogram.
@@ -1607,10 +1607,10 @@ func TestHistogramUniqueSpansAfterNextWithAtFloatHistogram(t *testing.T) {
 	app, err := c.Appender()
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 0, h1, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 0, h1, false)
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 1, h2, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 1, h2, false)
 	require.NoError(t, err)
 
 	// Create an iterator and advance to the first histogram.
@@ -1653,10 +1653,10 @@ func TestHistogramCustomValuesInternedAfterNextWithAtHistogram(t *testing.T) {
 	app, err := c.Appender()
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 0, h1, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 0, h1, false)
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 1, h2, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 1, h2, false)
 	require.NoError(t, err)
 
 	// Create an iterator and advance to the first histogram.
@@ -1699,10 +1699,10 @@ func TestHistogramCustomValuesInternedAfterNextWithAtFloatHistogram(t *testing.T
 	app, err := c.Appender()
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 0, h1, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 0, h1, false)
 	require.NoError(t, err)
 
-	_, _, _, err = app.AppendHistogram(nil, 1, h2, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 1, h2, false)
 	require.NoError(t, err)
 
 	// Create an iterator and advance to the first histogram.
@@ -1754,7 +1754,7 @@ func BenchmarkAppendable(b *testing.B) {
 		b.Fatal(err)
 	}
 
-	_, _, _, err = app.AppendHistogram(nil, 1, h, true)
+	_, _, _, err = app.AppendHistogram(nil, 0, 1, h, true)
 	if err != nil {
 		b.Fatal(err)
 	}
@@ -1791,7 +1791,7 @@ func TestIntHistogramEmptyBucketsWithGaps(t *testing.T) {
 	c := NewHistogramChunk()
 	app, err := c.Appender()
 	require.NoError(t, err)
-	_, _, _, err = app.AppendHistogram(nil, 1, h1, false)
+	_, _, _, err = app.AppendHistogram(nil, 0, 1, h1, false)
 	require.NoError(t, err)
 
 	h2 := &histogram.Histogram{
@@ -1804,7 +1804,7 @@ func TestIntHistogramEmptyBucketsWithGaps(t *testing.T) {
 	}
 	require.NoError(t, h2.Validate())
 
-	newC, recoded, _, err := app.AppendHistogram(nil, 2, h2, false)
+	newC, recoded, _, err := app.AppendHistogram(nil, 0, 2, h2, false)
 	require.NoError(t, err)
 	require.True(t, recoded)
 	require.NotNil(t, newC)
@@ -1839,7 +1839,7 @@ func TestHistogramIteratorFailIfSchemaInValid(t *testing.T) {
 			app, err := c.Appender()
 			require.NoError(t, err)
 
-			_, _, _, err = app.AppendHistogram(nil, 1, h, false)
+			_, _, _, err = app.AppendHistogram(nil, 0, 1, h, false)
 			require.NoError(t, err)
 
 			it := c.Iterator(nil)
@@ -1868,7 +1868,7 @@ func TestHistogramIteratorReduceSchema(t *testing.T) {
 			app, err := c.Appender()
 			require.NoError(t, err)
 
-			_, _, _, err = app.AppendHistogram(nil, 1, h, false)
+			_, _, _, err = app.AppendHistogram(nil, 0, 1, h, false)
 			require.NoError(t, err)
 
 			it := c.Iterator(nil)
diff --git a/tsdb/chunkenc/xor.go b/tsdb/chunkenc/xor.go
index bbe12a893b..5a9a59dc22 100644
--- a/tsdb/chunkenc/xor.go
+++ b/tsdb/chunkenc/xor.go
@@ -158,7 +158,7 @@ type xorAppender struct {
 	trailing uint8
 }
 
-func (a *xorAppender) Append(t int64, v float64) {
+func (a *xorAppender) Append(_, t int64, v float64) {
 	var tDelta uint64
 	num := binary.BigEndian.Uint16(a.b.bytes())
 	switch num {
@@ -225,11 +225,11 @@ func (a *xorAppender) writeVDelta(v float64) {
 	xorWrite(a.b, v, a.v, &a.leading, &a.trailing)
 }
 
-func (*xorAppender) AppendHistogram(*HistogramAppender, int64, *histogram.Histogram, bool) (Chunk, bool, Appender, error) {
+func (*xorAppender) AppendHistogram(*HistogramAppender, int64, int64, *histogram.Histogram, bool) (Chunk, bool, Appender, error) {
 	panic("appended a histogram sample to a float chunk")
 }
 
-func (*xorAppender) AppendFloatHistogram(*FloatHistogramAppender, int64, *histogram.FloatHistogram, bool) (Chunk, bool, Appender, error) {
+func (*xorAppender) AppendFloatHistogram(*FloatHistogramAppender, int64, int64, *histogram.FloatHistogram, bool) (Chunk, bool, Appender, error) {
 	panic("appended a float histogram sample to a float chunk")
 }
 
@@ -277,6 +277,10 @@ func (it *xorIterator) AtT() int64 {
 	return it.t
 }
 
+func (*xorIterator) AtST() int64 {
+	return 0
+}
+
 func (it *xorIterator) Err() error {
 	return it.err
 }
diff --git a/tsdb/chunkenc/xor_test.go b/tsdb/chunkenc/xor_test.go
index 904e536b49..b30c65283d 100644
--- a/tsdb/chunkenc/xor_test.go
+++ b/tsdb/chunkenc/xor_test.go
@@ -24,7 +24,7 @@ func BenchmarkXorRead(b *testing.B) {
 	app, err := c.Appender()
 	require.NoError(b, err)
 	for i := int64(0); i < 120*1000; i += 1000 {
-		app.Append(i, float64(i)+float64(i)/10+float64(i)/100+float64(i)/1000)
+		app.Append(0, i, float64(i)+float64(i)/10+float64(i)/100+float64(i)/1000)
 	}
 
 	b.ReportAllocs()
diff --git a/tsdb/chunks/chunks.go b/tsdb/chunks/chunks.go
index f8fc9a2e95..ce4c9d3d78 100644
--- a/tsdb/chunks/chunks.go
+++ b/tsdb/chunks/chunks.go
@@ -135,6 +135,7 @@ type Meta struct {
 }
 
 // ChunkFromSamples requires all samples to have the same type.
+// TODO(krajorama): test with ST when chunk formats support it.
 func ChunkFromSamples(s []Sample) (Meta, error) {
 	return ChunkFromSamplesGeneric(SampleSlice(s))
 }
@@ -164,9 +165,9 @@ func ChunkFromSamplesGeneric(s Samples) (Meta, error) {
 	for i := 0; i < s.Len(); i++ {
 		switch sampleType {
 		case chunkenc.ValFloat:
-			ca.Append(s.Get(i).T(), s.Get(i).F())
+			ca.Append(s.Get(i).ST(), s.Get(i).T(), s.Get(i).F())
 		case chunkenc.ValHistogram:
-			newChunk, _, ca, err = ca.AppendHistogram(nil, s.Get(i).T(), s.Get(i).H(), false)
+			newChunk, _, ca, err = ca.AppendHistogram(nil, s.Get(i).ST(), s.Get(i).T(), s.Get(i).H(), false)
 			if err != nil {
 				return emptyChunk, err
 			}
@@ -174,7 +175,7 @@ func ChunkFromSamplesGeneric(s Samples) (Meta, error) {
 				return emptyChunk, errors.New("did not expect to start a second chunk")
 			}
 		case chunkenc.ValFloatHistogram:
-			newChunk, _, ca, err = ca.AppendFloatHistogram(nil, s.Get(i).T(), s.Get(i).FH(), false)
+			newChunk, _, ca, err = ca.AppendFloatHistogram(nil, s.Get(i).ST(), s.Get(i).T(), s.Get(i).FH(), false)
 			if err != nil {
 				return emptyChunk, err
 			}
diff --git a/tsdb/chunks/head_chunks_test.go b/tsdb/chunks/head_chunks_test.go
index 17efd44aa6..c3cbc5a618 100644
--- a/tsdb/chunks/head_chunks_test.go
+++ b/tsdb/chunks/head_chunks_test.go
@@ -559,7 +559,7 @@ func randomChunk(t *testing.T) chunkenc.Chunk {
 	app, err := chunk.Appender()
 	require.NoError(t, err)
 	for range length {
-		app.Append(rand.Int63(), rand.Float64())
+		app.Append(0, rand.Int63(), rand.Float64())
 	}
 	return chunk
 }
diff --git a/tsdb/chunks/samples.go b/tsdb/chunks/samples.go
index 8097bcd72b..280f2dd606 100644
--- a/tsdb/chunks/samples.go
+++ b/tsdb/chunks/samples.go
@@ -25,6 +25,7 @@ type Samples interface {
 
 type Sample interface {
 	T() int64
+	ST() int64
 	F() float64
 	H() *histogram.Histogram
 	FH() *histogram.FloatHistogram
@@ -38,16 +39,20 @@ func (s SampleSlice) Get(i int) Sample { return s[i] }
 func (s SampleSlice) Len() int         { return len(s) }
 
 type sample struct {
-	t  int64
-	f  float64
-	h  *histogram.Histogram
-	fh *histogram.FloatHistogram
+	st, t int64
+	f     float64
+	h     *histogram.Histogram
+	fh    *histogram.FloatHistogram
 }
 
 func (s sample) T() int64 {
 	return s.t
 }
 
+func (s sample) ST() int64 {
+	return s.st
+}
+
 func (s sample) F() float64 {
 	return s.f
 }
diff --git a/tsdb/head.go b/tsdb/head.go
index 955c0ae5a7..213846aa35 100644
--- a/tsdb/head.go
+++ b/tsdb/head.go
@@ -2111,7 +2111,10 @@ func newSample(t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHi
 	return sample{t, v, h, fh}
 }
 
-func (s sample) T() int64                      { return s.t }
+func (s sample) T() int64 { return s.t }
+
+// TODO(krajorama): implement ST.
+func (sample) ST() int64                       { return 0 }
 func (s sample) F() float64                    { return s.f }
 func (s sample) H() *histogram.Histogram       { return s.h }
 func (s sample) FH() *histogram.FloatHistogram { return s.fh }
diff --git a/tsdb/head_append.go b/tsdb/head_append.go
index fceb80bd34..6a04fd16d9 100644
--- a/tsdb/head_append.go
+++ b/tsdb/head_append.go
@@ -1843,7 +1843,8 @@ func (s *memSeries) append(t int64, v float64, appendID uint64, o chunkOpts) (sa
 	if !sampleInOrder {
 		return sampleInOrder, chunkCreated
 	}
-	s.app.Append(t, v)
+	// TODO(krajorama): pass ST.
+	s.app.Append(0, t, v)
 
 	c.maxTime = t
 
@@ -1885,7 +1886,8 @@ func (s *memSeries) appendHistogram(t int64, h *histogram.Histogram, appendID ui
 		prevApp = nil
 	}
 
-	newChunk, recoded, s.app, _ = s.app.AppendHistogram(prevApp, t, h, false) // false=request a new chunk if needed
+	// TODO(krajorama): pass ST.
+	newChunk, recoded, s.app, _ = s.app.AppendHistogram(prevApp, 0, t, h, false) // false=request a new chunk if needed
 
 	s.lastHistogramValue = h
 	s.lastFloatHistogramValue = nil
@@ -1942,7 +1944,8 @@ func (s *memSeries) appendFloatHistogram(t int64, fh *histogram.FloatHistogram,
 		prevApp = nil
 	}
 
-	newChunk, recoded, s.app, _ = s.app.AppendFloatHistogram(prevApp, t, fh, false) // False means request a new chunk if needed.
+	// TODO(krajorama): pass ST.
+	newChunk, recoded, s.app, _ = s.app.AppendFloatHistogram(prevApp, 0, t, fh, false) // False means request a new chunk if needed.
 
 	s.lastHistogramValue = nil
 	s.lastFloatHistogramValue = fh
diff --git a/tsdb/ooo_head.go b/tsdb/ooo_head.go
index c6ae924372..bbb0f10e77 100644
--- a/tsdb/ooo_head.go
+++ b/tsdb/ooo_head.go
@@ -125,7 +125,8 @@ func (o *OOOChunk) ToEncodedChunks(mint, maxt int64) (chks []memChunk, err error
 		}
 		switch encoding {
 		case chunkenc.EncXOR:
-			app.Append(s.t, s.f)
+			// TODO(krajorama): pass ST.
+			app.Append(0, s.t, s.f)
 		case chunkenc.EncHistogram:
 			// Ignoring ok is ok, since we don't want to compare to the wrong previous appender anyway.
 			prevHApp, _ := prevApp.(*chunkenc.HistogramAppender)
@@ -133,7 +134,8 @@ func (o *OOOChunk) ToEncodedChunks(mint, maxt int64) (chks []memChunk, err error
 				newChunk chunkenc.Chunk
 				recoded  bool
 			)
-			newChunk, recoded, app, _ = app.AppendHistogram(prevHApp, s.t, s.h, false)
+			// TODO(krajorama): pass ST.
+			newChunk, recoded, app, _ = app.AppendHistogram(prevHApp, 0, s.t, s.h, false)
 			if newChunk != nil { // A new chunk was allocated.
 				if !recoded {
 					chks = append(chks, memChunk{chunk, cmint, cmaxt, nil})
@@ -148,7 +150,8 @@ func (o *OOOChunk) ToEncodedChunks(mint, maxt int64) (chks []memChunk, err error
 				newChunk chunkenc.Chunk
 				recoded  bool
 			)
-			newChunk, recoded, app, _ = app.AppendFloatHistogram(prevHApp, s.t, s.fh, false)
+			// TODO(krajorama): pass ST.
+			newChunk, recoded, app, _ = app.AppendFloatHistogram(prevHApp, 0, s.t, s.fh, false)
 			if newChunk != nil { // A new chunk was allocated.
 				if !recoded {
 					chks = append(chks, memChunk{chunk, cmint, cmaxt, nil})
diff --git a/tsdb/querier.go b/tsdb/querier.go
index 4a487aa568..ce0292bf24 100644
--- a/tsdb/querier.go
+++ b/tsdb/querier.go
@@ -788,6 +788,11 @@ func (p *populateWithDelSeriesIterator) AtT() int64 {
 	return p.curr.AtT()
 }
 
+// AtST TODO(krajorama): test AtST() when chunks support it.
+func (p *populateWithDelSeriesIterator) AtST() int64 {
+	return p.curr.AtST()
+}
+
 func (p *populateWithDelSeriesIterator) Err() error {
 	if err := p.populateWithDelGenericSeriesIterator.Err(); err != nil {
 		return err
@@ -862,6 +867,7 @@ func (p *populateWithDelChunkSeriesIterator) Next() bool {
 
 // populateCurrForSingleChunk sets the fields within p.currMetaWithChunk. This
 // should be called if the samples in p.currDelIter only form one chunk.
+// TODO(krajorama): test ST when chunks support it.
 func (p *populateWithDelChunkSeriesIterator) populateCurrForSingleChunk() bool {
 	valueType := p.currDelIter.Next()
 	if valueType == chunkenc.ValNone {
@@ -877,7 +883,7 @@ func (p *populateWithDelChunkSeriesIterator) populateCurrForSingleChunk() bool {
 	var (
 		newChunk chunkenc.Chunk
 		app      chunkenc.Appender
-		t        int64
+		st, t    int64
 		err      error
 	)
 	switch valueType {
@@ -893,7 +899,8 @@ func (p *populateWithDelChunkSeriesIterator) populateCurrForSingleChunk() bool {
 			}
 			var h *histogram.Histogram
 			t, h = p.currDelIter.AtHistogram(nil)
-			_, _, app, err = app.AppendHistogram(nil, t, h, true)
+			st = p.currDelIter.AtST()
+			_, _, app, err = app.AppendHistogram(nil, st, t, h, true)
 			if err != nil {
 				break
 			}
@@ -910,7 +917,8 @@ func (p *populateWithDelChunkSeriesIterator) populateCurrForSingleChunk() bool {
 			}
 			var v float64
 			t, v = p.currDelIter.At()
-			app.Append(t, v)
+			st = p.currDelIter.AtST()
+			app.Append(st, t, v)
 		}
 	case chunkenc.ValFloatHistogram:
 		newChunk = chunkenc.NewFloatHistogramChunk()
@@ -924,7 +932,8 @@ func (p *populateWithDelChunkSeriesIterator) populateCurrForSingleChunk() bool {
 			}
 			var h *histogram.FloatHistogram
 			t, h = p.currDelIter.AtFloatHistogram(nil)
-			_, _, app, err = app.AppendFloatHistogram(nil, t, h, true)
+			st = p.currDelIter.AtST()
+			_, _, app, err = app.AppendFloatHistogram(nil, st, t, h, true)
 			if err != nil {
 				break
 			}
@@ -950,6 +959,7 @@ func (p *populateWithDelChunkSeriesIterator) populateCurrForSingleChunk() bool {
 // populateChunksFromIterable reads the samples from currDelIter to create
 // chunks for chunksFromIterable. It also sets p.currMetaWithChunk to the first
 // chunk.
+// TODO(krajorama): test ST when chunks support it.
 func (p *populateWithDelChunkSeriesIterator) populateChunksFromIterable() bool {
 	p.chunksFromIterable = p.chunksFromIterable[:0]
 	p.chunksFromIterableIdx = -1
@@ -965,7 +975,7 @@ func (p *populateWithDelChunkSeriesIterator) populateChunksFromIterable() bool {
 
 	var (
 		// t is the timestamp for the current sample.
-		t     int64
+		st, t int64
 		cmint int64
 		cmaxt int64
 
@@ -1004,23 +1014,26 @@ func (p *populateWithDelChunkSeriesIterator) populateChunksFromIterable() bool {
 			{
 				var v float64
 				t, v = p.currDelIter.At()
-				app.Append(t, v)
+				st = p.currDelIter.AtST()
+				app.Append(st, t, v)
 			}
 		case chunkenc.ValHistogram:
 			{
 				var v *histogram.Histogram
 				t, v = p.currDelIter.AtHistogram(nil)
+				st = p.currDelIter.AtST()
 				// No need to set prevApp as AppendHistogram will set the
 				// counter reset header for the appender that's returned.
-				newChunk, recoded, app, err = app.AppendHistogram(nil, t, v, false)
+				newChunk, recoded, app, err = app.AppendHistogram(nil, st, t, v, false)
 			}
 		case chunkenc.ValFloatHistogram:
 			{
 				var v *histogram.FloatHistogram
 				t, v = p.currDelIter.AtFloatHistogram(nil)
+				st = p.currDelIter.AtST()
 				// No need to set prevApp as AppendHistogram will set the
 				// counter reset header for the appender that's returned.
-				newChunk, recoded, app, err = app.AppendFloatHistogram(nil, t, v, false)
+				newChunk, recoded, app, err = app.AppendFloatHistogram(nil, st, t, v, false)
 			}
 		}
 
@@ -1202,6 +1215,11 @@ func (it *DeletedIterator) AtT() int64 {
 	return it.Iter.AtT()
 }
 
+// AtST TODO(krajorama): test AtST() when chunks support it.
+func (it *DeletedIterator) AtST() int64 {
+	return it.Iter.AtST()
+}
+
 func (it *DeletedIterator) Seek(t int64) chunkenc.ValueType {
 	if it.Iter.Err() != nil {
 		return chunkenc.ValNone
diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go
index 6933aa617a..57a53c46fe 100644
--- a/tsdb/querier_test.go
+++ b/tsdb/querier_test.go
@@ -141,7 +141,7 @@ func createIdxChkReaders(t *testing.T, tc []seriesSamples) (IndexReader, ChunkRe
 				app, _ := chunk.Appender()
 				for _, smpl := range chk {
 					require.NotNil(t, smpl.fh, "chunk can only contain one type of sample")
-					_, _, _, err := app.AppendFloatHistogram(nil, smpl.t, smpl.fh, true)
+					_, _, _, err := app.AppendFloatHistogram(nil, 0, smpl.t, smpl.fh, true)
 					require.NoError(t, err, "chunk should be appendable")
 				}
 				chkReader[chunkRef] = chunk
@@ -150,7 +150,7 @@ func createIdxChkReaders(t *testing.T, tc []seriesSamples) (IndexReader, ChunkRe
 				app, _ := chunk.Appender()
 				for _, smpl := range chk {
 					require.NotNil(t, smpl.h, "chunk can only contain one type of sample")
-					_, _, _, err := app.AppendHistogram(nil, smpl.t, smpl.h, true)
+					_, _, _, err := app.AppendHistogram(nil, 0, smpl.t, smpl.h, true)
 					require.NoError(t, err, "chunk should be appendable")
 				}
 				chkReader[chunkRef] = chunk
@@ -160,7 +160,7 @@ func createIdxChkReaders(t *testing.T, tc []seriesSamples) (IndexReader, ChunkRe
 				for _, smpl := range chk {
 					require.Nil(t, smpl.h, "chunk can only contain one type of sample")
 					require.Nil(t, smpl.fh, "chunk can only contain one type of sample")
-					app.Append(smpl.t, smpl.f)
+					app.Append(0, smpl.t, smpl.f)
 				}
 				chkReader[chunkRef] = chunk
 			}
@@ -790,6 +790,10 @@ func (it *mockSampleIterator) AtT() int64 {
 	return it.s[it.idx].T()
 }
 
+func (it *mockSampleIterator) AtST() int64 {
+	return it.s[it.idx].ST()
+}
+
 func (it *mockSampleIterator) Next() chunkenc.ValueType {
 	if it.idx < len(it.s)-1 {
 		it.idx++
@@ -2096,7 +2100,7 @@ func TestDeletedIterator(t *testing.T) {
 	for i := range 1000 {
 		act[i].t = int64(i)
 		act[i].f = rand.Float64()
-		app.Append(act[i].t, act[i].f)
+		app.Append(0, act[i].t, act[i].f)
 	}
 
 	cases := []struct {
@@ -2156,7 +2160,7 @@ func TestDeletedIterator_WithSeek(t *testing.T) {
 	for i := range 1000 {
 		act[i].t = int64(i)
 		act[i].f = float64(i)
-		app.Append(act[i].t, act[i].f)
+		app.Append(0, act[i].t, act[i].f)
 	}
 
 	cases := []struct {

From 1e77d9ded85fd2c3db5a6be153d6d85e3d6853ab Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 08:57:54 +0100
Subject: [PATCH 253/439] storage/buffer.go: add ST to sample types and
 iterators
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Also fix the single multi line fSample definition to be one liner.

Signed-off-by: György Krajcsovits 
---
 storage/buffer.go      | 56 +++++++++++++++++++++++++++---------------
 storage/buffer_test.go |  5 +---
 2 files changed, 37 insertions(+), 24 deletions(-)

diff --git a/storage/buffer.go b/storage/buffer.go
index c6a32821d8..cdf8879f21 100644
--- a/storage/buffer.go
+++ b/storage/buffer.go
@@ -119,13 +119,16 @@ func (b *BufferedSeriesIterator) Next() chunkenc.ValueType {
 		return chunkenc.ValNone
 	case chunkenc.ValFloat:
 		t, f := b.it.At()
-		b.buf.addF(fSample{t: t, f: f})
+		st := b.it.AtST()
+		b.buf.addF(fSample{st: st, t: t, f: f})
 	case chunkenc.ValHistogram:
 		t, h := b.it.AtHistogram(&b.hReader)
-		b.buf.addH(hSample{t: t, h: h})
+		st := b.it.AtST()
+		b.buf.addH(hSample{st: st, t: t, h: h})
 	case chunkenc.ValFloatHistogram:
 		t, fh := b.it.AtFloatHistogram(&b.fhReader)
-		b.buf.addFH(fhSample{t: t, fh: fh})
+		st := b.it.AtST()
+		b.buf.addFH(fhSample{st: st, t: t, fh: fh})
 	default:
 		panic(fmt.Errorf("BufferedSeriesIterator: unknown value type %v", b.valueType))
 	}
@@ -157,23 +160,27 @@ func (b *BufferedSeriesIterator) AtT() int64 {
 	return b.it.AtT()
 }
 
+// AtST returns the current sample's start timestamp of the iterator.
+func (b *BufferedSeriesIterator) AtST() int64 {
+	return b.it.AtST()
+}
+
 // Err returns the last encountered error.
 func (b *BufferedSeriesIterator) Err() error {
 	return b.it.Err()
 }
 
 type fSample struct {
-	t int64
-	f float64
+	st, t int64
+	f     float64
 }
 
 func (s fSample) T() int64 {
 	return s.t
 }
 
-// TODO(krajorama): implement ST.
-func (fSample) ST() int64 {
-	return 0
+func (s fSample) ST() int64 {
+	return s.st
 }
 
 func (s fSample) F() float64 {
@@ -197,17 +204,16 @@ func (s fSample) Copy() chunks.Sample {
 }
 
 type hSample struct {
-	t int64
-	h *histogram.Histogram
+	st, t int64
+	h     *histogram.Histogram
 }
 
 func (s hSample) T() int64 {
 	return s.t
 }
 
-// TODO(krajorama): implement ST.
-func (hSample) ST() int64 {
-	return 0
+func (s hSample) ST() int64 {
+	return s.st
 }
 
 func (hSample) F() float64 {
@@ -227,21 +233,20 @@ func (hSample) Type() chunkenc.ValueType {
 }
 
 func (s hSample) Copy() chunks.Sample {
-	return hSample{t: s.t, h: s.h.Copy()}
+	return hSample{st: s.st, t: s.t, h: s.h.Copy()}
 }
 
 type fhSample struct {
-	t  int64
-	fh *histogram.FloatHistogram
+	st, t int64
+	fh    *histogram.FloatHistogram
 }
 
 func (s fhSample) T() int64 {
 	return s.t
 }
 
-// TODO(krajorama): implement ST.
-func (fhSample) ST() int64 {
-	return 0
+func (s fhSample) ST() int64 {
+	return s.st
 }
 
 func (fhSample) F() float64 {
@@ -261,7 +266,7 @@ func (fhSample) Type() chunkenc.ValueType {
 }
 
 func (s fhSample) Copy() chunks.Sample {
-	return fhSample{t: s.t, fh: s.fh.Copy()}
+	return fhSample{st: s.st, t: s.t, fh: s.fh.Copy()}
 }
 
 type sampleRing struct {
@@ -344,6 +349,7 @@ func (r *sampleRing) iterator() *SampleRingIterator {
 type SampleRingIterator struct {
 	r  *sampleRing
 	i  int
+	st int64
 	t  int64
 	f  float64
 	h  *histogram.Histogram
@@ -365,21 +371,25 @@ func (it *SampleRingIterator) Next() chunkenc.ValueType {
 	switch it.r.bufInUse {
 	case fBuf:
 		s := it.r.atF(it.i)
+		it.st = s.st
 		it.t = s.t
 		it.f = s.f
 		return chunkenc.ValFloat
 	case hBuf:
 		s := it.r.atH(it.i)
+		it.st = s.st
 		it.t = s.t
 		it.h = s.h
 		return chunkenc.ValHistogram
 	case fhBuf:
 		s := it.r.atFH(it.i)
+		it.st = s.st
 		it.t = s.t
 		it.fh = s.fh
 		return chunkenc.ValFloatHistogram
 	}
 	s := it.r.at(it.i)
+	it.st = s.ST()
 	it.t = s.T()
 	switch s.Type() {
 	case chunkenc.ValHistogram:
@@ -425,6 +435,10 @@ func (it *SampleRingIterator) AtT() int64 {
 	return it.t
 }
 
+func (it *SampleRingIterator) AtST() int64 {
+	return it.st
+}
+
 func (r *sampleRing) at(i int) chunks.Sample {
 	j := (r.f + i) % len(r.iBuf)
 	return r.iBuf[j]
@@ -666,6 +680,7 @@ func addH(s hSample, buf []hSample, r *sampleRing) []hSample {
 	}
 
 	buf[r.i].t = s.t
+	buf[r.i].st = s.st
 	if buf[r.i].h == nil {
 		buf[r.i].h = s.h.Copy()
 	} else {
@@ -710,6 +725,7 @@ func addFH(s fhSample, buf []fhSample, r *sampleRing) []fhSample {
 	}
 
 	buf[r.i].t = s.t
+	buf[r.i].st = s.st
 	if buf[r.i].fh == nil {
 		buf[r.i].fh = s.fh.Copy()
 	} else {
diff --git a/storage/buffer_test.go b/storage/buffer_test.go
index beb9d8e71c..e700231756 100644
--- a/storage/buffer_test.go
+++ b/storage/buffer_test.go
@@ -61,10 +61,7 @@ func TestSampleRing(t *testing.T) {
 
 		input := []fSample{}
 		for _, t := range c.input {
-			input = append(input, fSample{
-				t: t,
-				f: float64(rand.Intn(100)),
-			})
+			input = append(input, fSample{t: t, f: float64(rand.Intn(100))})
 		}
 
 		for i, s := range input {

From a00c0d6a660eb38380c96d801422c694f0506e23 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 09:37:15 +0100
Subject: [PATCH 254/439] auto update f/h/fh sample init with positional fields
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

find . -name "*.go" -type f \
-exec sed -E -i 's/((f|h|fh)Sample\{)([^,{:]+,[^,]+\})/\10, \3/g' {} +

Signed-off-by: György Krajcsovits 
---
 storage/merge_test.go  | 308 ++++++++++++++++++++---------------------
 storage/series.go      |   6 +-
 storage/series_test.go |  10 +-
 3 files changed, 162 insertions(+), 162 deletions(-)

diff --git a/storage/merge_test.go b/storage/merge_test.go
index 0060950d6f..5ffb0c4851 100644
--- a/storage/merge_test.go
+++ b/storage/merge_test.go
@@ -66,116 +66,116 @@ func TestMergeQuerierWithChainMerger(t *testing.T) {
 		{
 			name: "one querier, two series",
 			querierSeries: [][]Series{{
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			}},
 			expected: NewMockSeriesSet(
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			),
 		},
 		{
 			name: "two queriers, one different series each",
 			querierSeries: [][]Series{{
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
 			}, {
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			}},
 			expected: NewMockSeriesSet(
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			),
 		},
 		{
 			name: "two time unsorted queriers, two series each",
 			querierSeries: [][]Series{{
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{5, 5}, fSample{6, 6}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 5, 5}, fSample{0, 6, 6}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}, fSample{4, 4}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}, fSample{0, 4, 4}}),
 			}},
 			expected: NewMockSeriesSet(
 				NewListSeries(
 					labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}, fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}, fSample{0, 6, 6}},
 				),
 				NewListSeries(
 					labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "five queriers, only two queriers have two time unsorted series each",
 			querierSeries: [][]Series{{}, {}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{5, 5}, fSample{6, 6}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 5, 5}, fSample{0, 6, 6}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}, fSample{4, 4}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}, fSample{0, 4, 4}}),
 			}, {}},
 			expected: NewMockSeriesSet(
 				NewListSeries(
 					labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}, fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}, fSample{0, 6, 6}},
 				),
 				NewListSeries(
 					labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "two queriers, only two queriers have two time unsorted series each, with 3 noop and one nil querier together",
 			querierSeries: [][]Series{{}, {}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{5, 5}, fSample{6, 6}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 5, 5}, fSample{0, 6, 6}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}, fSample{4, 4}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}, fSample{0, 4, 4}}),
 			}, {}},
 			extraQueriers: []Querier{NoopQuerier(), NoopQuerier(), nil, NoopQuerier()},
 			expected: NewMockSeriesSet(
 				NewListSeries(
 					labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}, fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}, fSample{0, 6, 6}},
 				),
 				NewListSeries(
 					labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "two queriers, with two series, one is overlapping",
 			querierSeries: [][]Series{{}, {}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 21}, fSample{3, 31}, fSample{5, 5}, fSample{6, 6}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 21}, fSample{0, 3, 31}, fSample{0, 5, 5}, fSample{0, 6, 6}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}}),
 			}, {
-				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 22}, fSample{3, 32}}),
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}, fSample{4, 4}}),
+				NewListSeries(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 22}, fSample{0, 3, 32}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}, fSample{0, 4, 4}}),
 			}, {}},
 			expected: NewMockSeriesSet(
 				NewListSeries(
 					labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 21}, fSample{3, 31}, fSample{5, 5}, fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 21}, fSample{0, 3, 31}, fSample{0, 5, 5}, fSample{0, 6, 6}},
 				),
 				NewListSeries(
 					labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "two queries, one with NaN samples series",
 			querierSeries: [][]Series{{
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, math.NaN()}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, math.NaN()}}),
 			}, {
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{1, 1}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 1, 1}}),
 			}},
 			expected: NewMockSeriesSet(
-				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, math.NaN()}, fSample{1, 1}}),
+				NewListSeries(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, math.NaN()}, fSample{0, 1, 1}}),
 			),
 		},
 	} {
@@ -249,108 +249,108 @@ func TestMergeChunkQuerierWithNoVerticalChunkSeriesMerger(t *testing.T) {
 		{
 			name: "one querier, two series",
 			chkQuerierSeries: [][]ChunkSeries{{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			}},
 			expected: NewMockChunkSeriesSet(
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			),
 		},
 		{
 			name: "two secondaries, one different series each",
 			chkQuerierSeries: [][]ChunkSeries{{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
 			}, {
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			}},
 			expected: NewMockChunkSeriesSet(
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			),
 		},
 		{
 			name: "two secondaries, two not in time order series each",
 			chkQuerierSeries: [][]ChunkSeries{{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{5, 5}}, []chunks.Sample{fSample{6, 6}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 6, 6}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			}, {
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}}, []chunks.Sample{fSample{4, 4}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}}, []chunks.Sample{fSample{0, 4, 4}}),
 			}},
 			expected: NewMockChunkSeriesSet(
 				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 2}},
-					[]chunks.Sample{fSample{3, 3}},
-					[]chunks.Sample{fSample{5, 5}},
-					[]chunks.Sample{fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}},
+					[]chunks.Sample{fSample{0, 3, 3}},
+					[]chunks.Sample{fSample{0, 5, 5}},
+					[]chunks.Sample{fSample{0, 6, 6}},
 				),
 				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}},
-					[]chunks.Sample{fSample{2, 2}},
-					[]chunks.Sample{fSample{3, 3}},
-					[]chunks.Sample{fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}},
+					[]chunks.Sample{fSample{0, 2, 2}},
+					[]chunks.Sample{fSample{0, 3, 3}},
+					[]chunks.Sample{fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "five secondaries, only two have two not in time order series each",
 			chkQuerierSeries: [][]ChunkSeries{{}, {}, {
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{5, 5}}, []chunks.Sample{fSample{6, 6}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 6, 6}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			}, {
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}}, []chunks.Sample{fSample{4, 4}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}}, []chunks.Sample{fSample{0, 4, 4}}),
 			}, {}},
 			expected: NewMockChunkSeriesSet(
 				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 2}},
-					[]chunks.Sample{fSample{3, 3}},
-					[]chunks.Sample{fSample{5, 5}},
-					[]chunks.Sample{fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}},
+					[]chunks.Sample{fSample{0, 3, 3}},
+					[]chunks.Sample{fSample{0, 5, 5}},
+					[]chunks.Sample{fSample{0, 6, 6}},
 				),
 				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}},
-					[]chunks.Sample{fSample{2, 2}},
-					[]chunks.Sample{fSample{3, 3}},
-					[]chunks.Sample{fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}},
+					[]chunks.Sample{fSample{0, 2, 2}},
+					[]chunks.Sample{fSample{0, 3, 3}},
+					[]chunks.Sample{fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "two secondaries, with two not in time order series each, with 3 noop queries and one nil together",
 			chkQuerierSeries: [][]ChunkSeries{{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{5, 5}}, []chunks.Sample{fSample{6, 6}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}}, []chunks.Sample{fSample{2, 2}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 6, 6}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}}, []chunks.Sample{fSample{0, 2, 2}}),
 			}, {
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{3, 3}}, []chunks.Sample{fSample{4, 4}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 3, 3}}, []chunks.Sample{fSample{0, 4, 4}}),
 			}},
 			extraQueriers: []ChunkQuerier{NoopChunkedQuerier(), NoopChunkedQuerier(), nil, NoopChunkedQuerier()},
 			expected: NewMockChunkSeriesSet(
 				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-					[]chunks.Sample{fSample{1, 1}, fSample{2, 2}},
-					[]chunks.Sample{fSample{3, 3}},
-					[]chunks.Sample{fSample{5, 5}},
-					[]chunks.Sample{fSample{6, 6}},
+					[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}},
+					[]chunks.Sample{fSample{0, 3, 3}},
+					[]chunks.Sample{fSample{0, 5, 5}},
+					[]chunks.Sample{fSample{0, 6, 6}},
 				),
 				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"),
-					[]chunks.Sample{fSample{0, 0}, fSample{1, 1}},
-					[]chunks.Sample{fSample{2, 2}},
-					[]chunks.Sample{fSample{3, 3}},
-					[]chunks.Sample{fSample{4, 4}},
+					[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}},
+					[]chunks.Sample{fSample{0, 2, 2}},
+					[]chunks.Sample{fSample{0, 3, 3}},
+					[]chunks.Sample{fSample{0, 4, 4}},
 				),
 			),
 		},
 		{
 			name: "two queries, one with NaN samples series",
 			chkQuerierSeries: [][]ChunkSeries{{
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, math.NaN()}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, math.NaN()}}),
 			}, {
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{1, 1}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 1, 1}}),
 			}},
 			expected: NewMockChunkSeriesSet(
-				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, math.NaN()}}, []chunks.Sample{fSample{1, 1}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("foo", "bar"), []chunks.Sample{fSample{0, 0, math.NaN()}}, []chunks.Sample{fSample{0, 1, 1}}),
 			),
 		},
 	} {
@@ -431,9 +431,9 @@ func TestCompactingChunkSeriesMerger(t *testing.T) {
 		{
 			name: "single series",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
 		},
 		{
 			name: "two empty series",
@@ -446,55 +446,55 @@ func TestCompactingChunkSeriesMerger(t *testing.T) {
 		{
 			name: "two non overlapping",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{5, 5}}, []chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 		},
 		{
 			name: "two overlapping",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{8, 8}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 8, 8}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{7, 7}, fSample{8, 8}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 7, 7}, fSample{0, 8, 8}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 		},
 		{
 			name: "two duplicated",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
 		},
 		{
 			name: "three overlapping",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 2}, fSample{3, 3}, fSample{6, 6}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0}, fSample{4, 4}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 6, 6}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 4, 4}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0}, fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{4, 4}, fSample{5, 5}, fSample{6, 6}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 4, 4}, fSample{0, 5, 5}, fSample{0, 6, 6}}),
 		},
 		{
 			name: "three in chained overlap",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{4, 4}, fSample{6, 66}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{6, 6}, fSample{10, 10}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 4, 4}, fSample{0, 6, 66}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 6, 6}, fSample{0, 10, 10}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{4, 4}, fSample{5, 5}, fSample{6, 66}, fSample{10, 10}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 4, 4}, fSample{0, 5, 5}, fSample{0, 6, 66}, fSample{0, 10, 10}}),
 		},
 		{
 			name: "three in chained overlap complex",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0}, fSample{5, 5}}, []chunks.Sample{fSample{10, 10}, fSample{15, 15}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 2}, fSample{20, 20}}, []chunks.Sample{fSample{25, 25}, fSample{30, 30}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{18, 18}, fSample{26, 26}}, []chunks.Sample{fSample{31, 31}, fSample{35, 35}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 10, 10}, fSample{0, 15, 15}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 2}, fSample{0, 20, 20}}, []chunks.Sample{fSample{0, 25, 25}, fSample{0, 30, 30}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 18, 18}, fSample{0, 26, 26}}, []chunks.Sample{fSample{0, 31, 31}, fSample{0, 35, 35}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-				[]chunks.Sample{fSample{0, 0}, fSample{2, 2}, fSample{5, 5}, fSample{10, 10}, fSample{15, 15}, fSample{18, 18}, fSample{20, 20}, fSample{25, 25}, fSample{26, 26}, fSample{30, 30}},
-				[]chunks.Sample{fSample{31, 31}, fSample{35, 35}},
+				[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 2, 2}, fSample{0, 5, 5}, fSample{0, 10, 10}, fSample{0, 15, 15}, fSample{0, 18, 18}, fSample{0, 20, 20}, fSample{0, 25, 25}, fSample{0, 26, 26}, fSample{0, 30, 30}},
+				[]chunks.Sample{fSample{0, 31, 31}, fSample{0, 35, 35}},
 			),
 		},
 		{
@@ -534,13 +534,13 @@ func TestCompactingChunkSeriesMerger(t *testing.T) {
 			name: "histogram chunks overlapping with float chunks",
 			input: []ChunkSeries{
 				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{histogramSample(0), histogramSample(5)}, []chunks.Sample{histogramSample(10), histogramSample(15)}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{12, 12}}, []chunks.Sample{fSample{14, 14}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 12, 12}}, []chunks.Sample{fSample{0, 14, 14}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
 				[]chunks.Sample{histogramSample(0)},
-				[]chunks.Sample{fSample{1, 1}},
+				[]chunks.Sample{fSample{0, 1, 1}},
 				[]chunks.Sample{histogramSample(5), histogramSample(10)},
-				[]chunks.Sample{fSample{12, 12}, fSample{14, 14}},
+				[]chunks.Sample{fSample{0, 12, 12}, fSample{0, 14, 14}},
 				[]chunks.Sample{histogramSample(15)},
 			),
 		},
@@ -560,13 +560,13 @@ func TestCompactingChunkSeriesMerger(t *testing.T) {
 			name: "float histogram chunks overlapping with float chunks",
 			input: []ChunkSeries{
 				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{floatHistogramSample(0), floatHistogramSample(5)}, []chunks.Sample{floatHistogramSample(10), floatHistogramSample(15)}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{12, 12}}, []chunks.Sample{fSample{14, 14}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 12, 12}}, []chunks.Sample{fSample{0, 14, 14}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
 				[]chunks.Sample{floatHistogramSample(0)},
-				[]chunks.Sample{fSample{1, 1}},
+				[]chunks.Sample{fSample{0, 1, 1}},
 				[]chunks.Sample{floatHistogramSample(5), floatHistogramSample(10)},
-				[]chunks.Sample{fSample{12, 12}, fSample{14, 14}},
+				[]chunks.Sample{fSample{0, 12, 12}, fSample{0, 14, 14}},
 				[]chunks.Sample{floatHistogramSample(15)},
 			),
 		},
@@ -736,9 +736,9 @@ func TestConcatenatingChunkSeriesMerger(t *testing.T) {
 		{
 			name: "single series",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}}),
 		},
 		{
 			name: "two empty series",
@@ -751,70 +751,70 @@ func TestConcatenatingChunkSeriesMerger(t *testing.T) {
 		{
 			name: "two non overlapping",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 			},
-			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{5, 5}}, []chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 		},
 		{
 			name: "two overlapping",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{8, 8}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 8, 8}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-				[]chunks.Sample{fSample{1, 1}, fSample{2, 2}}, []chunks.Sample{fSample{3, 3}, fSample{8, 8}},
-				[]chunks.Sample{fSample{7, 7}, fSample{9, 9}}, []chunks.Sample{fSample{10, 10}},
+				[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}}, []chunks.Sample{fSample{0, 3, 3}, fSample{0, 8, 8}},
+				[]chunks.Sample{fSample{0, 7, 7}, fSample{0, 9, 9}}, []chunks.Sample{fSample{0, 10, 10}},
 			),
 		},
 		{
 			name: "two duplicated",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-				[]chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}},
-				[]chunks.Sample{fSample{2, 2}, fSample{3, 3}, fSample{5, 5}},
+				[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}},
+				[]chunks.Sample{fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}},
 			),
 		},
 		{
 			name: "three overlapping",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 2}, fSample{3, 3}, fSample{6, 6}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0}, fSample{4, 4}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 6, 6}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 4, 4}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-				[]chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}},
-				[]chunks.Sample{fSample{2, 2}, fSample{3, 3}, fSample{6, 6}},
-				[]chunks.Sample{fSample{0, 0}, fSample{4, 4}},
+				[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}},
+				[]chunks.Sample{fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 6, 6}},
+				[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 4, 4}},
 			),
 		},
 		{
 			name: "three in chained overlap",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{4, 4}, fSample{6, 66}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{6, 6}, fSample{10, 10}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 4, 4}, fSample{0, 6, 66}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 6, 6}, fSample{0, 10, 10}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-				[]chunks.Sample{fSample{1, 1}, fSample{2, 2}, fSample{3, 3}, fSample{5, 5}},
-				[]chunks.Sample{fSample{4, 4}, fSample{6, 66}},
-				[]chunks.Sample{fSample{6, 6}, fSample{10, 10}},
+				[]chunks.Sample{fSample{0, 1, 1}, fSample{0, 2, 2}, fSample{0, 3, 3}, fSample{0, 5, 5}},
+				[]chunks.Sample{fSample{0, 4, 4}, fSample{0, 6, 66}},
+				[]chunks.Sample{fSample{0, 6, 6}, fSample{0, 10, 10}},
 			),
 		},
 		{
 			name: "three in chained overlap complex",
 			input: []ChunkSeries{
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0}, fSample{5, 5}}, []chunks.Sample{fSample{10, 10}, fSample{15, 15}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{2, 2}, fSample{20, 20}}, []chunks.Sample{fSample{25, 25}, fSample{30, 30}}),
-				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{18, 18}, fSample{26, 26}}, []chunks.Sample{fSample{31, 31}, fSample{35, 35}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 0, 0}, fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 10, 10}, fSample{0, 15, 15}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 2, 2}, fSample{0, 20, 20}}, []chunks.Sample{fSample{0, 25, 25}, fSample{0, 30, 30}}),
+				NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"), []chunks.Sample{fSample{0, 18, 18}, fSample{0, 26, 26}}, []chunks.Sample{fSample{0, 31, 31}, fSample{0, 35, 35}}),
 			},
 			expected: NewListChunkSeriesFromSamples(labels.FromStrings("bar", "baz"),
-				[]chunks.Sample{fSample{0, 0}, fSample{5, 5}}, []chunks.Sample{fSample{10, 10}, fSample{15, 15}},
-				[]chunks.Sample{fSample{2, 2}, fSample{20, 20}}, []chunks.Sample{fSample{25, 25}, fSample{30, 30}},
-				[]chunks.Sample{fSample{18, 18}, fSample{26, 26}}, []chunks.Sample{fSample{31, 31}, fSample{35, 35}},
+				[]chunks.Sample{fSample{0, 0, 0}, fSample{0, 5, 5}}, []chunks.Sample{fSample{0, 10, 10}, fSample{0, 15, 15}},
+				[]chunks.Sample{fSample{0, 2, 2}, fSample{0, 20, 20}}, []chunks.Sample{fSample{0, 25, 25}, fSample{0, 30, 30}},
+				[]chunks.Sample{fSample{0, 18, 18}, fSample{0, 26, 26}}, []chunks.Sample{fSample{0, 31, 31}, fSample{0, 35, 35}},
 			),
 		},
 		{
@@ -1059,7 +1059,7 @@ func (*mockChunkSeriesSet) Warnings() annotations.Annotations { return nil }
 
 func TestChainSampleIterator(t *testing.T) {
 	for sampleType, sampleFunc := range map[string]func(int64) chunks.Sample{
-		"float":           func(ts int64) chunks.Sample { return fSample{ts, float64(ts)} },
+		"float":           func(ts int64) chunks.Sample { return fSample{0, ts, float64(ts)} },
 		"histogram":       func(ts int64) chunks.Sample { return histogramSample(ts, uk) },
 		"float histogram": func(ts int64) chunks.Sample { return floatHistogramSample(ts, uk) },
 	} {
@@ -1176,7 +1176,7 @@ func TestChainSampleIteratorHistogramCounterResetHint(t *testing.T) {
 
 func TestChainSampleIteratorSeek(t *testing.T) {
 	for sampleType, sampleFunc := range map[string]func(int64) chunks.Sample{
-		"float":           func(ts int64) chunks.Sample { return fSample{ts, float64(ts)} },
+		"float":           func(ts int64) chunks.Sample { return fSample{0, ts, float64(ts)} },
 		"histogram":       func(ts int64) chunks.Sample { return histogramSample(ts, uk) },
 		"float histogram": func(ts int64) chunks.Sample { return floatHistogramSample(ts, uk) },
 	} {
@@ -1224,13 +1224,13 @@ func TestChainSampleIteratorSeek(t *testing.T) {
 				switch merged.Seek(tc.seek) {
 				case chunkenc.ValFloat:
 					t, f := merged.At()
-					actual = append(actual, fSample{t, f})
+					actual = append(actual, fSample{0, t, f})
 				case chunkenc.ValHistogram:
 					t, h := merged.AtHistogram(nil)
-					actual = append(actual, hSample{t, h})
+					actual = append(actual, hSample{0, t, h})
 				case chunkenc.ValFloatHistogram:
 					t, fh := merged.AtFloatHistogram(nil)
-					actual = append(actual, fhSample{t, fh})
+					actual = append(actual, fhSample{0, t, fh})
 				}
 				s, err := ExpandSamples(merged, nil)
 				require.NoError(t, err)
@@ -1243,7 +1243,7 @@ func TestChainSampleIteratorSeek(t *testing.T) {
 
 func TestChainSampleIteratorSeekFailingIterator(t *testing.T) {
 	merged := ChainSampleIteratorFromIterators(nil, []chunkenc.Iterator{
-		NewListSeriesIterator(samples{fSample{0, 0.1}, fSample{1, 1.1}, fSample{2, 2.1}}),
+		NewListSeriesIterator(samples{fSample{0, 0, 0.1}, fSample{0, 1, 1.1}, fSample{0, 2, 2.1}}),
 		errIterator{errors.New("something went wrong")},
 	})
 
@@ -1253,7 +1253,7 @@ func TestChainSampleIteratorSeekFailingIterator(t *testing.T) {
 
 func TestChainSampleIteratorNextImmediatelyFailingIterator(t *testing.T) {
 	merged := ChainSampleIteratorFromIterators(nil, []chunkenc.Iterator{
-		NewListSeriesIterator(samples{fSample{0, 0.1}, fSample{1, 1.1}, fSample{2, 2.1}}),
+		NewListSeriesIterator(samples{fSample{0, 0, 0.1}, fSample{0, 1, 1.1}, fSample{0, 2, 2.1}}),
 		errIterator{errors.New("something went wrong")},
 	})
 
@@ -1263,7 +1263,7 @@ func TestChainSampleIteratorNextImmediatelyFailingIterator(t *testing.T) {
 	// Next() does some special handling for the first iterator, so make sure it handles the first iterator returning an error too.
 	merged = ChainSampleIteratorFromIterators(nil, []chunkenc.Iterator{
 		errIterator{errors.New("something went wrong")},
-		NewListSeriesIterator(samples{fSample{0, 0.1}, fSample{1, 1.1}, fSample{2, 2.1}}),
+		NewListSeriesIterator(samples{fSample{0, 0, 0.1}, fSample{0, 1, 1.1}, fSample{0, 2, 2.1}}),
 	})
 
 	require.Equal(t, chunkenc.ValNone, merged.Next())
@@ -1310,13 +1310,13 @@ func TestChainSampleIteratorSeekHistogramCounterResetHint(t *testing.T) {
 				switch merged.Seek(tc.seek) {
 				case chunkenc.ValFloat:
 					t, f := merged.At()
-					actual = append(actual, fSample{t, f})
+					actual = append(actual, fSample{0, t, f})
 				case chunkenc.ValHistogram:
 					t, h := merged.AtHistogram(nil)
-					actual = append(actual, hSample{t, h})
+					actual = append(actual, hSample{0, t, h})
 				case chunkenc.ValFloatHistogram:
 					t, fh := merged.AtFloatHistogram(nil)
-					actual = append(actual, fhSample{t, fh})
+					actual = append(actual, fhSample{0, t, fh})
 				}
 				s, err := ExpandSamples(merged, nil)
 				require.NoError(t, err)
diff --git a/storage/series.go b/storage/series.go
index d114438078..c16e628ba2 100644
--- a/storage/series.go
+++ b/storage/series.go
@@ -452,11 +452,11 @@ func ExpandSamples(iter chunkenc.Iterator, newSampleFn func(t int64, f float64,
 		newSampleFn = func(t int64, f float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample {
 			switch {
 			case h != nil:
-				return hSample{t, h}
+				return hSample{0, t, h}
 			case fh != nil:
-				return fhSample{t, fh}
+				return fhSample{0, t, fh}
 			default:
-				return fSample{t, f}
+				return fSample{0, t, f}
 			}
 		}
 	}
diff --git a/storage/series_test.go b/storage/series_test.go
index 954d62f1b3..3ad84be6b0 100644
--- a/storage/series_test.go
+++ b/storage/series_test.go
@@ -28,11 +28,11 @@ import (
 
 func TestListSeriesIterator(t *testing.T) {
 	it := NewListSeriesIterator(samples{
-		fSample{0, 0},
-		fSample{1, 1},
-		fSample{1, 1.5},
-		fSample{2, 2},
-		fSample{3, 3},
+		fSample{0, 0, 0},
+		fSample{0, 1, 1},
+		fSample{0, 1, 1.5},
+		fSample{0, 2, 2},
+		fSample{0, 3, 3},
 	})
 
 	// Seek to the first sample with ts=1.

From f616689f0980eb94e885ffd02e42fb59314b15db Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 09:46:56 +0100
Subject: [PATCH 255/439] tsdb/head.go: add start timestamp to sample type used
 in tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: György Krajcsovits 
---
 tsdb/head.go | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tsdb/head.go b/tsdb/head.go
index 213846aa35..8db9231124 100644
--- a/tsdb/head.go
+++ b/tsdb/head.go
@@ -2101,20 +2101,20 @@ func (s *stripeSeries) postCreation(lset labels.Labels) {
 }
 
 type sample struct {
+	st int64
 	t  int64
 	f  float64
 	h  *histogram.Histogram
 	fh *histogram.FloatHistogram
 }
 
-func newSample(t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample {
-	return sample{t, v, h, fh}
+func newSample(st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample {
+	return sample{st, t, v, h, fh}
 }
 
 func (s sample) T() int64 { return s.t }
 
-// TODO(krajorama): implement ST.
-func (sample) ST() int64                       { return 0 }
+func (s sample) ST() int64                     { return s.st }
 func (s sample) F() float64                    { return s.f }
 func (s sample) H() *histogram.Histogram       { return s.h }
 func (s sample) FH() *histogram.FloatHistogram { return s.fh }

From 28dca34f4ff257ec74121736ab29a600424d0dce Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 09:55:45 +0100
Subject: [PATCH 256/439] auto update head sample use in tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

find . -name "*.go" -type f -exec sed -E -i \
's/([^[:alpha:]]sample\{)([^,{:]+,[^,]+,[^,]+,[^,]+\})/\10, \2/g' {} +

I've omitted tsdb/ooo_head.go from the commit because I'm also adding todo
there.

Signed-off-by: György Krajcsovits 
---
 tsdb/block_test.go          |  16 +-
 tsdb/db_append_v2_test.go   |  16 +-
 tsdb/db_test.go             |  36 +-
 tsdb/head_append_v2_test.go |  22 +-
 tsdb/head_test.go           |  22 +-
 tsdb/querier_test.go        | 722 ++++++++++++++++++------------------
 6 files changed, 417 insertions(+), 417 deletions(-)

diff --git a/tsdb/block_test.go b/tsdb/block_test.go
index 855fa5638a..edd2df7415 100644
--- a/tsdb/block_test.go
+++ b/tsdb/block_test.go
@@ -176,7 +176,7 @@ func TestCorruptedChunk(t *testing.T) {
 		t.Run(tc.name, func(t *testing.T) {
 			tmpdir := t.TempDir()
 
-			series := storage.NewListSeries(labels.FromStrings("a", "b"), []chunks.Sample{sample{1, 1, nil, nil}})
+			series := storage.NewListSeries(labels.FromStrings("a", "b"), []chunks.Sample{sample{0, 1, 1, nil, nil}})
 			blockDir := createBlock(t, tmpdir, []storage.Series{series})
 			files, err := sequenceFiles(chunkDir(blockDir))
 			require.NoError(t, err)
@@ -236,7 +236,7 @@ func TestLabelValuesWithMatchers(t *testing.T) {
 		seriesEntries = append(seriesEntries, storage.NewListSeries(labels.FromStrings(
 			"tens", fmt.Sprintf("value%d", i/10),
 			"unique", fmt.Sprintf("value%d", i),
-		), []chunks.Sample{sample{100, 0, nil, nil}}))
+		), []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 	}
 
 	blockDir := createBlock(t, tmpdir, seriesEntries)
@@ -319,7 +319,7 @@ func TestBlockQuerierReturnsSortedLabelValues(t *testing.T) {
 	for i := 100; i > 0; i-- {
 		seriesEntries = append(seriesEntries, storage.NewListSeries(labels.FromStrings(
 			"__name__", fmt.Sprintf("value%d", i),
-		), []chunks.Sample{sample{100, 0, nil, nil}}))
+		), []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 	}
 
 	blockDir := createBlock(t, tmpdir, seriesEntries)
@@ -436,7 +436,7 @@ func BenchmarkLabelValuesWithMatchers(b *testing.B) {
 			"a_unique", fmt.Sprintf("value%d", i),
 			"b_tens", fmt.Sprintf("value%d", i/(metricCount/10)),
 			"c_ninety", fmt.Sprintf("value%d", i/(metricCount/10)/9), // "0" for the first 90%, then "1"
-		), []chunks.Sample{sample{100, 0, nil, nil}}))
+		), []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 	}
 
 	blockDir := createBlock(b, tmpdir, seriesEntries)
@@ -472,13 +472,13 @@ func TestLabelNamesWithMatchers(t *testing.T) {
 	for i := range 100 {
 		seriesEntries = append(seriesEntries, storage.NewListSeries(labels.FromStrings(
 			"unique", fmt.Sprintf("value%d", i),
-		), []chunks.Sample{sample{100, 0, nil, nil}}))
+		), []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 
 		if i%10 == 0 {
 			seriesEntries = append(seriesEntries, storage.NewListSeries(labels.FromStrings(
 				"tens", fmt.Sprintf("value%d", i/10),
 				"unique", fmt.Sprintf("value%d", i),
-			), []chunks.Sample{sample{100, 0, nil, nil}}))
+			), []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 		}
 
 		if i%20 == 0 {
@@ -486,7 +486,7 @@ func TestLabelNamesWithMatchers(t *testing.T) {
 				"tens", fmt.Sprintf("value%d", i/10),
 				"twenties", fmt.Sprintf("value%d", i/20),
 				"unique", fmt.Sprintf("value%d", i),
-			), []chunks.Sample{sample{100, 0, nil, nil}}))
+			), []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 		}
 	}
 
@@ -542,7 +542,7 @@ func TestBlockIndexReader_PostingsForLabelMatching(t *testing.T) {
 	testPostingsForLabelMatching(t, 2, func(t *testing.T, series []labels.Labels) IndexReader {
 		var seriesEntries []storage.Series
 		for _, s := range series {
-			seriesEntries = append(seriesEntries, storage.NewListSeries(s, []chunks.Sample{sample{100, 0, nil, nil}}))
+			seriesEntries = append(seriesEntries, storage.NewListSeries(s, []chunks.Sample{sample{0, 100, 0, nil, nil}}))
 		}
 
 		blockDir := createBlock(t, t.TempDir(), seriesEntries)
diff --git a/tsdb/db_append_v2_test.go b/tsdb/db_append_v2_test.go
index 344b1d6943..16134e8c93 100644
--- a/tsdb/db_append_v2_test.go
+++ b/tsdb/db_append_v2_test.go
@@ -372,7 +372,7 @@ func TestDeleteSimple_AppendV2(t *testing.T) {
 
 			expSamples := make([]chunks.Sample, 0, len(c.remaint))
 			for _, ts := range c.remaint {
-				expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil})
+				expSamples = append(expSamples, sample{0, ts, smpls[ts], nil, nil})
 			}
 
 			expss := newMockSeriesSet([]storage.Series{
@@ -507,7 +507,7 @@ func TestSkippingInvalidValuesInSameTxn_AppendV2(t *testing.T) {
 	ssMap := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))
 
 	require.Equal(t, map[string][]chunks.Sample{
-		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 1, nil, nil}},
+		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 0, 1, nil, nil}},
 	}, ssMap)
 
 	// Append Out of Order Value.
@@ -524,7 +524,7 @@ func TestSkippingInvalidValuesInSameTxn_AppendV2(t *testing.T) {
 	ssMap = query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))
 
 	require.Equal(t, map[string][]chunks.Sample{
-		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 1, nil, nil}, sample{10, 3, nil, nil}},
+		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 0, 1, nil, nil}, sample{0, 10, 3, nil, nil}},
 	}, ssMap)
 }
 
@@ -669,7 +669,7 @@ func TestDB_SnapshotWithDelete_AppendV2(t *testing.T) {
 
 			expSamples := make([]chunks.Sample, 0, len(c.remaint))
 			for _, ts := range c.remaint {
-				expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil})
+				expSamples = append(expSamples, sample{0, ts, smpls[ts], nil, nil})
 			}
 
 			expss := newMockSeriesSet([]storage.Series{
@@ -772,7 +772,7 @@ func TestDB_e2e_AppendV2(t *testing.T) {
 		for range numDatapoints {
 			v := rand.Float64()
 
-			series = append(series, sample{ts, v, nil, nil})
+			series = append(series, sample{0, ts, v, nil, nil})
 
 			_, err := app.Append(0, lset, 0, ts, v, nil, nil, storage.AOptions{})
 			require.NoError(t, err)
@@ -1094,7 +1094,7 @@ func TestTombstoneClean_AppendV2(t *testing.T) {
 
 		expSamples := make([]chunks.Sample, 0, len(c.remaint))
 		for _, ts := range c.remaint {
-			expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil})
+			expSamples = append(expSamples, sample{0, ts, smpls[ts], nil, nil})
 		}
 
 		expss := newMockSeriesSet([]storage.Series{
@@ -2310,7 +2310,7 @@ func TestCompactHead_AppendV2(t *testing.T) {
 		val := rand.Float64()
 		_, err := app.Append(0, labels.FromStrings("a", "b"), 0, int64(i), val, nil, nil, storage.AOptions{})
 		require.NoError(t, err)
-		expSamples = append(expSamples, sample{int64(i), val, nil, nil})
+		expSamples = append(expSamples, sample{0, int64(i), val, nil, nil})
 	}
 	require.NoError(t, app.Commit())
 
@@ -2337,7 +2337,7 @@ func TestCompactHead_AppendV2(t *testing.T) {
 		series = seriesSet.At().Iterator(series)
 		for series.Next() == chunkenc.ValFloat {
 			time, val := series.At()
-			actSamples = append(actSamples, sample{time, val, nil, nil})
+			actSamples = append(actSamples, sample{0, time, val, nil, nil})
 		}
 		require.NoError(t, series.Err())
 	}
diff --git a/tsdb/db_test.go b/tsdb/db_test.go
index 299ade8826..5e57982b5d 100644
--- a/tsdb/db_test.go
+++ b/tsdb/db_test.go
@@ -546,7 +546,7 @@ func TestDeleteSimple(t *testing.T) {
 
 			expSamples := make([]chunks.Sample, 0, len(c.remaint))
 			for _, ts := range c.remaint {
-				expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil})
+				expSamples = append(expSamples, sample{0, ts, smpls[ts], nil, nil})
 			}
 
 			expss := newMockSeriesSet([]storage.Series{
@@ -691,7 +691,7 @@ func TestSkippingInvalidValuesInSameTxn(t *testing.T) {
 	ssMap := query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))
 
 	require.Equal(t, map[string][]chunks.Sample{
-		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 1, nil, nil}},
+		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 0, 1, nil, nil}},
 	}, ssMap)
 
 	// Append Out of Order Value.
@@ -708,7 +708,7 @@ func TestSkippingInvalidValuesInSameTxn(t *testing.T) {
 	ssMap = query(t, q, labels.MustNewMatcher(labels.MatchEqual, "a", "b"))
 
 	require.Equal(t, map[string][]chunks.Sample{
-		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 1, nil, nil}, sample{10, 3, nil, nil}},
+		labels.New(labels.Label{Name: "a", Value: "b"}).String(): {sample{0, 0, 1, nil, nil}, sample{0, 10, 3, nil, nil}},
 	}, ssMap)
 }
 
@@ -853,7 +853,7 @@ func TestDB_SnapshotWithDelete(t *testing.T) {
 
 			expSamples := make([]chunks.Sample, 0, len(c.remaint))
 			for _, ts := range c.remaint {
-				expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil})
+				expSamples = append(expSamples, sample{0, ts, smpls[ts], nil, nil})
 			}
 
 			expss := newMockSeriesSet([]storage.Series{
@@ -956,7 +956,7 @@ func TestDB_e2e(t *testing.T) {
 		for range numDatapoints {
 			v := rand.Float64()
 
-			series = append(series, sample{ts, v, nil, nil})
+			series = append(series, sample{0, ts, v, nil, nil})
 
 			_, err := app.Append(0, lset, ts, v)
 			require.NoError(t, err)
@@ -1278,7 +1278,7 @@ func TestTombstoneClean(t *testing.T) {
 
 		expSamples := make([]chunks.Sample, 0, len(c.remaint))
 		for _, ts := range c.remaint {
-			expSamples = append(expSamples, sample{ts, smpls[ts], nil, nil})
+			expSamples = append(expSamples, sample{0, ts, smpls[ts], nil, nil})
 		}
 
 		expss := newMockSeriesSet([]storage.Series{
@@ -2863,11 +2863,11 @@ func assureChunkFromSamples(t *testing.T, samples []chunks.Sample) chunks.Meta {
 // TestChunkWriter_ReadAfterWrite ensures that chunk segment are cut at the set segment size and
 // that the resulted segments includes the expected chunks data.
 func TestChunkWriter_ReadAfterWrite(t *testing.T) {
-	chk1 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 1, nil, nil}})
-	chk2 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 2, nil, nil}})
-	chk3 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 3, nil, nil}})
-	chk4 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 4, nil, nil}})
-	chk5 := assureChunkFromSamples(t, []chunks.Sample{sample{1, 5, nil, nil}})
+	chk1 := assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 1, nil, nil}})
+	chk2 := assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 2, nil, nil}})
+	chk3 := assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 3, nil, nil}})
+	chk4 := assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 4, nil, nil}})
+	chk5 := assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 5, nil, nil}})
 	chunkSize := len(chk1.Chunk.Bytes()) + chunks.MaxChunkLengthFieldSize + chunks.ChunkEncodingSize + crc32.Size
 
 	tests := []struct {
@@ -3069,11 +3069,11 @@ func TestRangeForTimestamp(t *testing.T) {
 func TestChunkReader_ConcurrentReads(t *testing.T) {
 	t.Parallel()
 	chks := []chunks.Meta{
-		assureChunkFromSamples(t, []chunks.Sample{sample{1, 1, nil, nil}}),
-		assureChunkFromSamples(t, []chunks.Sample{sample{1, 2, nil, nil}}),
-		assureChunkFromSamples(t, []chunks.Sample{sample{1, 3, nil, nil}}),
-		assureChunkFromSamples(t, []chunks.Sample{sample{1, 4, nil, nil}}),
-		assureChunkFromSamples(t, []chunks.Sample{sample{1, 5, nil, nil}}),
+		assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 1, nil, nil}}),
+		assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 2, nil, nil}}),
+		assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 3, nil, nil}}),
+		assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 4, nil, nil}}),
+		assureChunkFromSamples(t, []chunks.Sample{sample{0, 1, 5, nil, nil}}),
 	}
 
 	tempDir := t.TempDir()
@@ -3133,7 +3133,7 @@ func TestCompactHead(t *testing.T) {
 		val := rand.Float64()
 		_, err := app.Append(0, labels.FromStrings("a", "b"), int64(i), val)
 		require.NoError(t, err)
-		expSamples = append(expSamples, sample{int64(i), val, nil, nil})
+		expSamples = append(expSamples, sample{0, int64(i), val, nil, nil})
 	}
 	require.NoError(t, app.Commit())
 
@@ -3160,7 +3160,7 @@ func TestCompactHead(t *testing.T) {
 		series = seriesSet.At().Iterator(series)
 		for series.Next() == chunkenc.ValFloat {
 			time, val := series.At()
-			actSamples = append(actSamples, sample{time, val, nil, nil})
+			actSamples = append(actSamples, sample{0, time, val, nil, nil})
 		}
 		require.NoError(t, series.Err())
 	}
diff --git a/tsdb/head_append_v2_test.go b/tsdb/head_append_v2_test.go
index 33bc3aec38..892a5b3bfe 100644
--- a/tsdb/head_append_v2_test.go
+++ b/tsdb/head_append_v2_test.go
@@ -312,8 +312,8 @@ func TestHeadAppenderV2_WALMultiRef(t *testing.T) {
 	// The samples before the new ref should be discarded since Head truncation
 	// happens only after compacting the Head.
 	require.Equal(t, map[string][]chunks.Sample{`{foo="bar"}`: {
-		sample{1700, 3, nil, nil},
-		sample{2000, 4, nil, nil},
+		sample{0, 1700, 3, nil, nil},
+		sample{0, 2000, 4, nil, nil},
 	}}, series)
 }
 
@@ -605,7 +605,7 @@ func TestHeadAppenderV2_DeleteUntilCurrMax(t *testing.T) {
 	it = exps.Iterator(nil)
 	resSamples, err := storage.ExpandSamples(it, newSample)
 	require.NoError(t, err)
-	require.Equal(t, []chunks.Sample{sample{11, 1, nil, nil}}, resSamples)
+	require.Equal(t, []chunks.Sample{sample{0, 11, 1, nil, nil}}, resSamples)
 	for res.Next() {
 	}
 	require.NoError(t, res.Err())
@@ -722,7 +722,7 @@ func TestHeadAppenderV2_Delete_e2e(t *testing.T) {
 			v := rand.Float64()
 			_, err := app.Append(0, ls, 0, ts, v, nil, nil, storage.AOptions{})
 			require.NoError(t, err)
-			series = append(series, sample{ts, v, nil, nil})
+			series = append(series, sample{0, ts, v, nil, nil})
 			ts += rand.Int63n(timeInterval) + 1
 		}
 		seriesMap[labels.New(l...).String()] = series
@@ -1520,7 +1520,7 @@ func TestDataMissingOnQueryDuringCompaction_AppenderV2(t *testing.T) {
 		ref, err = app.Append(ref, labels.FromStrings("a", "b"), 0, ts, float64(i), nil, nil, storage.AOptions{})
 		require.NoError(t, err)
 		maxt = ts
-		expSamples = append(expSamples, sample{ts, float64(i), nil, nil})
+		expSamples = append(expSamples, sample{0, ts, float64(i), nil, nil})
 	}
 	require.NoError(t, app.Commit())
 
@@ -2166,17 +2166,17 @@ func TestChunkSnapshot_AppenderV2(t *testing.T) {
 					aOpts.Exemplars = []exemplar.Exemplar{newExemplar(lbls, ts)}
 				}
 				val := rand.Float64()
-				expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil})
+				expSeries[lblStr] = append(expSeries[lblStr], sample{0, ts, val, nil, nil})
 				_, err := app.Append(0, lbls, 0, ts, val, nil, nil, aOpts)
 				require.NoError(t, err)
 
 				hist := histograms[int(ts)]
-				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil})
+				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{0, ts, 0, hist, nil})
 				_, err = app.Append(0, lblsHist, 0, ts, 0, hist, nil, storage.AOptions{})
 				require.NoError(t, err)
 
 				floatHist := floatHistogram[int(ts)]
-				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist})
+				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{0, ts, 0, nil, floatHist})
 				_, err = app.Append(0, lblsFloatHist, 0, ts, 0, nil, floatHist, storage.AOptions{})
 				require.NoError(t, err)
 
@@ -2244,17 +2244,17 @@ func TestChunkSnapshot_AppenderV2(t *testing.T) {
 					aOpts.Exemplars = []exemplar.Exemplar{newExemplar(lbls, ts)}
 				}
 				val := rand.Float64()
-				expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil})
+				expSeries[lblStr] = append(expSeries[lblStr], sample{0, ts, val, nil, nil})
 				_, err := app.Append(0, lbls, 0, ts, val, nil, nil, aOpts)
 				require.NoError(t, err)
 
 				hist := histograms[int(ts)]
-				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil})
+				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{0, ts, 0, hist, nil})
 				_, err = app.Append(0, lblsHist, 0, ts, 0, hist, nil, storage.AOptions{})
 				require.NoError(t, err)
 
 				floatHist := floatHistogram[int(ts)]
-				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist})
+				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{0, ts, 0, nil, floatHist})
 				_, err = app.Append(0, lblsFloatHist, 0, ts, 0, nil, floatHist, storage.AOptions{})
 				require.NoError(t, err)
 
diff --git a/tsdb/head_test.go b/tsdb/head_test.go
index acdf0ee000..d0928d64bf 100644
--- a/tsdb/head_test.go
+++ b/tsdb/head_test.go
@@ -841,8 +841,8 @@ func TestHead_WALMultiRef(t *testing.T) {
 	// The samples before the new ref should be discarded since Head truncation
 	// happens only after compacting the Head.
 	require.Equal(t, map[string][]chunks.Sample{`{foo="bar"}`: {
-		sample{1700, 3, nil, nil},
-		sample{2000, 4, nil, nil},
+		sample{0, 1700, 3, nil, nil},
+		sample{0, 2000, 4, nil, nil},
 	}}, series)
 }
 
@@ -1859,7 +1859,7 @@ func TestDeleteUntilCurMax(t *testing.T) {
 	it = exps.Iterator(nil)
 	resSamples, err := storage.ExpandSamples(it, newSample)
 	require.NoError(t, err)
-	require.Equal(t, []chunks.Sample{sample{11, 1, nil, nil}}, resSamples)
+	require.Equal(t, []chunks.Sample{sample{0, 11, 1, nil, nil}}, resSamples)
 	for res.Next() {
 	}
 	require.NoError(t, res.Err())
@@ -1976,7 +1976,7 @@ func TestDelete_e2e(t *testing.T) {
 			v := rand.Float64()
 			_, err := app.Append(0, ls, ts, v)
 			require.NoError(t, err)
-			series = append(series, sample{ts, v, nil, nil})
+			series = append(series, sample{0, ts, v, nil, nil})
 			ts += rand.Int63n(timeInterval) + 1
 		}
 		seriesMap[labels.New(l...).String()] = series
@@ -3838,7 +3838,7 @@ func TestDataMissingOnQueryDuringCompaction(t *testing.T) {
 		ref, err = app.Append(ref, labels.FromStrings("a", "b"), ts, float64(i))
 		require.NoError(t, err)
 		maxt = ts
-		expSamples = append(expSamples, sample{ts, float64(i), nil, nil})
+		expSamples = append(expSamples, sample{0, ts, float64(i), nil, nil})
 	}
 	require.NoError(t, app.Commit())
 
@@ -4503,17 +4503,17 @@ func TestChunkSnapshot(t *testing.T) {
 			// 240 samples should m-map at least 1 chunk.
 			for ts := int64(1); ts <= 240; ts++ {
 				val := rand.Float64()
-				expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil})
+				expSeries[lblStr] = append(expSeries[lblStr], sample{0, ts, val, nil, nil})
 				ref, err := app.Append(0, lbls, ts, val)
 				require.NoError(t, err)
 
 				hist := histograms[int(ts)]
-				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil})
+				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{0, ts, 0, hist, nil})
 				_, err = app.AppendHistogram(0, lblsHist, ts, hist, nil)
 				require.NoError(t, err)
 
 				floatHist := floatHistogram[int(ts)]
-				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist})
+				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{0, ts, 0, nil, floatHist})
 				_, err = app.AppendHistogram(0, lblsFloatHist, ts, nil, floatHist)
 				require.NoError(t, err)
 
@@ -4577,17 +4577,17 @@ func TestChunkSnapshot(t *testing.T) {
 			// 240 samples should m-map at least 1 chunk.
 			for ts := int64(241); ts <= 480; ts++ {
 				val := rand.Float64()
-				expSeries[lblStr] = append(expSeries[lblStr], sample{ts, val, nil, nil})
+				expSeries[lblStr] = append(expSeries[lblStr], sample{0, ts, val, nil, nil})
 				ref, err := app.Append(0, lbls, ts, val)
 				require.NoError(t, err)
 
 				hist := histograms[int(ts)]
-				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{ts, 0, hist, nil})
+				expHist[lblsHistStr] = append(expHist[lblsHistStr], sample{0, ts, 0, hist, nil})
 				_, err = app.AppendHistogram(0, lblsHist, ts, hist, nil)
 				require.NoError(t, err)
 
 				floatHist := floatHistogram[int(ts)]
-				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{ts, 0, nil, floatHist})
+				expFloatHist[lblsFloatHistStr] = append(expFloatHist[lblsFloatHistStr], sample{0, ts, 0, nil, floatHist})
 				_, err = app.AppendHistogram(0, lblsFloatHist, ts, nil, floatHist)
 				require.NoError(t, err)
 
diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go
index 57a53c46fe..9ff5124074 100644
--- a/tsdb/querier_test.go
+++ b/tsdb/querier_test.go
@@ -318,24 +318,24 @@ func TestBlockQuerier(t *testing.T) {
 			ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "a", ".*")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("b", "b"),
-					[]chunks.Sample{sample{1, 3, nil, nil}, sample{2, 2, nil, nil}, sample{3, 6, nil, nil}, sample{5, 1, nil, nil}, sample{6, 7, nil, nil}, sample{7, 2, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 3, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 6, nil, nil}, sample{0, 5, 1, nil, nil}, sample{0, 6, 7, nil, nil}, sample{0, 7, 2, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 4, nil, nil}}, []chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}}, []chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}}, []chunks.Sample{sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}}, []chunks.Sample{sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("b", "b"),
-					[]chunks.Sample{sample{1, 3, nil, nil}, sample{2, 2, nil, nil}, sample{3, 6, nil, nil}}, []chunks.Sample{sample{5, 1, nil, nil}, sample{6, 7, nil, nil}, sample{7, 2, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 3, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 6, nil, nil}}, []chunks.Sample{sample{0, 5, 1, nil, nil}, sample{0, 6, 7, nil, nil}, sample{0, 7, 2, nil, nil}},
 				),
 			}),
 		},
@@ -345,18 +345,18 @@ func TestBlockQuerier(t *testing.T) {
 			ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "a")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{2, 3, nil, nil}, sample{3, 4, nil, nil}}, []chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}}, []chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{2, 2, nil, nil}, sample{3, 3, nil, nil}}, []chunks.Sample{sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}}, []chunks.Sample{sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 		},
@@ -369,20 +369,20 @@ func TestBlockQuerier(t *testing.T) {
 			ms:    []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "a")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 4, nil, nil}},
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}},
-					[]chunks.Sample{sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 		},
@@ -395,18 +395,18 @@ func TestBlockQuerier(t *testing.T) {
 			ms:    []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "a")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 		},
@@ -454,24 +454,24 @@ func TestBlockQuerier_AgainstHeadWithOpenChunks(t *testing.T) {
 			ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "a", ".*")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("b", "b"),
-					[]chunks.Sample{sample{1, 3, nil, nil}, sample{2, 2, nil, nil}, sample{3, 6, nil, nil}, sample{5, 1, nil, nil}, sample{6, 7, nil, nil}, sample{7, 2, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 3, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 6, nil, nil}, sample{0, 5, 1, nil, nil}, sample{0, 6, 7, nil, nil}, sample{0, 7, 2, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("b", "b"),
-					[]chunks.Sample{sample{1, 3, nil, nil}, sample{2, 2, nil, nil}, sample{3, 6, nil, nil}, sample{5, 1, nil, nil}, sample{6, 7, nil, nil}, sample{7, 2, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 3, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 6, nil, nil}, sample{0, 5, 1, nil, nil}, sample{0, 6, 7, nil, nil}, sample{0, 7, 2, nil, nil}},
 				),
 			}),
 		},
@@ -481,18 +481,18 @@ func TestBlockQuerier_AgainstHeadWithOpenChunks(t *testing.T) {
 			ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "a")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{2, 3, nil, nil}, sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 3, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{2, 2, nil, nil}, sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+					[]chunks.Sample{sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 				),
 			}),
 		},
@@ -537,18 +537,18 @@ func TestBlockQuerier_TrimmingDoesNotModifyOriginalTombstoneIntervals(t *testing
 		ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "a", "a")},
 		exp: newMockSeriesSet([]storage.Series{
 			storage.NewListSeries(labels.FromStrings("a", "a"),
-				[]chunks.Sample{sample{3, 4, nil, nil}, sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+				[]chunks.Sample{sample{0, 3, 4, nil, nil}, sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 			),
 			storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-				[]chunks.Sample{sample{3, 3, nil, nil}, sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+				[]chunks.Sample{sample{0, 3, 3, nil, nil}, sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 			),
 		}),
 		expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 			storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-				[]chunks.Sample{sample{3, 4, nil, nil}}, []chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+				[]chunks.Sample{sample{0, 3, 4, nil, nil}}, []chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 			),
 			storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-				[]chunks.Sample{sample{3, 3, nil, nil}}, []chunks.Sample{sample{5, 3, nil, nil}, sample{6, 6, nil, nil}},
+				[]chunks.Sample{sample{0, 3, 3, nil, nil}}, []chunks.Sample{sample{0, 5, 3, nil, nil}, sample{0, 6, 6, nil, nil}},
 			),
 		}),
 	}
@@ -636,24 +636,24 @@ func TestBlockQuerierDelete(t *testing.T) {
 			ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchRegexp, "a", ".*")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{5, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("b", "b"),
-					[]chunks.Sample{sample{1, 3, nil, nil}, sample{2, 2, nil, nil}, sample{3, 6, nil, nil}, sample{5, 1, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 3, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 6, nil, nil}, sample{0, 5, 1, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}, sample{7, 4, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}, sample{0, 7, 4, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{5, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("b", "b"),
-					[]chunks.Sample{sample{1, 3, nil, nil}, sample{2, 2, nil, nil}, sample{3, 6, nil, nil}}, []chunks.Sample{sample{5, 1, nil, nil}},
+					[]chunks.Sample{sample{0, 1, 3, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 6, nil, nil}}, []chunks.Sample{sample{0, 5, 1, nil, nil}},
 				),
 			}),
 		},
@@ -663,18 +663,18 @@ func TestBlockQuerierDelete(t *testing.T) {
 			ms:   []*labels.Matcher{labels.MustNewMatcher(labels.MatchEqual, "a", "a")},
 			exp: newMockSeriesSet([]storage.Series{
 				storage.NewListSeries(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 				),
 				storage.NewListSeries(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{5, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}},
 				),
 			}),
 			expChks: newMockChunkSeriesSet([]storage.ChunkSeries{
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a"),
-					[]chunks.Sample{sample{5, 2, nil, nil}, sample{6, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 2, nil, nil}, sample{0, 6, 3, nil, nil}},
 				),
 				storage.NewListChunkSeriesFromSamples(labels.FromStrings("a", "a", "b", "b"),
-					[]chunks.Sample{sample{5, 3, nil, nil}},
+					[]chunks.Sample{sample{0, 5, 3, nil, nil}},
 				),
 			}),
 		},
@@ -875,15 +875,15 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "one chunk",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
 			},
 
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil},
+				sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil},
+					sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}},
@@ -891,19 +891,19 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two full chunks",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+				sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil},
+					sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+					sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}, {7, 9}},
@@ -911,23 +911,23 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "three full chunks",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
-				{sample{10, 22, nil, nil}, sample{203, 3493, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
+				{sample{0, 10, 22, nil, nil}, sample{0, 203, 3493, nil, nil}},
 			},
 
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil}, sample{9, 8, nil, nil}, sample{10, 22, nil, nil}, sample{203, 3493, nil, nil},
+				sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}, sample{0, 10, 22, nil, nil}, sample{0, 203, 3493, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil},
+					sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+					sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{10, 22, nil, nil}, sample{203, 3493, nil, nil},
+					sample{0, 10, 22, nil, nil}, sample{0, 203, 3493, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}, {7, 9}, {10, 203}},
@@ -943,8 +943,8 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks and seek beyond chunks",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			seek: 10,
 
@@ -953,27 +953,27 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks and seek on middle of first chunk",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			seek: 2,
 
 			seekSuccess: true,
 			expected: []chunks.Sample{
-				sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+				sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 			},
 		},
 		{
 			name: "two chunks and seek before first chunk",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			seek: -32,
 
 			seekSuccess: true,
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+				sample{0, 1, 2, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 			},
 		},
 		// Deletion / Trim cases.
@@ -985,20 +985,20 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks with trimmed first and last samples from edge chunks",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			intervals: tombstones.Intervals{{Mint: math.MinInt64, Maxt: 2}}.Add(tombstones.Interval{Mint: 9, Maxt: math.MaxInt64}),
 
 			expected: []chunks.Sample{
-				sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil},
+				sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{3, 5, nil, nil}, sample{6, 1, nil, nil},
+					sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 89, nil, nil},
+					sample{0, 7, 89, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{3, 6}, {7, 7}},
@@ -1006,20 +1006,20 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks with trimmed middle sample of first chunk",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			intervals: tombstones.Intervals{{Mint: 2, Maxt: 3}},
 
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+				sample{0, 1, 2, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 2, nil, nil}, sample{6, 1, nil, nil},
+					sample{0, 1, 2, nil, nil}, sample{0, 6, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+					sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}, {7, 9}},
@@ -1027,20 +1027,20 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks with deletion across two chunks",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			intervals: tombstones.Intervals{{Mint: 6, Maxt: 7}},
 
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{9, 8, nil, nil},
+				sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 9, 8, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil},
+					sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{9, 8, nil, nil},
+					sample{0, 9, 8, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 3}, {9, 9}},
@@ -1048,17 +1048,17 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks with first chunk deleted",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			intervals: tombstones.Intervals{{Mint: 1, Maxt: 6}},
 
 			expected: []chunks.Sample{
-				sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+				sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 89, nil, nil}, sample{9, 8, nil, nil},
+					sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{7, 9}},
@@ -1067,22 +1067,22 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "two chunks with trimmed first and last samples from edge chunks, seek from middle of first chunk",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			intervals: tombstones.Intervals{{Mint: math.MinInt64, Maxt: 2}}.Add(tombstones.Interval{Mint: 9, Maxt: math.MaxInt64}),
 
 			seek:        3,
 			seekSuccess: true,
 			expected: []chunks.Sample{
-				sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 89, nil, nil},
+				sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 89, nil, nil},
 			},
 		},
 		{
 			name: "one chunk where all samples are trimmed",
 			samples: [][]chunks.Sample{
-				{sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
-				{sample{7, 89, nil, nil}, sample{9, 8, nil, nil}},
+				{sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
+				{sample{0, 7, 89, nil, nil}, sample{0, 9, 8, nil, nil}},
 			},
 			intervals: tombstones.Intervals{{Mint: math.MinInt64, Maxt: 3}}.Add(tombstones.Interval{Mint: 4, Maxt: math.MaxInt64}),
 
@@ -1093,24 +1093,24 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one histogram chunk",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestHistogram(6), nil},
 				},
 			},
 			expected: []chunks.Sample{
-				sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-				sample{2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
-				sample{3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
-				sample{6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
+				sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+				sample{0, 2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
+				sample{0, 3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
+				sample{0, 6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-					sample{2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
-					sample{3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
-					sample{6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
+					sample{0, 3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
+					sample{0, 6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}},
@@ -1119,21 +1119,21 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one histogram chunk intersect with earlier deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestHistogram(6), nil},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 1, Maxt: 2}},
 			expected: []chunks.Sample{
-				sample{3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
-				sample{6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
+				sample{0, 3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
+				sample{0, 6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
-					sample{6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
+					sample{0, 3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
+					sample{0, 6, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{3, 6}},
@@ -1142,23 +1142,23 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one histogram chunk intersect with later deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestHistogram(6), nil},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 5, Maxt: 20}},
 			expected: []chunks.Sample{
-				sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-				sample{2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
-				sample{3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
+				sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+				sample{0, 2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
+				sample{0, 3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil},
-					sample{2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
-					sample{3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(2)), nil},
+					sample{0, 3, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(3)), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 3}},
@@ -1167,24 +1167,24 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one float histogram chunk",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
 				},
 			},
 			expected: []chunks.Sample{
-				sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-				sample{2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
-				sample{3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
-				sample{6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
+				sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+				sample{0, 2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
+				sample{0, 3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
+				sample{0, 6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
-					sample{3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
-					sample{6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
+					sample{0, 3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
+					sample{0, 6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}},
@@ -1193,21 +1193,21 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one float histogram chunk intersect with earlier deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 1, Maxt: 2}},
 			expected: []chunks.Sample{
-				sample{3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
-				sample{6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
+				sample{0, 3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
+				sample{0, 6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
-					sample{6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
+					sample{0, 3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
+					sample{0, 6, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{3, 6}},
@@ -1216,23 +1216,23 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one float histogram chunk intersect with later deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 5, Maxt: 20}},
 			expected: []chunks.Sample{
-				sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-				sample{2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
-				sample{3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
+				sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+				sample{0, 2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
+				sample{0, 3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
-					sample{3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(2))},
+					sample{0, 3, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(3))},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 3}},
@@ -1241,24 +1241,24 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one gauge histogram chunk",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 				},
 			},
 			expected: []chunks.Sample{
-				sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-				sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-				sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-				sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+				sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+				sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+				sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+				sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}},
@@ -1267,21 +1267,21 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one gauge histogram chunk intersect with earlier deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 1, Maxt: 2}},
 			expected: []chunks.Sample{
-				sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-				sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+				sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+				sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{3, 6}},
@@ -1290,23 +1290,23 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one gauge histogram chunk intersect with later deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
-					sample{6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+					sample{0, 6, 0, tsdbutil.GenerateTestGaugeHistogram(6), nil},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 5, Maxt: 20}},
 			expected: []chunks.Sample{
-				sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-				sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-				sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+				sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+				sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+				sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
-					sample{2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
-					sample{3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
+					sample{0, 1, 0, tsdbutil.GenerateTestGaugeHistogram(1), nil},
+					sample{0, 2, 0, tsdbutil.GenerateTestGaugeHistogram(2), nil},
+					sample{0, 3, 0, tsdbutil.GenerateTestGaugeHistogram(3), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 3}},
@@ -1315,24 +1315,24 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one gauge float histogram",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 				},
 			},
 			expected: []chunks.Sample{
-				sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-				sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-				sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-				sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+				sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+				sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+				sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+				sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}},
@@ -1341,21 +1341,21 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one gauge float histogram chunk intersect with earlier deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 1, Maxt: 2}},
 			expected: []chunks.Sample{
-				sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-				sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+				sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+				sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{3, 6}},
@@ -1364,23 +1364,23 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "one gauge float histogram chunk intersect with later deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
-					sample{6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+					sample{0, 6, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(6)},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 5, Maxt: 20}},
 			expected: []chunks.Sample{
-				sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-				sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-				sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+				sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+				sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+				sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
-					sample{2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
-					sample{3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
+					sample{0, 1, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(1)},
+					sample{0, 2, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(2)},
+					sample{0, 3, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 3}},
@@ -1388,31 +1388,31 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 		{
 			name: "three full mixed chunks",
 			samples: [][]chunks.Sample{
-				{sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}},
 				{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				},
 				{
-					sample{10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				},
 			},
 
 			expected: []chunks.Sample{
-				sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil}, sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil}, sample{10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)}, sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+				sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil}, sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil}, sample{0, 10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)}, sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{1, 2, nil, nil}, sample{2, 3, nil, nil}, sample{3, 5, nil, nil}, sample{6, 1, nil, nil},
+					sample{0, 1, 2, nil, nil}, sample{0, 2, 3, nil, nil}, sample{0, 3, 5, nil, nil}, sample{0, 6, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{1, 6}, {7, 9}, {10, 203}},
@@ -1421,30 +1421,30 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "three full mixed chunks in different order",
 			samples: [][]chunks.Sample{
 				{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				},
-				{sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil}},
+				{sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil}},
 				{
-					sample{100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				},
 			},
 
 			expected: []chunks.Sample{
-				sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil}, sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil}, sample{100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)}, sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+				sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil}, sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil}, sample{0, 100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)}, sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil},
+					sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{7, 9}, {11, 16}, {100, 203}},
@@ -1453,29 +1453,29 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "three full mixed chunks in different order intersect with deletion interval",
 			samples: [][]chunks.Sample{
 				{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 9, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				},
-				{sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil}},
+				{sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil}},
 				{
-					sample{100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 100, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				},
 			},
 			intervals: tombstones.Intervals{{Mint: 8, Maxt: 11}, {Mint: 15, Maxt: 150}},
 
 			expected: []chunks.Sample{
-				sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+				sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{12, 3, nil, nil}, sample{13, 5, nil, nil},
+					sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{7, 7}, {12, 13}, {203, 203}},
@@ -1484,30 +1484,30 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "three full mixed chunks overlapping",
 			samples: [][]chunks.Sample{
 				{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{12, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 12, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				},
-				{sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil}},
+				{sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil}},
 				{
-					sample{10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				},
 			},
 
 			expected: []chunks.Sample{
-				sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{12, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil}, sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil}, sample{10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)}, sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+				sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil}, sample{0, 12, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil}, sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil}, sample{0, 10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)}, sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
-					sample{12, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestGaugeHistogram(89), nil},
+					sample{0, 12, 0, tsdbutil.GenerateTestGaugeHistogram(8), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{11, 2, nil, nil}, sample{12, 3, nil, nil}, sample{13, 5, nil, nil}, sample{16, 1, nil, nil},
+					sample{0, 11, 2, nil, nil}, sample{0, 12, 3, nil, nil}, sample{0, 13, 5, nil, nil}, sample{0, 16, 1, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
-					sample{203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
+					sample{0, 10, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(22)},
+					sample{0, 203, 0, nil, tsdbutil.GenerateTestGaugeFloatHistogram(3493)},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{{7, 12}, {11, 16}, {10, 203}},
@@ -1516,56 +1516,56 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "int histogram iterables with counter resets",
 			samples: [][]chunks.Sample{
 				{
-					sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil},
-					sample{8, 0, tsdbutil.GenerateTestHistogram(9), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil},
+					sample{0, 8, 0, tsdbutil.GenerateTestHistogram(9), nil},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{12, 0, tsdbutil.GenerateTestHistogram(5), nil},
-					sample{15, 0, tsdbutil.GenerateTestHistogram(6), nil},
-					sample{16, 0, tsdbutil.GenerateTestHistogram(7), nil},
+					sample{0, 12, 0, tsdbutil.GenerateTestHistogram(5), nil},
+					sample{0, 15, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 16, 0, tsdbutil.GenerateTestHistogram(7), nil},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{17, 0, tsdbutil.GenerateTestHistogram(5), nil},
+					sample{0, 17, 0, tsdbutil.GenerateTestHistogram(5), nil},
 				},
 				{
-					sample{18, 0, tsdbutil.GenerateTestHistogram(6), nil},
-					sample{19, 0, tsdbutil.GenerateTestHistogram(7), nil},
+					sample{0, 18, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 19, 0, tsdbutil.GenerateTestHistogram(7), nil},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{20, 0, tsdbutil.GenerateTestHistogram(5), nil},
-					sample{21, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 20, 0, tsdbutil.GenerateTestHistogram(5), nil},
+					sample{0, 21, 0, tsdbutil.GenerateTestHistogram(6), nil},
 				},
 			},
 
 			expected: []chunks.Sample{
-				sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil},
-				sample{8, 0, tsdbutil.GenerateTestHistogram(9), nil},
-				sample{12, 0, tsdbutil.GenerateTestHistogram(5), nil},
-				sample{15, 0, tsdbutil.GenerateTestHistogram(6), nil},
-				sample{16, 0, tsdbutil.GenerateTestHistogram(7), nil},
-				sample{17, 0, tsdbutil.GenerateTestHistogram(5), nil},
-				sample{18, 0, tsdbutil.GenerateTestHistogram(6), nil},
-				sample{19, 0, tsdbutil.GenerateTestHistogram(7), nil},
-				sample{20, 0, tsdbutil.GenerateTestHistogram(5), nil},
-				sample{21, 0, tsdbutil.GenerateTestHistogram(6), nil},
+				sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil},
+				sample{0, 8, 0, tsdbutil.GenerateTestHistogram(9), nil},
+				sample{0, 12, 0, tsdbutil.GenerateTestHistogram(5), nil},
+				sample{0, 15, 0, tsdbutil.GenerateTestHistogram(6), nil},
+				sample{0, 16, 0, tsdbutil.GenerateTestHistogram(7), nil},
+				sample{0, 17, 0, tsdbutil.GenerateTestHistogram(5), nil},
+				sample{0, 18, 0, tsdbutil.GenerateTestHistogram(6), nil},
+				sample{0, 19, 0, tsdbutil.GenerateTestHistogram(7), nil},
+				sample{0, 20, 0, tsdbutil.GenerateTestHistogram(5), nil},
+				sample{0, 21, 0, tsdbutil.GenerateTestHistogram(6), nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil},
-					sample{8, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(9)), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil},
+					sample{0, 8, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(9)), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{12, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(5)), nil},
-					sample{15, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
-					sample{16, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(7)), nil},
+					sample{0, 12, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(5)), nil},
+					sample{0, 15, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
+					sample{0, 16, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(7)), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{17, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(5)), nil},
+					sample{0, 17, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(5)), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{18, 0, tsdbutil.GenerateTestHistogram(6), nil},
-					sample{19, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(7)), nil},
+					sample{0, 18, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 19, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(7)), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{20, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(5)), nil},
-					sample{21, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
+					sample{0, 20, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(5)), nil},
+					sample{0, 21, 0, tsdbutil.SetHistogramNotCounterReset(tsdbutil.GenerateTestHistogram(6)), nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{
@@ -1585,56 +1585,56 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "float histogram iterables with counter resets",
 			samples: [][]chunks.Sample{
 				{
-					sample{7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)},
-					sample{8, 0, nil, tsdbutil.GenerateTestFloatHistogram(9)},
+					sample{0, 7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)},
+					sample{0, 8, 0, nil, tsdbutil.GenerateTestFloatHistogram(9)},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{12, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
-					sample{15, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
-					sample{16, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
+					sample{0, 12, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
+					sample{0, 15, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 16, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{17, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
+					sample{0, 17, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
 				},
 				{
-					sample{18, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
-					sample{19, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
+					sample{0, 18, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 19, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{20, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
-					sample{21, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 20, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
+					sample{0, 21, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
 				},
 			},
 
 			expected: []chunks.Sample{
-				sample{7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)},
-				sample{8, 0, nil, tsdbutil.GenerateTestFloatHistogram(9)},
-				sample{12, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
-				sample{15, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
-				sample{16, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
-				sample{17, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
-				sample{18, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
-				sample{19, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
-				sample{20, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
-				sample{21, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+				sample{0, 7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)},
+				sample{0, 8, 0, nil, tsdbutil.GenerateTestFloatHistogram(9)},
+				sample{0, 12, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
+				sample{0, 15, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+				sample{0, 16, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
+				sample{0, 17, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
+				sample{0, 18, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+				sample{0, 19, 0, nil, tsdbutil.GenerateTestFloatHistogram(7)},
+				sample{0, 20, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)},
+				sample{0, 21, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)},
-					sample{8, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(9))},
+					sample{0, 7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)},
+					sample{0, 8, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(9))},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{12, 0, nil, tsdbutil.SetFloatHistogramCounterReset(tsdbutil.GenerateTestFloatHistogram(5))},
-					sample{15, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
-					sample{16, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(7))},
+					sample{0, 12, 0, nil, tsdbutil.SetFloatHistogramCounterReset(tsdbutil.GenerateTestFloatHistogram(5))},
+					sample{0, 15, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
+					sample{0, 16, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(7))},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{17, 0, nil, tsdbutil.SetFloatHistogramCounterReset(tsdbutil.GenerateTestFloatHistogram(5))},
+					sample{0, 17, 0, nil, tsdbutil.SetFloatHistogramCounterReset(tsdbutil.GenerateTestFloatHistogram(5))},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{18, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
-					sample{19, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(7))},
+					sample{0, 18, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)},
+					sample{0, 19, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(7))},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{20, 0, nil, tsdbutil.SetFloatHistogramCounterReset(tsdbutil.GenerateTestFloatHistogram(5))},
-					sample{21, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
+					sample{0, 20, 0, nil, tsdbutil.SetFloatHistogramCounterReset(tsdbutil.GenerateTestFloatHistogram(5))},
+					sample{0, 21, 0, nil, tsdbutil.SetFloatHistogramNotCounterReset(tsdbutil.GenerateTestFloatHistogram(6))},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{
@@ -1654,61 +1654,61 @@ func TestPopulateWithTombSeriesIterators(t *testing.T) {
 			name: "iterables with mixed encodings and counter resets",
 			samples: [][]chunks.Sample{
 				{
-					sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil},
-					sample{8, 0, tsdbutil.GenerateTestHistogram(9), nil},
-					sample{9, 0, nil, tsdbutil.GenerateTestFloatHistogram(10)},
-					sample{10, 0, nil, tsdbutil.GenerateTestFloatHistogram(11)},
-					sample{11, 0, nil, tsdbutil.GenerateTestFloatHistogram(12)},
-					sample{12, 13, nil, nil},
-					sample{13, 14, nil, nil},
-					sample{14, 0, tsdbutil.GenerateTestHistogram(8), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil},
+					sample{0, 8, 0, tsdbutil.GenerateTestHistogram(9), nil},
+					sample{0, 9, 0, nil, tsdbutil.GenerateTestFloatHistogram(10)},
+					sample{0, 10, 0, nil, tsdbutil.GenerateTestFloatHistogram(11)},
+					sample{0, 11, 0, nil, tsdbutil.GenerateTestFloatHistogram(12)},
+					sample{0, 12, 13, nil, nil},
+					sample{0, 13, 14, nil, nil},
+					sample{0, 14, 0, tsdbutil.GenerateTestHistogram(8), nil},
 					// Counter reset should be detected when chunks are created from the iterable.
-					sample{15, 0, tsdbutil.GenerateTestHistogram(7), nil},
+					sample{0, 15, 0, tsdbutil.GenerateTestHistogram(7), nil},
 				},
 				{
-					sample{18, 0, tsdbutil.GenerateTestHistogram(6), nil},
-					sample{19, 45, nil, nil},
+					sample{0, 18, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 19, 45, nil, nil},
 				},
 			},
 
 			expected: []chunks.Sample{
-				sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil},
-				sample{8, 0, tsdbutil.GenerateTestHistogram(9), nil},
-				sample{9, 0, nil, tsdbutil.GenerateTestFloatHistogram(10)},
-				sample{10, 0, nil, tsdbutil.GenerateTestFloatHistogram(11)},
-				sample{11, 0, nil, tsdbutil.GenerateTestFloatHistogram(12)},
-				sample{12, 13, nil, nil},
-				sample{13, 14, nil, nil},
-				sample{14, 0, tsdbutil.GenerateTestHistogram(8), nil},
-				sample{15, 0, tsdbutil.GenerateTestHistogram(7), nil},
-				sample{18, 0, tsdbutil.GenerateTestHistogram(6), nil},
-				sample{19, 45, nil, nil},
+				sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil},
+				sample{0, 8, 0, tsdbutil.GenerateTestHistogram(9), nil},
+				sample{0, 9, 0, nil, tsdbutil.GenerateTestFloatHistogram(10)},
+				sample{0, 10, 0, nil, tsdbutil.GenerateTestFloatHistogram(11)},
+				sample{0, 11, 0, nil, tsdbutil.GenerateTestFloatHistogram(12)},
+				sample{0, 12, 13, nil, nil},
+				sample{0, 13, 14, nil, nil},
+				sample{0, 14, 0, tsdbutil.GenerateTestHistogram(8), nil},
+				sample{0, 15, 0, tsdbutil.GenerateTestHistogram(7), nil},
+				sample{0, 18, 0, tsdbutil.GenerateTestHistogram(6), nil},
+				sample{0, 19, 45, nil, nil},
 			},
 			expectedChks: []chunks.Meta{
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil},
-					sample{8, 0, tsdbutil.GenerateTestHistogram(9), nil},
+					sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil},
+					sample{0, 8, 0, tsdbutil.GenerateTestHistogram(9), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{9, 0, nil, tsdbutil.GenerateTestFloatHistogram(10)},
-					sample{10, 0, nil, tsdbutil.GenerateTestFloatHistogram(11)},
-					sample{11, 0, nil, tsdbutil.GenerateTestFloatHistogram(12)},
+					sample{0, 9, 0, nil, tsdbutil.GenerateTestFloatHistogram(10)},
+					sample{0, 10, 0, nil, tsdbutil.GenerateTestFloatHistogram(11)},
+					sample{0, 11, 0, nil, tsdbutil.GenerateTestFloatHistogram(12)},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{12, 13, nil, nil},
-					sample{13, 14, nil, nil},
+					sample{0, 12, 13, nil, nil},
+					sample{0, 13, 14, nil, nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{14, 0, tsdbutil.GenerateTestHistogram(8), nil},
+					sample{0, 14, 0, tsdbutil.GenerateTestHistogram(8), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{15, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(7)), nil},
+					sample{0, 15, 0, tsdbutil.SetHistogramCounterReset(tsdbutil.GenerateTestHistogram(7)), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{18, 0, tsdbutil.GenerateTestHistogram(6), nil},
+					sample{0, 18, 0, tsdbutil.GenerateTestHistogram(6), nil},
 				}),
 				assureChunkFromSamples(t, []chunks.Sample{
-					sample{19, 45, nil, nil},
+					sample{0, 19, 45, nil, nil},
 				}),
 			},
 			expectedMinMaxTimes: []minMaxTimes{
@@ -1849,8 +1849,8 @@ func TestPopulateWithDelSeriesIterator_DoubleSeek(t *testing.T) {
 			valType: chunkenc.ValFloat,
 			chks: [][]chunks.Sample{
 				{},
-				{sample{1, 1, nil, nil}, sample{2, 2, nil, nil}, sample{3, 3, nil, nil}},
-				{sample{4, 4, nil, nil}, sample{5, 5, nil, nil}},
+				{sample{0, 1, 1, nil, nil}, sample{0, 2, 2, nil, nil}, sample{0, 3, 3, nil, nil}},
+				{sample{0, 4, 4, nil, nil}, sample{0, 5, 5, nil, nil}},
 			},
 		},
 		{
@@ -1858,8 +1858,8 @@ func TestPopulateWithDelSeriesIterator_DoubleSeek(t *testing.T) {
 			valType: chunkenc.ValHistogram,
 			chks: [][]chunks.Sample{
 				{},
-				{sample{1, 0, tsdbutil.GenerateTestHistogram(1), nil}, sample{2, 0, tsdbutil.GenerateTestHistogram(2), nil}, sample{3, 0, tsdbutil.GenerateTestHistogram(3), nil}},
-				{sample{4, 0, tsdbutil.GenerateTestHistogram(4), nil}, sample{5, 0, tsdbutil.GenerateTestHistogram(5), nil}},
+				{sample{0, 1, 0, tsdbutil.GenerateTestHistogram(1), nil}, sample{0, 2, 0, tsdbutil.GenerateTestHistogram(2), nil}, sample{0, 3, 0, tsdbutil.GenerateTestHistogram(3), nil}},
+				{sample{0, 4, 0, tsdbutil.GenerateTestHistogram(4), nil}, sample{0, 5, 0, tsdbutil.GenerateTestHistogram(5), nil}},
 			},
 		},
 		{
@@ -1867,8 +1867,8 @@ func TestPopulateWithDelSeriesIterator_DoubleSeek(t *testing.T) {
 			valType: chunkenc.ValFloatHistogram,
 			chks: [][]chunks.Sample{
 				{},
-				{sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)}, sample{2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)}, sample{3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)}},
-				{sample{4, 0, nil, tsdbutil.GenerateTestFloatHistogram(4)}, sample{5, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)}},
+				{sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(1)}, sample{0, 2, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)}, sample{0, 3, 0, nil, tsdbutil.GenerateTestFloatHistogram(3)}},
+				{sample{0, 4, 0, nil, tsdbutil.GenerateTestFloatHistogram(4)}, sample{0, 5, 0, nil, tsdbutil.GenerateTestFloatHistogram(5)}},
 			},
 		},
 	}
@@ -1902,7 +1902,7 @@ func TestPopulateWithDelSeriesIterator_SeekInCurrentChunk(t *testing.T) {
 			valType: chunkenc.ValFloat,
 			chks: [][]chunks.Sample{
 				{},
-				{sample{1, 2, nil, nil}, sample{3, 4, nil, nil}, sample{5, 6, nil, nil}, sample{7, 8, nil, nil}},
+				{sample{0, 1, 2, nil, nil}, sample{0, 3, 4, nil, nil}, sample{0, 5, 6, nil, nil}, sample{0, 7, 8, nil, nil}},
 				{},
 			},
 		},
@@ -1911,7 +1911,7 @@ func TestPopulateWithDelSeriesIterator_SeekInCurrentChunk(t *testing.T) {
 			valType: chunkenc.ValHistogram,
 			chks: [][]chunks.Sample{
 				{},
-				{sample{1, 0, tsdbutil.GenerateTestHistogram(2), nil}, sample{3, 0, tsdbutil.GenerateTestHistogram(4), nil}, sample{5, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil}},
+				{sample{0, 1, 0, tsdbutil.GenerateTestHistogram(2), nil}, sample{0, 3, 0, tsdbutil.GenerateTestHistogram(4), nil}, sample{0, 5, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil}},
 				{},
 			},
 		},
@@ -1920,7 +1920,7 @@ func TestPopulateWithDelSeriesIterator_SeekInCurrentChunk(t *testing.T) {
 			valType: chunkenc.ValFloatHistogram,
 			chks: [][]chunks.Sample{
 				{},
-				{sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)}, sample{3, 0, nil, tsdbutil.GenerateTestFloatHistogram(4)}, sample{5, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)}},
+				{sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(2)}, sample{0, 3, 0, nil, tsdbutil.GenerateTestFloatHistogram(4)}, sample{0, 5, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{0, 7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)}},
 				{},
 			},
 		},
@@ -1952,21 +1952,21 @@ func TestPopulateWithDelSeriesIterator_SeekWithMinTime(t *testing.T) {
 			name:    "float",
 			valType: chunkenc.ValFloat,
 			chks: [][]chunks.Sample{
-				{sample{1, 6, nil, nil}, sample{5, 6, nil, nil}, sample{6, 8, nil, nil}},
+				{sample{0, 1, 6, nil, nil}, sample{0, 5, 6, nil, nil}, sample{0, 6, 8, nil, nil}},
 			},
 		},
 		{
 			name:    "histogram",
 			valType: chunkenc.ValHistogram,
 			chks: [][]chunks.Sample{
-				{sample{1, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{5, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{6, 0, tsdbutil.GenerateTestHistogram(8), nil}},
+				{sample{0, 1, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{0, 5, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{0, 6, 0, tsdbutil.GenerateTestHistogram(8), nil}},
 			},
 		},
 		{
 			name:    "float histogram",
 			valType: chunkenc.ValFloatHistogram,
 			chks: [][]chunks.Sample{
-				{sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{5, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{6, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)}},
+				{sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{0, 5, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{0, 6, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)}},
 			},
 		},
 	}
@@ -1995,21 +1995,21 @@ func TestPopulateWithDelSeriesIterator_NextWithMinTime(t *testing.T) {
 			name:    "float",
 			valType: chunkenc.ValFloat,
 			chks: [][]chunks.Sample{
-				{sample{1, 6, nil, nil}, sample{5, 6, nil, nil}, sample{7, 8, nil, nil}},
+				{sample{0, 1, 6, nil, nil}, sample{0, 5, 6, nil, nil}, sample{0, 7, 8, nil, nil}},
 			},
 		},
 		{
 			name:    "histogram",
 			valType: chunkenc.ValHistogram,
 			chks: [][]chunks.Sample{
-				{sample{1, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{5, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{7, 0, tsdbutil.GenerateTestHistogram(8), nil}},
+				{sample{0, 1, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{0, 5, 0, tsdbutil.GenerateTestHistogram(6), nil}, sample{0, 7, 0, tsdbutil.GenerateTestHistogram(8), nil}},
 			},
 		},
 		{
 			name:    "float histogram",
 			valType: chunkenc.ValFloatHistogram,
 			chks: [][]chunks.Sample{
-				{sample{1, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{5, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)}},
+				{sample{0, 1, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{0, 5, 0, nil, tsdbutil.GenerateTestFloatHistogram(6)}, sample{0, 7, 0, nil, tsdbutil.GenerateTestFloatHistogram(8)}},
 			},
 		},
 	}

From 6647e512ad7c161ab8f0721773752185adbe1ff5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 10:01:55 +0100
Subject: [PATCH 257/439] update ExpandSamples
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: György Krajcsovits 
---
 storage/series.go | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/storage/series.go b/storage/series.go
index c16e628ba2..ebc5a16c07 100644
--- a/storage/series.go
+++ b/storage/series.go
@@ -447,16 +447,16 @@ func (e errChunksIterator) Err() error    { return e.err }
 // ExpandSamples iterates over all samples in the iterator, buffering all in slice.
 // Optionally it takes samples constructor, useful when you want to compare sample slices with different
 // sample implementations. if nil, sample type from this package will be used.
-func ExpandSamples(iter chunkenc.Iterator, newSampleFn func(t int64, f float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample) ([]chunks.Sample, error) {
+func ExpandSamples(iter chunkenc.Iterator, newSampleFn func(st, t int64, f float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample) ([]chunks.Sample, error) {
 	if newSampleFn == nil {
-		newSampleFn = func(t int64, f float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample {
+		newSampleFn = func(st, t int64, f float64, h *histogram.Histogram, fh *histogram.FloatHistogram) chunks.Sample {
 			switch {
 			case h != nil:
-				return hSample{0, t, h}
+				return hSample{st, t, h}
 			case fh != nil:
-				return fhSample{0, t, fh}
+				return fhSample{st, t, fh}
 			default:
-				return fSample{0, t, f}
+				return fSample{st, t, f}
 			}
 		}
 	}
@@ -468,17 +468,20 @@ func ExpandSamples(iter chunkenc.Iterator, newSampleFn func(t int64, f float64,
 			return result, iter.Err()
 		case chunkenc.ValFloat:
 			t, f := iter.At()
+			st := iter.AtST()
 			// NaNs can't be compared normally, so substitute for another value.
 			if math.IsNaN(f) {
 				f = -42
 			}
-			result = append(result, newSampleFn(t, f, nil, nil))
+			result = append(result, newSampleFn(st, t, f, nil, nil))
 		case chunkenc.ValHistogram:
 			t, h := iter.AtHistogram(nil)
-			result = append(result, newSampleFn(t, 0, h, nil))
+			st := iter.AtST()
+			result = append(result, newSampleFn(st, t, 0, h, nil))
 		case chunkenc.ValFloatHistogram:
 			t, fh := iter.AtFloatHistogram(nil)
-			result = append(result, newSampleFn(t, 0, nil, fh))
+			st := iter.AtST()
+			result = append(result, newSampleFn(st, t, 0, nil, fh))
 		}
 	}
 }

From a5ac0bff1d1fe8a167fc9b982f692b39190090a4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 10:02:13 +0100
Subject: [PATCH 258/439] update ooo_head.go but only with TODOs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: György Krajcsovits 
---
 tsdb/ooo_head.go | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tsdb/ooo_head.go b/tsdb/ooo_head.go
index bbb0f10e77..f9746c4c61 100644
--- a/tsdb/ooo_head.go
+++ b/tsdb/ooo_head.go
@@ -40,7 +40,8 @@ func (o *OOOChunk) Insert(t int64, v float64, h *histogram.Histogram, fh *histog
 	// try to append at the end first if the new timestamp is higher than the
 	// last known timestamp.
 	if len(o.samples) == 0 || t > o.samples[len(o.samples)-1].t {
-		o.samples = append(o.samples, sample{t, v, h, fh})
+		// TODO(krajorama): pass ST.
+		o.samples = append(o.samples, sample{0, t, v, h, fh})
 		return true
 	}
 
@@ -49,7 +50,8 @@ func (o *OOOChunk) Insert(t int64, v float64, h *histogram.Histogram, fh *histog
 
 	if i >= len(o.samples) {
 		// none found. append it at the end
-		o.samples = append(o.samples, sample{t, v, h, fh})
+		// TODO(krajorama): pass ST.
+		o.samples = append(o.samples, sample{0, t, v, h, fh})
 		return true
 	}
 
@@ -61,7 +63,8 @@ func (o *OOOChunk) Insert(t int64, v float64, h *histogram.Histogram, fh *histog
 	// Expand length by 1 to make room. use a zero sample, we will overwrite it anyway.
 	o.samples = append(o.samples, sample{})
 	copy(o.samples[i+1:], o.samples[i:])
-	o.samples[i] = sample{t, v, h, fh}
+	// TODO(krajorama): pass ST.
+	o.samples[i] = sample{0, t, v, h, fh}
 
 	return true
 }

From adf734db7a8a314b95ed8b88b960400b7e2f11d6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 10:02:26 +0100
Subject: [PATCH 259/439] update remaining tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: György Krajcsovits 
---
 tsdb/head_test.go    |  6 +++---
 tsdb/querier_test.go | 12 ++++++------
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tsdb/head_test.go b/tsdb/head_test.go
index d0928d64bf..bcf9b52f34 100644
--- a/tsdb/head_test.go
+++ b/tsdb/head_test.go
@@ -745,7 +745,7 @@ func TestHead_ReadWAL(t *testing.T) {
 			// Verify samples and exemplar for series 10.
 			c, _, _, err := s10.chunk(0, head.chunkDiskMapper, &head.memChunkPool)
 			require.NoError(t, err)
-			require.Equal(t, []sample{{100, 2, nil, nil}, {101, 5, nil, nil}}, expandChunk(c.chunk.Iterator(nil)))
+			require.Equal(t, []sample{{0, 100, 2, nil, nil}, {0, 101, 5, nil, nil}}, expandChunk(c.chunk.Iterator(nil)))
 
 			q, err := head.ExemplarQuerier(context.Background())
 			require.NoError(t, err)
@@ -758,14 +758,14 @@ func TestHead_ReadWAL(t *testing.T) {
 			// Verify samples for series 50
 			c, _, _, err = s50.chunk(0, head.chunkDiskMapper, &head.memChunkPool)
 			require.NoError(t, err)
-			require.Equal(t, []sample{{101, 6, nil, nil}}, expandChunk(c.chunk.Iterator(nil)))
+			require.Equal(t, []sample{{0, 101, 6, nil, nil}}, expandChunk(c.chunk.Iterator(nil)))
 
 			// Verify records for series 100 and its duplicate, series 101.
 			// The samples before the new series record should be discarded since a duplicate record
 			// is only possible when old samples were compacted.
 			c, _, _, err = s100.chunk(0, head.chunkDiskMapper, &head.memChunkPool)
 			require.NoError(t, err)
-			require.Equal(t, []sample{{101, 7, nil, nil}}, expandChunk(c.chunk.Iterator(nil)))
+			require.Equal(t, []sample{{0, 101, 7, nil, nil}}, expandChunk(c.chunk.Iterator(nil)))
 
 			q, err = head.ExemplarQuerier(context.Background())
 			require.NoError(t, err)
diff --git a/tsdb/querier_test.go b/tsdb/querier_test.go
index 9ff5124074..4387635959 100644
--- a/tsdb/querier_test.go
+++ b/tsdb/querier_test.go
@@ -574,22 +574,22 @@ var testData = []seriesSamples{
 	{
 		lset: map[string]string{"a": "a"},
 		chunks: [][]sample{
-			{{1, 2, nil, nil}, {2, 3, nil, nil}, {3, 4, nil, nil}},
-			{{5, 2, nil, nil}, {6, 3, nil, nil}, {7, 4, nil, nil}},
+			{{0, 1, 2, nil, nil}, {0, 2, 3, nil, nil}, {0, 3, 4, nil, nil}},
+			{{0, 5, 2, nil, nil}, {0, 6, 3, nil, nil}, {0, 7, 4, nil, nil}},
 		},
 	},
 	{
 		lset: map[string]string{"a": "a", "b": "b"},
 		chunks: [][]sample{
-			{{1, 1, nil, nil}, {2, 2, nil, nil}, {3, 3, nil, nil}},
-			{{5, 3, nil, nil}, {6, 6, nil, nil}},
+			{{0, 1, 1, nil, nil}, {0, 2, 2, nil, nil}, {0, 3, 3, nil, nil}},
+			{{0, 5, 3, nil, nil}, {0, 6, 6, nil, nil}},
 		},
 	},
 	{
 		lset: map[string]string{"b": "b"},
 		chunks: [][]sample{
-			{{1, 3, nil, nil}, {2, 2, nil, nil}, {3, 6, nil, nil}},
-			{{5, 1, nil, nil}, {6, 7, nil, nil}, {7, 2, nil, nil}},
+			{{0, 1, 3, nil, nil}, {0, 2, 2, nil, nil}, {0, 3, 6, nil, nil}},
+			{{0, 5, 1, nil, nil}, {0, 6, 7, nil, nil}, {0, 7, 2, nil, nil}},
 		},
 	},
 }

From 8067b3d60ac9231026513da5e520ddc54d804bec Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 12:13:59 +0100
Subject: [PATCH 260/439] add test coverage for buffer.go
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

I've checked that commenting out any one of the new lines produces an error.

Signed-off-by: György Krajcsovits 
---
 storage/buffer_test.go | 122 ++++++++++++++++++++++++++---------------
 1 file changed, 78 insertions(+), 44 deletions(-)

diff --git a/storage/buffer_test.go b/storage/buffer_test.go
index e700231756..61d1601bc0 100644
--- a/storage/buffer_test.go
+++ b/storage/buffer_test.go
@@ -61,7 +61,9 @@ func TestSampleRing(t *testing.T) {
 
 		input := []fSample{}
 		for _, t := range c.input {
-			input = append(input, fSample{t: t, f: float64(rand.Intn(100))})
+			// Randomize start timestamp to make sure it does not affect the
+			// outcome.
+			input = append(input, fSample{st: rand.Int63(), t: t, f: float64(rand.Intn(100))})
 		}
 
 		for i, s := range input {
@@ -87,6 +89,24 @@ func TestSampleRing(t *testing.T) {
 	}
 }
 
+func TestSampleRingFloatST(t *testing.T) {
+	r := newSampleRing(10, 5, chunkenc.ValNone)
+	require.Empty(t, r.fBuf)
+	require.Empty(t, r.hBuf)
+	require.Empty(t, r.fhBuf)
+	require.Empty(t, r.iBuf)
+
+	r.addF(fSample{st: 100, t: 11, f: 3.14})
+	it := r.iterator()
+
+	require.Equal(t, chunkenc.ValFloat, it.Next())
+	ts, f := it.At()
+	require.Equal(t, int64(11), ts)
+	require.Equal(t, 3.14, f)
+	require.Equal(t, int64(100), it.AtST())
+	require.Equal(t, chunkenc.ValNone, it.Next())
+}
+
 func TestSampleRingMixed(t *testing.T) {
 	h1 := tsdbutil.GenerateTestHistogram(1)
 	h2 := tsdbutil.GenerateTestHistogram(2)
@@ -99,39 +119,43 @@ func TestSampleRingMixed(t *testing.T) {
 	require.Empty(t, r.iBuf)
 
 	// But then mixed adds should work as expected.
-	r.addF(fSample{t: 1, f: 3.14})
-	r.addH(hSample{t: 2, h: h1})
+	r.addF(fSample{st: 10, t: 11, f: 3.14})
+	r.addH(hSample{st: 20, t: 21, h: h1})
 
 	it := r.iterator()
 
 	require.Equal(t, chunkenc.ValFloat, it.Next())
 	ts, f := it.At()
-	require.Equal(t, int64(1), ts)
+	require.Equal(t, int64(11), ts)
 	require.Equal(t, 3.14, f)
+	require.Equal(t, int64(10), it.AtST())
 	require.Equal(t, chunkenc.ValHistogram, it.Next())
 	var h *histogram.Histogram
 	ts, h = it.AtHistogram()
-	require.Equal(t, int64(2), ts)
+	require.Equal(t, int64(21), ts)
 	require.Equal(t, h1, h)
+	require.Equal(t, int64(20), it.AtST())
 	require.Equal(t, chunkenc.ValNone, it.Next())
 
 	r.reset()
 	it = r.iterator()
 	require.Equal(t, chunkenc.ValNone, it.Next())
 
-	r.addF(fSample{t: 3, f: 4.2})
-	r.addH(hSample{t: 4, h: h2})
+	r.addF(fSample{st: 30, t: 31, f: 4.2})
+	r.addH(hSample{st: 40, t: 41, h: h2})
 
 	it = r.iterator()
 
 	require.Equal(t, chunkenc.ValFloat, it.Next())
 	ts, f = it.At()
-	require.Equal(t, int64(3), ts)
+	require.Equal(t, int64(31), ts)
 	require.Equal(t, 4.2, f)
+	require.Equal(t, int64(30), it.AtST())
 	require.Equal(t, chunkenc.ValHistogram, it.Next())
 	ts, h = it.AtHistogram()
-	require.Equal(t, int64(4), ts)
+	require.Equal(t, int64(41), ts)
 	require.Equal(t, h2, h)
+	require.Equal(t, int64(40), it.AtST())
 	require.Equal(t, chunkenc.ValNone, it.Next())
 }
 
@@ -157,44 +181,50 @@ func TestSampleRingAtFloatHistogram(t *testing.T) {
 	it := r.iterator()
 	require.Equal(t, chunkenc.ValNone, it.Next())
 
-	r.addFH(fhSample{t: 1, fh: fh1})
-	r.addFH(fhSample{t: 2, fh: fh2})
+	r.addFH(fhSample{st: 10, t: 11, fh: fh1})
+	r.addFH(fhSample{st: 20, t: 21, fh: fh2})
 
 	it = r.iterator()
 
 	require.Equal(t, chunkenc.ValFloatHistogram, it.Next())
 	ts, fh = it.AtFloatHistogram(fh)
-	require.Equal(t, int64(1), ts)
+	require.Equal(t, int64(11), ts)
 	require.Equal(t, fh1, fh)
+	require.Equal(t, int64(10), it.AtST())
 	require.Equal(t, chunkenc.ValFloatHistogram, it.Next())
 	ts, fh = it.AtFloatHistogram(fh)
-	require.Equal(t, int64(2), ts)
+	require.Equal(t, int64(21), ts)
 	require.Equal(t, fh2, fh)
+	require.Equal(t, int64(20), it.AtST())
 	require.Equal(t, chunkenc.ValNone, it.Next())
 
 	r.reset()
 	it = r.iterator()
 	require.Equal(t, chunkenc.ValNone, it.Next())
 
-	r.addH(hSample{t: 3, h: h1})
-	r.addH(hSample{t: 4, h: h2})
+	r.addH(hSample{st: 30, t: 31, h: h1})
+	r.addH(hSample{st: 40, t: 41, h: h2})
 
 	it = r.iterator()
 
 	require.Equal(t, chunkenc.ValHistogram, it.Next())
 	ts, h = it.AtHistogram()
-	require.Equal(t, int64(3), ts)
+	require.Equal(t, int64(31), ts)
 	require.Equal(t, h1, h)
+	require.Equal(t, int64(30), it.AtST())
 	ts, fh = it.AtFloatHistogram(fh)
-	require.Equal(t, int64(3), ts)
+	require.Equal(t, int64(31), ts)
 	require.Equal(t, h1.ToFloat(nil), fh)
+	require.Equal(t, int64(30), it.AtST())
 	require.Equal(t, chunkenc.ValHistogram, it.Next())
 	ts, h = it.AtHistogram()
-	require.Equal(t, int64(4), ts)
+	require.Equal(t, int64(41), ts)
 	require.Equal(t, h2, h)
+	require.Equal(t, int64(40), it.AtST())
 	ts, fh = it.AtFloatHistogram(fh)
-	require.Equal(t, int64(4), ts)
+	require.Equal(t, int64(41), ts)
 	require.Equal(t, h2.ToFloat(nil), fh)
+	require.Equal(t, int64(40), it.AtST())
 	require.Equal(t, chunkenc.ValNone, it.Next())
 }
 
@@ -206,59 +236,63 @@ func TestBufferedSeriesIterator(t *testing.T) {
 		bit := it.Buffer()
 		for bit.Next() == chunkenc.ValFloat {
 			t, f := bit.At()
-			b = append(b, fSample{t: t, f: f})
+			st := bit.AtST()
+			b = append(b, fSample{st: st, t: t, f: f})
 		}
 		require.Equal(t, exp, b, "buffer mismatch")
 	}
-	sampleEq := func(ets int64, ev float64) {
+	sampleEq := func(est, ets int64, ev float64) {
 		ts, v := it.At()
+		st := it.AtST()
+		require.Equal(t, est, st, "start timestamp mismatch")
 		require.Equal(t, ets, ts, "timestamp mismatch")
 		require.Equal(t, ev, v, "value mismatch")
 	}
-	prevSampleEq := func(ets int64, ev float64, eok bool) {
+	prevSampleEq := func(est, ets int64, ev float64, eok bool) {
 		s, ok := it.PeekBack(1)
 		require.Equal(t, eok, ok, "exist mismatch")
+		require.Equal(t, est, s.ST(), "start timestamp mismatch")
 		require.Equal(t, ets, s.T(), "timestamp mismatch")
 		require.Equal(t, ev, s.F(), "value mismatch")
 	}
 
 	it = NewBufferIterator(NewListSeriesIterator(samples{
-		fSample{t: 1, f: 2},
-		fSample{t: 2, f: 3},
-		fSample{t: 3, f: 4},
-		fSample{t: 4, f: 5},
-		fSample{t: 5, f: 6},
-		fSample{t: 99, f: 8},
-		fSample{t: 100, f: 9},
-		fSample{t: 101, f: 10},
+		fSample{st: -1, t: 1, f: 2},
+		fSample{st: 1, t: 2, f: 3},
+		fSample{st: 2, t: 3, f: 4},
+		fSample{st: 3, t: 4, f: 5},
+		fSample{st: 3, t: 5, f: 6},
+		fSample{st: 50, t: 99, f: 8},
+		fSample{st: 99, t: 100, f: 9},
+		fSample{st: 100, t: 101, f: 10},
 	}), 2)
 
 	require.Equal(t, chunkenc.ValFloat, it.Seek(-123), "seek failed")
-	sampleEq(1, 2)
-	prevSampleEq(0, 0, false)
+	sampleEq(-1, 1, 2)
+	prevSampleEq(0, 0, 0, false)
 	bufferEq(nil)
 
 	require.Equal(t, chunkenc.ValFloat, it.Next(), "next failed")
-	sampleEq(2, 3)
-	prevSampleEq(1, 2, true)
-	bufferEq([]fSample{{t: 1, f: 2}})
+	sampleEq(1, 2, 3)
+	prevSampleEq(-1, 1, 2, true)
+	bufferEq([]fSample{{st: -1, t: 1, f: 2}})
 
 	require.Equal(t, chunkenc.ValFloat, it.Next(), "next failed")
 	require.Equal(t, chunkenc.ValFloat, it.Next(), "next failed")
 	require.Equal(t, chunkenc.ValFloat, it.Next(), "next failed")
-	sampleEq(5, 6)
-	prevSampleEq(4, 5, true)
-	bufferEq([]fSample{{t: 2, f: 3}, {t: 3, f: 4}, {t: 4, f: 5}})
+	sampleEq(3, 5, 6)
+	prevSampleEq(3, 4, 5, true)
+	bufferEq([]fSample{{st: 1, t: 2, f: 3}, {st: 2, t: 3, f: 4}, {st: 3, t: 4, f: 5}})
 
 	require.Equal(t, chunkenc.ValFloat, it.Seek(5), "seek failed")
-	sampleEq(5, 6)
-	prevSampleEq(4, 5, true)
-	bufferEq([]fSample{{t: 2, f: 3}, {t: 3, f: 4}, {t: 4, f: 5}})
+	sampleEq(3, 5, 6)
+	prevSampleEq(3, 4, 5, true)
+	bufferEq([]fSample{{st: 1, t: 2, f: 3}, {st: 2, t: 3, f: 4}, {st: 3, t: 4, f: 5}})
 
 	require.Equal(t, chunkenc.ValFloat, it.Seek(101), "seek failed")
-	sampleEq(101, 10)
-	prevSampleEq(100, 9, true)
-	bufferEq([]fSample{{t: 99, f: 8}, {t: 100, f: 9}})
+	sampleEq(100, 101, 10)
+	prevSampleEq(99, 100, 9, true)
+	bufferEq([]fSample{{st: 50, t: 99, f: 8}, {st: 99, t: 100, f: 9}})
 
 	require.Equal(t, chunkenc.ValNone, it.Next(), "next succeeded unexpectedly")
 	require.Equal(t, chunkenc.ValNone, it.Seek(1024), "seek succeeded unexpectedly")

From 5ecc0e706295fc795218c874b5f94ffe4fdceca4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 12:19:43 +0100
Subject: [PATCH 261/439] test that ChainSampleIterator passes on the AtST call
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: György Krajcsovits 
---
 storage/merge_test.go | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/storage/merge_test.go b/storage/merge_test.go
index 5ffb0c4851..e42a6a4ce1 100644
--- a/storage/merge_test.go
+++ b/storage/merge_test.go
@@ -387,13 +387,13 @@ func TestMergeChunkQuerierWithNoVerticalChunkSeriesMerger(t *testing.T) {
 func histogramSample(ts int64, hint histogram.CounterResetHint) hSample {
 	h := tsdbutil.GenerateTestHistogram(ts + 1)
 	h.CounterResetHint = hint
-	return hSample{t: ts, h: h}
+	return hSample{st: -ts, t: ts, h: h}
 }
 
 func floatHistogramSample(ts int64, hint histogram.CounterResetHint) fhSample {
 	fh := tsdbutil.GenerateTestFloatHistogram(ts + 1)
 	fh.CounterResetHint = hint
-	return fhSample{t: ts, fh: fh}
+	return fhSample{st: -ts, t: ts, fh: fh}
 }
 
 // Shorthands for counter reset hints.
@@ -1059,7 +1059,7 @@ func (*mockChunkSeriesSet) Warnings() annotations.Annotations { return nil }
 
 func TestChainSampleIterator(t *testing.T) {
 	for sampleType, sampleFunc := range map[string]func(int64) chunks.Sample{
-		"float":           func(ts int64) chunks.Sample { return fSample{0, ts, float64(ts)} },
+		"float":           func(ts int64) chunks.Sample { return fSample{-ts, ts, float64(ts)} },
 		"histogram":       func(ts int64) chunks.Sample { return histogramSample(ts, uk) },
 		"float histogram": func(ts int64) chunks.Sample { return floatHistogramSample(ts, uk) },
 	} {
@@ -1176,7 +1176,7 @@ func TestChainSampleIteratorHistogramCounterResetHint(t *testing.T) {
 
 func TestChainSampleIteratorSeek(t *testing.T) {
 	for sampleType, sampleFunc := range map[string]func(int64) chunks.Sample{
-		"float":           func(ts int64) chunks.Sample { return fSample{0, ts, float64(ts)} },
+		"float":           func(ts int64) chunks.Sample { return fSample{-ts, ts, float64(ts)} },
 		"histogram":       func(ts int64) chunks.Sample { return histogramSample(ts, uk) },
 		"float histogram": func(ts int64) chunks.Sample { return floatHistogramSample(ts, uk) },
 	} {
@@ -1224,13 +1224,13 @@ func TestChainSampleIteratorSeek(t *testing.T) {
 				switch merged.Seek(tc.seek) {
 				case chunkenc.ValFloat:
 					t, f := merged.At()
-					actual = append(actual, fSample{0, t, f})
+					actual = append(actual, fSample{merged.AtST(), t, f})
 				case chunkenc.ValHistogram:
 					t, h := merged.AtHistogram(nil)
-					actual = append(actual, hSample{0, t, h})
+					actual = append(actual, hSample{merged.AtST(), t, h})
 				case chunkenc.ValFloatHistogram:
 					t, fh := merged.AtFloatHistogram(nil)
-					actual = append(actual, fhSample{0, t, fh})
+					actual = append(actual, fhSample{merged.AtST(), t, fh})
 				}
 				s, err := ExpandSamples(merged, nil)
 				require.NoError(t, err)
@@ -1310,13 +1310,13 @@ func TestChainSampleIteratorSeekHistogramCounterResetHint(t *testing.T) {
 				switch merged.Seek(tc.seek) {
 				case chunkenc.ValFloat:
 					t, f := merged.At()
-					actual = append(actual, fSample{0, t, f})
+					actual = append(actual, fSample{merged.AtST(), t, f})
 				case chunkenc.ValHistogram:
 					t, h := merged.AtHistogram(nil)
-					actual = append(actual, hSample{0, t, h})
+					actual = append(actual, hSample{merged.AtST(), t, h})
 				case chunkenc.ValFloatHistogram:
 					t, fh := merged.AtFloatHistogram(nil)
-					actual = append(actual, fhSample{0, t, fh})
+					actual = append(actual, fhSample{merged.AtST(), t, fh})
 				}
 				s, err := ExpandSamples(merged, nil)
 				require.NoError(t, err)

From 6137de085e9615b1ef3f976de405550d493c9249 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gy=C3=B6rgy=20Krajcsovits?= 
Date: Wed, 14 Jan 2026 12:30:24 +0100
Subject: [PATCH 262/439] test ListSeriesIterator with ST
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: György Krajcsovits 
---
 storage/series_test.go | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/storage/series_test.go b/storage/series_test.go
index 3ad84be6b0..b33d6cb1b3 100644
--- a/storage/series_test.go
+++ b/storage/series_test.go
@@ -28,11 +28,11 @@ import (
 
 func TestListSeriesIterator(t *testing.T) {
 	it := NewListSeriesIterator(samples{
-		fSample{0, 0, 0},
-		fSample{0, 1, 1},
-		fSample{0, 1, 1.5},
-		fSample{0, 2, 2},
-		fSample{0, 3, 3},
+		fSample{-10, 0, 0},
+		fSample{-9, 1, 1},
+		fSample{-8, 1, 1.5},
+		fSample{-7, 2, 2},
+		fSample{-6, 3, 3},
 	})
 
 	// Seek to the first sample with ts=1.
@@ -40,30 +40,35 @@ func TestListSeriesIterator(t *testing.T) {
 	ts, v := it.At()
 	require.Equal(t, int64(1), ts)
 	require.Equal(t, 1., v)
+	require.Equal(t, int64(-9), it.AtST())
 
 	// Seek one further, next sample still has ts=1.
 	require.Equal(t, chunkenc.ValFloat, it.Next())
 	ts, v = it.At()
 	require.Equal(t, int64(1), ts)
 	require.Equal(t, 1.5, v)
+	require.Equal(t, int64(-8), it.AtST())
 
 	// Seek again to 1 and make sure we stay where we are.
 	require.Equal(t, chunkenc.ValFloat, it.Seek(1))
 	ts, v = it.At()
 	require.Equal(t, int64(1), ts)
 	require.Equal(t, 1.5, v)
+	require.Equal(t, int64(-8), it.AtST())
 
 	// Another seek.
 	require.Equal(t, chunkenc.ValFloat, it.Seek(3))
 	ts, v = it.At()
 	require.Equal(t, int64(3), ts)
 	require.Equal(t, 3., v)
+	require.Equal(t, int64(-6), it.AtST())
 
 	// And we don't go back.
 	require.Equal(t, chunkenc.ValFloat, it.Seek(2))
 	ts, v = it.At()
 	require.Equal(t, int64(3), ts)
 	require.Equal(t, 3., v)
+	require.Equal(t, int64(-6), it.AtST())
 
 	// Seek beyond the end.
 	require.Equal(t, chunkenc.ValNone, it.Seek(5))

From 3374d2e56fc9ce12d19259633f8dd3bbe9015e81 Mon Sep 17 00:00:00 2001
From: Bartlomiej Plotka 
Date: Wed, 14 Jan 2026 13:48:33 +0000
Subject: [PATCH 263/439] feat(teststorage)[PART4a]: Add AppendableV2 support
 for mock Appendable (#17834)

* feat(teststorage)[PART4a]: Add AppendableV2 support for mock Appendable

Signed-off-by: bwplotka 

* fix: adjusted AppenderV1 flow for reliability

Found in https://github.com/prometheus/prometheus/pull/17838 and by
Krajo comment

Signed-off-by: bwplotka 

* addressed comments

Signed-off-by: bwplotka 

* fix broken appV2 commit and rollback; added tests

Signed-off-by: bwplotka 

---------

Signed-off-by: bwplotka 
---
 util/teststorage/appender.go      | 193 +++++++++++++++++++++-------
 util/teststorage/appender_test.go | 202 +++++++++++++++++++++++++++---
 2 files changed, 334 insertions(+), 61 deletions(-)

diff --git a/util/teststorage/appender.go b/util/teststorage/appender.go
index 058a09561c..a98ff9c48f 100644
--- a/util/teststorage/appender.go
+++ b/util/teststorage/appender.go
@@ -65,13 +65,17 @@ func (s Sample) String() string {
 	// Print all value types on purpose, to catch bugs for appending multiple sample types at once.
 	h := ""
 	if s.H != nil {
-		h = s.H.String()
+		h = " " + s.H.String()
 	}
 	fh := ""
 	if s.FH != nil {
-		fh = s.FH.String()
+		fh = " " + s.FH.String()
 	}
-	b.WriteString(fmt.Sprintf("%s %v%v%v st@%v t@%v\n", s.L.String(), s.V, h, fh, s.ST, s.T))
+	b.WriteString(fmt.Sprintf("%s %v%v%v st@%v t@%v", s.L.String(), s.V, h, fh, s.ST, s.T))
+	if len(s.ES) > 0 {
+		b.WriteString(fmt.Sprintf(" %v", s.ES))
+	}
+	b.WriteString("\n")
 	return b.String()
 }
 
@@ -104,7 +108,8 @@ type Appendable struct {
 	rolledbackSamples []Sample
 
 	// Optional chain (Appender will collect samples, then run next).
-	next storage.Appendable
+	next   storage.Appendable
+	nextV2 storage.AppendableV2
 }
 
 // NewAppendable returns mock Appendable.
@@ -112,12 +117,18 @@ func NewAppendable() *Appendable {
 	return &Appendable{}
 }
 
-// Then chains another appender from the provided appendable for the Appender calls.
+// Then chains another appender from the provided Appendable for the Appender calls.
 func (a *Appendable) Then(appendable storage.Appendable) *Appendable {
 	a.next = appendable
 	return a
 }
 
+// ThenV2 chains another appenderV2 from the provided AppendableV2 for the AppenderV2 calls.
+func (a *Appendable) ThenV2(appendable storage.AppendableV2) *Appendable {
+	a.nextV2 = appendable
+	return a
+}
+
 // WithErrs allows injecting errors to the appender.
 func (a *Appendable) WithErrs(appendErrFn func(ls labels.Labels) error, appendExemplarsError, commitErr error) *Appendable {
 	a.appendErrFn = appendErrFn
@@ -130,6 +141,9 @@ func (a *Appendable) WithErrs(appendErrFn func(ls labels.Labels) error, appendEx
 func (a *Appendable) PendingSamples() []Sample {
 	a.mtx.Lock()
 	defer a.mtx.Unlock()
+	if len(a.pendingSamples) == 0 {
+		return nil
+	}
 
 	ret := make([]Sample, len(a.pendingSamples))
 	copy(ret, a.pendingSamples)
@@ -140,6 +154,9 @@ func (a *Appendable) PendingSamples() []Sample {
 func (a *Appendable) ResultSamples() []Sample {
 	a.mtx.Lock()
 	defer a.mtx.Unlock()
+	if len(a.resultSamples) == 0 {
+		return nil
+	}
 
 	ret := make([]Sample, len(a.resultSamples))
 	copy(ret, a.resultSamples)
@@ -150,6 +167,9 @@ func (a *Appendable) ResultSamples() []Sample {
 func (a *Appendable) RolledbackSamples() []Sample {
 	a.mtx.Lock()
 	defer a.mtx.Unlock()
+	if len(a.rolledbackSamples) == 0 {
+		return nil
+	}
 
 	ret := make([]Sample, len(a.rolledbackSamples))
 	copy(ret, a.rolledbackSamples)
@@ -205,28 +225,77 @@ func (a *Appendable) String() string {
 
 var errClosedAppender = errors.New("appender was already committed/rolledback")
 
-type appender struct {
-	err  error
-	next storage.Appender
+type baseAppender struct {
+	err error
 
-	a *Appendable
+	nextTr storage.AppenderTransaction
+	a      *Appendable
 }
 
-func (a *appender) checkErr() error {
+func (a *baseAppender) checkErr() error {
 	a.a.mtx.Lock()
 	defer a.a.mtx.Unlock()
 
 	return a.err
 }
 
+func (a *baseAppender) Commit() error {
+	if err := a.checkErr(); err != nil {
+		return err
+	}
+	defer a.a.openAppenders.Dec()
+
+	if a.a.commitErr != nil {
+		return a.a.commitErr
+	}
+
+	a.a.mtx.Lock()
+	a.a.resultSamples = append(a.a.resultSamples, a.a.pendingSamples...)
+	a.a.pendingSamples = a.a.pendingSamples[:0]
+	a.err = errClosedAppender
+	a.a.mtx.Unlock()
+
+	if a.nextTr != nil {
+		return a.nextTr.Commit()
+	}
+	return nil
+}
+
+func (a *baseAppender) Rollback() error {
+	if err := a.checkErr(); err != nil {
+		return err
+	}
+	defer a.a.openAppenders.Dec()
+
+	a.a.mtx.Lock()
+	a.a.rolledbackSamples = append(a.a.rolledbackSamples, a.a.pendingSamples...)
+	a.a.pendingSamples = a.a.pendingSamples[:0]
+	a.err = errClosedAppender
+	a.a.mtx.Unlock()
+
+	if a.nextTr != nil {
+		return a.nextTr.Rollback()
+	}
+	return nil
+}
+
+type appender struct {
+	baseAppender
+
+	next storage.Appender
+}
+
 func (a *Appendable) Appender(ctx context.Context) storage.Appender {
-	ret := &appender{a: a}
+	ret := &appender{baseAppender: baseAppender{a: a}}
 	if a.openAppenders.Inc() > 1 {
 		ret.err = errors.New("teststorage.Appendable.Appender() concurrent use is not supported; attempted opening new Appender() without Commit/Rollback of the previous one. Extend the implementation if concurrent mock is needed")
 	}
 
 	if a.next != nil {
-		ret.next = a.next.Appender(ctx)
+		app := a.next.Appender(ctx)
+		ret.next, ret.nextTr = app, app
+	} else if a.nextV2 != nil {
+		ret.err = errors.Join(ret.err, errors.New("teststorage.Appendable.Appender() invoked with .ThenV2 but no .Then was supplied; likely bug"))
 	}
 	return ret
 }
@@ -264,7 +333,7 @@ func computeOrCheckRef(ref storage.SeriesRef, ls labels.Labels) (storage.SeriesR
 
 	if storage.SeriesRef(h) != ref {
 		// Check for buggy ref while we at it.
-		return 0, errors.New("teststorage.appender: found input ref not matching labels; potential bug in Appendable user")
+		return 0, errors.New("teststorage.appender: found input ref not matching labels; potential bug in Appendable usage")
 	}
 	return ref, nil
 }
@@ -297,6 +366,7 @@ func (a *appender) AppendExemplar(ref storage.SeriesRef, l labels.Labels, e exem
 	if a.a.appendExemplarsError != nil {
 		return 0, a.a.appendExemplarsError
 	}
+	var appended bool
 
 	a.a.mtx.Lock()
 	// NOTE(bwplotka): Eventually exemplar has to be attached to a series and soon
@@ -306,11 +376,12 @@ func (a *appender) AppendExemplar(ref storage.SeriesRef, l labels.Labels, e exem
 	for ; i >= 0; i-- { // Attach exemplars to the last matching sample.
 		if ref == storage.SeriesRef(a.a.pendingSamples[i].L.Hash()) {
 			a.a.pendingSamples[i].ES = append(a.a.pendingSamples[i].ES, e)
+			appended = true
 			break
 		}
 	}
 	a.a.mtx.Unlock()
-	if i < 0 {
+	if !appended {
 		return 0, fmt.Errorf("teststorage.appender: exemplar appender without series; ref %v; l %v; exemplar: %v", ref, l, e)
 	}
 
@@ -336,6 +407,8 @@ func (a *appender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m meta
 		return 0, err
 	}
 
+	var updated bool
+
 	a.a.mtx.Lock()
 	// NOTE(bwplotka): Eventually metadata has to be attached to a series and soon
 	// the AppenderV2 will guarantee that for TSDB. Assume this from the mock perspective
@@ -344,11 +417,12 @@ func (a *appender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m meta
 	for ; i >= 0; i-- { // Attach metadata to the last matching sample.
 		if ref == storage.SeriesRef(a.a.pendingSamples[i].L.Hash()) {
 			a.a.pendingSamples[i].M = m
+			updated = true
 			break
 		}
 	}
 	a.a.mtx.Unlock()
-	if i < 0 {
+	if !updated {
 		return 0, fmt.Errorf("teststorage.appender: metadata update without series; ref %v; l %v; m: %v", ref, l, m)
 	}
 
@@ -358,42 +432,75 @@ func (a *appender) UpdateMetadata(ref storage.SeriesRef, l labels.Labels, m meta
 	return computeOrCheckRef(ref, l)
 }
 
-func (a *appender) Commit() error {
-	if err := a.checkErr(); err != nil {
-		return err
-	}
-	defer a.a.openAppenders.Dec()
+type appenderV2 struct {
+	baseAppender
 
-	if a.a.commitErr != nil {
-		return a.a.commitErr
-	}
-
-	a.a.mtx.Lock()
-	a.a.resultSamples = append(a.a.resultSamples, a.a.pendingSamples...)
-	a.a.pendingSamples = a.a.pendingSamples[:0]
-	a.err = errClosedAppender
-	a.a.mtx.Unlock()
-
-	if a.a.next != nil {
-		return a.next.Commit()
-	}
-	return nil
+	next storage.AppenderV2
 }
 
-func (a *appender) Rollback() error {
-	if err := a.checkErr(); err != nil {
-		return err
+func (a *Appendable) AppenderV2(ctx context.Context) storage.AppenderV2 {
+	ret := &appenderV2{baseAppender: baseAppender{a: a}}
+	if a.openAppenders.Inc() > 1 {
+		ret.err = errors.New("teststorage.Appendable.AppenderV2() concurrent use is not supported; attempted opening new AppenderV2() without Commit/Rollback of the previous one. Extend the implementation if concurrent mock is needed")
+	}
+
+	if a.nextV2 != nil {
+		app := a.nextV2.AppenderV2(ctx)
+		ret.next, ret.nextTr = app, app
+	} else if a.next != nil {
+		ret.err = errors.Join(ret.err, errors.New("teststorage.Appendable.AppenderV2() invoked with .Then but no .ThenV2 was supplied; likely bug"))
+	}
+	return ret
+}
+
+func (a *appenderV2) Append(ref storage.SeriesRef, ls labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (_ storage.SeriesRef, err error) {
+	if err := a.checkErr(); err != nil {
+		return 0, err
+	}
+
+	if a.a.appendErrFn != nil {
+		if err := a.a.appendErrFn(ls); err != nil {
+			return 0, err
+		}
 	}
-	defer a.a.openAppenders.Dec()
 
 	a.a.mtx.Lock()
-	a.a.rolledbackSamples = append(a.a.rolledbackSamples, a.a.pendingSamples...)
-	a.a.pendingSamples = a.a.pendingSamples[:0]
-	a.err = errClosedAppender
+	var es []exemplar.Exemplar
+	if len(opts.Exemplars) > 0 {
+		// As per AppenderV2 interface, opts.Exemplar slice is unsafe for reuse.
+		es = make([]exemplar.Exemplar, len(opts.Exemplars))
+		copy(es, opts.Exemplars)
+	}
+	a.a.pendingSamples = append(a.a.pendingSamples, Sample{
+		MF: opts.MetricFamilyName,
+		M:  opts.Metadata,
+		L:  ls,
+		ST: st, T: t,
+		V: v, H: h, FH: fh,
+		ES: es,
+	})
 	a.a.mtx.Unlock()
 
+	var partialErr error
+	if a.a.appendExemplarsError != nil {
+		var exErrs []error
+		for range opts.Exemplars {
+			exErrs = append(exErrs, a.a.appendExemplarsError)
+		}
+		if len(exErrs) > 0 {
+			partialErr = &storage.AppendPartialError{ExemplarErrors: exErrs}
+		}
+	}
+
 	if a.next != nil {
-		return a.next.Rollback()
+		ref, err = a.next.Append(ref, ls, st, t, v, h, fh, opts)
+		if err != nil {
+			return ref, err
+		}
 	}
-	return nil
+	ref, err = computeOrCheckRef(ref, ls)
+	if err != nil {
+		return ref, err
+	}
+	return ref, partialErr
 }
diff --git a/util/teststorage/appender_test.go b/util/teststorage/appender_test.go
index 8c2a825c3a..5b0e03483b 100644
--- a/util/teststorage/appender_test.go
+++ b/util/teststorage/appender_test.go
@@ -19,62 +19,191 @@ import (
 	"testing"
 
 	"github.com/google/go-cmp/cmp"
+	"github.com/prometheus/common/model"
 	"github.com/stretchr/testify/require"
 
 	"github.com/prometheus/prometheus/model/exemplar"
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/model/metadata"
+	"github.com/prometheus/prometheus/storage"
+	"github.com/prometheus/prometheus/tsdb/tsdbutil"
 	"github.com/prometheus/prometheus/util/testutil"
 )
 
+func testAppendableV1(t *testing.T, appTest *Appendable, a storage.Appendable) {
+	for _, commit := range []bool{true, false} {
+		appTest.ResultReset()
+
+		app := a.Appender(t.Context())
+
+		ref1, err := app.Append(0, labels.FromStrings(model.MetricNameLabel, "test_metric1", "app", "v1"), 1, 2)
+		require.NoError(t, err)
+
+		h := tsdbutil.GenerateTestHistogram(0)
+		_, err = app.AppendHistogram(0, labels.FromStrings(model.MetricNameLabel, "test_metric2", "app", "v1"), 2, h, nil)
+		require.NoError(t, err)
+
+		fh := tsdbutil.GenerateTestFloatHistogram(0)
+		_, err = app.AppendHistogram(0, labels.FromStrings(model.MetricNameLabel, "test_metric3", "app", "v1"), 3, nil, fh)
+		require.NoError(t, err)
+
+		// Update meta of first series.
+		m1 := metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}
+		_, err = app.UpdateMetadata(ref1, labels.FromStrings(model.MetricNameLabel, "test_metric1", "app", "v1"), m1)
+		require.NoError(t, err)
+
+		// Add exemplars to the first series.
+		e1 := exemplar.Exemplar{Labels: labels.FromStrings(model.MetricNameLabel, "yolo"), HasTs: true, Ts: 1}
+		_, err = app.AppendExemplar(ref1, labels.FromStrings(model.MetricNameLabel, "test_metric1", "app", "v1"), e1)
+		require.NoError(t, err)
+
+		exp := []Sample{
+			{L: labels.FromStrings(model.MetricNameLabel, "test_metric1", "app", "v1"), M: m1, T: 1, V: 2, ES: []exemplar.Exemplar{e1}},
+			{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "app", "v1"), T: 2, H: h},
+			{L: labels.FromStrings(model.MetricNameLabel, "test_metric3", "app", "v1"), T: 3, FH: fh},
+		}
+		testutil.RequireEqual(t, exp, appTest.PendingSamples())
+		require.Nil(t, appTest.ResultSamples())
+		require.Nil(t, appTest.RolledbackSamples())
+
+		if commit {
+			require.NoError(t, app.Commit())
+			require.Nil(t, appTest.PendingSamples())
+			testutil.RequireEqual(t, exp, appTest.ResultSamples())
+			require.Nil(t, appTest.RolledbackSamples())
+			break
+		}
+
+		require.NoError(t, app.Rollback())
+		require.Nil(t, appTest.PendingSamples())
+		require.Nil(t, appTest.ResultSamples())
+		testutil.RequireEqual(t, exp, appTest.RolledbackSamples())
+	}
+}
+
+func testAppendableV2(t *testing.T, appTest *Appendable, a storage.AppendableV2) {
+	for _, commit := range []bool{true, false} {
+		appTest.ResultReset()
+
+		app := a.AppenderV2(t.Context())
+
+		m1 := metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}
+		e1 := exemplar.Exemplar{Labels: labels.FromStrings(model.MetricNameLabel, "yolo"), HasTs: true, Ts: 1}
+		_, err := app.Append(0, labels.FromStrings(model.MetricNameLabel, "test_metric1", "app", "v2"), -1, 1, 2, nil, nil, storage.AOptions{
+			MetricFamilyName: "test_metric1",
+			Metadata:         m1,
+			Exemplars:        []exemplar.Exemplar{e1},
+		})
+		require.NoError(t, err)
+
+		h := tsdbutil.GenerateTestHistogram(0)
+		_, err = app.Append(0, labels.FromStrings(model.MetricNameLabel, "test_metric2", "app", "v2"), -2, 2, 0, h, nil, storage.AOptions{})
+		require.NoError(t, err)
+
+		fh := tsdbutil.GenerateTestFloatHistogram(0)
+		_, err = app.Append(0, labels.FromStrings(model.MetricNameLabel, "test_metric3", "app", "v2"), -3, 3, 0, nil, fh, storage.AOptions{})
+		require.NoError(t, err)
+
+		exp := []Sample{
+			{L: labels.FromStrings(model.MetricNameLabel, "test_metric1", "app", "v2"), MF: "test_metric1", M: m1, ST: -1, T: 1, V: 2, ES: []exemplar.Exemplar{e1}},
+			{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "app", "v2"), ST: -2, T: 2, H: h},
+			{L: labels.FromStrings(model.MetricNameLabel, "test_metric3", "app", "v2"), ST: -3, T: 3, FH: fh},
+		}
+		testutil.RequireEqual(t, exp, appTest.PendingSamples())
+		require.Nil(t, appTest.ResultSamples())
+		require.Nil(t, appTest.RolledbackSamples())
+
+		if commit {
+			require.NoError(t, app.Commit())
+			require.Nil(t, appTest.PendingSamples())
+			testutil.RequireEqual(t, exp, appTest.ResultSamples())
+			require.Nil(t, appTest.RolledbackSamples())
+			break
+		}
+
+		require.NoError(t, app.Rollback())
+		require.Nil(t, appTest.PendingSamples())
+		require.Nil(t, appTest.ResultSamples())
+		testutil.RequireEqual(t, exp, appTest.RolledbackSamples())
+	}
+}
+
+func TestAppendable(t *testing.T) {
+	appTest := NewAppendable()
+	testAppendableV1(t, appTest, appTest)
+	testAppendableV2(t, appTest, appTest)
+}
+
+func TestAppendable_Then(t *testing.T) {
+	nextAppTest := NewAppendable()
+	app := NewAppendable().Then(nextAppTest)
+
+	// Ensure next mock record all the appends when appending to app.
+	testAppendableV1(t, nextAppTest, app)
+
+	// V2 should fail as Then was supplied with Appendable V1.
+	require.Error(t, app.AppenderV2(t.Context()).Commit())
+}
+
+func TestAppendable_ThenV2(t *testing.T) {
+	nextAppTest := NewAppendable()
+	app := NewAppendable().ThenV2(nextAppTest)
+
+	// Ensure next mock record all the appends when appending to app.
+	testAppendableV2(t, nextAppTest, app)
+
+	// V1 should fail as ThenV2 was supplied with Appendable V2.
+	require.Error(t, app.Appender(t.Context()).Commit())
+}
+
 // TestSample_RequireEqual ensures standard testutil.RequireEqual is enough for comparisons.
 // This is thanks to the fact metadata has now Equals method.
 func TestSample_RequireEqual(t *testing.T) {
 	a := []Sample{
 		{},
-		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
-		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
-		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings(model.MetricNameLabel, "yolo")}}},
 	}
 	testutil.RequireEqual(t, a, a)
 
 	b1 := []Sample{
 		{},
-		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
-		{L: labels.FromStrings("__name__", "test_metric2_diff", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123}, // test_metric2_diff is different.
-		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric2_diff", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123}, // test_metric2_diff is different.
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings(model.MetricNameLabel, "yolo")}}},
 	}
 	requireNotEqual(t, a, b1)
 
 	b2 := []Sample{
 		{},
-		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
-		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
-		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo2")}}}, // exemplar is different.
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings(model.MetricNameLabel, "yolo2")}}}, // exemplar is different.
 	}
 	requireNotEqual(t, a, b2)
 
 	b3 := []Sample{
 		{},
-		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
-		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123, T: 123}, // Timestamp is different.
-		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123, T: 123}, // Timestamp is different.
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings(model.MetricNameLabel, "yolo")}}},
 	}
 	requireNotEqual(t, a, b3)
 
 	b4 := []Sample{
 		{},
-		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
-		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 456.456}, // Value is different.
-		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric_total"), M: metadata.Metadata{Type: "counter", Unit: "metric", Help: "some help text"}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 456.456}, // Value is different.
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings(model.MetricNameLabel, "yolo")}}},
 	}
 	requireNotEqual(t, a, b4)
 
 	b5 := []Sample{
 		{},
-		{L: labels.FromStrings("__name__", "test_metric_total"), M: metadata.Metadata{Type: "counter2", Unit: "metric", Help: "some help text"}}, // Different type.
-		{L: labels.FromStrings("__name__", "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
-		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings("__name__", "yolo")}}},
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric_total"), M: metadata.Metadata{Type: "counter2", Unit: "metric", Help: "some help text"}}, // Different type.
+		{L: labels.FromStrings(model.MetricNameLabel, "test_metric2", "foo", "bar"), M: metadata.Metadata{Type: "gauge", Unit: "", Help: "other help text"}, V: 123.123},
+		{ES: []exemplar.Exemplar{{Labels: labels.FromStrings(model.MetricNameLabel, "yolo")}}},
 	}
 	requireNotEqual(t, a, b5)
 }
@@ -129,3 +258,40 @@ func TestConcurrentAppender_ReturnsErrAppender(t *testing.T) {
 	require.Error(t, app.Commit())
 	require.Error(t, app.Rollback())
 }
+
+func TestConcurrentAppenderV2_ReturnsErrAppender(t *testing.T) {
+	a := NewAppendable()
+
+	// Non-concurrent multiple use if fine.
+	app := a.AppenderV2(t.Context())
+	require.Equal(t, int32(1), a.openAppenders.Load())
+	require.NoError(t, app.Commit())
+	// Repeated commit fails.
+	require.Error(t, app.Commit())
+
+	app = a.AppenderV2(t.Context())
+	require.NoError(t, app.Rollback())
+	// Commit after rollback fails.
+	require.Error(t, app.Commit())
+
+	a.WithErrs(
+		nil,
+		nil,
+		errors.New("commit err"),
+	)
+	app = a.AppenderV2(t.Context())
+	require.Error(t, app.Commit())
+
+	a.WithErrs(nil, nil, nil)
+	app = a.AppenderV2(t.Context())
+	require.NoError(t, app.Commit())
+	require.Equal(t, int32(0), a.openAppenders.Load())
+
+	// Concurrent use should return appender that errors.
+	_ = a.AppenderV2(t.Context())
+	app = a.AppenderV2(t.Context())
+	_, err := app.Append(0, labels.EmptyLabels(), 0, 0, 0, nil, nil, storage.AOptions{})
+	require.Error(t, err)
+	require.Error(t, app.Commit())
+	require.Error(t, app.Rollback())
+}

From 49c3aea56d37197e5575c336e60a3ee1c9d8a076 Mon Sep 17 00:00:00 2001
From: bwplotka 
Date: Mon, 12 Jan 2026 08:45:26 +0000
Subject: [PATCH 264/439] feat(storage)[PART4b]: add AppenderV2 to the rest of
 storage.Storage impl

Signed-off-by: bwplotka 
---
 cmd/prometheus/main.go      |  17 ++++++
 storage/fanout.go           |  64 ++++++++++++++++++++
 storage/fanout_test.go      | 116 +++++++++++++++++++++++++++++++++++-
 storage/interface.go        |  29 ++++++---
 storage/interface_append.go |   1 +
 storage/remote/storage.go   |   7 +++
 storage/remote/write.go     |  50 +++++++++++++---
 tsdb/head_append_v2.go      |   1 +
 8 files changed, 267 insertions(+), 18 deletions(-)

diff --git a/cmd/prometheus/main.go b/cmd/prometheus/main.go
index ee60e58b2e..8b82049f50 100644
--- a/cmd/prometheus/main.go
+++ b/cmd/prometheus/main.go
@@ -1746,6 +1746,14 @@ func (s *readyStorage) Appender(ctx context.Context) storage.Appender {
 	return notReadyAppender{}
 }
 
+// AppenderV2 implements the Storage interface.
+func (s *readyStorage) AppenderV2(ctx context.Context) storage.AppenderV2 {
+	if x := s.get(); x != nil {
+		return x.AppenderV2(ctx)
+	}
+	return notReadyAppenderV2{}
+}
+
 type notReadyAppender struct{}
 
 // SetOptions does nothing in this appender implementation.
@@ -1779,6 +1787,15 @@ func (notReadyAppender) Commit() error { return tsdb.ErrNotReady }
 
 func (notReadyAppender) Rollback() error { return tsdb.ErrNotReady }
 
+type notReadyAppenderV2 struct{}
+
+func (notReadyAppenderV2) Append(storage.SeriesRef, labels.Labels, int64, int64, float64, *histogram.Histogram, *histogram.FloatHistogram, storage.AOptions) (storage.SeriesRef, error) {
+	return 0, tsdb.ErrNotReady
+}
+func (notReadyAppenderV2) Commit() error { return tsdb.ErrNotReady }
+
+func (notReadyAppenderV2) Rollback() error { return tsdb.ErrNotReady }
+
 // Close implements the Storage interface.
 func (s *readyStorage) Close() error {
 	if x := s.get(); x != nil {
diff --git a/storage/fanout.go b/storage/fanout.go
index afcf993b3f..c5102b442f 100644
--- a/storage/fanout.go
+++ b/storage/fanout.go
@@ -136,6 +136,19 @@ func (f *fanout) Appender(ctx context.Context) Appender {
 	}
 }
 
+func (f *fanout) AppenderV2(ctx context.Context) AppenderV2 {
+	primary := f.primary.AppenderV2(ctx)
+	secondaries := make([]AppenderV2, 0, len(f.secondaries))
+	for _, storage := range f.secondaries {
+		secondaries = append(secondaries, storage.AppenderV2(ctx))
+	}
+	return &fanoutAppenderV2{
+		logger:      f.logger,
+		primary:     primary,
+		secondaries: secondaries,
+	}
+}
+
 // Close closes the storage and all its underlying resources.
 func (f *fanout) Close() error {
 	errs := []error{
@@ -278,3 +291,54 @@ func (f *fanoutAppender) Rollback() (err error) {
 	}
 	return nil
 }
+
+type fanoutAppenderV2 struct {
+	logger *slog.Logger
+
+	primary     AppenderV2
+	secondaries []AppenderV2
+}
+
+func (f *fanoutAppenderV2) Append(ref SeriesRef, l labels.Labels, st, t int64, v float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts AOptions) (SeriesRef, error) {
+	ref, err := f.primary.Append(ref, l, st, t, v, h, fh, opts)
+	if err != nil {
+		return ref, err
+	}
+
+	for _, appender := range f.secondaries {
+		if _, err := appender.Append(ref, l, st, t, v, h, fh, opts); err != nil {
+			return 0, err
+		}
+	}
+	return ref, nil
+}
+
+func (f *fanoutAppenderV2) Commit() (err error) {
+	err = f.primary.Commit()
+
+	for _, appender := range f.secondaries {
+		if err == nil {
+			err = appender.Commit()
+		} else {
+			if rollbackErr := appender.Rollback(); rollbackErr != nil {
+				f.logger.Error("Squashed rollback error on commit", "err", rollbackErr)
+			}
+		}
+	}
+	return err
+}
+
+func (f *fanoutAppenderV2) Rollback() (err error) {
+	err = f.primary.Rollback()
+
+	for _, appender := range f.secondaries {
+		rollbackErr := appender.Rollback()
+		switch {
+		case err == nil:
+			err = rollbackErr
+		case rollbackErr != nil:
+			f.logger.Error("Squashed rollback error on rollback", "err", rollbackErr)
+		}
+	}
+	return nil
+}
diff --git a/storage/fanout_test.go b/storage/fanout_test.go
index ed4cf17696..fb2f8dd553 100644
--- a/storage/fanout_test.go
+++ b/storage/fanout_test.go
@@ -132,6 +132,115 @@ func TestFanout_SelectSorted(t *testing.T) {
 	})
 }
 
+func TestFanout_SelectSorted_AppenderV2(t *testing.T) {
+	inputLabel := labels.FromStrings(model.MetricNameLabel, "a")
+	outputLabel := labels.FromStrings(model.MetricNameLabel, "a")
+
+	inputTotalSize := 0
+
+	priStorage := teststorage.New(t)
+	defer priStorage.Close()
+	app1 := priStorage.AppenderV2(t.Context())
+	_, err := app1.Append(0, inputLabel, 0, 0, 0, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	_, err = app1.Append(0, inputLabel, 0, 1000, 1, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	_, err = app1.Append(0, inputLabel, 0, 2000, 2, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	require.NoError(t, app1.Commit())
+
+	remoteStorage1 := teststorage.New(t)
+	defer remoteStorage1.Close()
+	app2 := remoteStorage1.AppenderV2(t.Context())
+	_, err = app2.Append(0, inputLabel, 0, 3000, 3, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	_, err = app2.Append(0, inputLabel, 0, 4000, 4, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	_, err = app2.Append(0, inputLabel, 0, 5000, 5, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	require.NoError(t, app2.Commit())
+
+	remoteStorage2 := teststorage.New(t)
+	defer remoteStorage2.Close()
+
+	app3 := remoteStorage2.AppenderV2(t.Context())
+	_, err = app3.Append(0, inputLabel, 0, 6000, 6, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	_, err = app3.Append(0, inputLabel, 0, 7000, 7, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+	_, err = app3.Append(0, inputLabel, 0, 8000, 8, nil, nil, storage.AOptions{})
+	require.NoError(t, err)
+	inputTotalSize++
+
+	require.NoError(t, app3.Commit())
+
+	fanoutStorage := storage.NewFanout(nil, priStorage, remoteStorage1, remoteStorage2)
+
+	t.Run("querier", func(t *testing.T) {
+		querier, err := fanoutStorage.Querier(0, 8000)
+		require.NoError(t, err)
+		defer querier.Close()
+
+		matcher, err := labels.NewMatcher(labels.MatchEqual, model.MetricNameLabel, "a")
+		require.NoError(t, err)
+
+		seriesSet := querier.Select(t.Context(), true, nil, matcher)
+
+		result := make(map[int64]float64)
+		var labelsResult labels.Labels
+		var iterator chunkenc.Iterator
+		for seriesSet.Next() {
+			series := seriesSet.At()
+			seriesLabels := series.Labels()
+			labelsResult = seriesLabels
+			iterator := series.Iterator(iterator)
+			for iterator.Next() == chunkenc.ValFloat {
+				timestamp, value := iterator.At()
+				result[timestamp] = value
+			}
+		}
+
+		require.Equal(t, labelsResult, outputLabel)
+		require.Len(t, result, inputTotalSize)
+	})
+	t.Run("chunk querier", func(t *testing.T) {
+		querier, err := fanoutStorage.ChunkQuerier(0, 8000)
+		require.NoError(t, err)
+		defer querier.Close()
+
+		matcher, err := labels.NewMatcher(labels.MatchEqual, model.MetricNameLabel, "a")
+		require.NoError(t, err)
+
+		seriesSet := storage.NewSeriesSetFromChunkSeriesSet(querier.Select(t.Context(), true, nil, matcher))
+
+		result := make(map[int64]float64)
+		var labelsResult labels.Labels
+		var iterator chunkenc.Iterator
+		for seriesSet.Next() {
+			series := seriesSet.At()
+			seriesLabels := series.Labels()
+			labelsResult = seriesLabels
+			iterator := series.Iterator(iterator)
+			for iterator.Next() == chunkenc.ValFloat {
+				timestamp, value := iterator.At()
+				result[timestamp] = value
+			}
+		}
+
+		require.NoError(t, seriesSet.Err())
+		require.Equal(t, labelsResult, outputLabel)
+		require.Len(t, result, inputTotalSize)
+	})
+}
+
 func TestFanoutErrors(t *testing.T) {
 	workingStorage := teststorage.New(t)
 	defer workingStorage.Close()
@@ -224,9 +333,10 @@ type errChunkQuerier struct{ errQuerier }
 func (errStorage) ChunkQuerier(_, _ int64) (storage.ChunkQuerier, error) {
 	return errChunkQuerier{}, nil
 }
-func (errStorage) Appender(context.Context) storage.Appender { return nil }
-func (errStorage) StartTime() (int64, error)                 { return 0, nil }
-func (errStorage) Close() error                              { return nil }
+func (errStorage) Appender(context.Context) storage.Appender     { return nil }
+func (errStorage) AppenderV2(context.Context) storage.AppenderV2 { return nil }
+func (errStorage) StartTime() (int64, error)                     { return 0, nil }
+func (errStorage) Close() error                                  { return nil }
 
 func (errQuerier) Select(context.Context, bool, *storage.SelectHints, ...*labels.Matcher) storage.SeriesSet {
 	return storage.ErrSeriesSet(errSelect)
diff --git a/storage/interface.go b/storage/interface.go
index 23b8b48a0c..d6ce895d58 100644
--- a/storage/interface.go
+++ b/storage/interface.go
@@ -61,7 +61,8 @@ type SeriesRef uint64
 
 // Appendable allows creating Appender.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// Appendable will be removed soon (ETA: Q2 2026).
 type Appendable interface {
 	// Appender returns a new appender for the storage.
 	//
@@ -77,10 +78,16 @@ type SampleAndChunkQueryable interface {
 }
 
 // Storage ingests and manages samples, along with various indexes. All methods
-// are goroutine-safe. Storage implements storage.Appender.
+// are goroutine-safe.
 type Storage interface {
 	SampleAndChunkQueryable
+
+	// Appendable allows appending to storage.
+	// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+	// Appendable will be removed soon (ETA: Q2 2026).
 	Appendable
+	// AppendableV2 allows appending to storage.
+	AppendableV2
 
 	// StartTime returns the oldest timestamp stored in the storage.
 	StartTime() (int64, error)
@@ -261,7 +268,8 @@ func (f QueryableFunc) Querier(mint, maxt int64) (Querier, error) {
 
 // AppendOptions provides options for implementations of the Appender interface.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// AppendOptions will be removed soon (ETA: Q2 2026).
 type AppendOptions struct {
 	// DiscardOutOfOrder tells implementation that this append should not be out
 	// of order. An OOO append MUST be rejected with storage.ErrOutOfOrderSample
@@ -278,7 +286,8 @@ type AppendOptions struct {
 // I.e. timestamp order within batch is not validated, samples are not reordered per timestamp or by float/histogram
 // type.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// Appender will be removed soon (ETA: Q2 2026).
 type Appender interface {
 	AppenderTransaction
 
@@ -315,7 +324,8 @@ type GetRef interface {
 // ExemplarAppender provides an interface for adding samples to exemplar storage, which
 // within Prometheus is in-memory only.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// ExemplarAppender will be removed soon (ETA: Q2 2026).
 type ExemplarAppender interface {
 	// AppendExemplar adds an exemplar for the given series labels.
 	// An optional reference number can be provided to accelerate calls.
@@ -333,7 +343,8 @@ type ExemplarAppender interface {
 
 // HistogramAppender provides an interface for appending histograms to the storage.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// HistogramAppender will be removed soon (ETA: Q2 2026).
 type HistogramAppender interface {
 	// AppendHistogram adds a histogram for the given series labels. An
 	// optional reference number can be provided to accelerate calls. A
@@ -365,7 +376,8 @@ type HistogramAppender interface {
 
 // MetadataUpdater provides an interface for associating metadata to stored series.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// MetadataUpdater will be removed soon (ETA: Q2 2026).
 type MetadataUpdater interface {
 	// UpdateMetadata updates a metadata entry for the given series and labels.
 	// A series reference number is returned which can be used to modify the
@@ -379,7 +391,8 @@ type MetadataUpdater interface {
 
 // StartTimestampAppender provides an interface for appending ST to storage.
 //
-// WARNING: Work AppendableV2 is in progress. Appendable will be removed soon (ETA: Q2 2026).
+// WARNING(bwplotka): Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632).
+// StartTimestampAppender will be removed soon (ETA: Q2 2026).
 type StartTimestampAppender interface {
 	// AppendSTZeroSample adds synthetic zero sample for the given st timestamp,
 	// which will be associated with given series, labels and the incoming
diff --git a/storage/interface_append.go b/storage/interface_append.go
index cc7045dbd5..f2dce8e52e 100644
--- a/storage/interface_append.go
+++ b/storage/interface_append.go
@@ -69,6 +69,7 @@ type AppendV2Options struct {
 	// Exemplars (optional) attached to the appended sample.
 	// Exemplar slice MUST be sorted by Exemplar.TS.
 	// Exemplar slice is unsafe for reuse.
+	// Duplicate exemplars errors MUST be ignored by implementations.
 	Exemplars []exemplar.Exemplar
 
 	// RejectOutOfOrder tells implementation that this append should not be out
diff --git a/storage/remote/storage.go b/storage/remote/storage.go
index f482597249..be75d23383 100644
--- a/storage/remote/storage.go
+++ b/storage/remote/storage.go
@@ -63,6 +63,8 @@ type Storage struct {
 	localStartTimeCallback startTimeCallback
 }
 
+var _ storage.Storage = &Storage{}
+
 // NewStorage returns a remote.Storage.
 func NewStorage(l *slog.Logger, reg prometheus.Registerer, stCallback startTimeCallback, walDir string, flushDeadline time.Duration, sm ReadyScrapeManager, enableTypeAndUnitLabels bool) *Storage {
 	if l == nil {
@@ -193,6 +195,11 @@ func (s *Storage) Appender(ctx context.Context) storage.Appender {
 	return s.rws.Appender(ctx)
 }
 
+// AppenderV2 implements storage.Storage.
+func (s *Storage) AppenderV2(ctx context.Context) storage.AppenderV2 {
+	return s.rws.AppenderV2(ctx)
+}
+
 // LowestSentTimestamp returns the lowest sent timestamp across all queues.
 func (s *Storage) LowestSentTimestamp() int64 {
 	return s.rws.LowestSentTimestamp()
diff --git a/storage/remote/write.go b/storage/remote/write.go
index 92f447d624..91000a1d25 100644
--- a/storage/remote/write.go
+++ b/storage/remote/write.go
@@ -238,8 +238,20 @@ func (rws *WriteStorage) ApplyConfig(conf *config.Config) error {
 // Appender implements storage.Storage.
 func (rws *WriteStorage) Appender(context.Context) storage.Appender {
 	return ×tampTracker{
-		writeStorage:         rws,
-		highestRecvTimestamp: rws.highestTimestamp,
+		baseTimestampTracker: baseTimestampTracker{
+			writeStorage:         rws,
+			highestRecvTimestamp: rws.highestTimestamp,
+		},
+	}
+}
+
+// AppenderV2 implements storage.Storage.
+func (rws *WriteStorage) AppenderV2(context.Context) storage.AppenderV2 {
+	return ×tampTrackerV2{
+		baseTimestampTracker: baseTimestampTracker{
+			writeStorage:         rws,
+			highestRecvTimestamp: rws.highestTimestamp,
+		},
 	}
 }
 
@@ -282,9 +294,9 @@ func (rws *WriteStorage) Close() error {
 	return nil
 }
 
-type timestampTracker struct {
-	writeStorage         *WriteStorage
-	appendOptions        *storage.AppendOptions
+type baseTimestampTracker struct {
+	writeStorage *WriteStorage
+
 	samples              int64
 	exemplars            int64
 	histograms           int64
@@ -292,6 +304,12 @@ type timestampTracker struct {
 	highestRecvTimestamp *maxTimestamp
 }
 
+type timestampTracker struct {
+	baseTimestampTracker
+
+	appendOptions *storage.AppendOptions
+}
+
 func (t *timestampTracker) SetOptions(opts *storage.AppendOptions) {
 	t.appendOptions = opts
 }
@@ -345,7 +363,7 @@ func (*timestampTracker) UpdateMetadata(storage.SeriesRef, labels.Labels, metada
 }
 
 // Commit implements storage.Appender.
-func (t *timestampTracker) Commit() error {
+func (t *baseTimestampTracker) Commit() error {
 	t.writeStorage.samplesIn.incr(t.samples + t.exemplars + t.histograms)
 
 	samplesIn.Add(float64(t.samples))
@@ -356,6 +374,24 @@ func (t *timestampTracker) Commit() error {
 }
 
 // Rollback implements storage.Appender.
-func (*timestampTracker) Rollback() error {
+func (*baseTimestampTracker) Rollback() error {
 	return nil
 }
+
+type timestampTrackerV2 struct {
+	baseTimestampTracker
+}
+
+func (t *timestampTrackerV2) Append(ref storage.SeriesRef, _ labels.Labels, _, ts int64, _ float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) {
+	switch {
+	case fh != nil, h != nil:
+		t.histograms++
+	default:
+		t.samples++
+	}
+	if ts > t.highestTimestamp {
+		t.highestTimestamp = ts
+	}
+	t.exemplars += int64(len(opts.Exemplars))
+	return ref, nil
+}
diff --git a/tsdb/head_append_v2.go b/tsdb/head_append_v2.go
index 241fb42e97..4a62d56741 100644
--- a/tsdb/head_append_v2.go
+++ b/tsdb/head_append_v2.go
@@ -323,6 +323,7 @@ func (a *headAppenderV2) appendExemplars(s *memSeries, exemplar []exemplar.Exemp
 		if err := a.head.exemplars.ValidateExemplar(s.labels(), e); err != nil {
 			if !errors.Is(err, storage.ErrDuplicateExemplar) && !errors.Is(err, storage.ErrExemplarsDisabled) {
 				// Except duplicates, return partial errors.
+				// TODO(bwplotka): Add exemplar info into error.
 				errs = append(errs, err)
 				continue
 			}

From 8a2921e3851d886367308aaedc7545ee4cd8138a Mon Sep 17 00:00:00 2001
From: bwplotka 
Date: Wed, 14 Jan 2026 13:57:48 +0000
Subject: [PATCH 265/439] addressed feedback

Signed-off-by: bwplotka 
---
 storage/fanout.go       | 4 ++--
 storage/remote/write.go | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/storage/fanout.go b/storage/fanout.go
index c5102b442f..95c6499952 100644
--- a/storage/fanout.go
+++ b/storage/fanout.go
@@ -289,7 +289,7 @@ func (f *fanoutAppender) Rollback() (err error) {
 			f.logger.Error("Squashed rollback error on rollback", "err", rollbackErr)
 		}
 	}
-	return nil
+	return err
 }
 
 type fanoutAppenderV2 struct {
@@ -340,5 +340,5 @@ func (f *fanoutAppenderV2) Rollback() (err error) {
 			f.logger.Error("Squashed rollback error on rollback", "err", rollbackErr)
 		}
 	}
-	return nil
+	return err
 }
diff --git a/storage/remote/write.go b/storage/remote/write.go
index 91000a1d25..6a336dc06b 100644
--- a/storage/remote/write.go
+++ b/storage/remote/write.go
@@ -382,6 +382,7 @@ type timestampTrackerV2 struct {
 	baseTimestampTracker
 }
 
+// Append implements storage.AppenderV2.
 func (t *timestampTrackerV2) Append(ref storage.SeriesRef, _ labels.Labels, _, ts int64, _ float64, h *histogram.Histogram, fh *histogram.FloatHistogram, opts storage.AOptions) (storage.SeriesRef, error) {
 	switch {
 	case fh != nil, h != nil:

From 06a59346fe3236cddbd5d739b38c2ac52fa686e4 Mon Sep 17 00:00:00 2001
From: George Krajcsovits 
Date: Wed, 14 Jan 2026 16:41:03 +0100
Subject: [PATCH 266/439] Update tsdb/chunkenc/chunk.go

Co-authored-by: Arve Knudsen 
Signed-off-by: George Krajcsovits 
---
 tsdb/chunkenc/chunk.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tsdb/chunkenc/chunk.go b/tsdb/chunkenc/chunk.go
index d5e028e681..711966ec39 100644
--- a/tsdb/chunkenc/chunk.go
+++ b/tsdb/chunkenc/chunk.go
@@ -152,7 +152,7 @@ type Iterator interface {
 	// Before the iterator has advanced, the behaviour is unspecified.
 	AtT() int64
 	// AtST returns the current start timestamp.
-	// Return 0 if the start timestamp is not implemented or not set.
+	// Returns 0 if the start timestamp is not implemented or not set.
 	// Before the iterator has advanced, the behaviour is unspecified.
 	AtST() int64
 	// Err returns the current error. It should be used only after the

From ccb7468b0917abbce8d932a83185366b5b7fc440 Mon Sep 17 00:00:00 2001
From: Julius Hinze 
Date: Wed, 14 Jan 2026 16:44:50 +0100
Subject: [PATCH 267/439] tsdb: fix grow/shrink nextIndex calculation (#17863)

Signed-off-by: Julius Hinze 
---
 tsdb/exemplar.go      | 21 +++++++++++--------
 tsdb/exemplar_test.go | 49 +++++++++++++++++++++++++++++++++++++++----
 2 files changed, 57 insertions(+), 13 deletions(-)

diff --git a/tsdb/exemplar.go b/tsdb/exemplar.go
index b58976c911..71589cf21e 100644
--- a/tsdb/exemplar.go
+++ b/tsdb/exemplar.go
@@ -327,7 +327,8 @@ func (ce *CircularExemplarStorage) grow(l int64) int {
 		{from: ce.nextIndex, to: oldSize},
 		{from: 0, to: ce.nextIndex},
 	}
-	ce.nextIndex = copyExemplarRanges(ce.index, newSlice, ce.exemplars, ranges)
+	totalCopied, _ := copyExemplarRanges(ce.index, newSlice, ce.exemplars, ranges)
+	ce.nextIndex = totalCopied
 	ce.exemplars = newSlice
 	return oldSize
 }
@@ -353,6 +354,7 @@ func (ce *CircularExemplarStorage) shrink(l int64) (migrated int) {
 
 	newSlice := make([]circularBufferEntry, int(l))
 
+	var totalCopied int
 	switch {
 	case deleteStart == deleteEnd:
 		// The entire buffer was cleared (shrink to zero). Note that we don't have to
@@ -363,18 +365,18 @@ func (ce *CircularExemplarStorage) shrink(l int64) (migrated int) {
 		return 0
 	case deleteStart < deleteEnd:
 		// We delete an "inner" section of the circular buffer.
-		migrated = copyExemplarRanges(ce.index, newSlice, ce.exemplars, []intRange{
+		totalCopied, migrated = copyExemplarRanges(ce.index, newSlice, ce.exemplars, []intRange{
 			{from: deleteEnd, to: oldSize},
 			{from: 0, to: deleteStart},
 		})
 	case deleteStart > deleteEnd:
 		// We keep an "inner" section of the circular buffer.
-		migrated = copyExemplarRanges(ce.index, newSlice, ce.exemplars, []intRange{
+		totalCopied, migrated = copyExemplarRanges(ce.index, newSlice, ce.exemplars, []intRange{
 			{from: deleteEnd, to: deleteStart},
 		})
 	}
 
-	ce.nextIndex = migrated % int(l)
+	ce.nextIndex = totalCopied % int(l)
 	ce.exemplars = newSlice
 	return migrated
 }
@@ -582,20 +584,21 @@ func (e intRange) contains(i int) bool {
 }
 
 // copyExemplarRanges copies non-overlapping ranges from src into dest and
-// adjusts list pointers in dest and index accordingly. Returns the number of
-// copied items.
+// adjusts list pointers in dest and index accordingly. Returns the total
+// number of slots copied (for nextIndex) and the number of non-empty entries
+// migrated.
 func copyExemplarRanges(
 	index map[string]*indexEntry,
 	dest, src []circularBufferEntry,
 	ranges []intRange,
-) int {
+) (totalCopied, migratedEntries int) {
 	offsets := make([]int, len(ranges))
 	n := 0
 	for i, rng := range ranges {
 		offsets[i] = n - rng.from
 		n += copy(dest[n:], src[rng.from:rng.to])
 	}
-	migratedEntries := n
+	migratedEntries = n
 	for di := range n {
 		e := &dest[di]
 		if e.ref == nil {
@@ -631,5 +634,5 @@ func copyExemplarRanges(
 			}
 		}
 	}
-	return migratedEntries
+	return n, migratedEntries
 }
diff --git a/tsdb/exemplar_test.go b/tsdb/exemplar_test.go
index 01ffeb9541..6ecba25489 100644
--- a/tsdb/exemplar_test.go
+++ b/tsdb/exemplar_test.go
@@ -390,7 +390,7 @@ func TestCircularExemplarStorage_Resize(t *testing.T) {
 				{Labels: series1, Value: 0.1, Ts: 1},
 				{Labels: series1, Value: 0.2, Ts: 2},
 			},
-			wantNextIndex: 2,
+			wantNextIndex: 3,
 		},
 		{
 			name: "in-order, shrink",
@@ -431,7 +431,7 @@ func TestCircularExemplarStorage_Resize(t *testing.T) {
 				{Labels: series1, Value: 0.2, Ts: 2},
 				{Labels: series1, Value: 0.3, Ts: 3},
 			},
-			wantNextIndex: 2,
+			wantNextIndex: 3,
 		},
 		{
 			name: "duplicate timestamps",
@@ -452,7 +452,7 @@ func TestCircularExemplarStorage_Resize(t *testing.T) {
 			exemplars:     []exemplar.Exemplar{},
 			resize:        10,
 			wantExemplars: []exemplar.Exemplar{},
-			wantNextIndex: 0,
+			wantNextIndex: 3,
 		},
 		{
 			name:          "empty input, shrink",
@@ -507,7 +507,7 @@ func TestCircularExemplarStorage_Resize(t *testing.T) {
 			wantExemplars: []exemplar.Exemplar{
 				{Labels: series1, Value: 0.1, Ts: 1},
 			},
-			wantNextIndex: 1,
+			wantNextIndex: 0,
 		},
 	}
 
@@ -660,6 +660,47 @@ func TestCircularExemplarStorage_Resize(t *testing.T) {
 				{Labels: series1, Value: 0.6, Ts: 6},
 			},
 		},
+		{
+			name: "grow non-full buffer then add entries",
+			addExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			resize1: 10,
+			wantExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			resize2: 10,
+			addExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+			wantExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+				{Labels: series1, Value: 0.3, Ts: 3},
+				{Labels: series1, Value: 0.4, Ts: 4},
+			},
+		},
+		{
+			name: "shrink non-full buffer then add entries",
+			addExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+			},
+			resize1: 2,
+			wantExemplars1: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+			},
+			resize2: 2,
+			addExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+			wantExemplars2: []exemplar.Exemplar{
+				{Labels: series1, Value: 0.1, Ts: 1},
+				{Labels: series1, Value: 0.2, Ts: 2},
+			},
+		},
 	}
 
 	for _, tc := range resizeTwiceCases {

From af3277f8326431808ecec2a2095404ad3422a929 Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Wed, 3 Dec 2025 18:46:35 +0100
Subject: [PATCH 268/439] PromQL: Add `fill*()` binop modifiers to provide
 default values for missing series

Signed-off-by: Julius Volz 
---
 promql/engine.go                              |   54 +-
 promql/parser/ast.go                          |   13 +
 promql/parser/generated_parser.y              |   73 +-
 promql/parser/generated_parser.y.go           | 1284 +++++++++--------
 promql/parser/lex.go                          |   31 +
 promql/parser/parse.go                        |    6 +
 promql/parser/printer.go                      |   13 +
 promql/parser/printer_test.go                 |   20 +
 web/api/v1/translate_ast.go                   |    4 +
 .../ExplainViews/BinaryExpr/VectorVector.tsx  |   73 +-
 web/ui/mantine-ui/src/promql/ast.ts           |    5 +
 web/ui/mantine-ui/src/promql/binOp.test.ts    |   24 +
 web/ui/mantine-ui/src/promql/binOp.ts         |   29 +-
 web/ui/mantine-ui/src/promql/format.tsx       |   43 +-
 web/ui/mantine-ui/src/promql/serialize.ts     |   18 +-
 .../src/promql/serializeAndFormat.test.ts     |   13 +
 .../src/complete/promql.terms.ts              |    4 +
 .../src/parser/vector.test.ts                 |   85 +-
 .../codemirror-promql/src/parser/vector.ts    |   35 +
 .../codemirror-promql/src/types/vector.ts     |    7 +
 web/ui/module/lezer-promql/src/promql.grammar |   24 +-
 web/ui/module/lezer-promql/src/tokens.js      |  140 +-
 22 files changed, 1296 insertions(+), 702 deletions(-)

diff --git a/promql/engine.go b/promql/engine.go
index 57a1f41bb8..b609dc4f0a 100644
--- a/promql/engine.go
+++ b/promql/engine.go
@@ -2862,7 +2862,8 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching *
 	if matching.Card == parser.CardManyToMany {
 		panic("many-to-many only allowed for set operators")
 	}
-	if len(lhs) == 0 || len(rhs) == 0 {
+	if (len(lhs) == 0 && len(rhs) == 0) ||
+		((len(lhs) == 0 || len(rhs) == 0) && matching.FillValues.RHS == nil && matching.FillValues.LHS == nil) {
 		return nil, nil // Short-circuit: nothing is going to match.
 	}
 
@@ -2910,17 +2911,9 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching *
 	}
 	matchedSigs := enh.matchedSigs
 
-	// For all lhs samples find a respective rhs sample and perform
-	// the binary operation.
 	var lastErr error
-	for i, ls := range lhs {
-		sigOrd := lhsh[i].sigOrdinal
-
-		rs, found := rightSigs[sigOrd] // Look for a match in the rhs Vector.
-		if !found {
-			continue
-		}
 
+	doBinOp := func(ls, rs Sample, sigOrd int) {
 		// Account for potentially swapped sidedness.
 		fl, fr := ls.F, rs.F
 		hl, hr := ls.H, rs.H
@@ -2931,7 +2924,7 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching *
 		floatValue, histogramValue, keep, info, err := vectorElemBinop(op, fl, fr, hl, hr, pos)
 		if err != nil {
 			lastErr = err
-			continue
+			return
 		}
 		if info != nil {
 			lastErr = info
@@ -2971,7 +2964,7 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching *
 		}
 
 		if !keep && !returnBool {
-			continue
+			return
 		}
 
 		enh.Out = append(enh.Out, Sample{
@@ -2981,6 +2974,43 @@ func (ev *evaluator) VectorBinop(op parser.ItemType, lhs, rhs Vector, matching *
 			DropName: returnBool,
 		})
 	}
+
+	// For all lhs samples, find a respective rhs sample and perform
+	// the binary operation.
+	for i, ls := range lhs {
+		sigOrd := lhsh[i].sigOrdinal
+
+		rs, found := rightSigs[sigOrd] // Look for a match in the rhs Vector.
+		if !found {
+			fill := matching.FillValues.RHS
+			if fill == nil {
+				continue
+			}
+			rs = Sample{
+				Metric: ls.Metric.MatchLabels(matching.On, matching.MatchingLabels...),
+				F:      *fill,
+			}
+		}
+
+		doBinOp(ls, rs, sigOrd)
+	}
+
+	// For any rhs samples which have not been matched, check if we need to
+	// perform the operation with a fill value from the lhs.
+	if fill := matching.FillValues.LHS; fill != nil {
+		for sigOrd, rs := range rightSigs {
+			if _, matched := matchedSigs[sigOrd]; matched {
+				continue // Already matched.
+			}
+			ls := Sample{
+				Metric: rs.Metric.MatchLabels(matching.On, matching.MatchingLabels...),
+				F:      *fill,
+			}
+
+			doBinOp(ls, rs, sigOrd)
+		}
+	}
+
 	return enh.Out, lastErr
 }
 
diff --git a/promql/parser/ast.go b/promql/parser/ast.go
index 130f9aefb7..6496095287 100644
--- a/promql/parser/ast.go
+++ b/promql/parser/ast.go
@@ -318,6 +318,19 @@ type VectorMatching struct {
 	// Include contains additional labels that should be included in
 	// the result from the side with the lower cardinality.
 	Include []string
+	// Fill-in values to use when a series from one side does not find a match on the other side.
+	FillValues VectorMatchFillValues
+}
+
+// VectorMatchFillValues contains the fill values to use for Vector matching
+// when one side does not find a match on the other side.
+// When a fill value is nil, no fill is applied for that side, and there
+// is no output for the match group if there is no match.
+type VectorMatchFillValues struct {
+	// RHS is the fill value to use for the right-hand side.
+	RHS *float64
+	// LHS is the fill value to use for the left-hand side.
+	LHS *float64
 }
 
 // Visitor allows visiting a Node and its child nodes. The Visit method is
diff --git a/promql/parser/generated_parser.y b/promql/parser/generated_parser.y
index 47776f53d0..71ab6ed4b3 100644
--- a/promql/parser/generated_parser.y
+++ b/promql/parser/generated_parser.y
@@ -139,6 +139,9 @@ BOOL
 BY
 GROUP_LEFT
 GROUP_RIGHT
+FILL
+FILL_LEFT
+FILL_RIGHT
 IGNORING
 OFFSET
 SMOOTHED
@@ -190,7 +193,7 @@ START_METRIC_SELECTOR
 %type  int
 %type  uint
 %type  number series_value signed_number signed_or_unsigned_number
-%type  step_invariant_expr aggregate_expr aggregate_modifier bin_modifier binary_expr bool_modifier expr function_call function_call_args function_call_body group_modifiers label_matchers matrix_selector number_duration_literal offset_expr anchored_expr smoothed_expr on_or_ignoring paren_expr string_literal subquery_expr unary_expr vector_selector duration_expr paren_duration_expr positive_duration_expr offset_duration_expr
+%type  step_invariant_expr aggregate_expr aggregate_modifier bin_modifier fill_modifiers binary_expr bool_modifier expr function_call function_call_args function_call_body group_modifiers fill_value label_matchers matrix_selector number_duration_literal offset_expr anchored_expr smoothed_expr on_or_ignoring paren_expr string_literal subquery_expr unary_expr vector_selector duration_expr paren_duration_expr positive_duration_expr offset_duration_expr
 
 %start start
 
@@ -302,7 +305,7 @@ binary_expr     : expr ADD     bin_modifier expr { $$ = yylex.(*parser).newBinar
 
 // Using left recursion for the modifier rules, helps to keep the parser stack small and
 // reduces allocations.
-bin_modifier    : group_modifiers;
+bin_modifier    : fill_modifiers;
 
 bool_modifier   : /* empty */
                         { $$ = &BinaryExpr{
@@ -346,6 +349,47 @@ group_modifiers: bool_modifier /* empty */
                         }
                 ;
 
+fill_modifiers: group_modifiers /* empty */
+                /* Only fill() */
+                | group_modifiers FILL fill_value
+                        {
+                        $$ = $1
+                        fill := $3.(*NumberLiteral).Val
+                        $$.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill
+                        $$.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill
+                        }
+                /* Only fill_left() */
+                | group_modifiers FILL_LEFT fill_value
+                        {
+                        $$ = $1
+                        fill := $3.(*NumberLiteral).Val
+                        $$.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill
+                        }
+                /* Only fill_right() */
+                | group_modifiers FILL_RIGHT fill_value
+                        {
+                        $$ = $1
+                        fill := $3.(*NumberLiteral).Val
+                        $$.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill
+                        }
+                /* fill_left() fill_right() */
+                | group_modifiers FILL_LEFT fill_value FILL_RIGHT fill_value
+                        {
+                        $$ = $1
+                        fill_left := $3.(*NumberLiteral).Val
+                        fill_right := $5.(*NumberLiteral).Val
+                        $$.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill_left
+                        $$.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill_right
+                        }
+                /* fill_right() fill_left() */
+                | group_modifiers FILL_RIGHT fill_value FILL_LEFT fill_value
+                        {
+                        fill_right := $3.(*NumberLiteral).Val
+                        fill_left := $5.(*NumberLiteral).Val
+                        $$.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill_left
+                        $$.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill_right
+                        }
+                ;
 
 grouping_labels : LEFT_PAREN grouping_label_list RIGHT_PAREN
                         { $$ = $2 }
@@ -387,6 +431,21 @@ grouping_label  : maybe_label
                         { yylex.(*parser).unexpected("grouping opts", "label"); $$ = Item{} }
                 ;
 
+fill_value      : LEFT_PAREN number_duration_literal RIGHT_PAREN
+                        {
+                        $$ = $2.(*NumberLiteral)
+                        }
+                | LEFT_PAREN unary_op number_duration_literal RIGHT_PAREN
+                        {
+                        nl := $3.(*NumberLiteral)
+                        if $2.Typ == SUB {
+                          nl.Val *= -1
+                        }
+                        nl.PosRange.Start = $2.Pos
+                        $$ = nl
+                        }
+                ;
+
 /*
  * Function calls.
  */
@@ -697,7 +756,7 @@ metric          : metric_identifier label_set
                 ;
 
 
-metric_identifier: AVG | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | IDENTIFIER |  LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | QUANTILE | STDDEV | STDVAR | SUM | TOPK | WITHOUT | START | END | LIMITK | LIMIT_RATIO | STEP | RANGE | ANCHORED | SMOOTHED;
+metric_identifier: AVG | BOTTOMK | BY | COUNT | COUNT_VALUES | FILL | FILL_LEFT | FILL_RIGHT | GROUP | IDENTIFIER |  LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | QUANTILE | STDDEV | STDVAR | SUM | TOPK | WITHOUT | START | END | LIMITK | LIMIT_RATIO | STEP | RANGE | ANCHORED | SMOOTHED;
 
 label_set       : LEFT_BRACE label_set_list RIGHT_BRACE
                         { $$ = labels.New($2...) }
@@ -954,7 +1013,7 @@ counter_reset_hint : UNKNOWN_COUNTER_RESET | COUNTER_RESET | NOT_COUNTER_RESET |
 aggregate_op    : AVG | BOTTOMK | COUNT | COUNT_VALUES | GROUP | MAX | MIN | QUANTILE | STDDEV | STDVAR | SUM | TOPK | LIMITK | LIMIT_RATIO;
 
 // Inside of grouping options label names can be recognized as keywords by the lexer. This is a list of keywords that could also be a label name.
-maybe_label     : AVG | BOOL | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | GROUP_LEFT | GROUP_RIGHT | IDENTIFIER | IGNORING | LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | ON | QUANTILE | STDDEV | STDVAR | SUM | TOPK | START | END | ATAN2 | LIMITK | LIMIT_RATIO | STEP | RANGE | ANCHORED | SMOOTHED;
+maybe_label     : AVG | BOOL | BOTTOMK | BY | COUNT | COUNT_VALUES | GROUP | GROUP_LEFT | GROUP_RIGHT | FILL | FILL_LEFT | FILL_RIGHT | IDENTIFIER | IGNORING | LAND | LOR | LUNLESS | MAX | METRIC_IDENTIFIER | MIN | OFFSET | ON | QUANTILE | STDDEV | STDVAR | SUM | TOPK | START | END | ATAN2 | LIMITK | LIMIT_RATIO | STEP | RANGE | ANCHORED | SMOOTHED;
 
 unary_op        : ADD | SUB;
 
@@ -1162,7 +1221,7 @@ offset_duration_expr    : number_duration_literal
                                 }
                         | duration_expr
                         ;
-                        
+
 min_max: MIN | MAX ;
 
 duration_expr   : number_duration_literal
@@ -1277,14 +1336,14 @@ duration_expr   : number_duration_literal
                 ;
 
 paren_duration_expr : LEFT_PAREN duration_expr RIGHT_PAREN
-                        { 
+                        {
                             yylex.(*parser).experimentalDurationExpr($2.(Expr))
                             if durationExpr, ok := $2.(*DurationExpr); ok {
                                 durationExpr.Wrapped = true
                                 $$ = durationExpr
                                 break
                             }
-                            $$ = $2 
+                            $$ = $2
                         }
                 ;
 
diff --git a/promql/parser/generated_parser.y.go b/promql/parser/generated_parser.y.go
index f5feec0b55..d20460ed5b 100644
--- a/promql/parser/generated_parser.y.go
+++ b/promql/parser/generated_parser.y.go
@@ -113,31 +113,34 @@ const BOOL = 57420
 const BY = 57421
 const GROUP_LEFT = 57422
 const GROUP_RIGHT = 57423
-const IGNORING = 57424
-const OFFSET = 57425
-const SMOOTHED = 57426
-const ANCHORED = 57427
-const ON = 57428
-const WITHOUT = 57429
-const keywordsEnd = 57430
-const preprocessorStart = 57431
-const START = 57432
-const END = 57433
-const STEP = 57434
-const RANGE = 57435
-const preprocessorEnd = 57436
-const counterResetHintsStart = 57437
-const UNKNOWN_COUNTER_RESET = 57438
-const COUNTER_RESET = 57439
-const NOT_COUNTER_RESET = 57440
-const GAUGE_TYPE = 57441
-const counterResetHintsEnd = 57442
-const startSymbolsStart = 57443
-const START_METRIC = 57444
-const START_SERIES_DESCRIPTION = 57445
-const START_EXPRESSION = 57446
-const START_METRIC_SELECTOR = 57447
-const startSymbolsEnd = 57448
+const FILL = 57424
+const FILL_LEFT = 57425
+const FILL_RIGHT = 57426
+const IGNORING = 57427
+const OFFSET = 57428
+const SMOOTHED = 57429
+const ANCHORED = 57430
+const ON = 57431
+const WITHOUT = 57432
+const keywordsEnd = 57433
+const preprocessorStart = 57434
+const START = 57435
+const END = 57436
+const STEP = 57437
+const RANGE = 57438
+const preprocessorEnd = 57439
+const counterResetHintsStart = 57440
+const UNKNOWN_COUNTER_RESET = 57441
+const COUNTER_RESET = 57442
+const NOT_COUNTER_RESET = 57443
+const GAUGE_TYPE = 57444
+const counterResetHintsEnd = 57445
+const startSymbolsStart = 57446
+const START_METRIC = 57447
+const START_SERIES_DESCRIPTION = 57448
+const START_EXPRESSION = 57449
+const START_METRIC_SELECTOR = 57450
+const startSymbolsEnd = 57451
 
 var yyToknames = [...]string{
 	"$end",
@@ -221,6 +224,9 @@ var yyToknames = [...]string{
 	"BY",
 	"GROUP_LEFT",
 	"GROUP_RIGHT",
+	"FILL",
+	"FILL_LEFT",
+	"FILL_RIGHT",
 	"IGNORING",
 	"OFFSET",
 	"SMOOTHED",
@@ -258,376 +264,403 @@ var yyExca = [...]int16{
 	-1, 1,
 	1, -1,
 	-2, 0,
-	-1, 41,
-	1, 150,
-	10, 150,
-	24, 150,
+	-1, 44,
+	1, 161,
+	10, 161,
+	24, 161,
 	-2, 0,
-	-1, 72,
-	2, 193,
-	15, 193,
-	79, 193,
-	87, 193,
-	-2, 107,
-	-1, 73,
-	2, 194,
-	15, 194,
-	79, 194,
-	87, 194,
-	-2, 108,
-	-1, 74,
-	2, 195,
-	15, 195,
-	79, 195,
-	87, 195,
-	-2, 110,
 	-1, 75,
-	2, 196,
-	15, 196,
-	79, 196,
-	87, 196,
-	-2, 111,
-	-1, 76,
-	2, 197,
-	15, 197,
-	79, 197,
-	87, 197,
-	-2, 112,
-	-1, 77,
-	2, 198,
-	15, 198,
-	79, 198,
-	87, 198,
-	-2, 117,
-	-1, 78,
-	2, 199,
-	15, 199,
-	79, 199,
-	87, 199,
-	-2, 119,
-	-1, 79,
-	2, 200,
-	15, 200,
-	79, 200,
-	87, 200,
-	-2, 121,
-	-1, 80,
-	2, 201,
-	15, 201,
-	79, 201,
-	87, 201,
-	-2, 122,
-	-1, 81,
-	2, 202,
-	15, 202,
-	79, 202,
-	87, 202,
-	-2, 123,
-	-1, 82,
-	2, 203,
-	15, 203,
-	79, 203,
-	87, 203,
-	-2, 124,
-	-1, 83,
 	2, 204,
 	15, 204,
 	79, 204,
-	87, 204,
-	-2, 125,
-	-1, 84,
+	90, 204,
+	-2, 115,
+	-1, 76,
 	2, 205,
 	15, 205,
 	79, 205,
-	87, 205,
-	-2, 129,
-	-1, 85,
+	90, 205,
+	-2, 116,
+	-1, 77,
 	2, 206,
 	15, 206,
 	79, 206,
-	87, 206,
+	90, 206,
+	-2, 118,
+	-1, 78,
+	2, 207,
+	15, 207,
+	79, 207,
+	90, 207,
+	-2, 119,
+	-1, 79,
+	2, 208,
+	15, 208,
+	79, 208,
+	90, 208,
+	-2, 123,
+	-1, 80,
+	2, 209,
+	15, 209,
+	79, 209,
+	90, 209,
+	-2, 128,
+	-1, 81,
+	2, 210,
+	15, 210,
+	79, 210,
+	90, 210,
 	-2, 130,
-	-1, 137,
-	41, 274,
-	42, 274,
-	52, 274,
-	53, 274,
-	57, 274,
+	-1, 82,
+	2, 211,
+	15, 211,
+	79, 211,
+	90, 211,
+	-2, 132,
+	-1, 83,
+	2, 212,
+	15, 212,
+	79, 212,
+	90, 212,
+	-2, 133,
+	-1, 84,
+	2, 213,
+	15, 213,
+	79, 213,
+	90, 213,
+	-2, 134,
+	-1, 85,
+	2, 214,
+	15, 214,
+	79, 214,
+	90, 214,
+	-2, 135,
+	-1, 86,
+	2, 215,
+	15, 215,
+	79, 215,
+	90, 215,
+	-2, 136,
+	-1, 87,
+	2, 216,
+	15, 216,
+	79, 216,
+	90, 216,
+	-2, 140,
+	-1, 88,
+	2, 217,
+	15, 217,
+	79, 217,
+	90, 217,
+	-2, 141,
+	-1, 140,
+	41, 288,
+	42, 288,
+	52, 288,
+	53, 288,
+	57, 288,
 	-2, 22,
-	-1, 251,
-	9, 259,
-	12, 259,
-	13, 259,
-	18, 259,
-	19, 259,
-	25, 259,
-	41, 259,
-	47, 259,
-	48, 259,
-	51, 259,
-	57, 259,
-	62, 259,
-	63, 259,
-	64, 259,
-	65, 259,
-	66, 259,
-	67, 259,
-	68, 259,
-	69, 259,
-	70, 259,
-	71, 259,
-	72, 259,
-	73, 259,
-	74, 259,
-	75, 259,
-	79, 259,
-	83, 259,
-	84, 259,
-	85, 259,
-	87, 259,
-	90, 259,
-	91, 259,
-	92, 259,
-	93, 259,
+	-1, 258,
+	9, 273,
+	12, 273,
+	13, 273,
+	18, 273,
+	19, 273,
+	25, 273,
+	41, 273,
+	47, 273,
+	48, 273,
+	51, 273,
+	57, 273,
+	62, 273,
+	63, 273,
+	64, 273,
+	65, 273,
+	66, 273,
+	67, 273,
+	68, 273,
+	69, 273,
+	70, 273,
+	71, 273,
+	72, 273,
+	73, 273,
+	74, 273,
+	75, 273,
+	79, 273,
+	82, 273,
+	83, 273,
+	84, 273,
+	86, 273,
+	87, 273,
+	88, 273,
+	90, 273,
+	93, 273,
+	94, 273,
+	95, 273,
+	96, 273,
 	-2, 0,
-	-1, 252,
-	9, 259,
-	12, 259,
-	13, 259,
-	18, 259,
-	19, 259,
-	25, 259,
-	41, 259,
-	47, 259,
-	48, 259,
-	51, 259,
-	57, 259,
-	62, 259,
-	63, 259,
-	64, 259,
-	65, 259,
-	66, 259,
-	67, 259,
-	68, 259,
-	69, 259,
-	70, 259,
-	71, 259,
-	72, 259,
-	73, 259,
-	74, 259,
-	75, 259,
-	79, 259,
-	83, 259,
-	84, 259,
-	85, 259,
-	87, 259,
-	90, 259,
-	91, 259,
-	92, 259,
-	93, 259,
+	-1, 259,
+	9, 273,
+	12, 273,
+	13, 273,
+	18, 273,
+	19, 273,
+	25, 273,
+	41, 273,
+	47, 273,
+	48, 273,
+	51, 273,
+	57, 273,
+	62, 273,
+	63, 273,
+	64, 273,
+	65, 273,
+	66, 273,
+	67, 273,
+	68, 273,
+	69, 273,
+	70, 273,
+	71, 273,
+	72, 273,
+	73, 273,
+	74, 273,
+	75, 273,
+	79, 273,
+	82, 273,
+	83, 273,
+	84, 273,
+	86, 273,
+	87, 273,
+	88, 273,
+	90, 273,
+	93, 273,
+	94, 273,
+	95, 273,
+	96, 273,
 	-2, 0,
 }
 
 const yyPrivate = 57344
 
-const yyLast = 1050
+const yyLast = 1224
 
 var yyAct = [...]int16{
-	58, 186, 413, 411, 341, 418, 286, 243, 197, 95,
-	189, 48, 355, 144, 70, 227, 93, 251, 252, 356,
-	159, 190, 65, 120, 17, 88, 127, 130, 128, 129,
-	22, 425, 426, 427, 428, 131, 249, 121, 124, 335,
-	250, 67, 132, 126, 408, 407, 377, 332, 125, 123,
-	331, 102, 126, 122, 336, 154, 324, 6, 397, 18,
-	19, 111, 112, 20, 135, 114, 137, 119, 101, 375,
-	337, 323, 375, 330, 11, 12, 14, 15, 16, 21,
-	23, 25, 26, 27, 28, 29, 33, 34, 43, 133,
-	329, 13, 116, 118, 117, 24, 38, 37, 146, 30,
-	402, 124, 31, 32, 35, 36, 130, 412, 138, 396,
-	194, 125, 123, 328, 131, 126, 365, 182, 239, 401,
-	193, 199, 204, 205, 206, 207, 208, 209, 177, 363,
-	362, 181, 200, 200, 200, 200, 200, 200, 200, 178,
-	120, 238, 223, 201, 201, 201, 201, 201, 201, 201,
-	212, 215, 134, 200, 136, 211, 210, 2, 3, 4,
-	5, 222, 233, 221, 201, 245, 235, 384, 333, 371,
-	228, 247, 229, 360, 370, 359, 246, 358, 188, 273,
-	140, 368, 114, 195, 119, 194, 277, 139, 62, 369,
-	268, 237, 229, 271, 185, 193, 441, 200, 61, 196,
-	367, 201, 273, 383, 155, 278, 279, 280, 201, 116,
-	118, 117, 231, 200, 236, 121, 124, 195, 382, 440,
-	86, 218, 230, 232, 201, 381, 125, 123, 276, 275,
-	126, 122, 231, 196, 274, 146, 87, 132, 439, 327,
-	429, 438, 230, 232, 248, 141, 184, 183, 419, 253,
-	254, 255, 256, 257, 258, 259, 260, 261, 262, 263,
-	264, 265, 266, 267, 334, 357, 191, 192, 214, 353,
-	354, 202, 203, 361, 121, 124, 88, 364, 283, 7,
-	39, 213, 282, 199, 200, 125, 123, 395, 200, 126,
-	122, 366, 10, 194, 200, 201, 394, 281, 393, 201,
-	392, 391, 90, 193, 390, 201, 160, 161, 162, 163,
-	164, 165, 166, 167, 168, 169, 170, 171, 172, 173,
-	174, 389, 194, 388, 120, 195, 373, 387, 386, 385,
-	153, 99, 193, 62, 442, 374, 376, 200, 378, 185,
-	56, 196, 40, 61, 379, 380, 89, 152, 201, 151,
-	1, 100, 102, 103, 195, 104, 105, 175, 71, 108,
-	109, 398, 111, 112, 113, 86, 114, 115, 119, 101,
-	196, 66, 200, 55, 9, 9, 54, 404, 8, 53,
-	406, 87, 41, 201, 52, 158, 410, 51, 414, 415,
-	416, 184, 183, 116, 118, 117, 421, 420, 423, 422,
-	417, 430, 50, 49, 289, 47, 156, 216, 147, 46,
-	431, 432, 200, 372, 299, 433, 202, 203, 145, 96,
-	305, 435, 157, 201, 403, 437, 326, 288, 147, 94,
-	436, 97, 45, 44, 57, 242, 434, 234, 145, 338,
-	443, 200, 97, 98, 121, 124, 143, 240, 284, 301,
-	302, 97, 201, 303, 91, 125, 123, 424, 187, 126,
-	122, 316, 287, 59, 290, 292, 294, 295, 296, 304,
-	306, 309, 310, 311, 312, 313, 317, 318, 142, 0,
-	291, 293, 297, 298, 300, 307, 322, 321, 308, 289,
-	96, 0, 314, 315, 319, 320, 226, 150, 405, 299,
-	94, 225, 149, 0, 0, 305, 0, 0, 92, 285,
-	0, 0, 288, 97, 224, 148, 62, 121, 124, 0,
-	0, 0, 272, 0, 0, 0, 61, 0, 125, 123,
-	0, 0, 126, 122, 301, 302, 0, 0, 303, 0,
-	0, 0, 0, 0, 0, 0, 316, 0, 86, 290,
-	292, 294, 295, 296, 304, 306, 309, 310, 311, 312,
-	313, 317, 318, 0, 87, 291, 293, 297, 298, 300,
-	307, 322, 321, 308, 184, 183, 0, 314, 315, 319,
-	320, 62, 0, 120, 60, 88, 0, 63, 0, 0,
-	22, 61, 0, 0, 217, 0, 0, 64, 0, 269,
-	270, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-	100, 102, 0, 86, 0, 0, 0, 0, 0, 18,
-	19, 111, 112, 20, 0, 114, 115, 119, 101, 87,
-	0, 0, 0, 0, 72, 73, 74, 75, 76, 77,
-	78, 79, 80, 81, 82, 83, 84, 85, 0, 0,
-	400, 13, 116, 118, 117, 24, 38, 37, 399, 30,
-	0, 0, 31, 32, 68, 69, 62, 42, 0, 60,
-	88, 0, 63, 0, 0, 22, 61, 121, 124, 0,
-	0, 0, 64, 0, 121, 124, 0, 0, 125, 123,
-	0, 0, 126, 122, 0, 125, 123, 0, 86, 126,
-	122, 0, 0, 0, 18, 19, 0, 0, 20, 0,
-	0, 0, 0, 0, 87, 0, 0, 0, 0, 72,
-	73, 74, 75, 76, 77, 78, 79, 80, 81, 82,
-	83, 84, 85, 0, 0, 0, 13, 0, 0, 220,
-	24, 38, 37, 0, 30, 0, 325, 31, 32, 68,
-	69, 62, 0, 0, 60, 88, 0, 63, 121, 124,
-	22, 61, 0, 0, 0, 0, 0, 64, 0, 125,
-	123, 0, 0, 126, 122, 0, 0, 0, 0, 0,
-	121, 124, 0, 86, 0, 0, 0, 0, 0, 18,
-	19, 125, 123, 20, 0, 126, 122, 0, 0, 87,
-	0, 0, 0, 0, 72, 73, 74, 75, 76, 77,
-	78, 79, 80, 81, 82, 83, 84, 85, 17, 39,
-	0, 13, 0, 0, 22, 24, 38, 37, 0, 30,
-	340, 0, 31, 32, 68, 69, 0, 339, 0, 0,
-	0, 343, 344, 342, 349, 351, 348, 350, 345, 346,
-	347, 352, 241, 18, 19, 0, 194, 20, 0, 244,
-	0, 0, 0, 247, 0, 0, 193, 0, 11, 12,
-	14, 15, 16, 21, 23, 25, 26, 27, 28, 29,
-	33, 34, 0, 0, 120, 13, 0, 0, 195, 24,
-	38, 37, 219, 30, 0, 0, 31, 32, 35, 36,
-	0, 0, 0, 120, 196, 0, 0, 0, 0, 0,
-	0, 100, 102, 103, 0, 104, 105, 106, 107, 108,
-	109, 110, 111, 112, 113, 0, 114, 115, 119, 101,
-	100, 102, 103, 0, 104, 105, 106, 107, 108, 109,
-	110, 111, 112, 113, 198, 114, 115, 119, 101, 120,
-	0, 62, 0, 116, 118, 117, 0, 185, 176, 0,
-	0, 61, 0, 0, 0, 62, 0, 0, 0, 0,
-	0, 185, 116, 118, 117, 61, 100, 102, 103, 0,
-	104, 105, 106, 86, 108, 109, 110, 111, 112, 113,
-	0, 114, 115, 119, 101, 0, 0, 86, 0, 87,
-	0, 0, 0, 0, 0, 0, 0, 0, 0, 184,
-	183, 0, 0, 87, 0, 0, 0, 0, 116, 118,
-	117, 0, 0, 184, 183, 409, 0, 0, 0, 0,
-	0, 0, 0, 0, 202, 203, 343, 344, 342, 349,
-	351, 348, 350, 345, 346, 347, 352, 0, 179, 180,
+	61, 363, 190, 429, 351, 436, 431, 293, 247, 201,
+	98, 51, 147, 193, 369, 96, 231, 412, 413, 370,
+	132, 133, 68, 130, 73, 163, 194, 131, 443, 444,
+	445, 446, 134, 135, 256, 253, 254, 255, 257, 258,
+	259, 129, 70, 426, 123, 425, 124, 127, 391, 342,
+	157, 458, 223, 198, 447, 389, 415, 128, 126, 345,
+	451, 129, 125, 197, 414, 465, 398, 138, 379, 140,
+	6, 103, 105, 106, 346, 107, 108, 109, 110, 111,
+	112, 113, 114, 115, 116, 199, 117, 118, 122, 104,
+	347, 136, 343, 46, 124, 127, 389, 133, 334, 251,
+	397, 200, 149, 377, 192, 128, 126, 199, 134, 129,
+	125, 198, 141, 333, 420, 396, 119, 121, 120, 123,
+	186, 197, 395, 200, 203, 208, 209, 210, 211, 212,
+	213, 181, 376, 419, 430, 204, 204, 204, 204, 204,
+	204, 204, 182, 199, 185, 227, 205, 205, 205, 205,
+	205, 205, 205, 216, 219, 215, 204, 341, 214, 200,
+	137, 117, 139, 122, 339, 385, 237, 205, 239, 464,
+	384, 249, 226, 2, 3, 4, 5, 91, 290, 225,
+	340, 123, 289, 280, 250, 383, 364, 338, 124, 127,
+	284, 119, 121, 120, 275, 195, 196, 288, 218, 128,
+	126, 204, 460, 129, 125, 205, 280, 278, 158, 105,
+	374, 217, 205, 286, 287, 423, 243, 204, 241, 114,
+	115, 124, 127, 117, 373, 122, 104, 372, 205, 222,
+	143, 437, 128, 126, 124, 127, 129, 125, 65, 242,
+	149, 240, 337, 142, 42, 128, 126, 418, 64, 129,
+	125, 285, 252, 119, 121, 120, 365, 366, 260, 261,
+	262, 263, 264, 265, 266, 267, 268, 269, 270, 271,
+	272, 273, 274, 344, 371, 127, 367, 368, 198, 283,
+	375, 124, 127, 282, 378, 128, 126, 281, 197, 129,
+	203, 204, 128, 126, 135, 204, 129, 125, 198, 380,
+	65, 204, 205, 144, 7, 409, 205, 408, 197, 407,
+	64, 406, 205, 164, 165, 166, 167, 168, 169, 170,
+	171, 172, 173, 174, 175, 176, 177, 178, 202, 232,
+	199, 233, 89, 156, 417, 65, 387, 405, 463, 233,
+	404, 189, 102, 224, 403, 64, 200, 204, 90, 388,
+	390, 10, 392, 124, 127, 393, 394, 462, 205, 402,
+	461, 93, 124, 127, 128, 126, 401, 89, 129, 125,
+	400, 235, 399, 128, 126, 416, 410, 129, 125, 235,
+	8, 234, 236, 90, 44, 59, 204, 411, 43, 234,
+	236, 92, 422, 188, 187, 1, 179, 205, 424, 155,
+	428, 154, 230, 432, 433, 434, 150, 229, 74, 335,
+	439, 438, 441, 440, 449, 450, 148, 435, 58, 452,
+	228, 206, 207, 448, 336, 57, 296, 56, 386, 100,
+	204, 69, 453, 454, 9, 9, 309, 455, 99, 55,
+	457, 205, 315, 124, 127, 162, 421, 150, 97, 295,
+	99, 54, 459, 53, 128, 126, 238, 148, 129, 125,
+	97, 100, 153, 204, 466, 146, 52, 152, 95, 50,
+	100, 311, 312, 100, 205, 313, 160, 220, 49, 161,
+	151, 48, 159, 326, 47, 60, 297, 299, 301, 302,
+	303, 314, 316, 319, 320, 321, 322, 323, 327, 328,
+	246, 456, 298, 300, 304, 305, 306, 307, 308, 310,
+	317, 332, 331, 318, 296, 348, 101, 324, 325, 329,
+	330, 245, 244, 291, 309, 198, 94, 442, 248, 191,
+	315, 350, 251, 294, 292, 197, 62, 295, 349, 145,
+	0, 0, 353, 354, 352, 359, 361, 358, 360, 355,
+	356, 357, 362, 0, 0, 0, 0, 199, 0, 311,
+	312, 0, 0, 313, 0, 0, 0, 0, 0, 0,
+	0, 326, 0, 200, 297, 299, 301, 302, 303, 314,
+	316, 319, 320, 321, 322, 323, 327, 328, 0, 0,
+	298, 300, 304, 305, 306, 307, 308, 310, 317, 332,
+	331, 318, 0, 0, 0, 324, 325, 329, 330, 65,
+	0, 0, 63, 91, 0, 66, 427, 0, 25, 64,
+	0, 0, 221, 0, 0, 67, 0, 353, 354, 352,
+	359, 361, 358, 360, 355, 356, 357, 362, 0, 0,
+	0, 89, 0, 0, 0, 0, 0, 21, 22, 0,
+	0, 23, 0, 0, 0, 0, 0, 90, 0, 0,
+	0, 0, 75, 76, 77, 78, 79, 80, 81, 82,
+	83, 84, 85, 86, 87, 88, 0, 0, 0, 13,
+	0, 0, 16, 17, 18, 0, 27, 41, 40, 0,
+	33, 0, 0, 34, 35, 71, 72, 65, 45, 0,
+	63, 91, 0, 66, 0, 0, 25, 64, 0, 0,
+	0, 0, 0, 67, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 89,
+	0, 0, 0, 0, 0, 21, 22, 0, 0, 23,
+	0, 0, 0, 0, 0, 90, 0, 0, 0, 0,
+	75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
+	85, 86, 87, 88, 0, 0, 0, 13, 0, 0,
+	16, 17, 18, 0, 27, 41, 40, 0, 33, 0,
+	0, 34, 35, 71, 72, 65, 0, 0, 63, 91,
+	0, 66, 0, 0, 25, 64, 0, 0, 0, 0,
+	0, 67, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 89, 0, 0,
+	0, 0, 0, 21, 22, 0, 0, 23, 0, 0,
+	0, 0, 0, 90, 0, 0, 0, 0, 75, 76,
+	77, 78, 79, 80, 81, 82, 83, 84, 85, 86,
+	87, 88, 0, 0, 0, 13, 0, 0, 16, 17,
+	18, 0, 27, 41, 40, 0, 33, 20, 91, 34,
+	35, 71, 72, 25, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 21, 22, 0, 0, 23, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 11, 12, 14,
+	15, 19, 24, 26, 28, 29, 30, 31, 32, 36,
+	37, 0, 0, 0, 13, 0, 0, 16, 17, 18,
+	0, 27, 41, 40, 0, 33, 20, 42, 34, 35,
+	38, 39, 25, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 21, 22, 0, 0, 23, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 11, 12, 14, 15,
+	19, 24, 26, 28, 29, 30, 31, 32, 36, 37,
+	123, 0, 0, 13, 0, 0, 16, 17, 18, 0,
+	27, 41, 40, 0, 33, 0, 0, 34, 35, 38,
+	39, 123, 0, 0, 0, 0, 0, 103, 105, 106,
+	0, 107, 108, 109, 110, 111, 112, 113, 114, 115,
+	116, 0, 117, 118, 122, 104, 0, 0, 103, 105,
+	106, 0, 107, 108, 109, 0, 111, 112, 113, 114,
+	115, 116, 382, 117, 118, 122, 104, 0, 0, 65,
+	0, 123, 119, 121, 120, 189, 65, 0, 0, 64,
+	0, 381, 189, 0, 0, 0, 64, 0, 0, 0,
+	0, 0, 0, 119, 121, 120, 0, 0, 103, 105,
+	106, 89, 107, 108, 0, 0, 111, 112, 89, 114,
+	115, 116, 180, 117, 118, 122, 104, 90, 0, 65,
+	0, 0, 0, 0, 90, 189, 65, 188, 187, 64,
+	0, 0, 279, 0, 188, 187, 64, 123, 0, 0,
+	0, 0, 0, 119, 121, 120, 0, 0, 0, 0,
+	0, 89, 0, 0, 0, 206, 207, 0, 89, 0,
+	0, 0, 206, 207, 103, 105, 0, 90, 0, 0,
+	0, 0, 0, 0, 90, 114, 115, 188, 187, 117,
+	118, 122, 104, 0, 188, 187, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 183, 184, 0, 0, 119,
+	121, 120, 276, 277,
 }
 
 var yyPact = [...]int16{
-	55, 269, 806, 806, 657, 12, -1000, -1000, -1000, 267,
-	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
-	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
-	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 488,
-	-1000, 329, -1000, 889, -1000, -1000, -1000, -1000, -1000, -1000,
-	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -4, 27,
-	222, -1000, -1000, 742, -1000, 742, 263, -1000, 172, 165,
-	230, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
-	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 426, -1000,
-	-1000, 495, -1000, -1000, 345, 326, -1000, -1000, 31, -1000,
-	-58, -58, -58, -58, -58, -58, -58, -58, -58, -58,
-	-58, -58, -58, -58, -58, -58, 956, -1000, -1000, 176,
-	942, 324, 324, 324, 324, 324, 324, 222, -52, -1000,
-	266, 266, 572, -1000, 870, 717, 126, -13, -1000, 141,
-	139, 324, 494, -1000, -1000, 168, 188, -1000, -1000, 417,
-	-1000, 189, -1000, 116, 847, 742, -1000, -46, -63, -1000,
-	742, 742, 742, 742, 742, 742, 742, 742, 742, 742,
-	742, 742, 742, 742, 742, -1000, -1000, -1000, 507, 219,
-	214, 213, -4, -1000, -1000, 324, -1000, 190, -1000, -1000,
-	-1000, -1000, -1000, -1000, -1000, 101, 101, 276, -1000, -4,
-	-1000, 324, 172, 165, 59, 59, -13, -13, -13, -13,
-	-1000, -1000, -1000, 487, -1000, -1000, 49, -1000, 889, -1000,
-	-1000, -1000, -1000, 739, -1000, 406, -1000, 88, -1000, -1000,
-	-1000, -1000, -1000, 48, -1000, -1000, -1000, -1000, -1000, -1000,
-	-1000, 21, 142, 13, -1000, -1000, -1000, 813, 9, 266,
-	266, 266, 266, 126, 126, 569, 569, 569, 310, 935,
-	569, 569, 310, 126, 126, 569, 126, 9, -1000, 162,
-	160, 158, 324, -13, 108, 107, 324, 717, 94, -1000,
-	-1000, -1000, 179, -1000, 167, -1000, -1000, -1000, -1000, -1000,
+	68, 294, 934, 934, 688, 855, -1000, -1000, -1000, 231,
 	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
 	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
 	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
-	-1000, -1000, -1000, -1000, 742, 324, -1000, -1000, -1000, -1000,
-	-1000, -1000, 53, 53, 20, 53, 155, 155, 201, 150,
-	-1000, -1000, 323, 322, 321, 317, 315, 298, 295, 294,
-	292, 290, 281, -1000, -1000, -1000, -1000, -1000, 87, 36,
-	324, 636, -1000, -1000, 643, -1000, 98, -1000, -1000, -1000,
-	402, -1000, 889, 476, -1000, -1000, -1000, 53, -1000, 19,
-	18, 1008, -1000, -1000, -1000, 50, 284, 284, 284, 101,
-	234, 234, 50, 234, 50, -65, -1000, -1000, 233, -1000,
-	324, -1000, -1000, -1000, -1000, -1000, -1000, 53, 53, -1000,
-	-1000, -1000, 53, -1000, -1000, -1000, -1000, -1000, -1000, 284,
-	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 324,
-	403, -1000, -1000, -1000, 217, -1000, 174, -1000, 313, -1000,
-	-1000, -1000, -1000, -1000,
+	-1000, -1000, 448, -1000, 340, -1000, 996, -1000, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, 5, 18, 279, -1000, -1000, 776, -1000, 776, 164,
+	-1000, 228, 215, 288, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, 445, -1000, -1000, 460, -1000, -1000, 397, 329, -1000,
+	-1000, 26, -1000, -53, -53, -53, -53, -53, -53, -53,
+	-53, -53, -53, -53, -53, -53, -53, -53, -53, 1120,
+	-1000, -1000, 102, 326, 1077, 1077, 1077, 1077, 1077, 1077,
+	279, -58, -1000, 196, 196, 600, -1000, 30, 321, 105,
+	-15, -1000, 157, 150, 1077, 400, -1000, -1000, 327, 335,
+	-1000, -1000, 436, -1000, 216, -1000, 214, 516, 776, -1000,
+	-47, -51, -41, -1000, 776, 776, 776, 776, 776, 776,
+	776, 776, 776, 776, 776, 776, 776, 776, 776, -1000,
+	-1000, -1000, 1127, 272, 268, 264, 5, -1000, -1000, 1077,
+	-1000, 236, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 269,
+	269, 176, -1000, 5, -1000, 1077, 228, 215, 233, 233,
+	-15, -15, -15, -15, -1000, -1000, -1000, 512, -1000, -1000,
+	91, -1000, 996, -1000, -1000, -1000, -1000, 402, -1000, 404,
+	-1000, 162, -1000, -1000, -1000, -1000, -1000, 155, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, 23, 66, 33, -1000, -1000,
+	-1000, 514, 167, 171, 171, 171, 196, 196, 196, 196,
+	105, 105, 1133, 1133, 1133, 1067, 1017, 1133, 1133, 1067,
+	105, 105, 1133, 105, 167, -1000, 212, 209, 195, 1077,
+	-15, 110, 81, 1077, 321, 46, -1000, -1000, -1000, 1070,
+	-1000, 163, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000,
+	-1000, -1000, -1000, -1000, 776, 1077, -1000, -1000, -1000, -1000,
+	-1000, -1000, 36, 36, 22, 36, 83, 83, 98, 49,
+	-1000, -1000, 366, 364, 360, 353, 338, 334, 331, 305,
+	303, 301, 299, -1000, 291, -67, -65, -1000, -1000, -1000,
+	-1000, -1000, 42, 34, 1077, 312, -1000, -1000, 240, -1000,
+	112, -1000, -1000, -1000, 424, -1000, 996, 193, -1000, -1000,
+	-1000, 36, -1000, 19, 17, 599, -1000, -1000, -1000, 77,
+	289, 289, 289, 269, 217, 217, 77, 217, 77, -71,
+	32, 229, 171, 171, -1000, -1000, 53, -1000, 1077, -1000,
+	-1000, -1000, -1000, -1000, -1000, 36, 36, -1000, -1000, -1000,
+	36, -1000, -1000, -1000, -1000, -1000, -1000, 289, -1000, -1000,
+	-1000, -1000, -1000, -1000, -1000, -1000, -1000, -1000, 29, -1000,
+	-1000, 1077, 180, -1000, -1000, -1000, 336, -1000, -1000, 147,
+	-1000, 44, -1000, -1000, -1000, -1000, -1000,
 }
 
 var yyPgo = [...]int16{
-	0, 478, 13, 463, 6, 15, 462, 371, 22, 458,
-	9, 457, 14, 292, 378, 454, 16, 448, 19, 12,
-	447, 443, 7, 439, 4, 5, 436, 3, 2, 10,
-	435, 21, 1, 434, 433, 26, 204, 432, 422, 88,
-	409, 407, 28, 406, 41, 405, 11, 403, 402, 387,
-	385, 384, 379, 376, 373, 340, 0, 358, 8, 357,
-	350, 342,
+	0, 539, 12, 536, 7, 16, 533, 431, 22, 529,
+	10, 527, 24, 351, 380, 526, 15, 523, 19, 14,
+	522, 516, 8, 515, 4, 5, 501, 3, 6, 13,
+	500, 26, 2, 485, 484, 23, 208, 482, 481, 479,
+	93, 478, 477, 27, 476, 1, 42, 469, 11, 466,
+	453, 451, 445, 439, 427, 425, 418, 385, 0, 408,
+	9, 396, 395, 388,
 }
 
 var yyR1 = [...]int8{
-	0, 60, 60, 60, 60, 60, 60, 60, 39, 39,
-	39, 39, 39, 39, 39, 39, 39, 39, 39, 39,
-	39, 39, 39, 34, 34, 34, 34, 35, 35, 37,
-	37, 37, 37, 37, 37, 37, 37, 37, 37, 37,
-	37, 37, 37, 37, 37, 36, 38, 38, 50, 50,
-	43, 43, 43, 43, 18, 18, 18, 18, 17, 17,
-	17, 4, 4, 4, 40, 42, 42, 41, 41, 41,
-	51, 58, 47, 47, 48, 49, 33, 33, 33, 9,
-	9, 45, 53, 53, 53, 53, 53, 53, 54, 55,
-	55, 55, 44, 44, 44, 1, 1, 1, 2, 2,
-	2, 2, 2, 2, 2, 14, 14, 7, 7, 7,
+	0, 62, 62, 62, 62, 62, 62, 62, 40, 40,
+	40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
+	40, 40, 40, 34, 34, 34, 34, 35, 35, 38,
+	38, 38, 38, 38, 38, 38, 38, 38, 38, 38,
+	38, 38, 38, 38, 38, 36, 39, 39, 52, 52,
+	44, 44, 44, 44, 37, 37, 37, 37, 37, 37,
+	18, 18, 18, 18, 17, 17, 17, 4, 4, 4,
+	45, 45, 41, 43, 43, 42, 42, 42, 53, 60,
+	49, 49, 50, 51, 33, 33, 33, 9, 9, 47,
+	55, 55, 55, 55, 55, 55, 56, 57, 57, 57,
+	46, 46, 46, 1, 1, 1, 2, 2, 2, 2,
+	2, 2, 2, 14, 14, 7, 7, 7, 7, 7,
 	7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
 	7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
-	7, 7, 7, 7, 7, 13, 13, 13, 13, 15,
-	15, 15, 16, 16, 16, 16, 16, 16, 16, 61,
-	21, 21, 21, 21, 20, 20, 20, 20, 20, 20,
-	20, 20, 20, 30, 30, 30, 22, 22, 22, 22,
-	23, 23, 23, 24, 24, 24, 24, 24, 24, 24,
-	24, 24, 24, 24, 25, 25, 26, 26, 26, 11,
-	11, 11, 11, 3, 3, 3, 3, 3, 3, 3,
-	3, 3, 3, 3, 3, 3, 3, 6, 6, 6,
+	7, 7, 7, 7, 7, 7, 13, 13, 13, 13,
+	15, 15, 15, 16, 16, 16, 16, 16, 16, 16,
+	63, 21, 21, 21, 21, 20, 20, 20, 20, 20,
+	20, 20, 20, 20, 30, 30, 30, 22, 22, 22,
+	22, 23, 23, 23, 24, 24, 24, 24, 24, 24,
+	24, 24, 24, 24, 24, 25, 25, 26, 26, 26,
+	11, 11, 11, 11, 3, 3, 3, 3, 3, 3,
+	3, 3, 3, 3, 3, 3, 3, 3, 6, 6,
 	6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
 	6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
 	6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
-	8, 8, 5, 5, 5, 5, 46, 46, 29, 29,
-	31, 31, 32, 32, 28, 27, 27, 52, 10, 19,
-	19, 59, 59, 59, 59, 59, 59, 59, 59, 59,
-	59, 12, 12, 56, 56, 56, 56, 56, 56, 56,
-	56, 56, 56, 56, 56, 57,
+	6, 6, 6, 6, 8, 8, 5, 5, 5, 5,
+	48, 48, 29, 29, 31, 31, 32, 32, 28, 27,
+	27, 54, 10, 19, 19, 61, 61, 61, 61, 61,
+	61, 61, 61, 61, 61, 12, 12, 58, 58, 58,
+	58, 58, 58, 58, 58, 58, 58, 58, 58, 59,
 }
 
 var yyR2 = [...]int8{
@@ -636,126 +669,131 @@ var yyR2 = [...]int8{
 	1, 1, 1, 3, 3, 2, 2, 2, 2, 4,
 	4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
 	4, 4, 4, 4, 4, 1, 0, 1, 3, 3,
-	1, 1, 3, 3, 3, 4, 2, 1, 3, 1,
-	2, 1, 1, 1, 2, 3, 2, 3, 1, 2,
-	3, 1, 3, 3, 2, 2, 3, 5, 3, 1,
-	1, 4, 6, 5, 6, 5, 4, 3, 2, 2,
-	1, 1, 3, 4, 2, 3, 1, 2, 3, 3,
-	1, 3, 3, 2, 1, 2, 1, 1, 1, 1,
+	1, 1, 3, 3, 1, 3, 3, 3, 5, 5,
+	3, 4, 2, 1, 3, 1, 2, 1, 1, 1,
+	3, 4, 2, 3, 2, 3, 1, 2, 3, 1,
+	3, 3, 2, 2, 3, 5, 3, 1, 1, 4,
+	6, 5, 6, 5, 4, 3, 2, 2, 1, 1,
+	3, 4, 2, 3, 1, 2, 3, 3, 1, 3,
+	3, 2, 1, 2, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-	1, 1, 1, 1, 1, 3, 4, 2, 0, 3,
-	1, 2, 3, 3, 1, 3, 3, 2, 1, 2,
-	0, 3, 2, 1, 1, 3, 1, 3, 4, 1,
-	3, 5, 5, 1, 1, 1, 4, 3, 3, 2,
-	3, 1, 2, 3, 3, 3, 3, 3, 3, 3,
-	3, 3, 3, 3, 4, 3, 3, 1, 2, 1,
+	1, 1, 1, 1, 1, 1, 3, 4, 2, 0,
+	3, 1, 2, 3, 3, 1, 3, 3, 2, 1,
+	2, 0, 3, 2, 1, 1, 3, 1, 3, 4,
+	1, 3, 5, 5, 1, 1, 1, 4, 3, 3,
+	2, 3, 1, 2, 3, 3, 3, 3, 3, 3,
+	3, 3, 3, 3, 3, 4, 3, 3, 1, 2,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
 	1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
-	2, 2, 1, 1, 1, 2, 1, 1, 1, 0,
-	1, 1, 2, 3, 3, 4, 4, 6, 7, 4,
-	1, 1, 1, 1, 2, 3, 3, 3, 3, 3,
-	3, 3, 3, 6, 1, 3,
+	1, 1, 1, 1, 2, 2, 1, 1, 1, 2,
+	1, 1, 1, 0, 1, 1, 2, 3, 3, 4,
+	4, 6, 7, 4, 1, 1, 1, 1, 2, 3,
+	3, 3, 3, 3, 3, 3, 3, 6, 1, 3,
 }
 
 var yyChk = [...]int16{
-	-1000, -60, 102, 103, 104, 105, 2, 10, -14, -7,
-	-13, 62, 63, 79, 64, 65, 66, 12, 47, 48,
-	51, 67, 18, 68, 83, 69, 70, 71, 72, 73,
-	87, 90, 91, 74, 75, 92, 93, 85, 84, 13,
-	-61, -14, 10, -39, -34, -37, -40, -45, -46, -47,
-	-48, -49, -51, -52, -53, -54, -55, -33, -56, -3,
-	12, 19, 9, 15, 25, -8, -7, -44, 92, 93,
-	-12, -57, 62, 63, 64, 65, 66, 67, 68, 69,
-	70, 71, 72, 73, 74, 75, 41, 57, 13, -55,
-	-13, -15, 20, -16, 12, -10, 2, 25, -21, 2,
-	41, 59, 42, 43, 45, 46, 47, 48, 49, 50,
-	51, 52, 53, 54, 56, 57, 83, 85, 84, 58,
-	14, 41, 57, 53, 42, 52, 56, -35, -42, 2,
-	79, 87, 15, -42, -39, -56, -39, -56, -44, 15,
-	15, 15, -1, 20, -2, 12, -10, 2, 20, 7,
-	2, 4, 2, 4, 24, -36, -43, -38, -50, 78,
-	-36, -36, -36, -36, -36, -36, -36, -36, -36, -36,
-	-36, -36, -36, -36, -36, -59, 2, -46, -8, 92,
-	93, -12, -56, 68, 67, 15, -32, -9, 2, -29,
-	-31, 90, 91, 19, 9, 41, 57, -58, 2, -56,
-	-46, -8, 92, 93, -56, -56, -56, -56, -56, -56,
-	-42, -35, -18, 15, 2, -18, -41, 22, -39, 22,
-	22, 22, 22, -56, 20, 7, 2, -5, 2, 4,
-	54, 44, 55, -5, 20, -16, 25, 2, 25, 2,
-	-20, 5, -30, -22, 12, -29, -31, 16, -39, 82,
-	86, 80, 81, -39, -39, -39, -39, -39, -39, -39,
-	-39, -39, -39, -39, -39, -39, -39, -39, -46, 92,
-	93, -12, 15, -56, 15, 15, 15, -56, 15, -29,
-	-29, 21, 6, 2, -17, 22, -4, -6, 25, 2,
-	62, 78, 63, 79, 64, 65, 66, 80, 81, 12,
-	82, 47, 48, 51, 67, 18, 68, 83, 86, 69,
-	70, 71, 72, 73, 90, 91, 59, 74, 75, 92,
-	93, 85, 84, 22, 7, 7, 20, -2, 25, 2,
+	-1000, -62, 105, 106, 107, 108, 2, 10, -14, -7,
+	-13, 62, 63, 79, 64, 65, 82, 83, 84, 66,
+	12, 47, 48, 51, 67, 18, 68, 86, 69, 70,
+	71, 72, 73, 90, 93, 94, 74, 75, 95, 96,
+	88, 87, 13, -63, -14, 10, -40, -34, -38, -41,
+	-47, -48, -49, -50, -51, -53, -54, -55, -56, -57,
+	-33, -58, -3, 12, 19, 9, 15, 25, -8, -7,
+	-46, 95, 96, -12, -59, 62, 63, 64, 65, 66,
+	67, 68, 69, 70, 71, 72, 73, 74, 75, 41,
+	57, 13, -57, -13, -15, 20, -16, 12, -10, 2,
+	25, -21, 2, 41, 59, 42, 43, 45, 46, 47,
+	48, 49, 50, 51, 52, 53, 54, 56, 57, 86,
+	88, 87, 58, 14, 41, 57, 53, 42, 52, 56,
+	-35, -43, 2, 79, 90, 15, -43, -40, -58, -40,
+	-58, -46, 15, 15, 15, -1, 20, -2, 12, -10,
+	2, 20, 7, 2, 4, 2, 4, 24, -36, -37,
+	-44, -39, -52, 78, -36, -36, -36, -36, -36, -36,
+	-36, -36, -36, -36, -36, -36, -36, -36, -36, -61,
+	2, -48, -8, 95, 96, -12, -58, 68, 67, 15,
+	-32, -9, 2, -29, -31, 93, 94, 19, 9, 41,
+	57, -60, 2, -58, -48, -8, 95, 96, -58, -58,
+	-58, -58, -58, -58, -43, -35, -18, 15, 2, -18,
+	-42, 22, -40, 22, 22, 22, 22, -58, 20, 7,
+	2, -5, 2, 4, 54, 44, 55, -5, 20, -16,
+	25, 2, 25, 2, -20, 5, -30, -22, 12, -29,
+	-31, 16, -40, 82, 83, 84, 85, 89, 80, 81,
+	-40, -40, -40, -40, -40, -40, -40, -40, -40, -40,
+	-40, -40, -40, -40, -40, -48, 95, 96, -12, 15,
+	-58, 15, 15, 15, -58, 15, -29, -29, 21, 6,
+	2, -17, 22, -4, -6, 25, 2, 62, 78, 63,
+	79, 64, 65, 66, 80, 81, 82, 83, 84, 12,
+	85, 47, 48, 51, 67, 18, 68, 86, 89, 69,
+	70, 71, 72, 73, 93, 94, 59, 74, 75, 95,
+	96, 88, 87, 22, 7, 7, 20, -2, 25, 2,
 	25, 2, 26, 26, -31, 26, 41, 57, -23, 24,
 	17, -24, 30, 28, 29, 35, 36, 37, 33, 31,
-	34, 32, 38, -18, -18, -19, -18, -19, 15, 15,
-	15, -56, 22, 22, -56, 22, -58, 21, 2, 22,
-	7, 2, -39, -56, -28, 19, -28, 26, -28, -22,
-	-22, 24, 17, 2, 17, 6, 6, 6, 6, 6,
-	6, 6, 6, 6, 6, 6, 22, 22, -56, 22,
-	7, 21, 2, 22, -4, 22, -28, 26, 26, 17,
-	-24, -27, 57, -28, -32, -32, -32, -29, -25, 14,
-	-25, -27, -25, -27, -11, 96, 97, 98, 99, 7,
-	-56, -28, -28, -28, -26, -32, -56, 22, 24, 21,
-	2, 22, 21, -32,
+	34, 32, 38, -45, 15, -45, -45, -18, -18, -19,
+	-18, -19, 15, 15, 15, -58, 22, 22, -58, 22,
+	-60, 21, 2, 22, 7, 2, -40, -58, -28, 19,
+	-28, 26, -28, -22, -22, 24, 17, 2, 17, 6,
+	6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
+	-48, -8, 84, 83, 22, 22, -58, 22, 7, 21,
+	2, 22, -4, 22, -28, 26, 26, 17, -24, -27,
+	57, -28, -32, -32, -32, -29, -25, 14, -25, -27,
+	-25, -27, -11, 99, 100, 101, 102, 22, -48, -45,
+	-45, 7, -58, -28, -28, -28, -26, -32, 22, -58,
+	22, 24, 21, 2, 22, 21, -32,
 }
 
 var yyDef = [...]int16{
-	0, -2, 138, 138, 0, 0, 7, 6, 1, 138,
-	106, 107, 108, 109, 110, 111, 112, 113, 114, 115,
-	116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
-	126, 127, 128, 129, 130, 131, 132, 133, 134, 0,
-	2, -2, 3, 4, 8, 9, 10, 11, 12, 13,
-	14, 15, 16, 17, 18, 19, 20, 21, 22, 0,
-	113, 246, 247, 0, 257, 0, 90, 91, 131, 132,
-	0, 284, -2, -2, -2, -2, -2, -2, -2, -2,
-	-2, -2, -2, -2, -2, -2, 240, 241, 0, 5,
-	105, 0, 137, 140, 0, 144, 148, 258, 149, 153,
-	46, 46, 46, 46, 46, 46, 46, 46, 46, 46,
-	46, 46, 46, 46, 46, 46, 0, 74, 75, 0,
-	0, 0, 0, 0, 0, 0, 0, 0, 25, 26,
-	0, 0, 0, 64, 0, 22, 88, -2, 89, 0,
-	0, 0, 0, 94, 96, 0, 100, 104, 135, 0,
-	141, 0, 147, 0, 152, 0, 45, 50, 51, 47,
+	0, -2, 149, 149, 0, 0, 7, 6, 1, 149,
+	114, 115, 116, 117, 118, 119, 120, 121, 122, 123,
+	124, 125, 126, 127, 128, 129, 130, 131, 132, 133,
+	134, 135, 136, 137, 138, 139, 140, 141, 142, 143,
+	144, 145, 0, 2, -2, 3, 4, 8, 9, 10,
+	11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
+	21, 22, 0, 124, 260, 261, 0, 271, 0, 98,
+	99, 142, 143, 0, 298, -2, -2, -2, -2, -2,
+	-2, -2, -2, -2, -2, -2, -2, -2, -2, 254,
+	255, 0, 5, 113, 0, 148, 151, 0, 155, 159,
+	272, 160, 164, 46, 46, 46, 46, 46, 46, 46,
+	46, 46, 46, 46, 46, 46, 46, 46, 46, 0,
+	82, 83, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 25, 26, 0, 0, 0, 72, 0, 22, 96,
+	-2, 97, 0, 0, 0, 0, 102, 104, 0, 108,
+	112, 146, 0, 152, 0, 158, 0, 163, 0, 45,
+	54, 50, 51, 47, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 80,
+	81, 275, 0, 0, 0, 0, 284, 285, 286, 0,
+	84, 0, 86, 266, 267, 87, 88, 262, 263, 0,
+	0, 0, 95, 79, 287, 0, 0, 0, 289, 290,
+	291, 292, 293, 294, 23, 24, 27, 0, 63, 28,
+	0, 74, 76, 78, 299, 295, 296, 0, 100, 0,
+	105, 0, 111, 256, 257, 258, 259, 0, 147, 150,
+	153, 156, 154, 157, 162, 165, 167, 170, 174, 175,
+	176, 0, 29, 0, 0, 0, 0, 0, -2, -2,
+	30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
+	40, 41, 42, 43, 44, 276, 0, 0, 0, 0,
+	288, 0, 0, 0, 0, 0, 264, 265, 89, 0,
+	94, 0, 62, 65, 67, 68, 69, 218, 219, 220,
+	221, 222, 223, 224, 225, 226, 227, 228, 229, 230,
+	231, 232, 233, 234, 235, 236, 237, 238, 239, 240,
+	241, 242, 243, 244, 245, 246, 247, 248, 249, 250,
+	251, 252, 253, 73, 77, 0, 101, 103, 106, 110,
+	107, 109, 0, 0, 0, 0, 0, 0, 0, 0,
+	180, 182, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 55, 0, 56, 57, 48, 49, 52,
+	274, 53, 0, 0, 0, 0, 277, 278, 0, 85,
+	0, 91, 93, 60, 0, 66, 75, 0, 166, 268,
+	168, 0, 171, 0, 0, 0, 178, 183, 179, 0,
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-	0, 0, 0, 0, 0, 72, 73, 261, 0, 0,
-	0, 0, 270, 271, 272, 0, 76, 0, 78, 252,
-	253, 79, 80, 248, 249, 0, 0, 0, 87, 71,
-	273, 0, 0, 0, 275, 276, 277, 278, 279, 280,
-	23, 24, 27, 0, 57, 28, 0, 66, 68, 70,
-	285, 281, 282, 0, 92, 0, 97, 0, 103, 242,
-	243, 244, 245, 0, 136, 139, 142, 145, 143, 146,
-	151, 154, 156, 159, 163, 164, 165, 0, 29, 0,
-	0, -2, -2, 30, 31, 32, 33, 34, 35, 36,
-	37, 38, 39, 40, 41, 42, 43, 44, 262, 0,
-	0, 0, 0, 274, 0, 0, 0, 0, 0, 250,
-	251, 81, 0, 86, 0, 56, 59, 61, 62, 63,
-	207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
-	217, 218, 219, 220, 221, 222, 223, 224, 225, 226,
-	227, 228, 229, 230, 231, 232, 233, 234, 235, 236,
-	237, 238, 239, 65, 69, 0, 93, 95, 98, 102,
-	99, 101, 0, 0, 0, 0, 0, 0, 0, 0,
-	169, 171, 0, 0, 0, 0, 0, 0, 0, 0,
-	0, 0, 0, 48, 49, 52, 260, 53, 0, 0,
-	0, 0, 263, 264, 0, 77, 0, 83, 85, 54,
-	0, 60, 67, 0, 155, 254, 157, 0, 160, 0,
-	0, 0, 167, 172, 168, 0, 0, 0, 0, 0,
-	0, 0, 0, 0, 0, 0, 265, 266, 0, 269,
-	0, 82, 84, 55, 58, 283, 158, 0, 0, 166,
-	170, 173, 0, 256, 174, 175, 176, 177, 178, 0,
-	179, 180, 181, 182, 183, 189, 190, 191, 192, 0,
-	0, 161, 162, 255, 0, 187, 0, 267, 0, 185,
-	188, 268, 184, 186,
+	0, 0, 0, 0, 279, 280, 0, 283, 0, 90,
+	92, 61, 64, 297, 169, 0, 0, 177, 181, 184,
+	0, 270, 185, 186, 187, 188, 189, 0, 190, 191,
+	192, 193, 194, 200, 201, 202, 203, 70, 0, 58,
+	59, 0, 0, 172, 173, 269, 0, 198, 71, 0,
+	281, 0, 196, 199, 282, 195, 197,
 }
 
 var yyTok1 = [...]int8{
@@ -773,7 +811,7 @@ var yyTok2 = [...]int8{
 	72, 73, 74, 75, 76, 77, 78, 79, 80, 81,
 	82, 83, 84, 85, 86, 87, 88, 89, 90, 91,
 	92, 93, 94, 95, 96, 97, 98, 99, 100, 101,
-	102, 103, 104, 105, 106,
+	102, 103, 104, 105, 106, 107, 108, 109,
 }
 
 var yyTok3 = [...]int8{
@@ -1298,44 +1336,83 @@ yydefault:
 			yyVAL.node.(*BinaryExpr).VectorMatching.Card = CardOneToMany
 			yyVAL.node.(*BinaryExpr).VectorMatching.Include = yyDollar[3].strings
 		}
-	case 54:
+	case 55:
+		yyDollar = yyS[yypt-3 : yypt+1]
+		{
+			yyVAL.node = yyDollar[1].node
+			fill := yyDollar[3].node.(*NumberLiteral).Val
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill
+		}
+	case 56:
+		yyDollar = yyS[yypt-3 : yypt+1]
+		{
+			yyVAL.node = yyDollar[1].node
+			fill := yyDollar[3].node.(*NumberLiteral).Val
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill
+		}
+	case 57:
+		yyDollar = yyS[yypt-3 : yypt+1]
+		{
+			yyVAL.node = yyDollar[1].node
+			fill := yyDollar[3].node.(*NumberLiteral).Val
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill
+		}
+	case 58:
+		yyDollar = yyS[yypt-5 : yypt+1]
+		{
+			yyVAL.node = yyDollar[1].node
+			fill_left := yyDollar[3].node.(*NumberLiteral).Val
+			fill_right := yyDollar[5].node.(*NumberLiteral).Val
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill_left
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill_right
+		}
+	case 59:
+		yyDollar = yyS[yypt-5 : yypt+1]
+		{
+			fill_right := yyDollar[3].node.(*NumberLiteral).Val
+			fill_left := yyDollar[5].node.(*NumberLiteral).Val
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.LHS = &fill_left
+			yyVAL.node.(*BinaryExpr).VectorMatching.FillValues.RHS = &fill_right
+		}
+	case 60:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.strings = yyDollar[2].strings
 		}
-	case 55:
+	case 61:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.strings = yyDollar[2].strings
 		}
-	case 56:
+	case 62:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.strings = []string{}
 		}
-	case 57:
+	case 63:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yylex.(*parser).unexpected("grouping opts", "\"(\"")
 			yyVAL.strings = nil
 		}
-	case 58:
+	case 64:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.strings = append(yyDollar[1].strings, yyDollar[3].item.Val)
 		}
-	case 59:
+	case 65:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.strings = []string{yyDollar[1].item.Val}
 		}
-	case 60:
+	case 66:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).unexpected("grouping opts", "\",\" or \")\"")
 			yyVAL.strings = yyDollar[1].strings
 		}
-	case 61:
+	case 67:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			if !model.UTF8Validation.IsValidLabelName(yyDollar[1].item.Val) {
@@ -1343,7 +1420,7 @@ yydefault:
 			}
 			yyVAL.item = yyDollar[1].item
 		}
-	case 62:
+	case 68:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			unquoted := yylex.(*parser).unquoteString(yyDollar[1].item.Val)
@@ -1354,13 +1431,28 @@ yydefault:
 			yyVAL.item.Pos++
 			yyVAL.item.Val = unquoted
 		}
-	case 63:
+	case 69:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yylex.(*parser).unexpected("grouping opts", "label")
 			yyVAL.item = Item{}
 		}
-	case 64:
+	case 70:
+		yyDollar = yyS[yypt-3 : yypt+1]
+		{
+			yyVAL.node = yyDollar[2].node.(*NumberLiteral)
+		}
+	case 71:
+		yyDollar = yyS[yypt-4 : yypt+1]
+		{
+			nl := yyDollar[3].node.(*NumberLiteral)
+			if yyDollar[2].item.Typ == SUB {
+				nl.Val *= -1
+			}
+			nl.PosRange.Start = yyDollar[2].item.Pos
+			yyVAL.node = nl
+		}
+	case 72:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			fn, exist := getFunction(yyDollar[1].item.Val, yylex.(*parser).functions)
@@ -1379,38 +1471,38 @@ yydefault:
 				},
 			}
 		}
-	case 65:
+	case 73:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = yyDollar[2].node
 		}
-	case 66:
+	case 74:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.node = Expressions{}
 		}
-	case 67:
+	case 75:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = append(yyDollar[1].node.(Expressions), yyDollar[3].node.(Expr))
 		}
-	case 68:
+	case 76:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.node = Expressions{yyDollar[1].node.(Expr)}
 		}
-	case 69:
+	case 77:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).addParseErrf(yyDollar[2].item.PositionRange(), "trailing commas not allowed in function call args")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 70:
+	case 78:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = &ParenExpr{Expr: yyDollar[2].node.(Expr), PosRange: mergeRanges(&yyDollar[1].item, &yyDollar[3].item)}
 		}
-	case 71:
+	case 79:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			if numLit, ok := yyDollar[1].node.(*NumberLiteral); ok {
@@ -1424,7 +1516,7 @@ yydefault:
 			}
 			yyVAL.node = yyDollar[1].node
 		}
-	case 72:
+	case 80:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			if numLit, ok := yyDollar[3].node.(*NumberLiteral); ok {
@@ -1435,41 +1527,41 @@ yydefault:
 			yylex.(*parser).addOffsetExpr(yyDollar[1].node, yyDollar[3].node.(*DurationExpr))
 			yyVAL.node = yyDollar[1].node
 		}
-	case 73:
+	case 81:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("offset", "number, duration, step(), or range()")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 74:
+	case 82:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).setAnchored(yyDollar[1].node)
 		}
-	case 75:
+	case 83:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).setSmoothed(yyDollar[1].node)
 		}
-	case 76:
+	case 84:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).setTimestamp(yyDollar[1].node, yyDollar[3].float)
 			yyVAL.node = yyDollar[1].node
 		}
-	case 77:
+	case 85:
 		yyDollar = yyS[yypt-5 : yypt+1]
 		{
 			yylex.(*parser).setAtModifierPreprocessor(yyDollar[1].node, yyDollar[3].item)
 			yyVAL.node = yyDollar[1].node
 		}
-	case 78:
+	case 86:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("@", "timestamp")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 81:
+	case 89:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			var errMsg string
@@ -1499,7 +1591,7 @@ yydefault:
 				EndPos:         yylex.(*parser).lastClosing,
 			}
 		}
-	case 82:
+	case 90:
 		yyDollar = yyS[yypt-6 : yypt+1]
 		{
 			var rangeNl time.Duration
@@ -1521,7 +1613,7 @@ yydefault:
 				EndPos:    yyDollar[6].item.Pos + 1,
 			}
 		}
-	case 83:
+	case 91:
 		yyDollar = yyS[yypt-5 : yypt+1]
 		{
 			var rangeNl time.Duration
@@ -1536,31 +1628,31 @@ yydefault:
 				EndPos:    yyDollar[5].item.Pos + 1,
 			}
 		}
-	case 84:
+	case 92:
 		yyDollar = yyS[yypt-6 : yypt+1]
 		{
 			yylex.(*parser).unexpected("subquery selector", "\"]\"")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 85:
+	case 93:
 		yyDollar = yyS[yypt-5 : yypt+1]
 		{
 			yylex.(*parser).unexpected("subquery selector", "number, duration, step(), range(), or \"]\"")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 86:
+	case 94:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yylex.(*parser).unexpected("subquery or range", "\":\" or \"]\"")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 87:
+	case 95:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("subquery or range selector", "number, duration, step(), or range()")
 			yyVAL.node = yyDollar[1].node
 		}
-	case 88:
+	case 96:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			if nl, ok := yyDollar[2].node.(*NumberLiteral); ok {
@@ -1573,7 +1665,7 @@ yydefault:
 				yyVAL.node = &UnaryExpr{Op: yyDollar[1].item.Typ, Expr: yyDollar[2].node.(Expr), StartPos: yyDollar[1].item.Pos}
 			}
 		}
-	case 89:
+	case 97:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			vs := yyDollar[2].node.(*VectorSelector)
@@ -1582,7 +1674,7 @@ yydefault:
 			yylex.(*parser).assembleVectorSelector(vs)
 			yyVAL.node = vs
 		}
-	case 90:
+	case 98:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			vs := &VectorSelector{
@@ -1593,14 +1685,14 @@ yydefault:
 			yylex.(*parser).assembleVectorSelector(vs)
 			yyVAL.node = vs
 		}
-	case 91:
+	case 99:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			vs := yyDollar[1].node.(*VectorSelector)
 			yylex.(*parser).assembleVectorSelector(vs)
 			yyVAL.node = vs
 		}
-	case 92:
+	case 100:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = &VectorSelector{
@@ -1608,7 +1700,7 @@ yydefault:
 				PosRange:      mergeRanges(&yyDollar[1].item, &yyDollar[3].item),
 			}
 		}
-	case 93:
+	case 101:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.node = &VectorSelector{
@@ -1616,7 +1708,7 @@ yydefault:
 				PosRange:      mergeRanges(&yyDollar[1].item, &yyDollar[4].item),
 			}
 		}
-	case 94:
+	case 102:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.node = &VectorSelector{
@@ -1624,7 +1716,7 @@ yydefault:
 				PosRange:      mergeRanges(&yyDollar[1].item, &yyDollar[2].item),
 			}
 		}
-	case 95:
+	case 103:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			if yyDollar[1].matchers != nil {
@@ -1633,144 +1725,144 @@ yydefault:
 				yyVAL.matchers = yyDollar[1].matchers
 			}
 		}
-	case 96:
+	case 104:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.matchers = []*labels.Matcher{yyDollar[1].matcher}
 		}
-	case 97:
+	case 105:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label matching", "\",\" or \"}\"")
 			yyVAL.matchers = yyDollar[1].matchers
 		}
-	case 98:
+	case 106:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.matcher = yylex.(*parser).newLabelMatcher(yyDollar[1].item, yyDollar[2].item, yyDollar[3].item)
 		}
-	case 99:
+	case 107:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.matcher = yylex.(*parser).newLabelMatcher(yyDollar[1].item, yyDollar[2].item, yyDollar[3].item)
 		}
-	case 100:
+	case 108:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.matcher = yylex.(*parser).newMetricNameMatcher(yyDollar[1].item)
 		}
-	case 101:
+	case 109:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label matching", "string")
 			yyVAL.matcher = nil
 		}
-	case 102:
+	case 110:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label matching", "string")
 			yyVAL.matcher = nil
 		}
-	case 103:
+	case 111:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label matching", "label matching operator")
 			yyVAL.matcher = nil
 		}
-	case 104:
+	case 112:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label matching", "identifier or \"}\"")
 			yyVAL.matcher = nil
 		}
-	case 105:
+	case 113:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			b := labels.NewBuilder(yyDollar[2].labels)
 			b.Set(labels.MetricName, yyDollar[1].item.Val)
 			yyVAL.labels = b.Labels()
 		}
-	case 106:
+	case 114:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.labels = yyDollar[1].labels
 		}
-	case 135:
+	case 146:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.labels = labels.New(yyDollar[2].lblList...)
 		}
-	case 136:
+	case 147:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.labels = labels.New(yyDollar[2].lblList...)
 		}
-	case 137:
+	case 148:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.labels = labels.New()
 		}
-	case 138:
+	case 149:
 		yyDollar = yyS[yypt-0 : yypt+1]
 		{
 			yyVAL.labels = labels.New()
 		}
-	case 139:
+	case 150:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.lblList = append(yyDollar[1].lblList, yyDollar[3].label)
 		}
-	case 140:
+	case 151:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.lblList = []labels.Label{yyDollar[1].label}
 		}
-	case 141:
+	case 152:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label set", "\",\" or \"}\"")
 			yyVAL.lblList = yyDollar[1].lblList
 		}
-	case 142:
+	case 153:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.label = labels.Label{Name: yyDollar[1].item.Val, Value: yylex.(*parser).unquoteString(yyDollar[3].item.Val)}
 		}
-	case 143:
+	case 154:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.label = labels.Label{Name: yyDollar[1].item.Val, Value: yylex.(*parser).unquoteString(yyDollar[3].item.Val)}
 		}
-	case 144:
+	case 155:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.label = labels.Label{Name: labels.MetricName, Value: yyDollar[1].item.Val}
 		}
-	case 145:
+	case 156:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label set", "string")
 			yyVAL.label = labels.Label{}
 		}
-	case 146:
+	case 157:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label set", "string")
 			yyVAL.label = labels.Label{}
 		}
-	case 147:
+	case 158:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label set", "\"=\"")
 			yyVAL.label = labels.Label{}
 		}
-	case 148:
+	case 159:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yylex.(*parser).unexpected("label set", "identifier or \"}\"")
 			yyVAL.label = labels.Label{}
 		}
-	case 149:
+	case 160:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).generatedParserResult = &seriesDescription{
@@ -1778,33 +1870,33 @@ yydefault:
 				values: yyDollar[2].series,
 			}
 		}
-	case 150:
+	case 161:
 		yyDollar = yyS[yypt-0 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{}
 		}
-	case 151:
+	case 162:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.series = append(yyDollar[1].series, yyDollar[3].series...)
 		}
-	case 152:
+	case 163:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.series = yyDollar[1].series
 		}
-	case 153:
+	case 164:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yylex.(*parser).unexpected("series values", "")
 			yyVAL.series = nil
 		}
-	case 154:
+	case 165:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{{Omitted: true}}
 		}
-	case 155:
+	case 166:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{}
@@ -1812,12 +1904,12 @@ yydefault:
 				yyVAL.series = append(yyVAL.series, SequenceValue{Omitted: true})
 			}
 		}
-	case 156:
+	case 167:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{{Value: yyDollar[1].float}}
 		}
-	case 157:
+	case 168:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{}
@@ -1826,7 +1918,7 @@ yydefault:
 				yyVAL.series = append(yyVAL.series, SequenceValue{Value: yyDollar[1].float})
 			}
 		}
-	case 158:
+	case 169:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{}
@@ -1836,12 +1928,12 @@ yydefault:
 				yyDollar[1].float += yyDollar[2].float
 			}
 		}
-	case 159:
+	case 170:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{{Histogram: yyDollar[1].histogram}}
 		}
-	case 160:
+	case 171:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.series = []SequenceValue{}
@@ -1851,7 +1943,7 @@ yydefault:
 				//$1 += $2
 			}
 		}
-	case 161:
+	case 172:
 		yyDollar = yyS[yypt-5 : yypt+1]
 		{
 			val, err := yylex.(*parser).histogramsIncreaseSeries(yyDollar[1].histogram, yyDollar[3].histogram, yyDollar[5].uint)
@@ -1860,7 +1952,7 @@ yydefault:
 			}
 			yyVAL.series = val
 		}
-	case 162:
+	case 173:
 		yyDollar = yyS[yypt-5 : yypt+1]
 		{
 			val, err := yylex.(*parser).histogramsDecreaseSeries(yyDollar[1].histogram, yyDollar[3].histogram, yyDollar[5].uint)
@@ -1869,7 +1961,7 @@ yydefault:
 			}
 			yyVAL.series = val
 		}
-	case 163:
+	case 174:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			if yyDollar[1].item.Val != "stale" {
@@ -1877,130 +1969,130 @@ yydefault:
 			}
 			yyVAL.float = math.Float64frombits(value.StaleNaN)
 		}
-	case 166:
+	case 177:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&yyDollar[2].descriptors)
 		}
-	case 167:
+	case 178:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&yyDollar[2].descriptors)
 		}
-	case 168:
+	case 179:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			m := yylex.(*parser).newMap()
 			yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&m)
 		}
-	case 169:
+	case 180:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			m := yylex.(*parser).newMap()
 			yyVAL.histogram = yylex.(*parser).buildHistogramFromMap(&m)
 		}
-	case 170:
+	case 181:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = *(yylex.(*parser).mergeMaps(&yyDollar[1].descriptors, &yyDollar[3].descriptors))
 		}
-	case 171:
+	case 182:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.descriptors = yyDollar[1].descriptors
 		}
-	case 172:
+	case 183:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yylex.(*parser).unexpected("histogram description", "histogram description key, e.g. buckets:[5 10 7]")
 		}
-	case 173:
+	case 184:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["schema"] = yyDollar[3].int
 		}
-	case 174:
+	case 185:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["sum"] = yyDollar[3].float
 		}
-	case 175:
+	case 186:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["count"] = yyDollar[3].float
 		}
-	case 176:
+	case 187:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["z_bucket"] = yyDollar[3].float
 		}
-	case 177:
+	case 188:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["z_bucket_w"] = yyDollar[3].float
 		}
-	case 178:
+	case 189:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["custom_values"] = yyDollar[3].bucket_set
 		}
-	case 179:
+	case 190:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["buckets"] = yyDollar[3].bucket_set
 		}
-	case 180:
+	case 191:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["offset"] = yyDollar[3].int
 		}
-	case 181:
+	case 192:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["n_buckets"] = yyDollar[3].bucket_set
 		}
-	case 182:
+	case 193:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["n_offset"] = yyDollar[3].int
 		}
-	case 183:
+	case 194:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.descriptors = yylex.(*parser).newMap()
 			yyVAL.descriptors["counter_reset_hint"] = yyDollar[3].item
 		}
-	case 184:
+	case 195:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.bucket_set = yyDollar[2].bucket_set
 		}
-	case 185:
+	case 196:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.bucket_set = yyDollar[2].bucket_set
 		}
-	case 186:
+	case 197:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.bucket_set = append(yyDollar[1].bucket_set, yyDollar[3].float)
 		}
-	case 187:
+	case 198:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.bucket_set = []float64{yyDollar[1].float}
 		}
-	case 246:
+	case 260:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.node = &NumberLiteral{
@@ -2008,7 +2100,7 @@ yydefault:
 				PosRange: yyDollar[1].item.PositionRange(),
 			}
 		}
-	case 247:
+	case 261:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			var err error
@@ -2023,12 +2115,12 @@ yydefault:
 				Duration: true,
 			}
 		}
-	case 248:
+	case 262:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.float = yylex.(*parser).number(yyDollar[1].item.Val)
 		}
-	case 249:
+	case 263:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			var err error
@@ -2039,17 +2131,17 @@ yydefault:
 			}
 			yyVAL.float = dur.Seconds()
 		}
-	case 250:
+	case 264:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.float = yyDollar[2].float
 		}
-	case 251:
+	case 265:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.float = -yyDollar[2].float
 		}
-	case 254:
+	case 268:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			var err error
@@ -2058,17 +2150,17 @@ yydefault:
 				yylex.(*parser).addParseErrf(yyDollar[1].item.PositionRange(), "invalid repetition in series values: %s", err)
 			}
 		}
-	case 255:
+	case 269:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			yyVAL.int = -int64(yyDollar[2].uint)
 		}
-	case 256:
+	case 270:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.int = int64(yyDollar[1].uint)
 		}
-	case 257:
+	case 271:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.node = &StringLiteral{
@@ -2076,7 +2168,7 @@ yydefault:
 				PosRange: yyDollar[1].item.PositionRange(),
 			}
 		}
-	case 258:
+	case 272:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			yyVAL.item = Item{
@@ -2085,12 +2177,12 @@ yydefault:
 				Val: yylex.(*parser).unquoteString(yyDollar[1].item.Val),
 			}
 		}
-	case 259:
+	case 273:
 		yyDollar = yyS[yypt-0 : yypt+1]
 		{
 			yyVAL.strings = nil
 		}
-	case 261:
+	case 275:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			nl := yyDollar[1].node.(*NumberLiteral)
@@ -2101,7 +2193,7 @@ yydefault:
 			}
 			yyVAL.node = nl
 		}
-	case 262:
+	case 276:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			nl := yyDollar[2].node.(*NumberLiteral)
@@ -2116,7 +2208,7 @@ yydefault:
 			nl.PosRange.Start = yyDollar[1].item.Pos
 			yyVAL.node = nl
 		}
-	case 263:
+	case 277:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2125,7 +2217,7 @@ yydefault:
 				EndPos:   yyDollar[3].item.PositionRange().End,
 			}
 		}
-	case 264:
+	case 278:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2134,7 +2226,7 @@ yydefault:
 				EndPos:   yyDollar[3].item.PositionRange().End,
 			}
 		}
-	case 265:
+	case 279:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2147,7 +2239,7 @@ yydefault:
 				StartPos: yyDollar[1].item.Pos,
 			}
 		}
-	case 266:
+	case 280:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2160,7 +2252,7 @@ yydefault:
 				StartPos: yyDollar[1].item.Pos,
 			}
 		}
-	case 267:
+	case 281:
 		yyDollar = yyS[yypt-6 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2171,7 +2263,7 @@ yydefault:
 				RHS:      yyDollar[5].node.(Expr),
 			}
 		}
-	case 268:
+	case 282:
 		yyDollar = yyS[yypt-7 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2187,7 +2279,7 @@ yydefault:
 				},
 			}
 		}
-	case 269:
+	case 283:
 		yyDollar = yyS[yypt-4 : yypt+1]
 		{
 			de := yyDollar[3].node.(*DurationExpr)
@@ -2202,7 +2294,7 @@ yydefault:
 			}
 			yyVAL.node = yyDollar[3].node
 		}
-	case 273:
+	case 287:
 		yyDollar = yyS[yypt-1 : yypt+1]
 		{
 			nl := yyDollar[1].node.(*NumberLiteral)
@@ -2213,7 +2305,7 @@ yydefault:
 			}
 			yyVAL.node = nl
 		}
-	case 274:
+	case 288:
 		yyDollar = yyS[yypt-2 : yypt+1]
 		{
 			switch expr := yyDollar[2].node.(type) {
@@ -2246,25 +2338,25 @@ yydefault:
 				break
 			}
 		}
-	case 275:
+	case 289:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr))
 			yyVAL.node = &DurationExpr{Op: ADD, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)}
 		}
-	case 276:
+	case 290:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr))
 			yyVAL.node = &DurationExpr{Op: SUB, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)}
 		}
-	case 277:
+	case 291:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr))
 			yyVAL.node = &DurationExpr{Op: MUL, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)}
 		}
-	case 278:
+	case 292:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr))
@@ -2275,7 +2367,7 @@ yydefault:
 			}
 			yyVAL.node = &DurationExpr{Op: DIV, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)}
 		}
-	case 279:
+	case 293:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr))
@@ -2286,13 +2378,13 @@ yydefault:
 			}
 			yyVAL.node = &DurationExpr{Op: MOD, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)}
 		}
-	case 280:
+	case 294:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[1].node.(Expr))
 			yyVAL.node = &DurationExpr{Op: POW, LHS: yyDollar[1].node.(Expr), RHS: yyDollar[3].node.(Expr)}
 		}
-	case 281:
+	case 295:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2301,7 +2393,7 @@ yydefault:
 				EndPos:   yyDollar[3].item.PositionRange().End,
 			}
 		}
-	case 282:
+	case 296:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2310,7 +2402,7 @@ yydefault:
 				EndPos:   yyDollar[3].item.PositionRange().End,
 			}
 		}
-	case 283:
+	case 297:
 		yyDollar = yyS[yypt-6 : yypt+1]
 		{
 			yyVAL.node = &DurationExpr{
@@ -2321,7 +2413,7 @@ yydefault:
 				RHS:      yyDollar[5].node.(Expr),
 			}
 		}
-	case 285:
+	case 299:
 		yyDollar = yyS[yypt-3 : yypt+1]
 		{
 			yylex.(*parser).experimentalDurationExpr(yyDollar[2].node.(Expr))
diff --git a/promql/parser/lex.go b/promql/parser/lex.go
index b3a82dc0c6..7149985767 100644
--- a/promql/parser/lex.go
+++ b/promql/parser/lex.go
@@ -137,6 +137,9 @@ var key = map[string]ItemType{
 	"ignoring":    IGNORING,
 	"group_left":  GROUP_LEFT,
 	"group_right": GROUP_RIGHT,
+	"fill":        FILL,
+	"fill_left":   FILL_LEFT,
+	"fill_right":  FILL_RIGHT,
 	"bool":        BOOL,
 
 	// Preprocessors.
@@ -1083,6 +1086,17 @@ Loop:
 			word := l.input[l.start:l.pos]
 			switch kw, ok := key[strings.ToLower(word)]; {
 			case ok:
+				// For fill/fill_left/fill_right, only treat as keyword if followed by '('
+				// This allows using these as metric names (e.g., "fill + fill").
+				// This could be done for other keywords as well, but for the new fill
+				// modifiers this is especially important so we don't break any existing
+				// queries.
+				if kw == FILL || kw == FILL_LEFT || kw == FILL_RIGHT {
+					if !l.peekFollowedByLeftParen() {
+						l.emit(IDENTIFIER)
+						break Loop
+					}
+				}
 				l.emit(kw)
 			case !strings.Contains(word, ":"):
 				l.emit(IDENTIFIER)
@@ -1098,6 +1112,23 @@ Loop:
 	return lexStatements
 }
 
+// peekFollowedByLeftParen checks if the next non-whitespace character is '('.
+// This is used for context-sensitive keywords like fill/fill_left/fill_right
+// that should only be treated as keywords when followed by '('.
+func (l *Lexer) peekFollowedByLeftParen() bool {
+	pos := l.pos
+	for {
+		if int(pos) >= len(l.input) {
+			return false
+		}
+		r, w := utf8.DecodeRuneInString(l.input[pos:])
+		if !isSpace(r) {
+			return r == '('
+		}
+		pos += posrange.Pos(w)
+	}
+}
+
 func isSpace(r rune) bool {
 	return r == ' ' || r == '\t' || r == '\n' || r == '\r'
 }
diff --git a/promql/parser/parse.go b/promql/parser/parse.go
index 817e0d02d9..a872706364 100644
--- a/promql/parser/parse.go
+++ b/promql/parser/parse.go
@@ -768,6 +768,9 @@ func (p *parser) checkAST(node Node) (typ ValueType) {
 			if len(n.VectorMatching.MatchingLabels) > 0 {
 				p.addParseErrf(n.PositionRange(), "vector matching only allowed between instant vectors")
 			}
+			if n.VectorMatching.FillValues.LHS != nil || n.VectorMatching.FillValues.RHS != nil {
+				p.addParseErrf(n.PositionRange(), "filling in missing series only allowed between instant vectors")
+			}
 			n.VectorMatching = nil
 		case n.Op.IsSetOperator(): // Both operands are Vectors.
 			if n.VectorMatching.Card == CardOneToMany || n.VectorMatching.Card == CardManyToOne {
@@ -776,6 +779,9 @@ func (p *parser) checkAST(node Node) (typ ValueType) {
 			if n.VectorMatching.Card != CardManyToMany {
 				p.addParseErrf(n.PositionRange(), "set operations must always be many-to-many")
 			}
+			if n.VectorMatching.FillValues.LHS != nil || n.VectorMatching.FillValues.RHS != nil {
+				p.addParseErrf(n.PositionRange(), "filling in missing series not allowed for set operators")
+			}
 		}
 
 		if (lt == ValueTypeScalar || rt == ValueTypeScalar) && n.Op.IsSetOperator() {
diff --git a/promql/parser/printer.go b/promql/parser/printer.go
index 01e2c46c1b..44ca15e532 100644
--- a/promql/parser/printer.go
+++ b/promql/parser/printer.go
@@ -172,6 +172,19 @@ func (node *BinaryExpr) getMatchingStr() string {
 			b.WriteString(")")
 			matching += b.String()
 		}
+
+		if vm.FillValues.LHS != nil || vm.FillValues.RHS != nil {
+			if vm.FillValues.LHS == vm.FillValues.RHS {
+				matching += fmt.Sprintf(" fill (%v)", *vm.FillValues.LHS)
+			} else {
+				if vm.FillValues.LHS != nil {
+					matching += fmt.Sprintf(" fill_left (%v)", *vm.FillValues.LHS)
+				}
+				if vm.FillValues.RHS != nil {
+					matching += fmt.Sprintf(" fill_right (%v)", *vm.FillValues.RHS)
+				}
+			}
+		}
 	}
 	return matching
 }
diff --git a/promql/parser/printer_test.go b/promql/parser/printer_test.go
index 4499fa7860..a5f254527e 100644
--- a/promql/parser/printer_test.go
+++ b/promql/parser/printer_test.go
@@ -113,6 +113,26 @@ func TestExprString(t *testing.T) {
 			in:  `a - ignoring() group_left c`,
 			out: `a - ignoring () group_left () c`,
 		},
+		{
+			in:  `a + fill(-23) b`,
+			out: `a + fill (-23) b`,
+		},
+		{
+			in:  `a + fill_left(-23) b`,
+			out: `a + fill_left (-23) b`,
+		},
+		{
+			in:  `a + fill_right(42) b`,
+			out: `a + fill_right (42) b`,
+		},
+		{
+			in:  `a + fill_left(-23) fill_right(42) b`,
+			out: `a + fill_left (-23) fill_right (42) b`,
+		},
+		{
+			in:  `a + on(b) group_left fill(-23) c`,
+			out: `a + on (b) group_left () fill (-23) c`,
+		},
 		{
 			in: `up > bool 0`,
 		},
diff --git a/web/api/v1/translate_ast.go b/web/api/v1/translate_ast.go
index 3cce0583f9..3c2bc09943 100644
--- a/web/api/v1/translate_ast.go
+++ b/web/api/v1/translate_ast.go
@@ -47,6 +47,10 @@ func translateAST(node parser.Expr) any {
 				"labels":  sanitizeList(m.MatchingLabels),
 				"on":      m.On,
 				"include": sanitizeList(m.Include),
+				"fillValues": map[string]*float64{
+					"lhs": m.FillValues.LHS,
+					"rhs": m.FillValues.RHS,
+				},
 			}
 		}
 
diff --git a/web/ui/mantine-ui/src/pages/query/ExplainViews/BinaryExpr/VectorVector.tsx b/web/ui/mantine-ui/src/pages/query/ExplainViews/BinaryExpr/VectorVector.tsx
index e70b7a3f3e..5c10357561 100644
--- a/web/ui/mantine-ui/src/pages/query/ExplainViews/BinaryExpr/VectorVector.tsx
+++ b/web/ui/mantine-ui/src/pages/query/ExplainViews/BinaryExpr/VectorVector.tsx
@@ -8,6 +8,7 @@ import {
   MatchErrorType,
   computeVectorVectorBinOp,
   filteredSampleValue,
+  MaybeFilledInstantSample,
 } from "../../../../promql/binOp";
 import { formatNode, labelNameList } from "../../../../promql/format";
 import {
@@ -177,11 +178,10 @@ const explanationText = (node: BinaryExpr): React.ReactNode => {
           
         ) : (
           
-            
-              group_{manySide}({labelNameList(matching.include)})
-            
-            : {matching.card} match. Each series from the {oneSide}-hand side is
-            allowed to match with multiple series from the {manySide}-hand side.
+            group_{manySide}
+            ({labelNameList(matching.include)}) : {matching.card} match. Each
+            series from the {oneSide}-hand side is allowed to match with
+            multiple series from the {manySide}-hand side.
             {matching.include.length !== 0 && (
               <>
                 {" "}
@@ -192,6 +192,55 @@ const explanationText = (node: BinaryExpr): React.ReactNode => {
             )}
           
         )}
+        {(matching.fillValues.lhs !== null ||
+          matching.fillValues.rhs !== null) &&
+          (matching.fillValues.lhs === matching.fillValues.rhs ? (
+            
+              fill(
+              
+                {matching.fillValues.lhs}
+              
+              ) : For series on either side missing a match, fill in the sample
+              value{" "}
+              
+                {matching.fillValues.lhs}
+              
+              .
+            
+          ) : (
+            <>
+              {matching.fillValues.lhs !== null && (
+                
+                  fill_left(
+                  
+                    {matching.fillValues.lhs}
+                  
+                  ) : For series on the left-hand side missing a match, fill in
+                  the sample value{" "}
+                  
+                    {matching.fillValues.lhs}
+                  
+                  .
+                
+              )}
+
+              {matching.fillValues.rhs !== null && (
+                
+                  fill_right
+                  (
+                  
+                    {matching.fillValues.rhs}
+                  
+                  ) : For series on the right-hand side missing a match, fill in
+                  the sample value{" "}
+                  
+                    {matching.fillValues.rhs}
+                  
+                  .
+                
+              )}
+            
+          ))}
         {node.bool && (
           
             bool: Instead of
@@ -239,7 +288,12 @@ const explainError = (
                   matching: {
                     ...(binOp.matching
                       ? binOp.matching
-                      : { labels: [], on: false, include: [] }),
+                      : {
+                          labels: [],
+                          on: false,
+                          include: [],
+                          fillValues: { lhs: null, rhs: null },
+                        }),
                     card:
                       err.dupeSide === "left"
                         ? vectorMatchCardinality.manyToOne
@@ -403,7 +457,7 @@ const VectorVectorBinaryExprExplainView: FC<
             );
 
             const matchGroupTable = (
-              series: InstantSample[],
+              series: MaybeFilledInstantSample[],
               seriesCount: number,
               color: string,
               colorOffset?: number
@@ -458,6 +512,11 @@ const VectorVectorBinaryExprExplainView: FC<
                                     )}
                                     format={true}
                                   />
+                                  {s.filled && (
+                                    
+                                      no match, filling in default value
+                                    
+                                  )}
                                 
                               
                               {showSampleValues && (
diff --git a/web/ui/mantine-ui/src/promql/ast.ts b/web/ui/mantine-ui/src/promql/ast.ts
index 94872c6db0..9f8c5cb102 100644
--- a/web/ui/mantine-ui/src/promql/ast.ts
+++ b/web/ui/mantine-ui/src/promql/ast.ts
@@ -104,11 +104,16 @@ export interface LabelMatcher {
   value: string;
 }
 
+export interface FillValues {
+  lhs: number | null;
+  rhs: number | null;
+}
 export interface VectorMatching {
   card: vectorMatchCardinality;
   labels: string[];
   on: boolean;
   include: string[];
+  fillValues: FillValues;
 }
 
 export type StartOrEnd = "start" | "end" | null;
diff --git a/web/ui/mantine-ui/src/promql/binOp.test.ts b/web/ui/mantine-ui/src/promql/binOp.test.ts
index 72ef16947b..9c5d59a94c 100644
--- a/web/ui/mantine-ui/src/promql/binOp.test.ts
+++ b/web/ui/mantine-ui/src/promql/binOp.test.ts
@@ -81,6 +81,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -247,6 +248,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1", "label2"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -413,6 +415,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: ["same"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -579,6 +582,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricB,
     rhs: testMetricC,
@@ -701,6 +705,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricB,
     rhs: testMetricC,
@@ -791,6 +796,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricB,
     rhs: testMetricC,
@@ -905,6 +911,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricC,
     rhs: testMetricB,
@@ -1019,6 +1026,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricC,
     rhs: testMetricB,
@@ -1107,6 +1115,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -1223,6 +1232,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -1409,6 +1419,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -1596,6 +1607,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -1763,6 +1775,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -1929,6 +1942,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -2022,6 +2036,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricB,
     rhs: testMetricC,
@@ -2105,6 +2120,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricB,
     rhs: testMetricC,
@@ -2156,6 +2172,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA,
     rhs: testMetricB,
@@ -2342,6 +2359,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA.slice(0, 3),
     rhs: testMetricB.slice(1, 4),
@@ -2474,6 +2492,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA.slice(0, 3),
     rhs: testMetricB.slice(1, 4),
@@ -2568,6 +2587,7 @@ const testCases: TestCase[] = [
       on: true,
       include: [],
       labels: ["label1"],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA.slice(0, 3),
     rhs: testMetricB.slice(1, 4),
@@ -2700,6 +2720,7 @@ const testCases: TestCase[] = [
       on: false,
       include: [],
       labels: [],
+      fillValues: { lhs: null, rhs: null },
     },
     lhs: testMetricA.slice(0, 3),
     rhs: testMetricB.slice(1, 4),
@@ -2886,6 +2907,7 @@ describe("binOp", () => {
         on: true,
         labels: ["label1"],
         include: [],
+        fillValues: { lhs: null, rhs: null },
       };
 
       const result = resultMetric(lhs, rhs, op, matching);
@@ -2911,6 +2933,7 @@ describe("binOp", () => {
         on: true,
         labels: ["label1"],
         include: [],
+        fillValues: { lhs: null, rhs: null },
       };
 
       const result = resultMetric(lhs, rhs, op, matching);
@@ -2931,6 +2954,7 @@ describe("binOp", () => {
         on: true,
         labels: ["label1"],
         include: ["label2"],
+        fillValues: { lhs: null, rhs: null },
       };
 
       const result = resultMetric(lhs, rhs, op, matching);
diff --git a/web/ui/mantine-ui/src/promql/binOp.ts b/web/ui/mantine-ui/src/promql/binOp.ts
index dbfa64be2c..f583bf81bb 100644
--- a/web/ui/mantine-ui/src/promql/binOp.ts
+++ b/web/ui/mantine-ui/src/promql/binOp.ts
@@ -45,13 +45,18 @@ export type VectorMatchError =
   | MultipleMatchesOnBothSidesError
   | MultipleMatchesOnOneSideError;
 
+export type MaybeFilledInstantSample = InstantSample & {
+  // If the sample was filled in via a fill(...) modifier, this is true.
+  filled?: boolean;
+};
+
 // A single match group as produced by a vector-to-vector binary operation, with all of its
 // left-hand side and right-hand side series, as well as a result and error, if applicable.
 export type BinOpMatchGroup = {
   groupLabels: Metric;
-  rhs: InstantSample[];
+  rhs: MaybeFilledInstantSample[];
   rhsCount: number; // Number of samples before applying limits.
-  lhs: InstantSample[];
+  lhs: MaybeFilledInstantSample[];
   lhsCount: number; // Number of samples before applying limits.
   result: {
     sample: InstantSample;
@@ -338,6 +343,26 @@ export const computeVectorVectorBinOp = (
     groups[sig].lhsCount++;
   });
 
+  // Check for any LHS / RHS with no series and fill in default values, if specified.
+  Object.values(groups).forEach((mg) => {
+    if (mg.lhs.length === 0 && matching.fillValues.lhs !== null) {
+      mg.lhs.push({
+        metric: {},
+        value: [0, formatPrometheusFloat(matching.fillValues.lhs as number)],
+        filled: true,
+      });
+      mg.lhsCount = 1;
+    }
+    if (mg.rhs.length === 0 && matching.fillValues.rhs !== null) {
+      mg.rhs.push({
+        metric: {},
+        value: [0, formatPrometheusFloat(matching.fillValues.rhs as number)],
+        filled: true,
+      });
+      mg.rhsCount = 1;
+    }
+  });
+
   // Annotate the match groups with errors (if any) and populate the results.
   Object.values(groups).forEach((mg) => {
     switch (matching.card) {
diff --git a/web/ui/mantine-ui/src/promql/format.tsx b/web/ui/mantine-ui/src/promql/format.tsx
index 75b1965b35..8602c65a82 100644
--- a/web/ui/mantine-ui/src/promql/format.tsx
+++ b/web/ui/mantine-ui/src/promql/format.tsx
@@ -265,6 +265,7 @@ const formatNodeInternal = (
     case nodeType.binaryExpr: {
       let matching = <>;
       let grouping = <>;
+      let fill = <>;
       const vm = node.matching;
       if (vm !== null) {
         if (
@@ -305,6 +306,45 @@ const formatNodeInternal = (
             
           );
         }
+
+        const lfill = vm.fillValues.lhs;
+        const rfill = vm.fillValues.rhs;
+        if (lfill !== null || rfill !== null) {
+          if (lfill === rfill) {
+            fill = (
+              <>
+                {" "}
+                fill
+                (
+                {lfill}
+                )
+              
+            );
+          } else {
+            fill = (
+              <>
+                {lfill !== null && (
+                  <>
+                    {" "}
+                    fill_left
+                    (
+                    {lfill}
+                    )
+                  
+                )}
+                {rfill !== null && (
+                  <>
+                    {" "}
+                    fill_right
+                    (
+                    {rfill}
+                    )
+                  
+                )}
+              
+            );
+          }
+        }
       }
 
       return (
@@ -327,7 +367,8 @@ const formatNodeInternal = (
             
           )}
           {matching}
-          {grouping}{" "}
+          {grouping}
+          {fill}{" "}
           {showChildren &&
             formatNode(
               maybeParenthesizeBinopChild(node.op, node.rhs),
diff --git a/web/ui/mantine-ui/src/promql/serialize.ts b/web/ui/mantine-ui/src/promql/serialize.ts
index 584e1ae9ff..50c32c49e4 100644
--- a/web/ui/mantine-ui/src/promql/serialize.ts
+++ b/web/ui/mantine-ui/src/promql/serialize.ts
@@ -135,6 +135,7 @@ const serializeNode = (
     case nodeType.binaryExpr: {
       let matching = "";
       let grouping = "";
+      let fill = "";
       const vm = node.matching;
       if (vm !== null) {
         if (
@@ -152,11 +153,26 @@ const serializeNode = (
         ) {
           grouping = ` group_${vm.card === vectorMatchCardinality.manyToOne ? "left" : "right"}(${labelNameList(vm.include)})`;
         }
+
+        const lfill = vm.fillValues.lhs;
+        const rfill = vm.fillValues.rhs;
+        if (lfill !== null || rfill !== null) {
+          if (lfill === rfill) {
+            fill = ` fill(${lfill})`;
+          } else {
+            if (lfill !== null) {
+              fill += ` fill_left(${lfill})`;
+            }
+            if (rfill !== null) {
+              fill += ` fill_right(${rfill})`;
+            }
+          }
+        }
       }
 
       return `${serializeNode(maybeParenthesizeBinopChild(node.op, node.lhs), childIndent, pretty)}${childSeparator}${ind}${
         node.op
-      }${node.bool ? " bool" : ""}${matching}${grouping}${childSeparator}${serializeNode(
+      }${node.bool ? " bool" : ""}${matching}${grouping}${fill}${childSeparator}${serializeNode(
         maybeParenthesizeBinopChild(node.op, node.rhs),
         childIndent,
         pretty
diff --git a/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts b/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts
index a3734d311f..f9ff039882 100644
--- a/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts
+++ b/web/ui/mantine-ui/src/promql/serializeAndFormat.test.ts
@@ -658,6 +658,7 @@ describe("serializeNode and formatNode", () => {
             labels: [],
             on: false,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -677,6 +678,7 @@ describe("serializeNode and formatNode", () => {
             labels: [],
             on: true,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -696,6 +698,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: true,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -715,6 +718,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: false,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -735,6 +739,7 @@ describe("serializeNode and formatNode", () => {
             labels: [],
             on: false,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -755,6 +760,7 @@ describe("serializeNode and formatNode", () => {
             labels: [],
             on: false,
             include: ["__name__"],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -774,6 +780,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: true,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -793,6 +800,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: true,
             include: ["label3"],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -812,6 +820,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: true,
             include: [],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -831,6 +840,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: true,
             include: ["label3"],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: false,
         },
@@ -864,6 +874,7 @@ describe("serializeNode and formatNode", () => {
             labels: ["label1", "label2"],
             on: true,
             include: ["label3"],
+            fillValues: { lhs: null, rhs: null },
           },
           bool: true,
         },
@@ -911,6 +922,7 @@ describe("serializeNode and formatNode", () => {
               include: ["c", "ü"],
               labels: ["b", "ö"],
               on: true,
+              fillValues: { lhs: null, rhs: null },
             },
             op: binaryOperatorType.div,
             rhs: {
@@ -948,6 +960,7 @@ describe("serializeNode and formatNode", () => {
             include: [],
             labels: ["e", "ö"],
             on: false,
+            fillValues: { lhs: null, rhs: null },
           },
           op: binaryOperatorType.add,
           rhs: {
diff --git a/web/ui/module/codemirror-promql/src/complete/promql.terms.ts b/web/ui/module/codemirror-promql/src/complete/promql.terms.ts
index d356268d74..3670fffff7 100644
--- a/web/ui/module/codemirror-promql/src/complete/promql.terms.ts
+++ b/web/ui/module/codemirror-promql/src/complete/promql.terms.ts
@@ -39,6 +39,10 @@ export const binOpModifierTerms = [
   { label: 'ignoring', info: 'Ignore specified labels for matching', type: 'keyword' },
   { label: 'group_left', info: 'Allow many-to-one matching', type: 'keyword' },
   { label: 'group_right', info: 'Allow one-to-many matching', type: 'keyword' },
+  { label: 'bool', info: 'Return boolean result (0 or 1) instead of filtering', type: 'keyword' },
+  { label: 'fill', info: 'Fill in missing series on both sides', type: 'keyword' },
+  { label: 'fill_left', info: 'Fill in missing series on the left side', type: 'keyword' },
+  { label: 'fill_right', info: 'Fill in missing series on the right side', type: 'keyword' },
 ];
 
 export const atModifierTerms = [
diff --git a/web/ui/module/codemirror-promql/src/parser/vector.test.ts b/web/ui/module/codemirror-promql/src/parser/vector.test.ts
index f628206538..c6eeb930ab 100644
--- a/web/ui/module/codemirror-promql/src/parser/vector.test.ts
+++ b/web/ui/module/codemirror-promql/src/parser/vector.test.ts
@@ -15,29 +15,31 @@ import { buildVectorMatching } from './vector';
 import { createEditorState } from '../test/utils-test';
 import { BinaryExpr } from '@prometheus-io/lezer-promql';
 import { syntaxTree } from '@codemirror/language';
-import { VectorMatchCardinality } from '../types';
+import { VectorMatchCardinality, VectorMatching } from '../types';
+
+const noFill = { fill: { lhs: null, rhs: null } };
 
 describe('buildVectorMatching test', () => {
-  const testCases = [
+  const testCases: { binaryExpr: string; expectedVectorMatching: VectorMatching }[] = [
     {
       binaryExpr: 'foo * bar',
-      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [] },
+      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [], ...noFill },
     },
     {
       binaryExpr: 'foo * sum',
-      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [] },
+      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [], ...noFill },
     },
     {
       binaryExpr: 'foo == 1',
-      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [] },
+      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [], ...noFill },
     },
     {
       binaryExpr: 'foo == bool 1',
-      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [] },
+      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [], ...noFill },
     },
     {
       binaryExpr: '2.5 / bar',
-      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [] },
+      expectedVectorMatching: { card: VectorMatchCardinality.CardOneToOne, matchingLabels: [], on: false, include: [], ...noFill },
     },
     {
       binaryExpr: 'foo and bar',
@@ -46,6 +48,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -55,6 +58,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -64,6 +68,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -75,6 +80,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -86,6 +92,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -95,6 +102,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: true,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -104,6 +112,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: true,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -113,6 +122,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: true,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -122,6 +132,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: true,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -131,6 +142,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -140,6 +152,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: [],
         on: false,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -149,6 +162,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['bar'],
         on: true,
         include: [],
+        ...noFill,
       },
     },
     {
@@ -158,6 +172,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: true,
         include: ['bar'],
+        ...noFill,
       },
     },
     {
@@ -167,6 +182,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: false,
         include: ['blub'],
+        ...noFill,
       },
     },
     {
@@ -176,6 +192,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: false,
         include: ['bar'],
+        ...noFill,
       },
     },
     {
@@ -185,6 +202,7 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: true,
         include: ['bar', 'foo'],
+        ...noFill,
       },
     },
     {
@@ -194,6 +212,57 @@ describe('buildVectorMatching test', () => {
         matchingLabels: ['test', 'blub'],
         on: false,
         include: ['bar', 'foo'],
+        ...noFill,
+      },
+    },
+    {
+      binaryExpr: 'foo + fill(23) bar',
+      expectedVectorMatching: {
+        card: VectorMatchCardinality.CardOneToOne,
+        matchingLabels: [],
+        on: false,
+        include: [],
+        fill: { lhs: 23, rhs: 23 },
+      },
+    },
+    {
+      binaryExpr: 'foo + fill_left(23) bar',
+      expectedVectorMatching: {
+        card: VectorMatchCardinality.CardOneToOne,
+        matchingLabels: [],
+        on: false,
+        include: [],
+        fill: { lhs: 23, rhs: null },
+      },
+    },
+    {
+      binaryExpr: 'foo + fill_right(23) bar',
+      expectedVectorMatching: {
+        card: VectorMatchCardinality.CardOneToOne,
+        matchingLabels: [],
+        on: false,
+        include: [],
+        fill: { lhs: null, rhs: 23 },
+      },
+    },
+    {
+      binaryExpr: 'foo + fill_left(23) fill_right(42) bar',
+      expectedVectorMatching: {
+        card: VectorMatchCardinality.CardOneToOne,
+        matchingLabels: [],
+        on: false,
+        include: [],
+        fill: { lhs: 23, rhs: 42 },
+      },
+    },
+    {
+      binaryExpr: 'foo + fill_right(23) fill_left(42) bar',
+      expectedVectorMatching: {
+        card: VectorMatchCardinality.CardOneToOne,
+        matchingLabels: [],
+        on: false,
+        include: [],
+        fill: { lhs: 42, rhs: 23 },
       },
     },
   ];
@@ -203,7 +272,7 @@ describe('buildVectorMatching test', () => {
       const node = syntaxTree(state).topNode.getChild(BinaryExpr);
       expect(node).toBeTruthy();
       if (node) {
-        expect(value.expectedVectorMatching).toEqual(buildVectorMatching(state, node));
+        expect(buildVectorMatching(state, node)).toEqual(value.expectedVectorMatching);
       }
     });
   });
diff --git a/web/ui/module/codemirror-promql/src/parser/vector.ts b/web/ui/module/codemirror-promql/src/parser/vector.ts
index c47ca1fb76..9fc31bf5c6 100644
--- a/web/ui/module/codemirror-promql/src/parser/vector.ts
+++ b/web/ui/module/codemirror-promql/src/parser/vector.ts
@@ -24,6 +24,11 @@ import {
   On,
   Or,
   Unless,
+  NumberDurationLiteral,
+  FillModifier,
+  FillClause,
+  FillLeftClause,
+  FillRightClause,
 } from '@prometheus-io/lezer-promql';
 import { VectorMatchCardinality, VectorMatching } from '../types';
 import { containsAtLeastOneChild } from './path-finder';
@@ -37,6 +42,10 @@ export function buildVectorMatching(state: EditorState, binaryNode: SyntaxNode):
     matchingLabels: [],
     on: false,
     include: [],
+    fill: {
+      lhs: null,
+      rhs: null,
+    },
   };
   const modifierClause = binaryNode.getChild(MatchingModifierClause);
   if (modifierClause) {
@@ -60,6 +69,32 @@ export function buildVectorMatching(state: EditorState, binaryNode: SyntaxNode):
     }
   }
 
+  const fillModifier = binaryNode.getChild(FillModifier);
+  if (fillModifier) {
+    const fill = fillModifier.getChild(FillClause);
+    const fillLeft = fillModifier.getChild(FillLeftClause);
+    const fillRight = fillModifier.getChild(FillRightClause);
+
+    const getFillValue = (node: SyntaxNode) => {
+      const valueNode = node.getChild(NumberDurationLiteral);
+      return valueNode ? parseFloat(state.sliceDoc(valueNode.from, valueNode.to)) : null;
+    };
+
+    if (fill) {
+      const value = getFillValue(fill);
+      result.fill.lhs = value;
+      result.fill.rhs = value;
+    }
+
+    if (fillLeft) {
+      result.fill.lhs = getFillValue(fillLeft);
+    }
+
+    if (fillRight) {
+      result.fill.rhs = getFillValue(fillRight);
+    }
+  }
+
   const isSetOperator = containsAtLeastOneChild(binaryNode, And, Or, Unless);
   if (isSetOperator && result.card === VectorMatchCardinality.CardOneToOne) {
     result.card = VectorMatchCardinality.CardManyToMany;
diff --git a/web/ui/module/codemirror-promql/src/types/vector.ts b/web/ui/module/codemirror-promql/src/types/vector.ts
index 4e7a4f4c45..709b0b76d6 100644
--- a/web/ui/module/codemirror-promql/src/types/vector.ts
+++ b/web/ui/module/codemirror-promql/src/types/vector.ts
@@ -18,6 +18,11 @@ export enum VectorMatchCardinality {
   CardManyToMany = 'many-to-many',
 }
 
+export interface FillValues {
+  lhs: number | null;
+  rhs: number | null;
+}
+
 export interface VectorMatching {
   // The cardinality of the two Vectors.
   card: VectorMatchCardinality;
@@ -30,4 +35,6 @@ export interface VectorMatching {
   // Include contains additional labels that should be included in
   // the result from the side with the lower cardinality.
   include: string[];
+  // Fill contains optional fill values for missing elements.
+  fill: FillValues;
 }
diff --git a/web/ui/module/lezer-promql/src/promql.grammar b/web/ui/module/lezer-promql/src/promql.grammar
index 5fe8d4d025..9308ad01be 100644
--- a/web/ui/module/lezer-promql/src/promql.grammar
+++ b/web/ui/module/lezer-promql/src/promql.grammar
@@ -101,11 +101,30 @@ MatchingModifierClause {
   ((GroupLeft | GroupRight) (!group GroupingLabels)?)?
 }
 
+FillClause {
+  Fill "(" NumberDurationLiteral ")"
+}
+
+FillLeftClause {
+  FillLeft "(" NumberDurationLiteral ")"
+}
+
+FillRightClause {
+  FillRight "(" NumberDurationLiteral ")"
+}
+
+FillModifier {
+  (FillClause | FillLeftClause | FillRightClause) |
+  (FillLeftClause FillRightClause) |
+  (FillRightClause FillLeftClause)
+}
+
 BoolModifier { Bool }
 
 binModifiers {
   BoolModifier?
   MatchingModifierClause?
+  FillModifier?
 }
 
 GroupingLabels {
@@ -366,7 +385,10 @@ NumberDurationLiteralInDurationContext {
   Start,
   End,
   Smoothed,
-  Anchored
+  Anchored,
+  Fill,
+  FillLeft,
+  FillRight
 }
 
 @external propSource promQLHighLight from "./highlight"
diff --git a/web/ui/module/lezer-promql/src/tokens.js b/web/ui/module/lezer-promql/src/tokens.js
index 523c306ae9..6fd681f1f8 100644
--- a/web/ui/module/lezer-promql/src/tokens.js
+++ b/web/ui/module/lezer-promql/src/tokens.js
@@ -12,82 +12,88 @@
 // limitations under the License.
 
 import {
-    And,
-    Avg,
-    Atan2,
-    Bool,
-    Bottomk,
-    By,
-    Count,
-    CountValues,
-    End,
-    Group,
-    GroupLeft,
-    GroupRight,
-    Ignoring,
-    inf,
-    Max,
-    Min,
-    nan,
-    Offset,
-    On,
-    Or,
-    Quantile,
-    LimitK,
-    LimitRatio,
-    Start,
-    Stddev,
-    Stdvar,
-    Sum,
-    Topk,
-    Unless,
-    Without,
-    Smoothed,
-    Anchored,
-} from './parser.terms.js';
+  And,
+  Avg,
+  Atan2,
+  Bool,
+  Bottomk,
+  By,
+  Count,
+  CountValues,
+  End,
+  Group,
+  GroupLeft,
+  GroupRight,
+  Ignoring,
+  inf,
+  Max,
+  Min,
+  nan,
+  Offset,
+  On,
+  Or,
+  Quantile,
+  LimitK,
+  LimitRatio,
+  Start,
+  Stddev,
+  Stdvar,
+  Sum,
+  Topk,
+  Unless,
+  Without,
+  Smoothed,
+  Anchored,
+  Fill,
+  FillLeft,
+  FillRight,
+} from "./parser.terms.js";
 
 const keywordTokens = {
-    inf: inf,
-    nan: nan,
-    bool: Bool,
-    ignoring: Ignoring,
-    on: On,
-    group_left: GroupLeft,
-    group_right: GroupRight,
-    offset: Offset,
+  inf: inf,
+  nan: nan,
+  bool: Bool,
+  ignoring: Ignoring,
+  on: On,
+  group_left: GroupLeft,
+  group_right: GroupRight,
+  offset: Offset,
 };
 
 export const specializeIdentifier = (value, stack) => {
-    return keywordTokens[value.toLowerCase()] || -1;
+  return keywordTokens[value.toLowerCase()] || -1;
 };
 
 const contextualKeywordTokens = {
-    avg: Avg,
-    atan2: Atan2,
-    bottomk: Bottomk,
-    count: Count,
-    count_values: CountValues,
-    group: Group,
-    max: Max,
-    min: Min,
-    quantile: Quantile,
-    limitk: LimitK,
-    limit_ratio: LimitRatio,
-    stddev: Stddev,
-    stdvar: Stdvar,
-    sum: Sum,
-    topk: Topk,
-    by: By,
-    without: Without,
-    and: And,
-    or: Or,
-    unless: Unless,
-    start: Start,
-    end: End,
-    smoothed: Smoothed,
-    anchored: Anchored,
+  avg: Avg,
+  atan2: Atan2,
+  bottomk: Bottomk,
+  count: Count,
+  count_values: CountValues,
+  group: Group,
+  max: Max,
+  min: Min,
+  quantile: Quantile,
+  limitk: LimitK,
+  limit_ratio: LimitRatio,
+  stddev: Stddev,
+  stdvar: Stdvar,
+  sum: Sum,
+  topk: Topk,
+  by: By,
+  without: Without,
+  and: And,
+  or: Or,
+  unless: Unless,
+  start: Start,
+  end: End,
+  smoothed: Smoothed,
+  anchored: Anchored,
+  fill: Fill,
+  fill_left: FillLeft,
+  fill_right: FillRight,
 };
 
 export const extendIdentifier = (value, stack) => {
-    return contextualKeywordTokens[value.toLowerCase()] || -1;
+  return contextualKeywordTokens[value.toLowerCase()] || -1;
 };

From 57dd1f18b4686647af93f83beead2d5fa4f2345e Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Wed, 10 Dec 2025 19:25:08 +0100
Subject: [PATCH 269/439] Add fill modifier PromQL tests

Signed-off-by: Julius Volz 
---
 promql/promqltest/testdata/fill-modifier.test | 343 ++++++++++++++++++
 1 file changed, 343 insertions(+)
 create mode 100644 promql/promqltest/testdata/fill-modifier.test

diff --git a/promql/promqltest/testdata/fill-modifier.test b/promql/promqltest/testdata/fill-modifier.test
new file mode 100644
index 0000000000..08c4396242
--- /dev/null
+++ b/promql/promqltest/testdata/fill-modifier.test
@@ -0,0 +1,343 @@
+# ==================== fill / fill_left / fill_right modifier tests ====================
+
+# Test data for fill modifier tests: vectors with partial overlap.
+load 5m
+  left_vector{label="a"} 10
+  left_vector{label="b"} 20
+  left_vector{label="c"} 30
+  right_vector{label="a"} 100
+  right_vector{label="b"} 200
+  right_vector{label="d"} 400
+
+# ---------- Arithmetic operators with fill modifiers ----------
+
+# fill(0): Fill both sides with 0 for addition.
+eval instant at 0m left_vector + fill(0) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="c"} 30
+  {label="d"} 400
+
+# fill_left(0): Only fill left side with 0.
+eval instant at 0m left_vector + fill_left(0) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="d"} 400
+
+# fill_right(0): Only fill right side with 0.
+eval instant at 0m left_vector + fill_right(0) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="c"} 30
+
+# fill_left and fill_right with different values.
+eval instant at 0m left_vector + fill_left(5) fill_right(7) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="c"} 37
+  {label="d"} 405
+
+# fill with NaN.
+eval instant at 0m left_vector + fill(NaN) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="c"} NaN
+  {label="d"} NaN
+
+# fill with Inf.
+eval instant at 0m left_vector + fill(Inf) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="c"} +Inf
+  {label="d"} +Inf
+
+# fill with -Inf.
+eval instant at 0m left_vector + fill(-Inf) right_vector
+  {label="a"} 110
+  {label="b"} 220
+  {label="c"} -Inf
+  {label="d"} -Inf
+
+# ---------- Comparison operators with fill modifiers ----------
+
+# fill with equality comparison.
+eval instant at 0m left_vector == fill(30) right_vector
+  left_vector{label="c"} 30
+
+# fill with inequality comparison.
+eval instant at 0m left_vector != fill(30) right_vector
+  left_vector{label="a"} 10
+  left_vector{label="b"} 20
+  {label="d"} 30
+
+# fill with greater than.
+eval instant at 0m left_vector > fill(25) right_vector
+  left_vector{label="c"} 30
+
+# ---------- Comparison operators with bool modifier and fill ----------
+
+# fill with equality comparison and bool.
+eval instant at 0m left_vector == bool fill(30) right_vector
+  {label="a"} 0
+  {label="b"} 0
+  {label="c"} 1
+  {label="d"} 0
+
+# fill with inequality comparison and bool.
+eval instant at 0m left_vector != bool fill(30) right_vector
+  {label="a"} 1
+  {label="b"} 1
+  {label="c"} 0
+  {label="d"} 1
+
+# fill with greater than and bool.
+eval instant at 0m left_vector > bool fill(25) right_vector
+  {label="a"} 0
+  {label="b"} 0
+  {label="c"} 1
+  {label="d"} 0
+
+# ---------- fill with on() and ignoring() modifiers ----------
+
+clear
+
+load 5m
+  left_vector{job="foo", instance="a"} 10
+  left_vector{job="foo", instance="b"} 20
+  left_vector{job="bar", instance="a"} 30
+  right_vector{job="foo", instance="a"} 100
+  right_vector{job="foo", instance="c"} 300
+
+# fill with on().
+eval instant at 0m left_vector + on(job, instance) fill(0) right_vector
+  {job="foo", instance="a"} 110
+  {job="foo", instance="b"} 20
+  {job="bar", instance="a"} 30
+  {job="foo", instance="c"} 300
+
+# fill_right with on().
+eval instant at 0m left_vector + on(job, instance) fill_right(0) right_vector
+  {job="foo", instance="a"} 110
+  {job="foo", instance="b"} 20
+  {job="bar", instance="a"} 30
+
+# fill_left with on().
+eval instant at 0m left_vector + on(job, instance) fill_left(0) right_vector
+  {job="foo", instance="a"} 110
+  {job="foo", instance="c"} 300
+
+# fill with ignoring() - requires group_left since ignoring(job) creates many-to-one matching
+# when two left_vector series have same instance but different jobs.
+eval instant at 0m left_vector + ignoring(job) group_left fill(0) right_vector
+  {instance="a", job="foo"} 110
+  {instance="a", job="bar"} 130
+  {instance="b", job="foo"} 20
+  {instance="c"} 300
+
+# ---------- fill with group_left / group_right (many-to-one / one-to-many) ----------
+
+clear
+
+load 5m
+  requests{method="GET", status="200"} 100
+  requests{method="POST", status="200"} 200
+  requests{method="GET", status="500"} 10
+  requests{method="POST", status="500"} 20
+  limits{status="200"} 1000
+  limits{status="404"} 500
+  limits{status="500"} 50
+
+# group_left with fill_right: fill missing "one" side series.
+eval instant at 0m requests / on(status) group_left fill_right(1) limits
+  {method="GET", status="200"} 0.1
+  {method="POST", status="200"} 0.2
+  {method="GET", status="500"} 0.2
+  {method="POST", status="500"} 0.4
+
+# group_left with fill_left: fill missing "many" side series.
+# For status="404", there's no matching requests, so a single series with the match group's labels is filled
+eval instant at 0m requests + on(status) group_left fill_left(0) limits
+  {method="GET", status="200"} 1100
+  {method="POST", status="200"} 1200
+  {method="GET", status="500"} 60
+  {method="POST", status="500"} 70
+  {status="404"} 500
+
+# group_left with fill on both sides.
+eval instant at 0m requests + on(status) group_left fill(0) limits
+  {method="GET", status="200"} 1100
+  {method="POST", status="200"} 1200
+  {method="GET", status="500"} 60
+  {method="POST", status="500"} 70
+  {status="404"} 500
+
+# group_right with fill_left: fill missing "one" side series.
+clear
+
+load 5m
+  cpu_info{instance="a", cpu="0"} 1
+  cpu_info{instance="a", cpu="1"} 1
+  cpu_info{instance="b", cpu="0"} 1
+  node_meta{instance="a"} 100
+  node_meta{instance="c"} 300
+
+# fill_left fills the "one" side (node_meta) when missing for a "many" side series.
+eval instant at 0m node_meta * on(instance) group_right fill_left(1) cpu_info
+  {instance="a", cpu="0"} 100
+  {instance="a", cpu="1"} 100
+  {instance="c"} 300
+
+# group_right with fill_right: fill missing "many" side series.
+eval instant at 0m node_meta * on(instance) group_right fill_right(0) cpu_info
+  {instance="a", cpu="0"} 100
+  {instance="a", cpu="1"} 100
+  {instance="b", cpu="0"} 0
+
+# group_right with fill on both sides.
+eval instant at 0m node_meta * on(instance) group_right fill(1) cpu_info
+  {instance="a", cpu="0"} 100
+  {instance="a", cpu="1"} 100
+  {instance="b", cpu="0"} 1
+  {instance="c"} 300
+
+# ---------- fill with group_left/group_right and extra labels ----------
+
+clear
+
+load 5m
+  requests{method="GET", status="200"} 100
+  requests{method="POST", status="200"} 200
+  limits{status="200", owner="team-a"} 1000
+  limits{status="500", owner="team-b"} 50
+
+# group_left with extra label and fill_right.
+# Note: when filling the "one" side, the joined label cannot be filled.
+eval instant at 0m requests + on(status) group_left(owner) fill_right(0) limits
+  {method="GET", status="200", owner="team-a"} 1100
+  {method="POST", status="200", owner="team-a"} 1200
+
+# ---------- Edge cases ----------
+
+clear
+
+load 5m
+  only_left{label="a"} 10
+  only_left{label="b"} 20
+  only_right{label="c"} 30
+  only_right{label="d"} 40
+
+# No overlap at all - fill creates all results.
+eval instant at 0m only_left + fill(0) only_right
+  {label="a"} 10
+  {label="b"} 20
+  {label="c"} 30
+  {label="d"} 40
+
+# No overlap - fill_left only creates right side results.
+eval instant at 0m only_left + fill_left(0) only_right
+  {label="c"} 30
+  {label="d"} 40
+
+# No overlap - fill_right only creates left side results.
+eval instant at 0m only_left + fill_right(0) only_right
+  {label="a"} 10
+  {label="b"} 20
+
+# Complete overlap - fill has no effect.
+clear
+
+load 5m
+  complete_left{label="a"} 10
+  complete_left{label="b"} 20
+  complete_right{label="a"} 100
+  complete_right{label="b"} 200
+
+eval instant at 0m complete_left + fill(99) complete_right
+  {label="a"} 110
+  {label="b"} 220
+
+# ---------- fill with range queries ----------
+
+clear
+
+load 5m
+  range_left{label="a"} 1 2 3 4 5
+  range_left{label="b"} 10 20 30 40 50
+  range_right{label="a"} 100 200 300 400 500
+  range_right{label="c"} 1000 2000 3000 4000 5000
+
+eval range from 0 to 20m step 5m range_left + fill(0) range_right
+  {label="a"} 101 202 303 404 505
+  {label="b"} 10 20 30 40 50
+  {label="c"} 1000 2000 3000 4000 5000
+
+eval range from 0 to 20m step 5m range_left + fill_right(0) range_right
+  {label="a"} 101 202 303 404 505
+  {label="b"} 10 20 30 40 50
+
+eval range from 0 to 20m step 5m range_left + fill_left(0) range_right
+  {label="a"} 101 202 303 404 505
+  {label="c"} 1000 2000 3000 4000 5000
+
+# Range queries with intermittently present series.
+clear
+
+load 5m
+  intermittent_left{label="a"}  1  _  3  _  5
+  intermittent_left{label="b"}  _  20 _  40 _
+  intermittent_right{label="a"} _  200 _ 400 _
+  intermittent_right{label="b"} 100 _ 300 _ 500
+  intermittent_right{label="c"} 1000 _ _ 4000 5000
+
+# When both sides have the same label but are present at different times,
+# fill creates results at all timestamps where at least one side is present.
+eval range from 0 to 20m step 5m intermittent_left + fill(0) intermittent_right
+  {label="a"} 1 200 3 400 5
+  {label="b"} 100 20 300 40 500
+  {label="c"} 1000 _ _ 4000 5000
+
+# fill_right only fills the right side when it's missing.
+# Output only exists when left side is present (right side filled with 0 if missing).
+eval range from 0 to 20m step 5m intermittent_left + fill_right(0) intermittent_right
+  {label="a"} 1 _ 3 _ 5
+  {label="b"} _ 20 _ 40 _
+
+# fill_left only fills the left side when it's missing.
+# Output only exists when right side is present (left side filled with 0 if missing).
+eval range from 0 to 20m step 5m intermittent_left + fill_left(0) intermittent_right
+  {label="a"} _ 200 _ 400 _
+  {label="b"} 100 _ 300 _ 500
+  {label="c"} 1000 _ _ 4000 5000
+
+# ---------- fill with vectors where one side is empty ----------
+
+clear
+
+load 5m
+  non_empty{label="a"} 10
+  non_empty{label="b"} 20
+
+# Empty right side - fill_right has no effect (nothing to add).
+eval instant at 0m non_empty + fill_right(0) nonexistent
+  {label="a"} 10
+  {label="b"} 20
+
+# Empty right side - fill_left creates nothing (no right side labels to use).
+eval instant at 0m non_empty + fill_left(0) nonexistent
+
+# Empty left side - fill_left has no effect.
+eval instant at 0m nonexistent + fill_left(0) non_empty
+  {label="a"} 10
+  {label="b"} 20
+
+# Empty left side - fill_right creates nothing.
+eval instant at 0m nonexistent + fill_right(0) non_empty
+
+# fill both sides with one side empty.
+eval instant at 0m non_empty + fill(0) nonexistent
+  {label="a"} 10
+  {label="b"} 20
+
+eval instant at 0m nonexistent + fill(0) non_empty
+  {label="a"} 10
+  {label="b"} 20

From ce26370eeb2e19bdfac68c40a1f21913a046fddd Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Wed, 10 Dec 2025 20:07:43 +0100
Subject: [PATCH 270/439] Add PromLens binop matching explain view tests

Signed-off-by: Julius Volz 
---
 web/ui/mantine-ui/src/promql/binOp.test.ts | 431 +++++++++++++++++++++
 web/ui/mantine-ui/src/promql/binOp.ts      |   4 +-
 2 files changed, 433 insertions(+), 2 deletions(-)

diff --git a/web/ui/mantine-ui/src/promql/binOp.test.ts b/web/ui/mantine-ui/src/promql/binOp.test.ts
index 9c5d59a94c..76dd24fa79 100644
--- a/web/ui/mantine-ui/src/promql/binOp.test.ts
+++ b/web/ui/mantine-ui/src/promql/binOp.test.ts
@@ -2163,6 +2163,437 @@ const testCases: TestCase[] = [
       numGroups: 2,
     },
   },
+  {
+    // metric_a - fill(0) metric_b
+    desc: "subtraction with fill(0) but no missing series",
+    op: binaryOperatorType.sub,
+    matching: {
+      card: vectorMatchCardinality.oneToOne,
+      on: false,
+      include: [],
+      labels: [],
+      fillValues: { lhs: 0, rhs: 0 },
+    },
+    lhs: testMetricA,
+    rhs: testMetricB,
+    result: {
+      groups: {
+        [fnv1a(["a", "x", "same"])]: {
+          groupLabels: { label1: "a", label2: "x", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "a",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "1"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "a",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "10"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "a", label2: "x", same: "same" },
+                value: [0, "-9"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["a", "y", "same"])]: {
+          groupLabels: { label1: "a", label2: "y", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "a",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "2"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "a",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "20"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "a", label2: "y", same: "same" },
+                value: [0, "-18"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["b", "x", "same"])]: {
+          groupLabels: { label1: "b", label2: "x", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "b",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "3"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "b",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "30"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "b", label2: "x", same: "same" },
+                value: [0, "-27"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["b", "y", "same"])]: {
+          groupLabels: { label1: "b", label2: "y", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "b",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "4"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "b",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "40"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "b", label2: "y", same: "same" },
+                value: [0, "-36"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+      },
+      numGroups: 4,
+    },
+  },
+  {
+    // metric_a[0..2] - fill_left(23) fill_right(42) metric_b[1...3]
+    desc: "subtraction with different fill values and missing series on each side",
+    op: binaryOperatorType.sub,
+    matching: {
+      card: vectorMatchCardinality.oneToOne,
+      on: false,
+      include: [],
+      labels: [],
+      fillValues: { lhs: 23, rhs: 42 },
+    },
+    lhs: testMetricA.slice(0, 3),
+    rhs: testMetricB.slice(1, 4),
+    result: {
+      groups: {
+        [fnv1a(["a", "x", "same"])]: {
+          groupLabels: { label1: "a", label2: "x", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "a",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "1"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                label1: "a",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "42"],
+              filled: true,
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "a", label2: "x", same: "same" },
+                value: [0, "-41"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["a", "y", "same"])]: {
+          groupLabels: { label1: "a", label2: "y", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "a",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "2"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "a",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "20"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "a", label2: "y", same: "same" },
+                value: [0, "-18"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["b", "x", "same"])]: {
+          groupLabels: { label1: "b", label2: "x", same: "same" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_a",
+                label1: "b",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "3"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "b",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "30"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "b", label2: "x", same: "same" },
+                value: [0, "-27"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["b", "y", "same"])]: {
+          groupLabels: { label1: "b", label2: "y", same: "same" },
+          lhs: [
+            {
+              metric: {
+                label1: "b",
+                label2: "y",
+                same: "same",
+              },
+              filled: true,
+              value: [0, "23"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "b",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "40"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "b", label2: "y", same: "same" },
+                value: [0, "-17"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+      },
+      numGroups: 4,
+    },
+  },
+  {
+    // metric_b[0...1] - on(label1) group_left fill(0) metric_c
+    desc: "many-to-one matching with matching labels specified, group_left, and fill specified",
+    op: binaryOperatorType.sub,
+    matching: {
+      card: vectorMatchCardinality.manyToOne,
+      on: true,
+      include: [],
+      labels: ["label1"],
+      fillValues: { lhs: 0, rhs: 0 },
+    },
+    lhs: testMetricB.slice(0, 2),
+    rhs: testMetricC,
+    result: {
+      groups: {
+        [fnv1a(["a"])]: {
+          groupLabels: { label1: "a" },
+          lhs: [
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "a",
+                label2: "x",
+                same: "same",
+              },
+              value: [0, "10"],
+            },
+            {
+              metric: {
+                __name__: "metric_b",
+                label1: "a",
+                label2: "y",
+                same: "same",
+              },
+              value: [0, "20"],
+            },
+          ],
+          lhsCount: 2,
+          rhs: [
+            {
+              metric: { __name__: "metric_c", label1: "a" },
+              value: [0, "100"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "a", label2: "x", same: "same" },
+                value: [0, "-90"],
+              },
+              manySideIdx: 0,
+            },
+            {
+              sample: {
+                metric: { label1: "a", label2: "y", same: "same" },
+                value: [0, "-80"],
+              },
+              manySideIdx: 1,
+            },
+          ],
+          error: null,
+        },
+        [fnv1a(["b"])]: {
+          groupLabels: { label1: "b" },
+          lhs: [
+            {
+              metric: {
+                label1: "b",
+              },
+              filled: true,
+              value: [0, "0"],
+            },
+          ],
+          lhsCount: 1,
+          rhs: [
+            {
+              metric: { __name__: "metric_c", label1: "b" },
+              value: [0, "200"],
+            },
+          ],
+          rhsCount: 1,
+          result: [
+            {
+              sample: {
+                metric: { label1: "b" },
+                value: [0, "-200"],
+              },
+              manySideIdx: 0,
+            },
+          ],
+          error: null,
+        },
+      },
+      numGroups: 2,
+    },
+  },
   {
     // metric_a and metric b
     desc: "and operator with no matching labels and matching groups",
diff --git a/web/ui/mantine-ui/src/promql/binOp.ts b/web/ui/mantine-ui/src/promql/binOp.ts
index f583bf81bb..9ebee90f64 100644
--- a/web/ui/mantine-ui/src/promql/binOp.ts
+++ b/web/ui/mantine-ui/src/promql/binOp.ts
@@ -347,7 +347,7 @@ export const computeVectorVectorBinOp = (
   Object.values(groups).forEach((mg) => {
     if (mg.lhs.length === 0 && matching.fillValues.lhs !== null) {
       mg.lhs.push({
-        metric: {},
+        metric: mg.groupLabels,
         value: [0, formatPrometheusFloat(matching.fillValues.lhs as number)],
         filled: true,
       });
@@ -355,7 +355,7 @@ export const computeVectorVectorBinOp = (
     }
     if (mg.rhs.length === 0 && matching.fillValues.rhs !== null) {
       mg.rhs.push({
-        metric: {},
+        metric: mg.groupLabels,
         value: [0, formatPrometheusFloat(matching.fillValues.rhs as number)],
         filled: true,
       });

From 4c9795221073defa19d662c366c0ff1f0fca0e97 Mon Sep 17 00:00:00 2001
From: Julius Volz 
Date: Thu, 11 Dec 2025 12:29:48 +0100
Subject: [PATCH 271/439] Document new fill binop modifiers

Signed-off-by: Julius Volz 
---
 docs/querying/operators.md | 117 ++++++++++++++++++++++++++-----------
 1 file changed, 82 insertions(+), 35 deletions(-)

diff --git a/docs/querying/operators.md b/docs/querying/operators.md
index b320d8e86e..c5b01aff71 100644
--- a/docs/querying/operators.md
+++ b/docs/querying/operators.md
@@ -47,9 +47,9 @@ special values like `NaN`, `+Inf`, and `-Inf`.
 scalar that is the result of the operator applied to both scalar operands.
 
 **Between an instant vector and a scalar**, the operator is applied to the
-value of every data sample in the vector. 
+value of every data sample in the vector.
 
-If the data sample is a float, the operation is performed between that float and the scalar. 
+If the data sample is a float, the operation is performed between that float and the scalar.
 For example, if an instant vector of float samples is multiplied by 2,
 the result is another vector of float samples in which every sample value of
 the original vector is multiplied by 2.
@@ -81,8 +81,9 @@ following:
 **Between two instant vectors**, a binary arithmetic operator is applied to
 each entry in the LHS vector and its [matching element](#vector-matching) in
 the RHS vector. The result is propagated into the result vector with the
-grouping labels becoming the output label set. Entries for which no matching
-entry in the right-hand vector can be found are not part of the result.
+grouping labels becoming the output label set. By default, series for which
+no matching entry in the opposite vector can be found are not part of the
+result. This behavior can be adjusted using [fill modifiers](#filling-in-missing-matches).
 
 If two float samples are matched, the arithmetic operator is applied to the two
 input values.
@@ -97,7 +98,7 @@ If two histogram samples are matched, only `+` and `-` are valid operations,
 each adding or subtracting all matching bucket populations and the count and
 the sum of observations. All other operations result in the removal of the
 corresponding element from the output vector, flagged by an info-level
-annotation. The `+` and -` operations should generally only be applied to gauge
+annotation. The `+` and `-` operations should generally only be applied to gauge
 histograms, but PromQL allows them for counter histograms, too, to cover
 specific use cases, for which special attention is required to avoid problems
 with unaligned counter resets. (Certain incompatibilities of counter resets can
@@ -106,7 +107,7 @@ two counter histograms results in a counter histogram. All other combination of
 operands and all subtractions result in a gauge histogram.
 
 **In any arithmetic binary operation involving vectors**, the metric name is
-dropped. This occurs even if `__name__` is explicitly mentioned in `on` 
+dropped. This occurs even if `__name__` is explicitly mentioned in `on`
 (see https://github.com/prometheus/prometheus/issues/16631 for further discussion).
 
 **For any arithmetic binary operation that may result in a negative
@@ -156,9 +157,9 @@ info-level annotation.
 applied to matching entries. Vector elements for which the expression is not
 true or which do not find a match on the other side of the expression get
 dropped from the result, while the others are propagated into a result vector
-with the grouping labels becoming the output label set. 
+with the grouping labels becoming the output label set.
 
-Matches between two float samples work as usual. 
+Matches between two float samples work as usual.
 
 Matches between a float sample and a histogram sample are invalid, and the
 corresponding element is removed from the result vector, flagged by an info-level
@@ -171,8 +172,8 @@ comparison binary operations are again invalid.
 modifier changes the behavior in the following ways:
 
 * Vector elements which find a match on the other side of the expression but for
-  which the expression is false instead have the value `0` and vector elements
-  that do find a match and for which the expression is true have the value `1`. 
+  which the expression is false instead have the value `0`, and vector elements
+  that do find a match and for which the expression is true have the value `1`.
   (Note that elements with no match or invalid operations involving histogram
   samples still return no result rather than the value `0`.)
 * The metric name is dropped.
@@ -216,11 +217,10 @@ matching behavior: One-to-one and many-to-one/one-to-many.
 
 ### Vector matching keywords
 
-These vector matching keywords allow for matching between series with different label sets
-providing:
+These vector matching keywords allow for matching between series with different label sets:
 
-* `on`
-* `ignoring`
+* `on(