Move the lua httpclient code from hlua.c to http_client.c
The code is almost the same but the registering of the class which is
done in hlua_http_client_init_state(), from REGISTER_HLUA_STATE_INIT()
check_args() calls have been replaced by hlua_check_args().
hlua_httpclient_destroy_all() is exported so it can be called in hlua.c.
hlua_httpclient_table_to_hdrs() is made static.
hlua_pusherror() and check_args() are being exported.
check_args() is now a macro to hlua_check_args() so it's not confusing
when called outside hlua.c.
When a dynamic server is added with consistent hash balancing on the
backend, its lb_nodes elements are allocated and associated with a
calculated server key. This operation is performed in add server handler
via srv_alloc_lb(). By default, the server key is based on its ID.
However, automatic server ID is calculated later in add server handler,
which means the initial lb_nodes are not valid.
This could cause load balacing issue but in fact this is not directly
visible as the server key is recalculated when the dynamic server is
enabled via chash_queue_dequeue_srv() : all server lb_nodes are dequeued
and requeued with the now proper key.
Thus, "add server" handler must be corrected as it is buggy when
considering it alone. The simplest solution of the current patch is to
initialize server ID before srv_alloc_lb() is invoked. There is no issue
as handler runs under thread isolation so there is no risk of multiple
servers manipulating the same ID. Server insertion in proxy ID tree is
still performed at the end of the handler when all fallible operation
are completed.
The fact that server key is recalculated when the server is set to ready
state is a side effect of the following patch which was introduced in
3.0. What this means though is that users of older releases are facing a
bigger issue, with load-balancing not working as expected. Thus,
this patch is even more crucial for 2.8 and older releases.
faa8c3e024
MEDIUM: lb-chash: Deterministic node hashes based on server address
This should fix github issue #3413. Thanks to Joao Morais for is
analysis on the problem.
This must be backported to all stable releases.
When we receive a PUSH_PROMISE frame while we don't expect it, flag it
as a connection error, do not just set ret to H3_ERR_ID_ERROR, as it
would just be considered the number of bytes we read, and could lead to
random corruption. This should only happen with backend connections.
This should be backported whenever commit 4a8bb2fe5 is backported.
Now that there is no longer a shared wake queue, chances are if a shared task
is scheduled, it will always end up on the same thread. In
wake_expired_tasks(), when a task has to be waken up, randomly look to
three other threads, and if the runqueue of the current thread is at least
two time bigger than the runqueue of one of the other threads, then give
that task to that thread, so that our load gets reduced.
If we're giving the task to another thread, then we have to add the
TASK_RUNNING flag until we waked it up, otherwise the other thread could
just run it, if it gets waken up from another path, and free it while
we're still not done with it.
2 times has been chosen somewhat arbitrarily, and may be tweaked at a
later date if deemed not optimal.
Modify task_instant_wakeup() to use __task_set_state_and_tid().
It uses the new ownership behavior, but that's okay because
task_instant_wakeup() was not used anywhere.
Totally remove the per-thread group wait queue. This was potentially a
source of contention, because there were only a global lock for all
those wait queues.
Instead, for shared tasks, there is now the concept of ownership for the
task. When a task is in the wait queue, run queue, or is running on that
particular thread, the task's tid is set to -2 - thread_tid, and only
that thread will be responsible for it until it is no longer running,
and in none of its queue.
When a shared task is scheduled to be run at a later time, if its
current tid is -1, then the current thread will take ownership, and put
it in its own wait queue. If it is already owned, then TASK_WOKEN_WQ is
added to the task's state, and a task_wakeup() is done, so that the
owner thread will add it in its wait queue.
If there is any owner, then a task_wakeup() will just add the task to
the owner's runqueue, otherwise the current thread will become the
owner.
Introduce a new function, __task_get_current_owner, that returns the
owner of a task based on its current tid.
-1 means there is no current owner, otherwise either the tid is >= 0, in
which case it will just return it, or it's < -1, in which case it will
return -2 - tid, the tid of the thread with the current ownership.
Introduce __task_get_new_tid_field(), that provides the tid to be used
for a task.
For shared task, to mark temporary ownership of a task, instead of -1,
the tid will be set to -2-tid, tid being the tid of the current thread.
Introduce a new function, __task_set_state_and_tid, that atomically can
set a task's state and its tid. This will be used later, as the tid will
be used to indicate task ownership even for shared tasks.
This Lua script automates dns-01 ACME challenges using the Gandi LiveDNS
API v5. It subscribes to the ACME_DEPLOY event to set the required
_acme-challenge TXT record via the Gandi REST API, signals HAProxy that
the challenge is ready using ACME.challenge_ready(), then cleans up the
TXT record once the certificate is issued on ACME_NEWCERT.
The API key is read from the GANDI_API_KEY environment variable at
startup. Zone discovery is automatic: the script probes parent zones
from longest to shortest until Gandi accepts the record, which handles
both apex and wildcard certificates transparently.
Add EVENT_HDL_SUB_ACME_DEPLOY to the ACME family. It is published in
the dns-01 challenge path after the TXT record information has been
prepared, carrying the certificate store name, domain, account
thumbprint, dns_record value, and optionally the provider and vars
strings.
Lua subscribers using core.event_sub() receive the event data as an
AcmeEvent object, which is the same class used for ACME_NEWCERT and
carries the fields relevant to the event type.
Add a new EVENT_HDL_SUB_ACME_NEWCERT event type in the ACME family.
It is published after a new certificate has been successfully fetched
and installed. The event carries the certificate store name, allowing
subscribers to act on newly available certificates.
Lua subscribers using core.event_sub() receive the event data as an
AcmeEvent object with a crtname field containing the certificate store
name.
Using ha_diag_warning() to report the number of threads created resulted
in warnings being counted and possibly an error being fired when combined
with -dW:
$ printf "global\nstats socket /tmp/sock1\n" | ./haproxy -dD -dW -c -f /dev/stdin; echo $?
[NOTICE] (10406) : haproxy version is 3.5-dev0-5091ac-35
[NOTICE] (10406) : path to executable is ./haproxy
[DIAG] (10406) : Created 20 threads split into 2 groups
[ALERT] (10406) : Some warnings were found and 'zero-warning' is set. Aborting.
1
Now that we have ha_diag_notice(), let's use it:
$ printf "global\nstats socket /tmp/sock1\n" | ./haproxy -dD -dW -c -f /dev/stdin; echo $?
[DIAG] (10513) : Created 20 threads split into 2 groups
0
It would make sense to backport this to 3.2 because it helps validate configs
against diag warnings without triggering a false positive. It depends on
this previous patch:
MINOR: errors: add ha_diag_notice() to report diag-level notifications
Right now the only way to report info that is only displayed in diag
mode with -dD is to use ha_diag_warning(). The problem is that this is
then counted as a warning and may result in errors when combined with
-dW, as happens for the CPU topology info:
$ printf "global\nstats socket /tmp/sock1\n" | ./haproxy -dD -dW -c -f /dev/stdin; echo $?
[NOTICE] (10406) : haproxy version is 3.5-dev0-5091ac-35
[NOTICE] (10406) : path to executable is ./haproxy
[DIAG] (10406) : Created 20 threads split into 2 groups
[ALERT] (10406) : Some warnings were found and 'zero-warning' is set. Aborting.
1
We need another level. This commit introduces ha_diag_notice() which only
emits a notification that doesn't count as a warning. Note that we could
even introduce an info level and revisit various messages so that notice
only reports certain events while info is for anything (like versions
above). That could be a future improvement.
The Linux tls module requires a socket to be in TCP_ESTABLISHED state
before we can enable the TLS ULP on the socket, if the socket is in any
other state, then the setsockopt() call will fail, and we won't use
kTLS on that socket.
To make sure we're not doing it too early, defer it until the TLS
handshake is done, which means the TCP connection is established.
This should be backported up to 3.3.
Signed-off-by: Karol Kucharski <kkucharski@fastlogic.pl>
Add a new ACME global Lua table with a challenge_ready(crt, dns) method
that wraps acme_challenge_ready(). It marks the ACME challenge for domain
<dns> in certificate <crt> as ready and returns the number of remaining
challenges, or 0 when all challenges are ready and validation has been
triggered. A Lua error is raised if the certificate or domain is not found.
The ACME table is registered for each lua_State via the new
REGISTER_HLUA_STATE_INIT() mechanism.
__LJMP, WILL_LJMP() and MAY_LJMP() were defined locally in hlua.c,
making them unavailable to other modules that implement Lua bindings.
Move them to include/haproxy/hlua.h so they can be used outside of
hlua.c.
Add a registration mechanism so that modules outside of hlua.c can hook
into each lua_State creation. Modules call hap_register_hlua_state_init()
(or the REGISTER_HLUA_STATE_INIT() macro) with a callback of the form:
int my_init(lua_State *L, char **errmsg);
The callback returns an ERR_* code. ERR_ALERT and ERR_WARN trigger
ha_alert()/ha_warning() respectively; any other non-zero errmsg is
emitted via ha_notice(). ERR_FATAL or ERR_ABORT cause exit(1).
Registered entries are freed in hlua_deinit().
Output HTTP/3 header traces on the backend side. As previous commit,
this relies on h3_trace_header() function.
Extra calls are added for fields extracted from the request start-line
which produce an HTTP/3 pseudo-header.
Output trace for HTTP/3 headers manipulated on the frontend side. This
is implement via a new utility function h3_trace_header(), largely
inspired from existing h2_trace_header().
An extra call is added for HTTP/3 response :status which is extracted
from the HTX start line.
Define two new values for HTTP/3 trace verbosity : simple and advanced.
For now, these values are unused. However advanced level will become
useful to implement HTTP/3 header traces.
Extract the challenge-readiness logic from cli_acme_chall_ready_parse()
into a new acme_challenge_ready(crt, dns) function so it can be called
from other contexts such as Lua event handlers.
It slightly changes the messages on the CLI.
When an ACME order is re-used or when a domain was recently validated,
the CA may return status "valid" for an authorization without requiring
any challenge to be solved. In acme_res_auth(), this is handled by
setting auth->validated = 1 and jumping to out — but auth->ready is
never initialized and stays 0.
This became a bug in 3.4 when the "challenge-ready" option and the
ACME_CLI_WAIT state were introduced (commit 2b0c510aff). ACME_CLI_WAIT
computes:
all_cond_ready &= auth->ready;
across all authorizations. A single auth->ready == 0 drives the AND
to zero and the task waits indefinitely for a readiness signal that
will never arrive, since no challenge was published and no external
agent will ever call challenge_ready() for that domain.
Fix it by setting auth->ready = ctx->cfg->cond_ready for already-valid
authorizations, marking them as satisfying all required readiness
conditions so ACME_CLI_WAIT can proceed normally.
This should be backported to 3.4.
Commit 3c923d075 introduced a C99ism by declaring a variable in a for loop,
don't do that, especially since there already is a variable named "i"
declared.
This should fix the build when -std=c89 is used.
This should be backported if commit 3c923d075 is backported.
QUIC packets using a long header contains a Length field. Its value is
the length of the content following it, i.e. the packet number field and
the remaining payload (QUIC frames and TLS AEAD tag).
Computation to determine the packet length is performed in
qc_do_build_pkt(). However this calculation is incorrect when Initial
padding is added on a small enough Initial packet. As length field is
encoded as a varint, it changes the field size which grow from one to
two bytes, reducing in effect the total required padding length from one
byte. However, length value is not updated and thus is one byte bigger
than the final packet payload with padding.
Fix this calculation by reducing the length value after padding size has
been adjusted.
This bug caused the peer to reject such faulty packets. However, its
impact is minor as it only happened only for Initial with small enough
payload. Packets used for ClientHello/ServerHello exchanges should be
large enough and typically not concerned by this bug, except maybe in
case of fragmentation.
This bug was detected by testing QUIC backend with a quiche server. The
server endpoint reported the faulty packets with the following trace :
[2026-06-09T13:42:13.694179158Z ERROR quiche_server] 1b1b961c9c4ae1f470f3687510b120da1f5d5f5a recv failed: InvalidPacket
This must be backported up to 3.0.
Having a single task to take care of idle connection cleanup across all
servers leads to high contention. It uses a lock to maintain its tree of
servers to track, and then can acquire the idle_conns lock for each thread.
Instead, have one task per thread. Each thread will maintain its own
tree, so there will be no need for any lock, and it will just acquire
its own idle_conns lock, so it will lead to less contention.
This is a performance improvement, so backporting is optional, but may be
considered if it is worth it. That would require backporting commit
6f8dab2583 too.
When tune.max-checks-per-thread is used, checks that should run are
queued, to avoid having too many checks running at the same time.
But if the check is about to be purged, because the server is being
deleted, we have to explicitly remove it from the queue as that memory is
about to be freed, otherwise it will cause a use-after-free.
Also, queued checks have not yet incremented th_ctx->running_checks, so
don't decrement it if we're queued.
This should be backported up to 3.0.
For certain calls like strdup(), certain libc call the malloc() symbol
themselves, resulting in both strdup() and malloc() accounting for the
allocation while a single free() call is accounted for. Usually it's
not very hard to spot as these allocations are done inside libc, but
yet they complicate the tracing of allocations.
Let's note when we enter a handler and refrain from doing the accounting
again in this case. This way, the strdup() call place will be accountable
for the allocation and the libc's internal malloc() will not be seen.
It's not convenient to use it as it is now because it may only be
used to count passes via the memprof init code. Let's turn it to
a bitfield instead so that we can also check what we're doing there.
This is safe because all callers of memprof_init() check for the
bit being zero first so it's not reentrant.
As reported by @broxio in issue #3411, when trying to delete an ACL by
its name, in case of error the message says "unknown map identifier".
We need to check the type to decide between map and ACL as in other
messages.
This can be backported to all stable branches. Thanks to @broxio for
reporting the issue with a reproducer and providing this tested fix.
In order to preventively avoid issues that complicate debugging, let's
report to developers early if a pool name is not acceptable. This patch
does it in create_pool_from_reg() which catches both direct and declared
registrations. Aside the previous case, this didn't catch any other
occurrence.
Using "show pools detailed" on the CLI breaks the column alignment on
"sess_priv_conns" because the pool name contains spaces: "session priv
conns list", which is not welcome as pool names are truncated after the
12th chars anyway. Let's shorten it to the pool's name as done for many
other ones: sess_priv_conns.
This can be backported as far as 3.0 where this name was introduced,
because it helps when trying to sum or graph certain metrics during
debugging.
conn_get_ssl_sock_ctx() retrieves the ssl_sock_ctx of a connection by
calling conn->xprt->get_ssl_sock_ctx(). Only ssl_sock implements this
method, and it returns conn->xprt_ctx. This works because for every
existing XPRT combination the SSL layer is the topmost one: even
xprt_handshake (SOCKS4, PROXY, NetScaler CIP) is installed *below*
ssl_sock, so conn->xprt keeps pointing to ssl_sock.
Qmux changes this assumption: xprt_qmux is stacked *on top of* ssl_sock
and keeps the SSL layer as its lower layer to exchange the QUIC transport
parameters over the established TLS stream. During the qmux handshake,
conn->xprt therefore points to xprt_qmux, which does not implement
get_ssl_sock_ctx(), making conn_get_ssl_sock_ctx() return NULL for the
whole connection, affecting every caller that inspects the SSL layer
(sample fetches, logging, ssl_sock_infocbk(), ...).
The visible consequence was a crash: when the peer sends a TLS alert
during the qmux handshake, the SSL library calls ssl_sock_infocbk(),
which recovers a valid connection but a NULL ctx, rightfully triggering
the "BUG_ON(!ctx)" early in the function.
This patch implements xprt_qmux_get_ssl_sock_ctx() so that it returns
the ssl_sock_ctx of the lower layer when it is the SSL layer, just like
ssl_sock_get_ctx() does. conn_get_ssl_sock_ctx() then works again for
all callers while the qmux handshake is in progress. After the handshake,
conn->xprt is restored to the SSL layer so nothing else changes.
This should be backported to 3.4.
In thread_detect_count(), avoid any usage of thread_cpu_enable_at_boot
if we're building without thread support. That variable is only defined
when building with threads, and those tests make little sense when
building with no thread, anyway.
This was submitted by: ririnto <ririnto@kakao.com>
This should fix github issue #3408.
This should be backported to 3.4.
As reported by @zhanhb in github issue #3410, since 3.3 with commit
fda6dc959 ("MINOR: regex: use a thread-local match pointer for pcre2"),
the local_pcre2_match array is initialized too late for use by Lua. If
a lua-load makes use of regex, it may segfault (actually using PCRE2
is fine but PCRE2_JIT will crash):
Let's change the init sequence so that the first thread's context is
initialized early at boot and other threads are initialized when they
are created. For lua-load-per-thread, all extra threads will run on
the first thread's temporary storage during init but that's not a
problem since the sole purpose is to avoid concurrent accesses.
Thanks to @zhanbb for the detailed report and quick tests. This needs
to be backported to 3.3.
When an external check was configured at the proxy level, the healthcheck
section set on a server was not considered. The main reason was that the
check type of the server was always inherited for the proxy one.
To fix the issue, when a healthcheck section is set on a server line, the
check type for the server is forced to TCPCHK.
This patch must be backported to 3.4.
Prior to this patch, qcc_io_recv() stream decoding loop was interrupted
on the first decoding error or if incomplete data could not be parsed.
This patch adjusts this part so that loop is stopped only on a
connection level error. In case of a stream level error or on incomplete
data, decoding continues on the next QCS entry.
Without this patch, there is a risk that a QCS decode is not performed
as expected, with a possible client timeout firing. This is pretty
unlikely though. However this patch is still necessary to remove
completely this possibility.
This should be backported up to 3.2.
When a RESET_STREAM is received, QCS Rx channel is closed and pending Rx
data and buf are cleared without being transmitted to upper stream
layer.
This patch complements this by removing the QCS from recv_list if
present in it. This is a small optimization nothing would be performed
for such QCS on qcc_io_recv().
When a RESET_STREAM is received, QCS Rx channel is closed and pending Rx
data and buf are cleared without being transmitted to upper stream
layer.
This can cause an issue if this QCS instance is present in the QCC
recv_list. When qcc_io_recv() is executed after reset handling, an
infinite loop is triggered for the QCS instance as qcs_rx_avail_data()
always return 0.
This issue happened due to the poor writing of the while loop in
qcc_io_recv() which is not correctly protected against infinite
execution.
To prevent this issue, this patch rewrites the loop. Crucially,
LIST_DEL_INIT() is now performed unconditionally outside of the inner
loop. This guarantees that even if the inner loop is not executed, the
stream will be removed from QCC recv_list and iteration will progress.
This is functionally correct as a QCS should not be present in recv_list
if there is no avail data or demux is currently blocked. For the first
condition, qcc_decode_qcs() will be called again when new data is read
unless demux is blocked. In this case, QCS will be reinserted in the
list on unblocking, with a rescheduling to invoke qcc_decode_qcs().
In the context of the currently found reproducer linked to stream reset,
the QCS instance can be safely removed from the recv_list without
implication.
This must be backported up to 3.2.
The healthcheck keyword could be parsed on default-server lines but not
copied during server initialization, making it ineffective. But there is
also a true issue by setting it on a default-server. The pseudo server used
to parse the default-server line is not initialized via the new_server()
function, as regular servers. So there is no tcpcheck information inherited
from the proxy. We must take care of that when the "healthcheck" keyword is
parsed to avoid crashes.
This patch must be backported to 3.4.
