Until now, every form of "this server is referenced by something in the
running config" was collapsed onto a single flag, SRV_F_NON_PURGEABLE,
which prevents the server from being removed via 'del server'. This
catches everything but conflates two distinct properties:
- the server object itself is pinned by another runtime structure
(e.g. DNS resolution attached to it), versus
- the server's *name* is referenced statically (use-server rules,
track chains, sample-fetch arguments of type ARGT_SRV)
These differ for any operation that touches the name but not the
object identity, e.g. the runtime rename feature added next. Removing
a name-referenced server is still forbidden (the rule text would
dangle), but renaming such a server should also be forbidden for the
same reason - while renaming a resolver-pinned server is fine, since
the resolver holds the object pointer and doesn't care about the name.
Introduce SRV_F_NAME_REFD for the name-reference case and move the
three name-based setters (sample.c ARGT_SRV resolution, proxy.c
use-server resolution, server.c track chain setup) from
SRV_F_NON_PURGEABLE to SRV_F_NAME_REFD. The resolvers.c call site
keeps SRV_F_NON_PURGEABLE since it is the object-pinned case.
Adjust 'del server' to check both flags so the set of servers it
refuses to remove is unchanged: same observable behavior, just a
richer internal taxonomy.
A subsequent patch introducing 'set server name' will gate on
SRV_F_NAME_REFD only.
These ones were deprecated in 3.3-dev2 with commits 5c15ba5eff ("MEDIUM:
proxy: mark the "dispatch" directive as deprecated") and e93f3ea3f8
("MEDIUM: proxy: deprecate the "transparent" and "option transparent"
directives"), and were planned for removal in 3.5. See also:
https://github.com/orgs/haproxy/discussions/2921
as well as the wiki page about breaking changes.
They've lived their lives and always cause internal limitations
(exceptions between connecting to server or connecting to proxy), and
are even confusing to some extents (especially "transparent" which users
often get wrong).
This commit removes the ability to configure them, tests based on them
and all the doc related to them. The keywords remain detected by the
parser and indicate how to proceed instead.
It's likely that other deeper parts will be changed as well (e.g.
conn->target will no longer be of OBJ_TYPE_PROXY). This will be done
over the long term.
It's already possible to report that some options are not supported due
to build options by passing 0 instead of PR_CAP_* in the option's cap
field. Let's extend that by passing a non-zero value in the val field,
where the 3rd byte will be the major version and the 4th one the minor.
In this case haproxy will now indicate that support for that option was
removed in that version.
When threads are disabled, "static __decl_spinlock(foo);" ends up as
"static;", causing a build warning when threads are disabled. We don't
need it to be static so let's drop "static" here. No backport is needed,
this is 3.4-only.
When stream elasticity is active, the maximum number of concurrent bidi
streams advertised via transport parameters is now reduced depending on
the connection load. This is implemented via conn_calc_max_streams()
which returns the value to use.
This is not applied on listeners with enabled 0-RTT. Indeed, for such
connections, clients are expected to reuse the previously seen transport
parameters. The server on the other hand must not decrease several
values on the newly advertised params, in particular for the maximum
number of concurrent bidi streams. The simplest way to prevent 0-RTT
failure is to not mix stream elasticity with it.
Note that the 0-RTT limitation is only applied for the initial value :
during the connection lifetime, stream elasticity can still be used by
the MUX to dynamically reduce the stream window. This will be
implemented in a future patch.
Add a postparsing check on TCP ALPN bind and server setting. An error is
reported if the token "h3" is present and expose-experimental-directives
is not globally activated. This ensures that QMux protocol won't be
selected if experimental features are not explicitely requested.
The check is not performed though if "proto qmux" is explicitely
defined, as this setting already checks for experimental support.
Currently, it's not possible to activate QMux without any explicit
"proto qmux" config. However, this will be implemented in a next patch,
so this check will become necessary.
In conn_get_best_mux() and conn_get_best_mux_entry(), the mux name was
provided sometimes based on the "proto" directive, sometimes based on
the ALPN, but in any case, it was compared again the mux_proto_list
mux_proto field. This is not correct, as ALPN can be different from the
internal mux_proto. So enhance those functions so that they wll accept
an ALPN as well. If a mux_proto is provided, that will be used, if not,
and if an ALPN is provided, then that will be used, and compared against
the ALPN provided by the mux, if any.
In struct mux_proto_list, rename the "token" field to "mux_proto". That
field should only be used to match the name provided in the "proto"
directive, and it will be soon.
This should be a no-op.
As initially planned, if no trouble was reported on dynamic backend
commands on the CLI, the experimental status could be dropped before the
release. The feedback was not very broad, but was conclusive in that the
operations work as expected and the current syntax can be preserved even
for future evolutions. So we can drop the experimental status.
In parse_new_proxy(), when proxy_defproxy_cpy() fails, the error path used
ha_free(&curproxy) to release the partially constructed proxy. However, the
proxy was allocated via alloc_new_proxy() which performs significant setup:
- setup_new_proxy() inserts it into the proxy_by_name tree (proxy_store_name)
- It appends to the global proxies list (LIST_APPEND)
- proxy_take() increments its refcount
Additionally, proxy_defproxy_cpy() may have allocated further resources
(strdup'd strings, compression structures, email alert fields, etc).
Using ha_free() only freed the proxy struct itself, leaving:
- The proxy still registered in the name tree (dangling pointer)
- The proxy still linked in the global proxies list
- All strdup'd strings and other allocations leaked
This is visible with ASAN when causing random allocation errors:
[NOTICE] (27033) : haproxy version is 3.4-dev12-b15468-11
[NOTICE] (27033) : path to executable is ./haproxy
[ALERT] (27033) : config : parsing [/dev/stdin:5015] : proxy 'bk3': failed to duplicate tcpcheck preset-vars
[ALERT] (27033) : config : Error(s) found in configuration file : /dev/stdin
=================================================================
==27033==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 4 byte(s) in 1 object(s) allocated from:
#0 0x7f113e518e20 in strdup (/usr/lib64/libasan.so.8+0x118e20)
#1 0x000000955410 in setup_new_proxy src/proxy.c:3178
#2 0x000000955816 in alloc_new_proxy src/proxy.c:3221
#3 0x000000956c33 in parse_new_proxy src/proxy.c:3554
#4 0x000000a24d03 in cfg_parse_listen src/cfgparse-listen.c:495
#5 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#6 0x0000009e0bb9 in read_cfg src/haproxy.c:1142
#7 0x000000447e8c in main src/haproxy.c:3474
#8 0x7f113d82ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#9 0x7fff65b4e320 ([stack]+0x20320)
SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
The fix replaces ha_free(&curproxy) with proxy_drop(curproxy), which
properly calls deinit_proxy() to release all internal resources, removes
the proxy from trees and lists, decrements the refcount, and frees the
struct.
No backport is needed since proxy_drop() is only in 3.4.
One is s/keyworld/keyword in the retry-on parser. The other one is a
wrong argument "len" being printed in case of parse error for
"declare capture" instead of the length itself.
These can be backported though they are not important.
This lets lb_ops specify the conditions necessary to bind to this set of
ops. The condition is expressed as a list of mask and match fields on
the algorithm flags. This is then used in proxy_finalize() to locate the
lb_ops corresponding to the current configuration, by iterating over
the list of lb_ops structures. This list is implemented using the same
mechanisms used for configuration keywords: an INITCALL1 macro to a
registration function.
This also moves the lookup and property flags into the lb_ops structure
that were previously applied manually on a case by case basis.
The HTTP transaction is moved in an union. For now, it is the only possible
transaction that can be allocated. But that will change. Thanks to this
commit and the next one, it will be possible to deal with different kind of
transactions for a stream.
This patch looks quite huge, but it is more or less a renaming of all
accesses to "txn" field by "txn.http".
This is another pre-requisite work for upcoming decompression filter.
In this patch we implement the "filter-sequence" directive which can be
used in proxy section (frontend,backend,listen) and takes 2 parameters
The first one is the direction (request or response), the second one
is a comma separated list of filter names previously declared on the
proxy using the "filter" keyword.
The main goal of this directive is to be able to instruct haproxy in which
order the filters should be executed on request and response paths,
especially if the ordering between request and response handling must
differ, and without relying on the filter declaration ordering (within
the proxy) which is used by default by haproxy.
Another benefit of this feature is that it becomes possible to "ignore"
a previously declared filter on the proxy. Indeed, when filter-sequence
is defined for a given direction (request/response), then it will be used
over the implicit filter ordering, but if a filter which was previously
declared is not specified in the related filter-sequence, it will not be
executed on purpose. This can be used as a way to temporarily disable a
filter without completely removing its configuration.
Documentation was updated (check examples for more info)
Thanks to this patch, it is now possible to specify an healthcheck section
on the server line. In that case, the server will use the tcpcheck as
defined in the correspoding healthcheck section instead of the proxy's one.
The proxy flag PR_O_TCPCHK_SSL is replaced by a flag on the tcpcheck
itself. When TCPCHK_FL_USE_SSL flag is set, it means the healthcheck will
use an SSL connection and the SSL xprt must be prepared for the server.
disable-on-404 and send-state options, configured on an HTTP healtcheck,
were handled as proxy options. Now, these options are handled in the
tcp-check itself. So the corresponding PR_O and PR_02 flags are removed.
The tcpcheck_rules structure is replaced by the tcpcheck structure. The main
difference is that the ruleset is now referenced in the tcpcheck structure,
instead of the rules list. The flags about the ruleset type are moved into
the ruleset structure and flags to track unused rules remains on the
tcpcheck structure. So it should be easier to track unused rulesets. But it
should be possible to configure a set of tcpcheck rules outside of the proxy
scope.
The main idea of these changes is to prepare the parsing of a new
healthcheck section. So this patch is quite huge, but it is mainly about
renaming some fields.
When picking a mux, pay attention to its MX_FL_FRAMED. If it is set,
then it means we explicitely want QUIC, so don't use that mux for any
protocol that is not QUIC.
Use proxy_check_http_errors() on defaults proxy instances. This will
emit alert messages for errorfiles directives referencing a non-existing
http-errors section, or a warning if an explicitely listed status code
is not present in the target section.
This is a small behavior changes, as previouly this was only performed
for regular proxies. Thus, errorfile/errorfiles directives in an unused
defaults were never checked.
This may prevent startup of haproxy with a configuration file previously
considered as valid. However, this change is considered as necessary to
be able to use http-errors with dynamic backends. Any invalid defaults
will be detected on startup, rather than having to discover it at
runtime via "add backend" invokation.
Thus, any restriction on http-errors usage is now lifted for the
creation of dynamic backends.
Function proxy_check_errors() is used when configuration parsing is
over. This patch splits it in two newly named ones.
The first function is named proxy_check_http_errors(). It is responsible
to check for the validity of any "errorfiles" directive which could
reference non-existent http-errors section or code not defined in such
section. This function is now called via proxy_finalize().
The second function is named proxy_finalize_http_errors(). It converts
each conf_errors type used during parsing in a proper http_reply type
for runtime usage. This function is still called via post-proxy-check,
after proxy_finalize().
This patch does not bring any functional change. However, it will become
necessary to ensure http-errors can be used as expected with dynamic
backends.
In check_config_validity() and proxy_finalize() we check the consistency
of all rule sets, but the quic_initial rules were not placed there. This
currently has little to no impact, however we're going to use that to
also finalize certain debugging info so better call the function. This
can be backported to 3.1 (proxy_finalize is 3.4-only).
Update beconn separately from the call to COUNTERS_UPDATE_MAX(), as soon
there will be an option to get COUNTERS_UPDATE_MAX() to do nothing, and
we still want beconn to be properly updated, as it is used for other
purposes.
Introduce COUNTERS_UPDATE_MAX(), and use it instead of using
HA_ATOMIC_UPDATE_MAX() directly.
For now it just calls HA_ATOMIC_UPDATE_MAX(), but will later be modified
so that we can disable max calculation.
This can be backported up to 2.8 if the usage of COUNTERS_UPDATE_MAX()
generates too many conflicts.
This patch finalizes "del backend" handler by implementing the proper
proxy deletion.
After ensuring backend deletion can be performed, several steps are
executed. First, any watcher elements are updated to point on the next
proxy instance. The backend is then removed from ID and name global
trees and is finally detached from proxies_list.
Once the backend instance is removed from proxies_list, the backend
cannot be found by new elements. Thread isolation is lifted and
proxy_drop() is called, which will purge the proxy if its refcount is
null. Thanks to recently introduced PROXIES_DEL_LOCK, proxy_drop() is
thread safe.
Default proxy refcount <def_ref> is used to comptabilize reference on a
default proxy instance by standard proxies. Currently, this is necessary
when a default proxy defines TCP/HTTP rules or a tcpcheck ruleset.
Transform every access on <def_ref> so that atomic operations are now
used. Currently, this is not strictly needed as default proxies
references are only manipulated at init or deinit in single thread mode.
However, when dynamic backends deletion will be implemented, <def_ref>
will be decremented at runtime also.
This patch is similar to the previous one, but this time it deals with
functions related to defaults proxies instances. Lock PROXIES_DEL_LOCK
is used to protect accesses on global collections.
This patch will be necessary to implement dynamic backend deletion, even
if defaults won't be use as direct target of a "del backend" CLI.
However, a backend may have a reference on a default instance. When the
backend is freed, this references is released, which can in turn cause
the freeing of the default proxy instance. All of this will occur at
runtime, outside of thread isolation.
Define a new lock with label PROXIES_DEL_LOCK. Its purpose is to protect
operations performed on global lists or trees while a proxy is freed.
Currently, this lock is unneeded as proxies are only freed on
single-thread init or deinit. However, with the incoming dynamic backend
deletion, this operation will be also performed at runtime, outside of
thread isolation.
Ensure a backend instance cannot be removed if there is still server in
it. This is checked via be_check_for_deletion() to ensure "del backend"
cannot be executed. The only solution is to use "del server" to remove
on the servers instances.
This check only covers servers not yet targetted via "del server". For
deleted servers not yet purged (due to their refcount), the proxy
refcount is incremented but this does not block "del backend"
invokation.
Define a new proxy flag PR_FL_NON_PURGEABLE. This is used to mark every
proxy instance explicitely referenced in the config. Such instances
cannot be deleted at runtime.
Static use_backend/default_backend rules are handled in
proxy_finalize(). Also, sample expression proxy references are protected
via smp_resolve_args().
Note that this last case also incidentally protects any proxies
referenced via a CLI "set var" expression. This should not be the case
as in this case variable value is instantly resolved so the proxy
reference is not needed anymore. This also affects dynamic servers.
Prevent removal of a backend which relies on features not compatible
with dynamic backends. This is the case if either dispatch or
transparent option is used, or if a stick-table is declared.
These limitations are similar to the "add backend" ones.
Rename proxy conf <refcount> to <def_ref>. This field only serves for
defaults proxy instances. The objective is to avoid confusion with the
newly introduced <refcount> field used for dynamic backends.
As an optimization, it could be possible to remove <def_ref> and only
use <refcount> also for defaults proxies usage. However for now the
simplest solution is implemented.
This patch does not bring any functional change.
Implement refcount notion into proxy structure. The objective is to be
able to increment refcount on proxy to prevent its deletion temporarily.
This is similar to the server refcount : "del backend" is not blocked
and will remove the targetted instance from the global proxies_list.
However, the final free operation is delayed until the refcount is null.
As stated above, the API is similar to servers. Proxies are initialized
with a refcount of 1. Refcount can be incremented via proxy_take(). When
no longer useful, refcount is decremented via proxy_drop() which
replaces the older free_proxy(). Deinit is only performed once refcount
is null.
This commit also defines flag PR_FL_DELETED. It is set when a proxy
instance has been removed via a "del backend" CLI command. This should
serve as indication to modules which may still have a refcount on the
target proxy so that they can release it as soon as possible.
Note that this new refcount is completely ignored for a default proxy
instance. For them, proxy_take() is pure noop. Free is immediately
performed on first proxy_drop() invokation.
Define a new member watcher_list in proxy. It will be used to register
modules which iterate over the proxies list. This will ensure that the
operation is safe even if a backend is removed in parallel.
Add "del backend" handler which is restricted to admin level. Along with
it, a new function be_check_for_deletion() is used to test if the
backend is removable.
Add missing proxy_index_id() call in "add backend" handler. This step is
responsible to store the newly created proxy instance in the
used_proxy_id global tree.
No need to backport.
We'll need to permit any user to update its own tgroup's extra counters
instead of the global ones. For this we now store the per-tgroup step
between two consecutive data storages, for when they're stored in a
tgroup array. When shared (e.g. resolvers or listeners), we just store
zero to indicate that it doesn't scale with tgroups. For now only the
registration was handled, it's not used yet.
Servers, proxies, listeners and resolvers all use extra_counters. We'll
need to move the storage to per-tgroup for those where it matters. Now
we're relying on an external storage, and the data member of the struct
was replaced with a pointer to that pointer to data called datap. When
the counters are registered, these datap are set to point to relevant
locations. In the case of proxies and servers, it points to the first
tgrp's storage. For listeners and resolvers, it points to a local
storage. The rationale here is that listeners are limited to a single
group anyway, and that resolvers have a low enough load so that we do
not care about contention there.
Nothing should change for the user at this point.
During proxy_finalize(), a lookup is performed over the servers by name
tree to detect any collision. Only the first conflict for each server
instance is reported to avoid a combinatory explosion with too many
alerts shown.
Previously, this was written using a for loop without any iteration.
Replace this by a simple if statement as this is cleaner.
This should fix github issue #3276.
Auto SNI configuration is configured during check config validity.
However, nothing was implemented for dynamic servers.
Fix this by implementing auto SNI configuration during "add server" CLI
handler. Auto SNI configuration code is moved in a dedicated function
srv_configure_auto_sni() called both for static and dynamic servers.
Along with this, allows the keyword "no-sni-auto" on dynamic servers, so
that this process can be deactivated if wanted. Note that "sni-auto"
remains unavailable as it only makes sense with default-servers which
are never used for dynamic server creation.
This must be backported up to 3.3.
There was no check on the result of strdup() used to setup auto SNI on a
server instance during check config validity. In case of failure, the
error would be silently ignored as the following server_parse_exprs()
does nothing when <sni_expr> server field is NULL. Hence, no SNI would
be used on the server, without any error nor warning reported.
Fix this by adding a check on strdup() return value. On error, ERR_ABORT
is reported along with an alert, parsing should be interrupted as soon
as possible.
This must be backported up to 3.3. Note that the related code in this
case is present in cfgparse.c source file.
Add a pointer to function to proxies as ->stream_new_from_sc proxy
struct member to instantiate stream from connection as this is done by
all the muxes when they call sc_new_from_endp(). The default value for
this pointer is obviously stream_new() which is exported by this patch.
At many places, we rely on global.tune.bufsize value instead of using the buffer
size. For now, it is not a problem. But if we want to be able to deal with
buffers of different sizes, it is good to reduce as far as possible dependencies
on the global value. most of time, we can use b_size() or c_size()
functions. The main change is performed on the error snapshot where the buffer
size was added into the error_snapshot structure.
For "add backend" implementation, postparsing code in
check_config_validity() from cfgparse.c has been extracted in a new
dedicated function named proxy_finalize() into proxy.c.
This has caused unexpected compilation issue as in the latter file
TLSEXT_TYPE_application_layer_protocol_negotiation macro may be
undefined, in particular when building without QUIC support. Thus, code
related to default ALPN on binds is discarded after the preprocessing
stage.
Fix this by including openssl-compat header file into proxy source file.
This should be sufficient to ensure SSL related defines are properly
included.
This should fix recent issues on SSL regtests.
No need to backport.
When a backend is created at runtime, the new proxy instance is inserted
at the end of proxies_list. This operation is buggy if this list is
empty : the code causes a null dereference which will lead to a crash.
This causes the following compilation error :
CC src/proxy.o
src/proxy.c: In function 'cli_parse_add_backend':
src/proxy.c:4933:36: warning: null pointer dereference [-Wnull-dereference]
4933 | proxies_list->next = px;
| ~~~~~~~~~~~~~~~~~~~^~~~
This patch fixes this issue. Note that in reality it cannot occur at
this moment as proxies_list cannot be empty (haproxy requires at least
one frontend to start, and the list also always contains internal
proxies).
No need to backport.
This patch fixes the following compilation error :
src/proxy.c:4954:12: error: format string is not a string literal
(potentially insecure) [-Werror,-Wformat-security]
4954 | ha_notice(msg);
| ^~~
No need to backport.
Implement proxy ID generation for dynamic backends. This is performed
through the already function existing proxy_get_next_id().
As an optimization, lookup will performed starting from a global
variable <dynpx_next_id>. It is initialized to the greatest ID assigned
after parsing, and updated each time a backend instance is created. When
backend deletion will be implemented, it could be lowered to the newly
available slot.
