During H1 message parsing, the Upgrade header values are checked to detect
"websocket" prototol, to properly handle websocket upgrades between H1 and
H2 and to possibly reject messages if mandatory headers are missing.
However, the flag is reset for each new Upgrade header and the information
may be lost. So never reset it.
This patch must be backported as far as 2.4.
As initially planned, if no trouble was reported on dynamic backend
commands on the CLI, the experimental status could be dropped before the
release. The feedback was not very broad, but was conclusive in that the
operations work as expected and the current syntax can be preserved even
for future evolutions. So we can drop the experimental status.
When an async connect() fails in sock_conn_check(), it returns an errno
that will not be retrieved later by a subsequent getsockopt(SO_ERROR).
The problem is that this errno is then definitely lost. This is visible
in the 4be_1srv_smtpchk_httpchk_layer47errors regtest that fails on
certain systems (e.g. glibc 2.31 on arm32 running Linux 6.1), where the
connect() error is systematically lost and the "Connection refused" is
never seen in the check status. It also matches a few random reports of
the past indicating that the connection error was sometimes not reported
in the stats page in front of a down server.
Ideally we should store errno in connections as soon as the error is
seen. However this would require significant changes that are not
acceptable yet for 3.4 nor stable releases. A more acceptable fix is to
make use of the extra CO_ER_* flags set by conn_set_errno() as soon as
the error is detected. This will recognize a sufficiently large number
of errors and the check status will report them (here we'll have
"ECONNREFUSED" in the check). Note that on systems where the error is
seen synchronously, we can have "ECONNREFUSED (Connection refused)",
but this is not a problem.
This fix adds the missing conn_set_errno() call to sock_conn_check(),
that is thus sufficient to catch this error. In addition, the two
affected regtests were updated to search for ECONNREFUSED here.
This might be backported to older releases if users request it, but it
is probably not necessary.
This test is compatible with QUIC_OPENSSL_COMPAT but the "limited-quic"
directive was not set, making it fail on older libs with no QUIC support
despite being declared as compatible.
In domain-based routing and policy rules, suffix matching on hostnames is
often easier to express as a prefix match on reversed labels. A dedicated
converter makes this convenient with existing fetches and matchers.
This also has a performance benefit for large maps. Prefix string matches use
the prefix-tree index (PAT_MATCH_BEG with pat_idx_tree_pfx), while end matches
use the string-list index (PAT_MATCH_END with pat_idx_list_str), so
reversed-label lookups can avoid linear suffix scans.
This patch adds "reverse_dom", a string converter that reverses domain labels,
ignores one optional trailing dot on input, and rejects empty labels. It
intentionally leaves trailing-dot handling to the caller so configurations can
choose between exact matches, subdomain-only matches, or an explicit dotted
form built with "concat(.)" for prefix lookups.
Examples:
example.com -> com.example
mail.example.com -> com.example.mail
The documentation is updated and a reg-test covers the converter itself, the
explicit dotted form for "map_beg()", and the subdomain-only "-m beg" case.
Some use cases benefit from reversing a string before passing it to other
converters or lookups. While reverse_dom addresses domain-specific label
reversal, a generic byte-wise string reversal remains useful on its own and can
also be combined with other converters such as concat().
A common lookup use case is turning a suffix match on the original string into
a prefix match on the reversed string. Prefix string matches use the
prefix-tree index (PAT_MATCH_BEG with pat_idx_tree_pfx), while end matches use
the string-list index (PAT_MATCH_END with pat_idx_list_str), so reversing
before map_beg can avoid linear suffix scans for large maps.
This patch adds a new string converter named "reverse". It reverses the input
string byte by byte and returns the resulting string unchanged otherwise. It
does not apply any domain-specific semantics or character-encoding semantics.
The documentation is updated and a reg-test is added to cover the basic
conversion as well as a simple composition with concat(.).
The test doesn't need a real nameserver and in a isolated, restricted
test environment it might not be able to reach one at all, like with a
network sandbox. So lets just use 127.0.0.1:53. Even if there is none,
that's not a problem for this particular test.
Signed-off-by: Christian Ruppert <idl0r@qasl.de>
Tests may be excuted in sandboxed or minimalistic / restricted
environments, so incosistencies might cause trouble, like missing
permissions. So lets use the tmpdir variable instead, so the user might
define some path
Signed-off-by: Christian Ruppert <idl0r@qasl.de>
In RFC8725, section 3.2, they suggest to "Avoid all RSA-PKCS1 v1.5
encryption algorithms" so this algorithm gets disabled by default.
Tokens having this "alg" won't be decrypted unless it is explicitly
reenabled thanks to 'jwt.decrypt_alg_list' global option.
Thanks to Omkhar Arasaratnam for raising our awareness about this!
As reported by Huangbin Zhan (@zhanhb) in github issue #3355, latest
commit 96f7ff4fdd ("MINOR: mux-h2: add a new message flag to indicate
ext connect support") was not correct and can break RFC8441-compliant
clients, as it did for them with a variant of Chrome 142.
The problem is that while RFC9113 says that new pseudo-headers are only
permitted with *negotiated* extensions, and RFC8441 doesn't indicate
whether or not SETTINGS_ENABLE_CONNECT_PROTOCOL is needed from clients,
it only says that clients know that servers support the extension when
seeing it in their settings and can use it, which seems to imply that
they don't need to send it to indicate their willingness to use it.
This also means that the server cannot know if a client is expected to
use it or not by default. It only know that a client is not allowed to
use it if the server didn't emit support mentioning it, which haproxy
can do using h2-workaround-bogus-websocket-clients.
Thus the fix proposed by @zhanhb is right, when presetting the flag for
the parser to indicate whether or not we're willing to accept RFC8441's
:protocol pseudo-header, we should:
- consider the received setting on the backend side (though the
pseudo-header is neither used nor supported there, but at least
we pass the info regarding the support of the extension)
- consider the configuration for the frontend (since it's the only
place where we can decide on support or not)
This patch does just that and reverts the accompanying changes to the
regtests that made them want to see the client's setting. It must be
backported to 2.6.
In the mean time, placing this option in the global section will force
the clients to downgrade to h1:
h2-workaround-bogus-websocket-clients
Many thanks again to @zhanbb this feedback and proposing a tested fix.
This reverts commit 64383e655b.
As reported by Alexander Stephan in issue #3351, it causes problems.
First, as seen in the issue, the "reload" operation, handled by an applet
local to the master process, is being interrupted by the timeout so that
the client never gets the result (though the timeout is applied). A fix
for this was found (ignore client-fin/server-fin on applets, as they make
no sense there), but it only hides a deeper problem. Indeed, issuing
"@1 debug dev delay 2000" still stops at 1s with an error, indicating
that commands are systematically being sent with a shutdown, and thus
that the server-fin always applies. This is a problem because it means
that any long command will now be interrupted after one second.
All of this needs to be put back into perspective before progressing
further on this issue, and the reason for sending the shutdown should
be reconsidered in the context of the current version, as it looks
like this was once necessary but no longer is.
In addition, the issue encountered by Alexander, of a frozen worker,
was essentially reported once in many years, so it's totally acceptable
to leave older versions unfixed and figure what's the best solution for
modern versions only.
Let's just revert to the pre-fix situation so as to avoid causing
breakage everywhere. This revert should be backported to all versions
(2.4 included).
With VTEST, It seems possible to receive the H2 preface in 2 packets. So the
preface cannot be matched and the H1 to H2 upgrade is not performed as
expected. The script was fixed by forcing the H2 proto on the first bind
line.
The problem with the preface matching will be reviewed later.
If h1 to h2 upgrades are not allowed, a 405-method-not-allowed error is now
returned from the H1 multiplexer itself instead of dealing with "PRI *
HTTP/2.0\r\n\r\n" request as a normal request.
Before this kind of request was caught by the HTTP analyzers and a
400-bad-request was returned. This was added before the multiplexers era to
protect backend apps against unexpected H1 to H2 uprade on server side.
Now, it is possible to handle the error in the H1 multiplexer. One benefit
is to be able to increment the glichtes counters. However, the error is
still handled in HTTP analyzers to be sure to detect unwanted upgrades that
can be hidden in H2 or H3 requests.
There is a special case. TCP > H1 > H2 upgrades. In that case, a H1 stream
exist. So we must report an error to the upper layer too.
A reg-test script was added to validate the feature. In addition,
tcp_to_http_upgrade.vtc was updated accordingly.
As reported by Huangbin Zhan in github issue #3355, we're too lax on
the :protocol pseudo header. It is currently accepted with regular
CONNECT as well as non-CONNECT methods while it only ought to be
accepted with extended CONNECT (i.e. CONNECT after the connection
negotiated the RFC8441 extension). Let's refine the check in H2 by
leveraging the new flag H2_MSGF_EXT_CONN_OK that is passed by the
caller when the connection supports the extension. This is sufficient
to sort the various cases.
The proto upgrade regtest was updated to verify that CONNECT with
:protocol without nego and another method with nego and :protocol
both fail.
Thanks to Huangbin Zhan (@zhanhb) for the report and helpful reproducer.
This needs to be backported to all versions. It relies on these patches
first:
REGTESTS: http-messaging: always send RFC8441 client settings to use ext connect
BUG/MINOR: mux-h2: condition the processing of 8441 extension to global setting
MINOR: mux-h2: add a new message flag to indicate ext connect support
The tests were validating extended connect without sending the setting
in the client settings frame. It currently works due to a bug, so let's
fix the vtc first.
This test first performs two successive requests over the same
connection where reuse is expected, then perform two 401 which must
both work, testing both the transition from null->sess, and sess->sess.
This test could be backported to detect changes related to private
sessions.
Thanks to Omkhar Arasaratnam for the test.
In master-worker mode the master CLI proxy (mworker_proxy) has a
hardcoded maxconn of 10. When a client connects to the master CLI
socket and issues a command that gets forwarded to an unresponsive
worker (e.g. one that is stuck or very slow), the connection hangs
waiting for the worker's response. If the client then disconnects
(timeout, Ctrl-C, etc.), the connection slot is never released because
the client-side FIN is never acknowledged by the unresponsive worker.
After 10 such leaked slots the master CLI socket becomes completely
unreachable, returning "Resource temporarily unavailable" to any new
connection attempt. In containerized deployments this means readiness
probes start failing and the pod gets restarted.
The fix adds a timeout server-fin of 1s to the mworker_proxy. When
the client disconnects while waiting for a worker response, this
timeout ensures the dangling backend connection is cleaned up after
1s, freeing the connection slot. This does not affect normal CLI
operations since the timeout only starts after the client has already
closed its side of the connection.
A regression test is included that blocks the worker CLI thread using
"debug dev delay" with nbthread 1, fills all 10 master CLI slots,
waits for client-side timeouts, then verifies a new connection still
succeeds.
This fixes GH issue #3351.
This should be backported to all stable branches.
Co-authored-by: Martin Strenge <github@trixer.net>
Co-authored-by: William Lallemand <wlallemand@haproxy.com>
Add a new QUIC regtest to test the new frontend stream.max-total
setting.
This test relies on two haproxy instances, as QUIC client and server.
New setting stream.max-total is set to 3 on the server side. In total, 6
requests are performed, with a check to ensure that a new connection has
been reopened for the last ones.
This allows to use the `unique-id` fetch within `tcp-check` or `http-check`
ruleset. The format is taken from the checked server's backend (which is
naturally inherited from the corresponding `defaults` section).
This is particularly useful with
http-check send ... hdr request-id %[unique-id]
to ensure all requests sent by HAProxy have a unique ID header attached.
This resolves GitHub Issue #3307.
Reviewed-by: Volker Dusch <github@wallbash.com>
Ensure that all Lua regression tests exercise the restricted library
mode by setting "tune.lua.openlibs none" in their global section.
Only txn_get_priv-thread.vtc requires "string,table"
We add a reg-test, filter_sequence.vtc, with associated lua file
dummy_filters.lua to cover the "filter-sequence" directive and
ensure it is working as expected, both for request and responses
paths.
This regtest will only be effective starting with 3.4-dev0
When ECDH-ES algorithm is used in a JWE token, no cek is provided and
one must be built in order to decrypt the contents of the token. The
decrypting key is built by deriving a temporary key out of a public key
provided in the token and the private key provided by the user and
performing a concatKDF operation.
The 'jwt_tokenize' function that can be used to split a JWT token into
its subparts can either fully process the token (from beginning to end)
when we need to check its signature, or only partially when using the
jwt_header_query or jwt_member_query converters. In this case we relied
on the fact that the return value of the 'jwt_tokenize' function was not
checked because a '-1' was returned (which was not actually an error).
In order to make this logic more explicit, the 'jwt_tokenize' function
now has a way to warn the caller that the token was invalid (less
subparts than the specified 'item_num') or that the token was not
processed in full (enough subparts found without parsing the token all
the way).
The function will now only return 0 if we found strictly the same number
of subparts as 'item_num'.
Complete delete backend regtests by checking deletion of a proxy with a
reference on an unnamed defaults instance. This operation is sensible as
the defaults refcount is decremented, and when the last backend is
removed, the defaults is also freed.
Add a reg-tests to test "del backend" CLI command. First, checks are
performed to ensure a backend cannot be deleted if not in the expected
state.
Then, a "del backend" success is tested. Stats are dumped to ensure the
backend instance is indeed removed.
Remove extra quote in feature cmd used to test SSL compatibility with
set_ssl_cafile QUIC regtest. Due to this syntax error, the test was
never executed.
No need to backport.
the TCP_MD5SIG ifdef is not enough to check if the feature is usable.
The code might compile but the OS could prevent to use it.
This patch tries to use the TCP_MD5SIG setsockopt before adding
HAVE_WORKING_TCP_MD5SIG in the feature list. so it would prevent to
start reg-tests if the OS can't run it.
Test the new "jwt_decrypt_jwk" converter that takes a JWK as argument,
either as a string or in a variable.
Only "RSA" and "oct" types are managed for now.
Add a new regtests to validate backend creation at runtime. A server is
then added and requests made to validate the newly created instance
before (with force-be-switch) and after publishing.
Move backend compatibility checks performed during 'add server' in a
dedicated function be_supports_dynamic_srv(). This should simplify
addition of future restriction.
This function will be reused when implementing backend creation at
runtime.
OpenSSL 4.0 changed the way it stores objects in X509_STORE structures
and are not allowing anymore to iterate on objects in insertion order.
Meaning that the order of the object are not the same before and after
OpenSSL 4.0, and the reg-tests need to handle both cases.
Since commit eb5279b15 ("BUG/MEDIUM: ssl: fix generate-certificates
option when SNI greater than 64bytes") the LibreSSL job does not seem to
work anymore.
Indeed the reg-tests was modified to add a SNI longer than 64 bytes,
without any concern about the DNS standard, which allows only 63 bytes
per label.
LibreSSL is stricter than the other libraries about that, and checks
that the SNI is compliant with the DNS RFC in the
tlsext_sni_is_valid_hostname() function
https://github.com/libressl/openbsd/blob/OPENBSD_7_8/src/lib/libssl/ssl_tlsext.c#L710
This patch fixes the issue by splitting the SNI with a second label to
reach more than 64 bytes.
Must be backported with eb5279b15 in every stable branches.
The problem is that the certificate is generated with a CN greater than
64 bytes when the SNI is too long, which is not suppose to be supported,
and will end up with a handshake failure.
The patch fixes the issue by avoiding to add a CN when the SNI is longer than
64 bytes. Indeed this is not a mandatory field anymore and was deprecated more
than 20 years ago. The SAN DNS is enough for this case.
Must be backported in every stable branches.
Many tests use the A128KW algorithm which is not supported by AWS-LC but
instead of removing those tests we will just have a hardcoded value set
by default in this case.
This test mimics what was already done for the aes_gcm converters. Some
data is encrypted and directly decrypted and we ensure that the output
was not changed.
A recent patch has introduced a new state for proxies : unpublished
backends. Such backends won't be eligilible for traffic, thus
use_backend/default_backend rules which target them won't match and
content switching rules processing will continue.
This patch defines a new frontend keywords 'force-be-switch'. This
keyword allows to ignore unpublished or disabled state. Thus,
use_backend/default_backend will match even if the target backend is
unpublished or disabled. This is useful to be able to test a backend
instance before exposing it outside.
This new keyword is converted into a persist rule of new type
PERSIST_TYPE_BE_SWITCH, stored in persist_rules list proxy member. This
is the only persist rule applicable to frontend side. Prior to this
commit, pure frontend proxies persist_rules list were always empty.
This new features requires adjustment in process_switching_rules(). Now,
when a use_backend/default_backend rule matches with an non eligible
backend, frontend persist_rules are inspected to detect if a
force-be-switch is present so that the backend may be selected.
Define a new set of CLI commands publish/unpublish backend <be>. The
objective is to be able to change the status of a backend to
unpublished. Such a backend is considered ineligible to traffic : this
allows to skip use_backend rules which target it.
Note that contrary to disabled/stopped proxies, an unpublished backend
still has server checks running on it.
Internally, a new proxy flags PR_FL_BE_UNPUBLISHED is defined. CLI
commands handler "publish backend" and "unpublish backend" are executed
under thread isolation. This guarantees that the flag can safely be set
or remove in the CLI handlers, and read during content-switching
processing.
A proxy can be marked as disabled using the keyword with the same name.
The doc mentions that it won't process any traffic. However, this is not
really the case for backends as they may still be selected via switching
rules during stream processing.
In fact, currently access to disabled backends will be conducted up to
assign_server(). However, no eligible server is found at this stage,
resulting in a connection closure or an HTTP 503, which is expected. So
in the end, servers in disabled backends won't receive any traffic. But
this is only because post-parsing steps are not performed on such
backends. Thus, this can be considered as functional but only via
side-effects.
This patch clarifies the handling of disable backends, so that they are
never selected via switching rules. Now, process_switching_rules() will
ignore disable backends and continue rules evaluation.
As this is a behavior change, this patch is labelled as medium. The
documentation manuel for use_backend is updated accordingly.
Create a new test to ensure that switching rules selection is fine.
Currently, this checks that dynamic backend switching works as expected.
If a matching rule is resolved to an unexisting backend, the default
backend is used instead.
This regtest should be useful as switching-rules will be extended in a
future set of patches to add new abilities on backends, linked to
dynamic backend support.
OpenSSL changed the output from "Server Temp Key" in prior versions to
"Peer Temp Key" in recent ones.
a39dc27c25
It looks like it affects OpenSSL >=3.5.0
This broke the reg-test for e.g. Debian 13 builds, using OpenSSL 3.5.1
Fixes bug #3238
Could be backported in every branches.
Signed-off-by: Christian Ruppert <idl0r@qasl.de>
