In resolv_validate_dns_response(), when matching an additional A/AAAA
record to an SRV record, the code checked tmp_record->ar_item == NULL
then called pool_free(resolv_answer_item_pool, tmp_record->ar_item).
This is a copy-paste mistake from similar patterns elsewhere since
the pointer is confirmed to be NULL a few lines above, so let's just
drop the confusing pool_free.
In resolv_validate_dns_response(), the second DNS record parsing path
manually constructs a 32-bit big-endian TTL value from four individual
bytes using the expression:
reader[0] * 16777216 + reader[1] * 65536 + reader[2] * 256 + reader[3]
We have read_n32() to do this, and it's more robust against unexpected
signedness surprises (which should not happen right here since reader is
unsigned char and we use -fwrapv so the result is defined). Also, let's
make the ttl an uint instead of an int. The TTL is only retrieved and not
used for now, so better clean it now.
In 2.5, commit da0264a96 ("MINOR: sample: Add be2hex converter")
introduced the be2hex() converter, which reads input data of a given
chunk size, processes it as a big endian block and turns it to hex
output.
There's an issue if the configured chunk_size (2nd argument) is larger
than tune.bufsize/2, because the max_size calculation will underflow,
and the later loop will always match since it compares a size_t to an
int (BTW, compilers love to annoy us with useless warnings but I never
found how to see some for these ones). This can result in overflowing
the output trash if the input sample is at least as large as half a
buffer.
Let's add an explicit check for this, and change the max_size type to
size_t so that the comparison is always right. While we're at it, let's
ask the trash buffer to be twice as large, just like bin2hex() does, as
it may result in offering a larger buffer in 3.4. thanks to the large
buffers support.
Despite the risk, this is marked as minor because a config with that
large an argument in the converter makes absolutely no sense.
This should be backported to 2.6. The *2 for the trash allocation will
conflict and have to be dropped in stable versions, which is safe.
When not set, the cluster secret is randomly generated by two
consecutive calls to ha_random64(). However, the random64 PRNG may be
partially observed on a fully idle machine (QUIC retry tokens, UUID,
WS key), and it could be rolled back to the initial call that produced
the secret. This is purely theoretical as a normally loaded system
wouldn't reveal meaningful sequences, but better address this while
it's still easy.
The first here consists in isolating the cluster_secret from the PRNG
sequence. When RAND_bytes() is available and works, it's used. Otherwise
ha_random64() is mixed with uncorrelated bits from random().
This could be backported to stable releases.
In dict_insert(), when ebis_insert() returns an existing node n indicating
that another thread inserted the same key concurrently, the code freed its
own newly-allocated entry and returned the winner without bumping its
refcount. Both callers then held a reference with refcount=1 instead of 2,
so when one expires the other becomes a use-after-free or double-free.
The bug likely comes from the fact that new_dict_entry() creates an entry
with a refcount preset to 1 (saves an atomic op) and that because of this
there is no refcount increment upon a successful insertion in the tree,
resulting in requiring different code paths for collision and normal
insertion.
A simple fix consists in bumping the refcount under the lock and unlocking
only at the end, but this would mean performing two free() calls under a
lock, which we always try to avoid. The code was slightly rearranged so
that we can now bump the existing entry's refcount under the lock in case
of duplicate, or unlock immediately in the common case, so that the free()
call is done out of the lock.
The probably of the race is very low (at peers connection setup only),
reason why it's marked low. This should be backported to all versions.
In parse_log_message(), the first loop looks for '>' that finishes the
priority field, and unfortunately it stops once it has checked the first
byte after the end of the buffer. This means that a priority made only
of digits for the whole buffer would read one extra byte. In practice
since pools have a tag at the end this is only detectable when using ASAN,
but this should be fixed nevertheless.
This can be backported to all versions.
It's worth noting that RFC5424 now says that the PRI field is 1..3
digits only, so maybe at some point we could seriously limit the
length as well.
When the PRIORITY flag is present on a HEADERS frame, the frame must
contain a stream dependency and a weight, for a total of 5 bytes. The
length is checked after reading the stream dep field so theoretically
such a frame could cause up to 4-byte OOB read at the end of the buffer,
though in practice buffers allocated from pools never end on a page
boundary (one extra word at the end) and the anomaly is still detected
after reading the stream ID and the connection aborted with the glitch
count incremented. Thus while not technically correct, practically
speaking it's harmless.
This should be backported to all stable releases.
The previous fix 75f72c2eb ("BUG/MEDIUM: resolvers: Fix test on dn label
size in resolv_dn_label_to_str()") may still leave garbage from the input
buffer into the response: if a component length is passed as zero, it
should mark the end, but instead a dot will be emitted, and whatever
follows it in the input buffer would continue to be appended as extra
components. While having no direct consequences beyond the domain not
being properly decoded, it could at least complicate troubleshooting.
This should be backported where the fix above is backported.
The previous fix 75f72c2eb ("BUG/MEDIUM: resolvers: Fix test on dn label
size in resolv_dn_label_to_str()") can still be fooled by an input exactly
the size of str_len, in which case the trailing zero appended at the end
was not being accounted for. Let's add 1 to the condition to prepare for
it.
This needs to be backported wherever the fix above is backported.
When lf_expr_compile() fails in cfg_parse_log_profile, the code leaves
without freeing the previously strdup()'d strings in target_lf->str and
target_lf->conf.file. Let's add a call to lf_expr_deinit() there to
release it.
It was harmless anyway since the startup will abort when this happens,
but better clean it because with increasingly dynamic setups, one day
it could become a runtime leak.
No backport is needed.
When a primary cache hit has a Vary secondary_key_signature, the code calls
retain_entry() and shctx_row_detach() before performing the secondary lookup.
If get_secondary_entry() returns NULL (no stored variant matches), res is set
to NULL and the function falls through to return ACT_RET_CONT without calling
release_entry() or shctx_row_reattach(). Each such request leaks one refcount
and pins one shctx row permanently, eventually exhausting the cache if this
happens to all objects. This is visible when requesting a secondary key
covered by vary for an object that is already stored without that key.
"show cache" then shows the object's refcount increasing after each request.
In order to fix this we must do like when no secondary key could be built
and release everything. We only reattach to the row if we previously
detached.
The issue was introduced in 2.4 with commit 1785f3dd9 ("MEDIUM: cache: Add
the Vary header support"). The code changed a bit in 2.9 with commit
48f81ec09 ("MAJOR: cache: Delay cache entry delete in reserve_hot function"),
so in order to backport to 2.8 and older, the patch will have to be manually
applied (no test on detached).
tcpcheck_spop_expect_hello() stores the SPOA agent-supplied status-code
varint directly into check->code (signed short) without range validation.
The code is later used as an index into spop_err_reasons[100]. Let's
just replace invalid status codes with SPOP_ERR_UNKNOWN to avoid any
problem.
The SPOP tcp-check was introduced in 3.1 so this fix must be backported
to 3.2.
In 3.3 with commit fda6dc959 ("MINOR: regex: use a thread-local match
pointer for pcre2") we got a thread-local match that saves us from having
to allocate a match array with each match. However something was clearly
overlooked or misunderstood in the pcre2 API because the local match
array was initialized via pcre2_match_data_create() for MAX_MATCH-1
entries instead of MAX_MATCH, despite the commit message mentioning
MAX_MATCH entries. It was possibly confused with an index. Due to this
there is a risk of crash when matching more than 9 groups in a regex.
This fix must be backported to 3.3.
In 2.3, in preparation for log forwarding, commit 546488559 ("MEDIUM:
log/sink: re-work and merge of build message API.") extended the log
send API to be able to use metadata from an existing header. However
the month number is parsed from the passed meta-data and compared
against 11 but there's no check for negative values which could in
theory cause a negative monthname[] index.
It can be a problem when the date is received as RFC5424 and forced
to RFC3164 because certain characters in the month field could result
in a negative month value. Let's fix it by turning the month to unsigned
to make sure we only accept months 0..11.
This should be backported to all branches.
Cross-compilation on m68k fails in ssl_sock_resize_passphrase_cache()
where the compiler noticed the SIZE_MAX passed to realloc() in the
error path and complained that it's larger than PTRDIFF_MAX. This can
be disabled with -Walloc-size-larger-than=SIZE_MAX but in practice we
can simply hide the value and keep the warning to detect real failures
elsewhere. Let's pass it through DISGUISE() and also take this
opportunity for doing that inside an unlikely() clause since it's never
supposed to happen.
acme_res_certificate() passes the httpclient response buffer to
ssl_sock_load_pem_into_ckch(), which will then call BIO_new_mem_buf(buf, -1).
The "-1" flag will make the OpenSSL PEM parser determine the length by
using strlen(). However, the httpclient populates the response buffer with
__b_putblk() without writing a trailing NUL to it. The byte at area[data]
is whatever data previously resided there in the memory pool.
Thus, a malicious or compromised ACME CA can perform an arbitrary-length
out-of-bounds read until hitting the first NULL byte past the response
body. The OpenSSL PEM loader will try to iterate to load the chain
certificates, thus the PEM-looking garbage found in freed memory chunks
can be erroneously loaded as additional intermediate certificates. The
presence of a single NUL inside the valid response body will result in
silent truncation of the certificate.
Make sure that the area[data] contains a terminating NULL before passing
the buffer to the parser. Fail on insufficient room for the NUL terminator.
No backport required: The ACME client has been added in 3.x and this
code path didn't exist in 2.x.
When the dedidacted buffer to store the command payload was added (c5ae0da62
"MEDIUM: cli: Make a buffer for the command payload"), an bug was
introduced. When the pattern finishing the command payload is found, it is
removed from the buffer. A NULL-bytes is added before it, skipping the
previous newline character.
It worked well in all cases before the commit above, because the commandline
was already parsed and was placed at the beginning of the cmdline buffer.
So, there is always a line before the payload.
Now, the payload is stored in a dedicated buffer. So there is nothing
preceeding it in a buffer. If the payload is empty, we cannot rewind to the
previous line to set the NULL-byte character. We must handle this case to
avoid integer underflow on the payload buffer length.
It is a 3.4-specific bug. No backport needed.
In hlua_socket_receive_yield(), when we try to get a line, the trailing CRLF is
stripped by decrementing the block length. The '\n' is first skipped, then,
possible a preceeding '\r'. But the block lenght is never checked. If an empty
line is returned, this leads to an integer underflow and most probably to a
crash because this length is used to copy data into a LUA string.
To fix the issue, the block length is now properly tested against 0 before
decrementing it.
This patch must be backported to all stable versions.
When parsing the agent-check reply, we first loop on the response to find
the newline character, to add a NULL-byte at the end of the line. However,
this loop is not bounded to the data available in the buffer. So it is
possible to read bytes outside the buffer and eventually write a NULL-byte
ouside the buffer.
So let's check for the end of the buffer when looping on the agent-check
reply.
This patch must be backported to all stable versions.
In dict_entry_unref(), the write lock on d->rwlock was only acquired after
decrementing the refcount. However, between the decrement and the lock,
another thread could increment it by calling dict_insert(). That could lead
to a UAF.
To fix the issue, the call to HA_ATOMIC_SUB_FETCH is moved inside the write
lock.
This patch must be backported to all stable versions.
In haproxy, when an Initial packet is received, a new connection may be
created and a DCID must be attributed. This CID is derived from the
original DCID used by the client in its first packet. This is an
optimization to avoid storing two CIDs values in the CID tree.
On CID lookup, if the DCID used is not found, derivation is performed
again. This should permit to retrieve the DCID node. However, this
operation is not performed as expected in quic_get_cid_tid(), as the
wrong value is used on the second lookup. Fix this function by using
derive CID for it. Note that retrieve_qc_conn_from_cid() performs the
same lookup but the bug was not present there.
The impact of this bug is relatively low as most clients send a single
Initial packet. Even in case of multiple packets in a single datagram,
this does not cause any issue as the current thread is assigned as
default.
This should be backported up to 2.8.
In ssl_sock_generate_certificate(), if the LRU cache for generated
certificates is used, the LRU tree is not unlocked on cache miss if the
certificate generation failed. So let's unlock it on error path.
The bug was introduced by the commit fbc98ebcd ("BUG/MEDIUM: ssl: fix error
path on generate-certificates"). So this patch must be backported with the
commit above, so to all stable versions.
In resolv_dn_label_to_str(), size for a dn label was stored into an integer
from a signed char without a cast to unsigned. So dn label with a size of
128 bytes or more become negative, skipping this way the copy loop and
desynchronizing input vs output.
In addition, the size of the destination string was only checked at the
begining, against the dn string length. But it must also be checked for
every dn label, to be sure. The dn string can be forged to copied more bytes
than expected.
This patch must be backported to all stable versions.
when appctx_rcv_buf() function was called to get data from the applet, but
to get zero bytes, nothing was performed and the function early
returned. However, we must at least take care to set SE_FL_WANT_ROOM if
necessary. Otherwise, if data are still blocked in the applet's output
buffer while the EOI/EOS are pending, the information can be reported to the
upper layer and remaining data can be lost.
Indeed, in such case, SE_FL_WANT_ROOM flag is here to specify the applet has
more data to deliver. Thanks to this flag, the stream will wait before
closing. But when appctx_rcv_buf() function is called, this flag is removed by
the stconn. It is the function responsibility to set it again when necessary.
This patch should fix second part of the issue #3366. It must be backported
to 3.0.
Completes qmux_parse_frm() to ensure every frames allowed by QMux
protocol are listed. For now, nothing is implemented except a CHECK_IF()
to report such events.
This is necessary to prevent a crash on abort. Frames not supported by
QMux should already have been rejected prior via qmux_is_frm_valid().
Handle reception of a MAX_STREAMS frame for unidirectional stream usage
when using QMux. This simply consists in using qcc_recv_max_streams() as
with QUIC protocol.
Ensure reception of STOP_SENDING via QMux protocol is properly handled.
This simply consists in using qcc_recv_stop_sending() which will update
the associated QCS if found.
The check on the OCSP response expire time is based on the "Next Update"
field of the response, converted by my_timegm function that returns a
time_t (signed long). It is then stored in the 'expire' field of the
certificate_ocsp structure which is typed as a signed long.
When loading an OCSP response, if the "Next Update" time is too far in
the future and we are running on a 32 bits machine, we might end up with
negative times ireturned by my_timegm, which make the comparison with
the current date fail and raises the "OCSP single response: no longer
valid." error message.
This problem typically happens in the ocsp_auto_update.vtc regtest since
the loaded OCSP response have a "Next Update" field in 2050.
This patch simply changes the type of the expire field to an unsigned
long since the 'my_timegm' function does not return '-1' in case of
error, contrary to the standard 'timegm' one.
Ths patch can be backported to all stable branches.
Use xprt_add_l6hs() at the end of connect_server() if selected MUX layer
relies on a temporary handshake prior to its initialization. This
functions is noop is SSL layer is active.
This change is necessary to support clear QMux on the backend side.
Recently defined <init_xprt> from mux_proto_list is used to render the
code as generic as possible.
Activates xprt_qmux layer if necessary via session_accept_fd(). This is
necessary to be able to support QMux in clear. This operation is noop if
SSL is active, as in this case xprt_qmux will be activated after the SSL
handshake completion.
To ensure MUX init is delayed when running with clear QMux, mask
CO_FL_WAIT_XPRT_L6 is added to test if the embryonic task must be
started instead.
Define a new connection flag mask CO_FL_WAIT_XPRT_L6. This will be used
to indicate that a XPRT layer is running on top of layer 6. For now,
only xprt_qmux implements this method of operation.
Extend get_alpn() for xprt_qmux layer. If lower layer does not implement
ALPN negotiation, return a statically default protocol value. Currently
this is set to "h3".
This change is required to support QMux in clear without SSL. In the
future, it could be useful to configure the default protocol, for
example by extending the syntax for the "proto" keyword.
When QMux protocol is used, xprt_qmux layer is setup after SSL handshake
completion but prior to the MUX initialization. Once transport
parameters exchange is successful, the layer is removed and the MUX is
started.
The layer setup operation was performed directly on ssl_sock_io_cb().
Simplify the code by extracting it in a dedicated function
xprt_add_l6hs(). The function is generic so the requested XPRT layer
must be passed as argument.
The code is mostly identical. One difference is that a check is
performed to ensure no SSL handshake is pending. If this is the case,
the function is a noop. This will become useful to support QMux
transparently both in clear or on top of SSL.
Another minor addition is that CO_FL_XPRT_READY flag is automatically
resetted by xprt_add_l6hs(). This allows the code to use
conn_xprt_start() standard function after XPRT init.
A recent patch has introduced <init_xprt> mux_proto_list member. This
allows to activate QMux on SSL handshake completion without explicit
"proto qmux" setting.
Thanks to this change, on SSL handshake completion it is not necessary
anymore to check for CO_FL_QMUX_* flags.
The various tcp_option_* converters rely on tcp_fullhdr_find_opt() to
find the option. However, the same bug as fixed in commit dbf471f99a
("BUG/MAJOR: net_helper: ip.fp infinite loop on malformed tcp options")
was also present there, by which an option of length 0 could be looped
over indefinitely. In practice this does not happen since such options
are not valid, but if passed encoded in an HTTP header for example, it
could possibly be passed.
While fixing it, let's check for length >1 in all 3 locations insteead
of only non-zero, since there's no point processing a malformed option
that wouldn't even be properly skipped.
This fix doesn't need to be backported, unless the ip.fp series is.
Thanks to @Vincent55 for reporting this issue.
When threads are disabled, "static __decl_spinlock(foo);" ends up as
"static;", causing a build warning when threads are disabled. We don't
need it to be static so let's drop "static" here. No backport is needed,
this is 3.4-only.
Released version 3.4-dev13 with the following main changes :
- BUG/MINOR: backend: correct parameter value validation in get_server_ph_post()
- BUG/MINOR: config/dns: properly fail on duplicate nameserver name detection
- BUG/MEDIUM: dns: fix long loops in additional records parse on name failure
- BUG/MEDIUM: resolvers: fix name compression pointer validation in resolv_read_name()
- BUG/MEDIUM: dns: fix memory leak of sockaddr in dns_session_init() error path
- CLEANUP: proxy: fix tiny mistakes in parse error messages
- CLEANUP: dns: fix misleading error messages in dns_stream_init()
- BUG/MINOR: server: better handling of OOM in srv_set_fqdn()
- BUG/MINOR: servers: use proper source of pool_conn_name in srv_settings_cpy()
- BUG/MEDIUM: server/cli: unlock server lock on failure in cli_parse_set_server
- BUG/MINOR: resolvers: fix dangling list pointer in resolvers_new() error paths
- BUG/MINOR: dns: fix dangling dgram pointer on dns_dgram_init() failure path
- BUG/MINOR: proxy: use proxy_drop() in parse_new_proxy() error path
- CLEANUP: resolvers: properly initialize the sample in resolv_action_do_resolve()
- BUG/MINOR: resolvers: report the expression error in the do-resolve() action parser
- BUG/MINOR: resolvers: fix leaked dgram and dns_ring struct in parse_resolve_conf()
- BUG/MINOR: resolvers: fix leaked fields on cfg_parse_resolvers() error paths
- BUG/MINOR: resolvers: fix missing task_idle destruction in resolvers_destroy()
- CLEANUP: proxy: fix duplicate declaration of cli_find_frontend in proxy.h
- CLEANUP: address a few typos and copy-paste errors in httpclient and dns
- DOC: internal: add a few rules about internal core principles
- BUG/MINOR: session/trace: use distinct flags for SESS_EV_END and _ERR
- CLEANUP: stick-table: uniformize the different action_inc_gpc*()
- REGTESTS: do not run quic/tls13_ssl_crt-list_filters in quic openssl compat mode
- REGTESTS: quic/issuers_chain_path: do not forget to enable QUIC compat mode
- BUG/MINOR: sock: store the connection error status
- BUG/MINOR: check: properly report errno in chk_report_conn_err()
- CLEANUP: tcpcheck: mention that we're a bit far for a sync errno
- BUG/MINOR: jwt: fix possible memory leak in convert_ecdsa_sig() error path
- CLEANUP: jwe: fix theoretical overflow in AAD length calculation
- DOC: config: further clarify that resolvers "default" exists
- MINOR: proxy: remove the experimental status on dynamic backends
- BUG/MEDIUM: limits: properly account for global.maxpipes in compute_ideal_maxconn()
- BUG/MINOR: jws: fix OpenSSL 3.0 version check from > to >=
- BUG/MINOR: jws: Add missing return value check (EVP_PKEY_get_bn_param)
- BUG/MINOR: server: Properly handle init-state value during haproxy startup
- BUG/MINOR: httpclient-cli: Destroy http-client context if failing to start it
- BUG/MEDIUM: h1: Skip all h2c values from Upgrade headers during parsing
- BUG/MINOR: h1: Don't mask websocket protocol if multiple protocols used
- MINOR: haterm: Don't init haterm master pipe if not used
- CLEANUP: haterm: Remove "(too old kernel)" from warning message during init
- BUG/MINOR: httpclient-cli: fix uninit variable in error label
- MINOR: mux: Rename the "token" from mux_proto_list to mux_proto
- MEDIUM: connections: Use both mux_proto and alpn to pick a mux
- MINOR: connection: define conn_select_mux_fe()
- MINOR: connection: define conn_select_mux_be()
- MINOR: connection/mux_quic: add MUX <init_xprt> field for QMux handshake
- MINOR: proxy/server: reject TCP ALPN h3 without experimental
- MEDIUM: ssl: allow h3/QMux negotiation without explicit proto
- BUG/MINOR: server: accept server IDs above 2^31 and clarify error message
- BUG/MINOR: backend: fix balance hash calculation when using hash-type none
- MINOR: server: support hash-key id32 for a cleaner distribution
- MINOR: backend: support hash-key guid for a stabler distribution
- MINOR: startup: support unprivileged chroot if possible
- MEDIUM: startup: add automatic chroot feature
- MINOR: h2: explain committed_extra_streams dec on h2_init() error
- OPTIM: h2: do not update committed streams if elasticity disabled
- MINOR: mux_quic: implement basic committed_extra_streams accounting
- MINOR: quic: use stream elasticity value for initial advertisement
- MINOR: mux_quic: define ms_bidi_rel QCC member
- MAJOR: mux_quic: support stream elasticity during connection lifetime
- BUG/MEDIUM: servers: Store the connection hash with the parameter cache
- BUG/MINOR: prevent conn leak in case of xprt_qmux init failure
- BUILD: traces: set a few __maybe_unused on vars used only for traces
- BUILD: traces: add USE_TRACE allowing to disable traces
- MINOR: startup: do not execute chroot() when "/"
- MEDIUM: startup: warn when chroot is not set for root
- BUG/MEDIUM: servers: Don't forget to set srv_hash when needed
- DOC: fix typo on QUIC stream.max-concurrent reference
- BUG/MINOR: mux_quic: do not exceed stream.max-concurrent on backend side
- BUG/MINOR: htx: Fix value of HTX_XFER_HDRS_ONLY flag
- MEDIUM: htx: Improve htx_xfer API to not count HTX meta-data
- BUG/MEDIUM: applet: Fix transfer of HTX data to the applet
- BUG/MEDIUM: htx: Alloc a chunk of right size in htx_replace_blk_value()
- MEDIUM: stick-tables: Avoid freeing elements while holding a lock
- MINOR: intops: add a multiply overflow detection for ulong and size_t
- CLEANUP: tree-wide: use array_size_or_fail() in array size for allocations
- DOC: update supported gcc and openssl versions in INSTALL
Instead of relying on malloc(n*size), we now pass array_size_or_fail(n,m)
so that it becomes possible to detect overflow. This is particularly
interesting for global settings that might be set large enough to cause
overflows on 32-bit systems for example, resulting in small values that
then cause trouble. Now the overflow will be detected at allocation time.
Around 25 locations were updated.
Sometimes we'd like to know if some products overflow, so let's add a
pair of functions for this, for ulong and for size_t. For recent enough
compilers (gcc >= 5, clang >= 3.4) we just use __builtin_mul_overflow()
otherwise we rely on a division and a comparison before performing the
operation.
A third function, array_size_or_fail() computes the size of an array
of m elements of n bytes each, and returns the total size if it fits
in a size_t, otherwise ~0 if it does not so that passing this to
malloc() or any other variant would fail by trying to exhaust the
entire memory space.
In stksess_trash_oldest(), and process_tables_expire(), avoid freeing
elements while holding two locks, as it could be very costly.
Instead, build a linked list of elements to be free'd, and do so once we
no longer hold any lock.
This may help with github issue #3380, and may be backported to 3.3.
Since support for large buffers was added, we must be careful when chunks
are allocated. Indeed, depending on the context a large chunks may be
required if data are copied from a large buffer.
In htx_replace_blk_value() function, when a defragmentation is necessary,
the data to be replaced are copied to a chunk before the
defragmentation. However, I forgot to get large chunk when necessary by
calling alloc_trash_chunk_sz() instead of alloc_trash_chunk(). Because of
this issue, it is possible to copy data to a too small chunk, leading to a
crash.
So let's fix the issue.
Thanks to Vincent55 for finding and reporting this.
No backport needed.
appctx_htx_snd_buf() function is relying on htx_xfer() function to transfer
HTX blocks when a swap of buffers is not possible. However, it was not
properly using this function.
Indeed, originally htx_xfer() was designed to transfer blocks with a limit,
the <count> parameter, which included the blocks payload and the
meta-data. It was aligned with all calls, except for the transfer of HTX
data to the applet, in appctx_htx_snd_buf() function. In that case, the
<count> parameter is the amount of data forwarded by the stream to the
applet. So meta-data are not included.
Thanks to the previous commit ("MEDIUM: htx: Improve htx_xfer API to not count
HTX meta-data"), it is now possible to instruct htx_xfer() function that
<count> parameter does not include the meta-data.
Because of this bug, crashes can be experienced when transferring HTX data
to an applet. At first glance, lua HTTP applets and the http client are
concerned.
Stable versions from 3.3 to 3.0 are also affected. But this patch cannot be
backported as is because htx_xfer() function does not exist on these
versions.
Thaks to Yon Harlicaj for finding and reporting this.
(https://x.com/nvmb3r - https://www.linkedin.com/in/eljon-harlicaj/)
This patch add the ability to the htx_xfer() function to transfer data
without acounting the meta-data. By default, the <count> variable includes
the meta-data. But by setting the flag HTX_XFER_NO_METADATA, It is possible
to transfer HTX blocks without count meta-data. In that case, <count> will
not contain the blocks meta-data and the return value will not include them.
HTX_XFER_* flags must be declared as a bitfield. However, value of
HTX_XFER_HDRS_ONLY was set of 0x03 while it should be 0x04. So let's fix it.
This patch must be backported where the htx_xfer() function was backported
(5ead611cc "MEDIUM: htx: Add htx_xfer function to replace htx_xfer_blks").
Fix usage of stream.max-concurrent QUIC setting on the backend side.
Contrary to frontend connections, this limit must be enforced by QUIC
MUX directly. This is necessary as the peer may allow a larger number of
concurrent streams via its flow control.
First, QUIC TP initial max bidi streams value is now set to 0. This is
fine as only the HTTP/3 client is expected to open bidirectional
streams.
The most important changes is performed in qcm_avail_streams(). The
value first depends on the peer flow control. Now, it is further reduced
if necessary to not exceed the configured BE stream.max-concurrent.
Note that this new behavior may further increases current limitation on
QUIC BE reuse when a QCS instance is kept while its upper stream layer
is detached. In this case there is a risk that the connection is not
reinserted in the correct server pool, as an idle or avail one.
This is a breaking change as BE stream.max-concurrent keyword setting
meaning is changed in effect. However, this does not necessitate extra
warnings as the previous usage was in effect useless. Furthermore, QUIC
on the backend side is still considered as experimental.
This can be backported up to 3.3.
Commit 8aa854ab26a7daa613a17548f1fe1d0adb8cf61b made it so we'd store
the hash corresponding to the server parameters, so that we could detect
if we're still talking to the same server, and not use those parameters
if not.
However, when updating those parameters, we forgot to store the new
hash, which would result in the new parameters never be used, and
breakling 0RTT.
Fix that by properly update the hash when needed.
This should be backported when 8aa854ab26a7daa613a17548f1fe1d0adb8cf61b
is backported.
We're still regularly seeing insecure configs where chroot is missing.
Now that we have "chroot auto", there's no excuse for not knowing where
to chroot, so let's detect that we're starting as root, detect that the
process is allowed to chroot (i.e. no capability issue, or some hardened
containers), and if no chroot is set, let's emit a warning explaining how
to silence it, i.e. either "chroot auto" or "chroot /".
Most likely we'll start using "chroot auto" by default in 3.5 if no
usability issue is reported.
We'll recommend to use "chroot /" to explicitly disable chroot, however
there might be configurations where it would cause problems to just issue
the syscall (typically some hardened containers), so let's make sure that
"chroot /" is a nop in this case.
This reduces the total code size by 6-10% and speeds up the build a
bit. It can be further reduced by disabling the trace decoding code
inside certain subsystems like muxes. But at least like this it will
help users on small systems to reduce the footprint when not needed
by explicitly passing USE_TRACE=0 (they remain enabled by default).
In case of XPRT_QMUX init failure on the frontend side, the connection
must immediately be released. This is not the case on the backend side
as a stream can supervize the connection lifetime.
This patch performs the connection free via conn_complete_session(). As
conn is flagged with CO_FL_ERROR, this will automatically fail and
invoke session_kill_embryonic(), which ensures the session and its
connection are both freed as wanted in this case.
No need to backport.
When we store the negociated server parameters, such as the ALPN, also
store the calculated hash with the connection. If it is different, as
can happen because the IP address is different because set-dst was used,
we certainly do not want to reuse the information in the cache,
otherwise we could end up using the wrong ALPN and mux.
That means we already have to calculate the hash in connect_server()
now, while before we would not do it for Websockets, if we could not do
connection reuse, as that's all the hash was used for.
This should fix Github issue #3386
This should be backported as far as 3.2.
qcc_release_remote_stream() is called each time a remote stream is
closed. Flow control accounting is updated and when necessary, a
MAX_STREAMS_BIDI frame is prepared to allow the peer to initiate new
streams.
This patch extends stream elasticity features with the QUIC bidirection
stream flow control mechanism. The announced value can now be possibly
reduced depending on conn_calc_max_streams().
The first step is to decrement closed streams from the global committed
extra streams total. This must be performed conn_calc_max_streams() to
ensure the calculation will be valid.
Then, there is two cases depending on conn_calc_max_streams() result. If
the value is less than the peer still remaining stream window, nothing
more is performed. If the opposite case, flow control must be increased
and a MAX_STREAMS_BIDI frame is prepared, with the value adjusted to not
exceed the stream elasticity limit. Global extra streams total is then
finally incremented.
This calcul also ensures that when all streams are closed, global extra
streams accounting operations are decremented by 1, as a connection
always has access to one stream which is excluded from the global total.
Note that if stream elasticity is not active, flow control increases
principle is unchanged and remains statically performed.
This patch is labelled as major as it complexifies bidirectional stream
flow control mechanisme. This is a sensitive operation as there is a
risk of connection freeze if flow control updates are inadvertently
skipped.
Add a new QCC member <ms_bidi_rel>. This represents the number of
concurrent streams advertised similarly to ms_bidi, but as a relative
value.
This patch does not introduce any functional change. For now,
<ms_bidi_rel> will be equal to <ms_bidi_init>. However, with the
implementation of stream elasticity and dynamic adjustment for
concurrent max-streams-bidi, the former will be required to keep the
last advertised value.
When stream elasticity is active, the maximum number of concurrent bidi
streams advertised via transport parameters is now reduced depending on
the connection load. This is implemented via conn_calc_max_streams()
which returns the value to use.
This is not applied on listeners with enabled 0-RTT. Indeed, for such
connections, clients are expected to reuse the previously seen transport
parameters. The server on the other hand must not decrease several
values on the newly advertised params, in particular for the maximum
number of concurrent bidi streams. The simplest way to prevent 0-RTT
failure is to not mix stream elasticity with it.
Note that the 0-RTT limitation is only applied for the initial value :
during the connection lifetime, stream elasticity can still be used by
the MUX to dynamically reduce the stream window. This will be
implemented in a future patch.
Account QUIC frontend connections into committed_extra_streams when
stream elasticity setting is active. This is performed in QCC init and
release functions.
This patch has no impact on QUIC subsystem for now. Connections will
still allow a static number of concurrent streams based on
tune.quic.fe.stream.max-concurrent. However, this has a direct
repercussion on H2 subsystem, as a higher count of QUIC connections will
reduce the concurrent streams allowed there.
When streams-elasticity is enabled in the configuration, H2 mux is
responsible to update the global committed_extra_streams value.
Adjust these operations to ensure they are skipped if streams-elasticity
is disabled, which is the current default. This prevents unnecessary
atomic operations in this case.
No need to backport unless streams-elasticity feature is picked in older
releases.
h2_init() is now responsible to increment committed_extra_streams for
new frontend connections, in relation to the newly implemented
stream-elasticity feature. In case of an early error, a mirroring
decrement is executed on fail_stream label.
However, for now this error label can only be selected via BE conns. In
fact, it's not yet possible for h2_init() to fail after the extra
streams increment.
However, the decrement operation is kept to prevent any omissions in
case of future evolutions of h2_init() error path. To prevent reporting
of a possible dead code, add an extra comment which summarizes the
situation.
It is now possible to use "chroot auto" in the configuration. This lets
haproxy create an anonymous (cleaned up after the process terminates)
and read-only directory for chroot. This directory is created in /tmp;
we might want to support creating it in a different directory in the
future, either by respecting $TMPDIR or by allowing an optional
directory after the "auto" keyword.
When server fleets are constantly updated, using a stable distribution
across a bunch of load balancers can be convenient. The addr and port
already provide a bit of this but for situations were addresses might
differ between sites or change dynamically this does not work. The guid
is perfect for this because by definition it's supposed to designate a
single server and be unique. So when two servers anywhere have the same,
the tool that provisionned them promises that they are the same server.
So here we introduce "hash-key guid" which performs a 32-bit hash on
the GUID value. When no guid is provided, a fallback is performed on
ID, as is done for other keys.
The "id" hash-key scales the ID by a factor of 16 that tries to leave
room between the nodes on the 32-bit space to permit smooth weight
variations (e.g. during slowstart). However this does not deal well
with overlaps between server IDs. For example, assigning IDs that are
only multiples of 256 million to 16 servers yields traffic only on
one since in practice they all have the same 28 lower bits.
The new "id32" hash key bridges this gap by using the full 32-bit ID
of the server as the key. On the other hand, the user must be careful
not to switch the hash function to "none" when using incremental IDs
because in this case they might be very poorly distributed. But this
can be convenient for automated provisionning systems which assign
IDs themselves, as the full 32 bits are used now.
The "hash-type xxx none" is broken for keys that are not in type string
because the sample fetch call casts them to SMP_T_BIN, that tends to
preserve the original format (integers, IP addresses etc), but the
gen_hash() function in case of BE_LB_HFCN_NONE expects to read a string
representing a number, that it parses to retrieve the value, and just
fails on many binary types. For example, the following just always
returns key 0:
balance hash rand()
hash type consistent none
An ugly workaround is to make sure the expression returns a string, for
example this:
balance hash rand(),concat()
hash type consistent none
In order to fix most cases here, we force the conversion to type string
when using BE_LB_HFCN_NONE, but a better approach would require a larger
rework and split gen_hash() or change it to accept an integer as well,
so that the caller could cast to SMP_T_INT for BE_LB_HFCN_NONE and pass
the resulting number already parsed with the least information loss. In
this case even IPv4 addresses would be preserved.
The current approach at least addresses the initially envisioned use
cases, and the limitations have been added to the doc. This can be
backported to 3.0 though it's not really important.
Due to the check of the stored value instead of the parsed one, it was not
permitted to use server IDs above 2^31 while they are perfectly possible.
Let's refine the parsing and also update the error message to indicate the
range. The doc was also refined to reflect the relation with hash-key.
This may be backported though it wouldn't have any effect on working
configs.
Implements automatic selection of QMux MUX if "h3" ALPN has been
negotiated on top of TCP/SSL.
The first part of this change is to define "alpn" member of
mux_proto_list. This is necessary so that conn_get_best_mux_entry() can
select it when "h3" has been chosen. As a side-effect, this also
automatically sets a default ALPN to "h3" for bind lines with "proto
qmux".
The most important change is to adapt the SSL layer. On handshake
completion, the eligible MUX is retrieved via conn_select_mux_fe/be()
functions. If xprt_qmux is required by it, MUX init is delayed and QMux
handshake is started first.
This last change is necessary as connection flags CO_FL_QMUX_RECV/SEND
are only set if "proto qmux" is explicitely set. In case xprt_qmux is
activated via pure ALPN negotiation, these flags are also set on
xprt_qmux_init(). This is mandatory to ensure emission/reception of QMux
transport parameters will be performed as expected.
Add a postparsing check on TCP ALPN bind and server setting. An error is
reported if the token "h3" is present and expose-experimental-directives
is not globally activated. This ensures that QMux protocol won't be
selected if experimental features are not explicitely requested.
The check is not performed though if "proto qmux" is explicitely
defined, as this setting already checks for experimental support.
Currently, it's not possible to activate QMux without any explicit
"proto qmux" config. However, this will be implemented in a next patch,
so this check will become necessary.
The first part of this patch defines a new mux_proto_list field named
<xprt_init>. This allows to define an extra XPRT layer which should be
activated first prior to the MUX creation both on frontend and backend
sides.
This is immediately used for QMux mux_proto_list to require XPRT_QMUX
handshake. With this change, activation of QMux connection flags in
session_accept_fd() and connect_server() are adjusted to take into
account <init_xprt> field. This approach is much more evolutive than
relying on the previous MUX name.
Change in connect_server() will also be necessary to support QMux
activation on a TCP server with h3 ALPN without explicit "proto qmux".
This guarantees that MUX initialization is delayed after QMux handshake.
This patch is similar to the previous one but this time for backend
connections. The MUX selection code is directly extracted from
conn_install_mux_chk() and conn_install_mux_be().
Define a new function conn_select_mux_fe().
The objective is to have a preliminary function to determine the MUX
which will be used without initializing it. This will be useful for MUX
which relies on a specific XPRT handshake prior to its startup, which is
the case for QMux protocol.
The code of conn_select_mux_fe() is identical to the beginning of
conn_install_mux_fe() with a similar MUX selection logic. However,
connection MUX initialization is not performed in this case. In a future
patch, both functions should be merged together to reduce code
duplication.
In conn_get_best_mux() and conn_get_best_mux_entry(), the mux name was
provided sometimes based on the "proto" directive, sometimes based on
the ALPN, but in any case, it was compared again the mux_proto_list
mux_proto field. This is not correct, as ALPN can be different from the
internal mux_proto. So enhance those functions so that they wll accept
an ALPN as well. If a mux_proto is provided, that will be used, if not,
and if an ALPN is provided, then that will be used, and compared against
the ALPN provided by the mux, if any.
In struct mux_proto_list, rename the "token" field to "mux_proto". That
field should only be used to match the name provided in the "proto"
directive, and it will be soon.
This should be a no-op.
The following patch fixes a leak in case of httpclient_start() failure
in the httpclient_cli code by adding httpclient_destroy() call on error
path.
c53256adbc
BUG/MINOR: httpclient-cli: Destroy http-client context if failing to start it
However, error label may be selected prior to httpclient allocation if
CLI arguments are incorrect. This can cause a crash due to a deferencing
of an uninitialized variable. This has been detected via a compilation
error :
src/httpclient_cli.c: In function 'hc_cli_parse':
src/httpclient_cli.c:162:2: error: 'hc' may be used uninitialized in this function [-Werror=maybe-uninitialized]
162 | httpclient_destroy(hc);
| ^~~~~~~~~~~~~~~~~~~~~~
This must be backported along the above patch, which is scheduled up to
the 2.6 release.
During initialization of the haterm master pipe, If its size is limited
(lower than the configured one * 5/4), a warning is emitted. In this
warning, it is specified this happened because the kernel is too old. But it
is unrelated. So let's remove this part.
There is no reason to initialize the haterm master pipe if haterm is not
used. So now, it is only performed if a non-disabled haterm frontend is
found. To do so, in addition to test the proxy's flags and capabilities, we
also check if "stream_new_from_sc" points on "hstream_new".
During H1 message parsing, the Upgrade header values are checked to detect
"websocket" prototol, to properly handle websocket upgrades between H1 and
H2 and to possibly reject messages if mandatory headers are missing.
However, the flag is reset for each new Upgrade header and the information
may be lost. So never reset it.
This patch must be backported as far as 2.4.
During the H1 message parsing, the Upgrade header values are checked to
detect "h2c" and "h2" tokens and skip them. To do so, we rely on
H1_MF_UPG_H2C flag, set during the parsing. And during the request
post-parsing, if this flag was set, all Upgrade headers are removed.
This was fixed by the commit 7b89aa5b1 ("BUG/MINOR: h1: do not forward h2c
upgrade header token").
However, there are two issues here and the commit above must be refined.
First, the flag is reset for each new Upgrade header. So "h2c" or "h2"
tokens will be properly detected if all tokens are set on the same Upgrade
header. But if splitted on several headers, previously detected tokens will
be hidden by a next ones.
Concretly, the following will be properly caught
Connection: upgrade
Upgrade: foo, h2c, bar
But then following not:
Connection: upgrade:
Upgrade: foo, h2c
Upgrade: bar
Then, when a "h2c" or "h2" token is finally reported, all Upgrade headers
are removed, regardless other tokens.
So, to fix the both issues, everything is now handled during the message
parsing by skipping "h2c" and "h2" tokens, rebuilding the Upgrade header
value without then offending tokens. The same was already performed for the
Connection header, to skip "keep-alive" and "close" value. So it is not a so
fancy change.
Thanks to this change, it is no longer necessary to handle H1_MF_UPG_H2C
during the request post-parsing. And in fact, this flag is no longer
necessary. So let's remove it too.
Thanks to Vincent55 for finding and reporting this.
This patch must be backported as far as 2.4.
When the call to httpclient_start() failed, it is the caller responsibilty
to destroy the http-client context by calling httpclient_destroy(). It is
performed at several places but it was missing in the httpclient_cli
code. So let's fix it.
This patch must be backported as far as 2.6. On 3.2 and lower, it must be
applied on http_client.c.
Unlike stated in the configuration manual, the server 'init-state' parameter
was not evaluated during haproxy startup/reload. After a review, it appeared
there were also issues if combined with the 'track' parameter. In addtition,
this parameter was only evaluated when health-checks were enabled for the
server, leading to unexpected behavior if the serve settings are dynamically
changed via the CLI.
To fix those issues, behavior of the 'init-state' parameter was slightly
adapted. It is always evaluated, even when there is no running health-checks
for the server. An error is reported if the 'track' parameter is also
defined. Both cannot work together.
In addition, the "none" state was introduced to be able to restore the
default behavior. It will be especially useful when the parameter is
inherited from a 'default-server' directive.
This patch should fix the issue #3298. It must be backported as far as 3.2.
Three #if directives used > 0x30000000L which excluded OpenSSL 3.0.0
exactly from the modern code path, treating it as pre-3.0. Changed all
three to >= 0x30000000L to match jwe.c and openssl-compat.h conventions.
This affects EC key thumbprint generation, RSA JWK generation, and
JWS algorithm detection for OpenSSL 3.0.0.
Starting a config with maxpipes and no maxconn always ended up in error
because the number of FDs needed for pipes was not deduced from the total
number of FDs when calculating maxconn, and was later found to exceed the
total number of allocatable FD during final checks.
When global.maxpipes is set, it must be used during compute_ideal_maxconn()
so that it's properly deduced.
Without this, just having "maxpipes 500" in a config prevents it from
starting. With the fix, it properly starts with a maxconn adjusted
depending on the number of splice-enabled proxies.
This should be backported, theoretically everywhere, but preferably
progressively. The following config should fail on affected versions
and load with fixed ones:
global
maxpipes 500
frontend srv1
bind :8001
As initially planned, if no trouble was reported on dynamic backend
commands on the CLI, the experimental status could be dropped before the
release. The feedback was not very broad, but was conclusive in that the
operations work as expected and the current syntax can be preserved even
for future evolutions. So we can drop the experimental status.
It was explained in the general presentation of resolvers but not in
the "resolvers" keyword description itself, which might be where users
could be looking for that info, so let's quickly repeat that info there.
The expression items[JWE_ELT_JOSE].length << 3 performs the shift on an
unsigned int (32-bit) before being cast to uint64_t instead of after.
This means that we don't cover for a possible overflow (which would
never happen as it would need a header length beyond 512MB). At least
fixing it will avoid code check reports.
The allocated ec_R and ec_S were not released in case one of the two
would fail to be allocated/created, and would cause a memory leak. Let's
add the missing BN_free(). This may be backported to 2.4.
The collection of errno in tcpcheck_eval_connect() and tcpcheck_main()
is quite far from the production location, and the risk of having a
zero errno is definitely not null. Tests show that this works, so
better not try to fix something not broken, but at least place a
comment there indicating that it's not necessarily super-reliable.
This would need to be revisited the day we finally store errno in
the connection.
When in 2.2, with commit c8dc20a825 ("BUG/MINOR: checks: refine which
errno values are really errors."), errno reporting was refined, an
extra check was added before calling retrieve_errno_from_socket(), and
by mistake the test on !errno got inverted so that we only call the
function to retrieve the error from the socket when errno is set!
The first test in the function detects it and returns without changing
anything, so this didn't have much effect, however when errno is not
set (certain call places purposely pass zero so that getsockopt() is
used), this wasn't called so the error wasn't reported. Apparently it
only happened when called from process_chk_conn() after an async
error was detected, so probably just cases where POLLERR is reported,
which remains infrequent.
Let's fix the direction of this flag. It can be backported if needed
but it's unlikely anyone really noticed.
When an async connect() fails in sock_conn_check(), it returns an errno
that will not be retrieved later by a subsequent getsockopt(SO_ERROR).
The problem is that this errno is then definitely lost. This is visible
in the 4be_1srv_smtpchk_httpchk_layer47errors regtest that fails on
certain systems (e.g. glibc 2.31 on arm32 running Linux 6.1), where the
connect() error is systematically lost and the "Connection refused" is
never seen in the check status. It also matches a few random reports of
the past indicating that the connection error was sometimes not reported
in the stats page in front of a down server.
Ideally we should store errno in connections as soon as the error is
seen. However this would require significant changes that are not
acceptable yet for 3.4 nor stable releases. A more acceptable fix is to
make use of the extra CO_ER_* flags set by conn_set_errno() as soon as
the error is detected. This will recognize a sufficiently large number
of errors and the check status will report them (here we'll have
"ECONNREFUSED" in the check). Note that on systems where the error is
seen synchronously, we can have "ECONNREFUSED (Connection refused)",
but this is not a problem.
This fix adds the missing conn_set_errno() call to sock_conn_check(),
that is thus sufficient to catch this error. In addition, the two
affected regtests were updated to search for ECONNREFUSED here.
This might be backported to older releases if users request it, but it
is probably not necessary.
This test is compatible with QUIC_OPENSSL_COMPAT but the "limited-quic"
directive was not set, making it fail on older libs with no QUIC support
despite being declared as compatible.
While action_inc_gpc1() explicitly checks if s->stkctr or sess->stkctr
are set since 2.8 with commit 6c0117168 ("MEDIUM: stick-table: set the
track-sc limit at boottime via tune.stick-counters"), action_inc_gpc0()
and the generic action_inc_gpc() still stuck to the old approach of not
checking them, causing confusion when reviewing the code.
Upon closer inspection, the only case where the pointer may be NULL is
when global.tune.nb_stk_ctr is zero, which happens when the global
section contains "tune.stick-counters 0". However in this case, the
config parser "parse_inc_gpc()" will reject any reference to any stick
counter, so in theory there is no problem.
Regardless, the difference of treatment between sibling functions remains
confusing and the check is cheap, so let's generalize it, it will save a
future reader from the need to inspect stream_new() and session_new().
Session traces were brought in 3.1 by commit abb07af67 ("MINOR:
session/trace: enable very minimal session tracing") though there was
an issue, because SESS_EV_END and SESS_EV_ERR have the same value (it's
a copy-paste mistake).
This can be backported to 3.2.
The new file core-principles.txt quickly enumerates a number of rules
and invariants across the project. These can be used as quick reminders
as well as basic rules for reviews. It's still lacking a lot of info but
should be a good start.
The function cli_find_frontend was declared twice identically at lines 98-99
of include/haproxy/proxy.h. The second declaration should have been for
cli_find_backend, which is defined in src/proxy.c and used in several places
but was missing from the header's exported symbols.
This is a simple copy-paste mistake where line 99 duplicated line 98 verbatim
instead of declaring cli_find_backend.
When destroying a stream-based DNS nameserver, task_req and task_rsp
were destroyed but task_idle was missed, causing a task object leak.
This doesn't necessarily have to be backported since it's only upon
exit that it is visible.
cfg_parse_resolvers() has many error paths on allocation failure when
parsing "nameserver". These paths handle their own cleanup instead of
centralizing it. The result is that some errors paths leak some fields.
The most complex ones are the strdup() failures which require to check
for stream or dgram to figure what to free. These can be detected via
ASAN on a dummy strdup() allocation failure:
Indirect leak of 131080 byte(s) in 1 object(s) allocated from:
#0 0x7f0b7ed1f0ab in malloc (/usr/lib64/libasan.so.8+0x11f0ab)
#1 0x000000c73e19 in dns_ring_new src/dns_ring.c:59
#2 0x000000af1848 in dns_dgram_init src/dns.c:480
#3 0x000000922005 in cfg_parse_resolvers src/resolvers.c:3792
#4 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#5 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#6 0x000000447e8c in main src/haproxy.c:3474
#7 0x7f0b7e02ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#8 0x7ffd35f1531c ([stack]+0x2031c)
Indirect leak of 304 byte(s) in 1 object(s) allocated from:
#0 0x7f0b7ed1ea23 in calloc (/usr/lib64/libasan.so.8+0x11ea23)
#1 0x000000af1681 in dns_dgram_init src/dns.c:468
#2 0x000000922005 in cfg_parse_resolvers src/resolvers.c:3792
#3 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#4 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#5 0x000000447e8c in main src/haproxy.c:3474
#6 0x7f0b7e02ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#7 0x7ffd35f1531c ([stack]+0x2031c)
Indirect leak of 104 byte(s) in 1 object(s) allocated from:
#0 0x7f0b7ed1ea23 in calloc (/usr/lib64/libasan.so.8+0x11ea23)
#1 0x000000921f83 in cfg_parse_resolvers src/resolvers.c:3772
#2 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#3 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#4 0x000000447e8c in main src/haproxy.c:3474
#5 0x7f0b7e02ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#6 0x7ffd35f1531c ([stack]+0x2031c)
Indirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x7f0b7ed1f0ab in malloc (/usr/lib64/libasan.so.8+0x11f0ab)
#1 0x000000c73e09 in dns_ring_new src/dns_ring.c:55
#2 0x000000af1848 in dns_dgram_init src/dns.c:480
#3 0x000000922005 in cfg_parse_resolvers src/resolvers.c:3792
#4 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#5 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#6 0x000000447e8c in main src/haproxy.c:3474
#7 0x7f0b7e02ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#8 0x7ffd35f1531c ([stack]+0x2031c)
Indirect leak of 15 byte(s) in 1 object(s) allocated from:
#0 0x7f0b7ed18e20 in strdup (/usr/lib64/libasan.so.8+0x118e20)
#1 0x00000092203b in cfg_parse_resolvers src/resolvers.c:3798
#2 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#3 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#4 0x000000447e8c in main src/haproxy.c:3474
#5 0x7f0b7e02ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#6 0x7ffd35f1531c ([stack]+0x2031c)
This should be completely reworked so that the cleanup is performed in
a central place, as the risk to get it wrong remains high.
This patch does the minimal changes to clean this up. It does not need
to be backported since it only triggers on boot OOM.
Some strdup() failures in parse_resolve_conf() do not release everything
due to the way the function is built, resulting in leaks on error that are
caught by ASAN:
Direct leak of 304 byte(s) in 1 object(s) allocated from:
#0 0x7fe74231ea23 in calloc (/usr/lib64/libasan.so.8+0x11ea23)
#1 0x000000af1681 in dns_dgram_init src/dns.c:468
#2 0x00000091cbbf in parse_resolve_conf src/resolvers.c:3559
#3 0x00000092179e in cfg_parse_resolvers src/resolvers.c:3815
#4 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#5 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#6 0x000000447e8c in main src/haproxy.c:3474
#7 0x7fe74162ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#8 0x7ffc0a43e31f ([stack]+0x2031f)
Indirect leak of 131080 byte(s) in 1 object(s) allocated from:
#0 0x7fe74231f0ab in malloc (/usr/lib64/libasan.so.8+0x11f0ab)
#1 0x000000c73e19 in dns_ring_new src/dns_ring.c:59
#2 0x000000af1848 in dns_dgram_init src/dns.c:480
#3 0x00000091cbbf in parse_resolve_conf src/resolvers.c:3559
#4 0x00000092179e in cfg_parse_resolvers src/resolvers.c:3815
#5 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#6 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#7 0x000000447e8c in main src/haproxy.c:3474
#8 0x7fe74162ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#9 0x7ffc0a43e31f ([stack]+0x2031f)
Indirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x7fe74231f0ab in malloc (/usr/lib64/libasan.so.8+0x11f0ab)
#1 0x000000c73e09 in dns_ring_new src/dns_ring.c:55
#2 0x000000af1848 in dns_dgram_init src/dns.c:480
#3 0x00000091cbbf in parse_resolve_conf src/resolvers.c:3559
#4 0x00000092179e in cfg_parse_resolvers src/resolvers.c:3815
#5 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#6 0x0000009e0a39 in read_cfg src/haproxy.c:1142
#7 0x000000447e8c in main src/haproxy.c:3474
#8 0x7fe74162ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#9 0x7ffc0a43e31f ([stack]+0x2031f)
SUMMARY: AddressSanitizer: 131448 byte(s) leaked in 3 allocation(s).
Let's free the dgram and the dns ring. This can be backported though it's
not important as it only happens on OOM condition during boot.
When an expression is used for do-resolve(), an error may be reported.
Unfortunately it was scratched and replaced by the do-resolve() error,
leaving no chance to know exactly what was wrong. Let's report the
contents of the error when available. It will indicate identifiers that
are not found or invalid ranges or types being used.
This can be backported to all versions.
The sample used to pass the IP address only had its data, px, sess and
strm fields initialized before being passed to vars_set_by_name(). It
turns out that this latter one doesn't seem to touch ctx, flags nor opt
but nothing guarantees it. Let's at least initialize the fields properly
to avoid passing random garbage.
No backport is needed.
In parse_new_proxy(), when proxy_defproxy_cpy() fails, the error path used
ha_free(&curproxy) to release the partially constructed proxy. However, the
proxy was allocated via alloc_new_proxy() which performs significant setup:
- setup_new_proxy() inserts it into the proxy_by_name tree (proxy_store_name)
- It appends to the global proxies list (LIST_APPEND)
- proxy_take() increments its refcount
Additionally, proxy_defproxy_cpy() may have allocated further resources
(strdup'd strings, compression structures, email alert fields, etc).
Using ha_free() only freed the proxy struct itself, leaving:
- The proxy still registered in the name tree (dangling pointer)
- The proxy still linked in the global proxies list
- All strdup'd strings and other allocations leaked
This is visible with ASAN when causing random allocation errors:
[NOTICE] (27033) : haproxy version is 3.4-dev12-b15468-11
[NOTICE] (27033) : path to executable is ./haproxy
[ALERT] (27033) : config : parsing [/dev/stdin:5015] : proxy 'bk3': failed to duplicate tcpcheck preset-vars
[ALERT] (27033) : config : Error(s) found in configuration file : /dev/stdin
=================================================================
==27033==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 4 byte(s) in 1 object(s) allocated from:
#0 0x7f113e518e20 in strdup (/usr/lib64/libasan.so.8+0x118e20)
#1 0x000000955410 in setup_new_proxy src/proxy.c:3178
#2 0x000000955816 in alloc_new_proxy src/proxy.c:3221
#3 0x000000956c33 in parse_new_proxy src/proxy.c:3554
#4 0x000000a24d03 in cfg_parse_listen src/cfgparse-listen.c:495
#5 0x00000089d33e in parse_cfg src/cfgparse.c:2202
#6 0x0000009e0bb9 in read_cfg src/haproxy.c:1142
#7 0x000000447e8c in main src/haproxy.c:3474
#8 0x7f113d82ad13 in __libc_start_call_main (/lib64/libc.so.6+0x2ad13)
#9 0x7fff65b4e320 ([stack]+0x20320)
SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
The fix replaces ha_free(&curproxy) with proxy_drop(curproxy), which
properly calls deinit_proxy() to release all internal resources, removes
the proxy from trees and lists, decrements the refcount, and frees the
struct.
No backport is needed since proxy_drop() is only in 3.4.
In dns_dgram_init(), the newly created dgram is assigned to the name server
before the ring is attached. In case of errors, e.g. due to too many watchers,
the dgram is released but not removed from ns->dgram. Let's only assign the
pointer on success to avoid this, as it's not needed before. The problem
was introduced in 2.4 with commit c943799c86 ("MEDIUM: resolvers/dns: split
dns.c into dns.c and resolvers.c"), and was possibly there before. The fix
may be backported to all stable versions.
The resolver 'r' is appended to the global sec_resolvers list, but upon failure
later, pointers are released but the element remains in the list, corrupting it,
and possibly causing a crash during deinit() when releasing remaining ones.
Adding a LIST_DEL_INIT() on the error unrolling path is sufficient.
Note that the issue will only happen on failure to allocate memory via
strdup() so the risk is low. The bug was introduced in 2.6 by commit
e7f5776800 ("MINOR: resolvers: resolvers_new() create a resolvers with
default values"), so the fix may be backported to several releases, but
does not necessarily have to go that far.
In cli_parse_set_server()'s 'ssl' branch, the server lock is taken,
and not released in case srv_set_ssl() fails, resulting in a dead lock
and a panic the next time an attempt to touch this server is made. The
lock must be released on all error paths.
This was introduced in 3.3 by commit f8f94ffc9 ("BUG/MEDIUM: server:
Use sni as pool connection name for SSL server only") which was marked
for backporting to 3.0, so this must likely be backported that far.
The condition 'if (srv->pool_conn_name)' was checking the destination
instead of the source 'src->pool_conn_name', meaning the strdup() would
never fire (since newly calloc'd servers start with NULL pool_conn_name),
and the pool_conn_name setting from default-server was silently ignored.
Introduced in 3.2 with commit f0f1816f1 ("MINOR: check: implement
check-pool-conn-name srv keyword") when pool_conn_name support was added
to srv_settings_cpy(). The bug caused any 'pool-conn-name' setting in a
'default-server' line to be lost for all servers inheriting from it.
Note that it's not the first time this function induces such a bug due
to the poor choice of "srv" vs "src" that should be renamed to avoid
keyboard mistakes and visual confusion.
This needs to be backported to 3.2.
This function may face an OOM on strdup() in the middle of the hostname
or hostname_dn replacement, leaving NULLs in either or both of the server's
fields, which is definitely not good for other call places.
Let's perform a safe replacement instead: we first allocate the new
values, and only if they are successful, then we release the previous
ones and replace them.
It is not necessary to backport this unless the issue is reported (it
was found via code review).
All task allocation errors report "memory allocation error initializing
the ring" when the actual failure was task_new_anywhere() returning NULL.
This clearly is a copy-paste. Let's fix the error messages to help when
debugging. Since it's only about allocation failures during init, there
is probably no point in backporting this.
One is s/keyworld/keyword in the retry-on parser. The other one is a
wrong argument "len" being printed in case of parse error for
"declare capture" instead of the length itself.
These can be backported though they are not important.
In dns_session_init(), sockaddr_alloc() allocates 'addr' from the sockaddr
pool, but on failure of appctx_finalize_startup() we jump to the error label
without calling sockaddr_free(&addr), leaking the allocation. Let's add the
missing sockaddr_free() on the error branch.
This must be backported to 2.6.
The original DNS code would only use the 8 lower bits of the compression
offset. This was fixed in 2.0 with commit 2fa66c3b9 ("BUG/MEDIUM: dns:
overflowed dns name start position causing invalid dns error") but it was
not sufficient because the anti-loop check continues to use only 8 of the
14 bits, thus a crafted response where the 8 lower bits pass the check and
the 6 higher should fail it would be accepted. The impacts remains limited
thanks to the bounds check and the recursion limits, but such invalid
responses could still cost a lot to process. Let's compute the 14-bit
offset once for all and use it everywhere.
In resolv_validate_dns_response(), the additional records loop calls
resolv_read_name(). When it returns zero due to a bad response, the main
loop does a "continue" without making the "reader" pointer progress, so it
evaluates the exact same field again and again. Fortunately this is limited
by arcount which is 16 bits, but it means it can still iterate 65535 times
there, allocating and releasing an answer_record at each turn. Let's just
jump to the invalid_resp label that handles the cleaning. There was the
same pattern (without the allocation) with nscount a few lines above BTW.
These can possibly explain some situations where a high CPU usage observed
processing responses.
Seems like these were introduced in 2.2 with commit 37950c8d2
("BUG/MEDIUM: dns: improper parsing of aditional records")
This must be backported to stable versions.
In cfg_parse_resolvers(), two duplicate name checks set err_code but lacked
'goto out', allowing execution to fall through and create the duplicate entry.
This would result in new resolvers and nameservers to be created after the
error was displayed, and a leak of the previous one. It's mostly harmless
since we're exiting after such errors. This can be backported if desired.
In the inner while loop that validates each character of a POST parameter
value, the code checks *p via HTTP_IS_TOKEN() and HTTP_IS_LWS() instead
of *end, while the loop condition only advances "end", so only the first
character of each value is validated.
This means spaces or binary data embedded in parameter values after the
first character goes undetected. Fix by replacing both references to *p
with *end to properly scan through all characters as intended.
This bug was introduced in 1.5-dev20 by commit 98634f0c7 ("MEDIUM:
backend: Enhance hash-type directive with an algorithm options") so
the fix must be backported to all versions.
Released version 3.4-dev12 with the following main changes :
- SCRIPTS: announce-release: add a link to the OpenTelemetry filter
- BUG/MEDIUM: servers: Only requeue servers if they are up
- MINOR: tinfo: store the number of committed extra streams in the tgroup
- MINOR: connection: add a function to calculate elastic streams limit
- MINOR: mux-h2: consider the elastic streams limit on frontend
- MINOR: lb: make LB initialization even more declarative
- BUG/MINOR: cfgparse-listen: do not emit extraneous line in rule order warnings
- CLEANUP: tree-wide: fix typos in non user-visible comments in 15 files
- CLEANUP: h1/htx: fix a few typos in warning, debug and trace messages
- BUG/MINOR: mux-h1: only check h1s if not NULL
- BUG/MINOR: http-fetch: fix smp_fetch_hdr_ip()'s handling of brackets for IPv6
- BUG/MINOR: http-fetch: make http_first_req() check for HTTP first
- BUG/MINOR: http-act: set-status() must check the response message, not the request
- BUG/MINOR: tools: fix memory leak in env_expand() error path
- BUG/MINOR: auth: free user groups on error paths in userlist_postinit()
- BUG/MINOR: uri-auth: avoid leaks on initialization error
- BUG/MINOR: cache: fix memory leak in parse_cache_rule error path
- BUG/MINOR: cfgcond: make KQUEUE check for GTUNE_USE_KQUEUE not GTUNE_USE_EPOLL
- BUG/MINOR: mqtt: connack parser returns MQTT_NEED_MORE_DATA on unknown property
- BUG/MINOR: mqtt: connect parser uses wrong bit field for TOPIC_ALIAS_MAXIMUM
- BUG/MINOR: mqtt: connack parser uses wrong bit for SUBSCRIPTION_IDENTIFIERS_AVAILABLE
- BUG/MINOR: mqtt: fix PUBLISH flags validation that want all bits to be set
- CLEANUP: http_htx: rename inner 'type' to 'ptype' to avoid variable shadowing
- CLEANUP: mux-h2: fix minor output debugging format issues
- CLEANUP: http-rules: fix a few '&' vs '&&' checks for clarity
- CLEANUP: auth: remove undeclared auth_resolve_groups() from auth.h
- CLEANUP: cache: remove redundant res_htx assignment in http_cache_io_handler()
- CLEANUP: channel: remove bogus and unused definition of channel_empty()
- CLEANUP: flt_http_comp: remove duplicate rate limit and CPU usage checks
- CLEANUP: mqtt: remove duplicate MQTT_FN_BIT_USER_PROPERTY in CONNECT fields
- BUG/MINOR: uri-auth: fix possible null-deref in latest fix for leaks
- BUG/MEDIUM: tasks: Keep the TASK_RUNNING flag until queued
- CLEANUP: mqtt: fix spelling of shared_subscription_available
- CLEANUP: regex: pre-initialize error variable in regex_comp() to calm analysis
- BUILD: compiler: fix redefinition of __nonstring
- CLEANUP: defaults: adjust MAX_THREADS multiplier number in comment
- CLEANUP: src/cpuset.c: fix missing return in functions returning int
- REGTESTS: Use ${tmpdir} instead of hardcoded /tmp/
- REGTESTS: Don't try to use real nameservers for testcases
- CLEANUP: tree-wide: fix typos in non user-visible comments in 3 more files
- MINOR: cli: improve forward compatibility for show fd
- DOC: management: document the <tgid>/<fd> form of show fd
- CLEANUP: tree-wide: fix more typos and outdated explanations in comments
- BUG/MEDIUM: dict: hold read lock while incrementing refcount in dict_insert
- BUG/MEDIUM: http-client: Only consume input buffer when hc one is empty
- BUG/MINOR: xprt_qstrm: fix conflicting prototype
- REORG: mux_quic: use newer qcm prefix for legacy qmux files
- MINOR: mux_quic: use qcm prefix for mux callbacks
- MINOR: mux_quic: use qcm prefix for mux functions
- MINOR: mux_quic: use qcm prefix for traces functions/structs
- MINOR: mux_quic: rename qstrm files to qmux
- MINOR: mux_quic: remove qstrm naming in QUIC MUX
- MINOR: connection: rename QMux related flags
- MINOR: xprt_qmux: use qmux instead of qstrm naming
- MINOR: trace: implement source alias
- MEDIUM: mux_quic: rename qmux traces to qcm
- MINOR: sample: add a generic reverse converter
- MINOR: sample: add a reverse_dom converter
- DOC: proxy-protocol: clarify UDP usage
- BUILD: 51d.c: cleanup, fix preprocessor ifdefs
- CLEANUP: tree-wide: fix typos in user-invisible files
- CLEANUP: htx: Adjust numbering of HTX blocks' types in the description
Support of pseudo-headers was removed as unused, but mention of it
in the description remains and disrupt the numbering in comment, which
can be confusing.
The ifdef spans over function boundaries, making some combinations produce
code that does not build:
addons/51degrees/51d.c:638:1: error: Unmatched '}'. Configuration: ''. [syntaxError]
the proxy protocol spec didn't specify UDP and therefore most
implementations treat it as a TCP connection and re-use the last send
information for a ip/port pair.
This change makes it more clear.
In domain-based routing and policy rules, suffix matching on hostnames is
often easier to express as a prefix match on reversed labels. A dedicated
converter makes this convenient with existing fetches and matchers.
This also has a performance benefit for large maps. Prefix string matches use
the prefix-tree index (PAT_MATCH_BEG with pat_idx_tree_pfx), while end matches
use the string-list index (PAT_MATCH_END with pat_idx_list_str), so
reversed-label lookups can avoid linear suffix scans.
This patch adds "reverse_dom", a string converter that reverses domain labels,
ignores one optional trailing dot on input, and rejects empty labels. It
intentionally leaves trailing-dot handling to the caller so configurations can
choose between exact matches, subdomain-only matches, or an explicit dotted
form built with "concat(.)" for prefix lookups.
Examples:
example.com -> com.example
mail.example.com -> com.example.mail
The documentation is updated and a reg-test covers the converter itself, the
explicit dotted form for "map_beg()", and the subdomain-only "-m beg" case.
Some use cases benefit from reversing a string before passing it to other
converters or lookups. While reverse_dom addresses domain-specific label
reversal, a generic byte-wise string reversal remains useful on its own and can
also be combined with other converters such as concat().
A common lookup use case is turning a suffix match on the original string into
a prefix match on the reversed string. Prefix string matches use the
prefix-tree index (PAT_MATCH_BEG with pat_idx_tree_pfx), while end matches use
the string-list index (PAT_MATCH_END with pat_idx_list_str), so reversing
before map_beg can avoid linear suffix scans for large maps.
This patch adds a new string converter named "reverse". It reverses the input
string byte by byte and returns the resulting string unchanged otherwise. It
does not apply any domain-specific semantics or character-encoding semantics.
The documentation is updated and a reg-test is added to cover the basic
conversion as well as a simple composition with concat(.).
This patch is part of a renaming affecting mux_quic and related files.
Naming "qcm" will become the new marker for common mux_quic stuffs,
shared both on QUIC and QMux protocols. The final objective is to limit
"qmux" naming for stuffs related to the new experimental protocol of the
same name.
The current patch renames mux_quic traces token to "qcm". This is the
only change with external visible impacts in the whole renaming series.
However, to preserve compatibility, trace source alias is defined with
the older name "qmux". This relies on the previous patch which
introduced "alias" new trace_source member.
Add a new "alias" member in trace_source structure. Its purpose is to be
an alternative to the member "name". This will be used in the next patch
to allow renaming of QUIC mux traces while preserving compatibility.
This new member is only used in trace_find_source() which is the helper
used to retrieve a trace source from its name.
This is a follow-up on the QUIC MUX renaming process.
The current patch performs renaming in xprt_qmux layer. Older "qstrm"
identifier is replaced by the new name "qmux". Every remaining functions
and structures in xprt_qmux are changed. Outside effects are only
present in QUIC MUX which directly uses some of these functions.
This is a follow-up on the QUIC MUX renaming process.
The current patch performs renaming of "qstrm" to "qmux" in connection
flags. These flags are only used in linked with the xprt_qmux layer.
This has an impact on every files which manipulates these flags, namely
backend, session and ssl_sock sources.
Also, internal xprt identifier is renamed from XPRT_QSTRM to XPRT_QMUX,
This is a follow-up on the QUIC MUX renaming process.
The current patch replaces "qstrm" naming with "qmux" in QUIC MUX source
file. Some members are impacted in qcc and qcs structures, as well as
some internal functions used for QMux receive/send. Internal mux_ops is
also rename to qmux_ops. This is not a breaking change as its externally
visible name was already set to "qmux" originally.
This is a follow-up on the QUIC MUX renaming process. Now most of "qmux"
generic usages as been replaced in favor of "qcm" naming. The next part
of the renaming is to replace "qstrm" naming with "qmux" for stuffs
related to the new QMux protocol specifically.
This is first applied on filenames. As with the previous renaming,
Makefile and include statements are updated as well to prevent
compilation issues.
This is a follow-up on the QUIC MUX renaming process.
This patch renames several definitions in QUIC MUX traces source code.
This is only an internal change. Trace source name is kept to "qmux" for
now, but will be changed in a dedicated patch.
This is a follow-up on the QUIC MUX renaming process.
A previous patch already renames MUX ops callbacks. The current patch
proceed to a similar operation for the rest of the MUX functions.
This is a follow-up on the QUIC MUX renaming process.
The current patch renames all MUX functions used as stream ops
callbacks. Also, internally defined mux_ops is also renamed from
"qmux_ops" to "quic_ops". There is no breaking change as mux_ops name
field remain set to "QUIC".
This patch is the first one of the renaming serie, affecting the QUIC
MUX module. The objective is to remove older "qmux" naming which was
used as a generic identifier. Now it should be restricted to the QMux
experimental protocol. A new "qcm" naming will replace the generic
usage.
The current patch renames the files themselves. Token "qmux" is replaced
by the new "qcm" identifier. Makefile and include statements are
adjusted as required.
This patch adds the missing include of xprt_qstrm header into its
companion source file. This helped to detect an incoherence in the
xprt_qstrm_xfer_rxbuf() prototype which is now fixed.
Header files is also updated with mandatory include statements and
forward declaration.
No backport needed.
Since http-client applet uses its own buffers, it is possible to have data
stuck in the applet input buffer while the http-client response buffer is
full, preventing the applet to consume these data. If this happens on the
last part of the response payload, the upper stream can decide to shut the
applet. In this case, the applet using the http client will not be able to
retrieve these last data because they will never be move into the hc
response buffer.
The main reason for this bug is that, for now, the applets cannot survive
the upper stream unlike multiplexers. It could be a good improvement for the
3.5. However, some applets still uses the stream-connector and the upper
stream (peer and stat applets for instance). So it is not an easy task.
In the mean time, to fix the issue on stable branches, the http-client
applet now stops to consume data when the hc response buffer is not empty.
This way, the applet shut will be deferred. Data will be consumed when they
can be fully moved in the httpclient response buffer.
This patch should fix the issue #3366. It must be backported to 3.3.
In dict_insert(), the read lock on d->rwlock was released before
incrementing the entry's refcount. Between the RDUNLOCK and the
HA_ATOMIC_INC, another thread could call dict_entry_unref() to drop
the refcount to zero, acquire the write lock, delete the entry from
the tree, and free it. The subsequent HA_ATOMIC_INC would then be a
use-after-free on freed memory.
The fix moves the HA_ATOMIC_INC inside the read lock, matching the
pattern used in stick_table.c for identical refcount-then-unlock
sequences.
It can be backported to the branches where this is relevant.
Some outdated comments, as well as typos were fixed in the following files:
dgram.h protocol.h queue-t.h cpu_topo.c debug.c dict.c
protocol.c queue.c raw_sock.c trace.c wdt.c
Add the syntax description, including the wildcard forms and the
note that <tgid> is currently parsed but ignored pending future
support for per-thread-group fd tables.
The "<tgid>/" and "/" wildcard forms previously produced no output.
This isn't a bug since they are new, but a script written for future
versions (where the slash form will gain per-thread-group semantics)
would not work the same on 3.4. Make them produce output by dropping
the redundant ctx->fd = -1 wildcard sentinel; also tighten tgid
validation to reject values <= 0.
The test doesn't need a real nameserver and in a isolated, restricted
test environment it might not be able to reach one at all, like with a
network sandbox. So lets just use 127.0.0.1:53. Even if there is none,
that's not a problem for this particular test.
Signed-off-by: Christian Ruppert <idl0r@qasl.de>
Tests may be excuted in sandboxed or minimalistic / restricted
environments, so incosistencies might cause trouble, like missing
permissions. So lets use the tmpdir variable instead, so the user might
define some path
Signed-off-by: Christian Ruppert <idl0r@qasl.de>
Cppcheck found the issue described in github #2124, which can cause these
errors if no CPUSET implementation is supported (and CPUSET_USE_ULONG is
not enabled):
src/cpuset.c:21:11: error: Found an exit path from function with non-void return type that has missing return statement [missingReturn]
src/cpuset.c:36:11: error: Found an exit path from function with non-void return type that has missing return statement [missingReturn]
src/cpuset.c💯1: error: Found an exit path from function with non-void return type that has missing return statement [missingReturn]
src/cpuset.c:124:1: error: Found an exit path from function with non-void return type that has missing return statement [missingReturn]
src/cpuset.c:152:1: error: Found an exit path from function with non-void return type that has missing return statement [missingReturn]
src/cpuset.c:163:1: error: Found an exit path from function with non-void return type that has missing return statement [missingReturn]
This can be backported.
Dmitry Sivachenko reported a build warning on FreeBSD -dev, where
__nonstring is apparently already defined. Let's guard our own
definition to avoid such issues. It could make sense to backport
this to recent stable versions which may soon be exposed to modern
compilers.
In regex_comp(), the error variable is either a const char* (USE_PCRE)
or a a uchar[] (USE_PCRE2), and navigating through the ifdefs is quite a
mess, making it hard to figure if it's always properly initialized when
printing an error message. Let's just preset it to NULL to clarify what
comes from where.
In task_schedule(), it is not enough to get the TASK_RUNNING flag before
setting the expire field, we also have to keep it while queueing the
taks, otherwise the task may run in the meanwhile and set expire to 0,
triggering the BUG_ON() in __task_queue() again. So now, only drop the
running flag once it's done.
This should be backported up to 2.8.
Latest commit 2dfbc311a8 ("BUG/MINOR: uri-auth: avoid leaks on
initialization error") left a possible null-deref case which was
surprisingly only detected by certain compiler combinations. No
backport needed.
In comp_prepare_compress_request(), the compression rate limit and CPU
usage checks were duplicated. The first set runs before selecting the
algorithm, and the second set runs after. That's definitely a copy-paste
issue or a patch being applied twice. Let's just drop one.
In http_re{q,s}_get_intercept_rule(), there are two occurrences of '&'
being used instead of '&&' which fortunately work thanks to the tests
being negations (hence 0/1 on each branch). Let's fix that and take this
opportunity for adding explicit precedence in http_apply_redirect_rule().
In h2_dump_h2s_info(), the tl.calls was being printed as signed instead
of unsigned, which is not correct but harmless (only used with "show
fd"). In the same function, we don't check if h2s->sd is valid while
dereferencing it. In practise it is valid since "show fd" is run under
thread isolation, but it's far from being obvious, and if conditions
would later change, we don't know it could be printed between h2s_new()
and h2s_frt_stream_new(). Finally in h2s_make_data() a wrong set of
H2_EV_RX_* flags were used instead of H2_EV_TX_* to emit traces.
In http_add_header() there are "type" variables of the same type at two
levels, which is a bit confusing. The inner one is for the "prev" block,
so let's rename it "ptype" by analogy with "pblk".
The definition of the PUBLISH message type indicates that the LSB are
independent, but uses a value of 0xF that clearly shows an attempt to
use a mask instead, but it results in all messages not having all flags
set to be rejected. A sane approach would have been to check for a mask
and an expected value. Let's just add a special case for it in function
mqtt_read_fixed_hdr() since that's for a single message type.
This can be backported anywhere.
In mqtt_parse_connack(), the MQTT_PROP_SUBSCRIPTION_IDENTIFIERS_AVAILABLE
case was checking and setting MQTT_FN_BIT_SUBSCRIPTION_IDENTIFIER instead
of MQTT_FN_BIT_SUBSCRIPTION_IDENTIFIERS_AVAILABLE, due to a copy-paste
mistake. This can be backported where needed.
In mqtt_parse_connect(), the MQTT_PROP_TOPIC_ALIAS_MAXIMUM case was checking
and setting MQTT_FN_BIT_TOPIC_ALIAS instead of MQTT_FN_BIT_TOPIC_ALIAS_MAXIMUM.
This means duplicate detection for Topic-Alias-Maximum property was using the
wrong bitmask, and the actual Topic-Alias-Maximum bit was never set, making
duplicate detection ineffective for this property. The CONNACK parser already
had this correct.
In mqtt_parse_connack(), the switch statement's default case for unknown
MQTT properties was using 'return 0' which returns MQTT_NEED_MORE_DATA.
This is misleading: an unknown property should be treated as an invalid
message (MQTT_INVALID_MESSAGE), like other functions do. This branches to
the "end" label without touching the preset return value instead. This can
be backported if needed.
In cfg_eval_cond_enabled(), the "KQUEUE" option incorrectly checks
GTUNE_USE_EPOLL instead of GTUNE_USE_KQUEUE. This is a copy-paste bug
from the preceding EPOLL case. It can be backported though it's harmless.
When the filter config (fconf) allocation fails in parse_cache_rule,
the previously allocated cache_flt_conf (cconf) and its strdup'd name
string are not freed. The error path only freed cconf but not
cconf->c.name, causing a memory leak.
No backport is needed.
When stats_add_scope() and stats_add_auth() fail to initialize a field,
they just leave a partially allocated and initialized structure behind
them that is leaked. The whole architecture doesn't provide clean
unrolling abilities since everything is shared and assigned unconditionally,
but let's at least release what was just allocated. The whole approach would
probably deserve being revisited if one day this becomes more dynamic.
No backport needed.
In userlist_postinit(), when an error occurs (missing group, missing user, or
allocation failure), the function returned immediately without freeing the
auth_groups_list linked lists that were built for all users in the first loop.
Each user's curuser->u.groups pointed to these allocated nodes, which leaked
on every error path.
Fix by replacing direct returns with a goto to a centralized cleanup label
that frees all users' groups lists before returning the error. Also fix a
trailing double space in one error return statement while refactoring.
Note that the impact is very low since we're supposed to fail to boo after
such errors.
When my_realloc2() fails in env_expand(), the code jumps to 'leave:' and
returns NULL, but the original input 'in' is never freed (it's only freed
at line 4919 in the success case). Given that callers typically pass it
the direct return of strdup(), it looks like it is expected to always be
freed. This can be backported everywhere.
action_http_set_status() checks for soft rewrite on the request message
by mistake instead of the response message. This could possibly cause a
rewrite failure when soft rewrite is enabled since it will not be seen
there, though the impact is extremely low. It can be backported.
smp_fetch_http_first_req() reads ->txn.http->flags without first
checking if txn.http is properly allocated. In theory if called from
the wrong context it could crash, even though tests where it's called
from "tcp-request content" don't seem to have any effect. Let's fix
it regardless, at least to dissipate the doubt. It can be backported
everywhere.
IPv6 addresses can be read enclosed in brackets, but the length of the
string is not checked before checking them. If by lack of luck, the
buffer is empty but already contains '[' in the first place, we'd read
the byte at position -1, possibly crashing (even though in practice it
will not since allocated blocks will be precedeed by the malloc meta-
data). At least it could make asan/valgrind unhappy.
This can be backported to all versions.
Since we can emit glitches during an H2 upgrade, we no longer have a
guaranteed h1s, so _h1_report_glitch() must check h1s before
dereferencing it. No backport is needed as this arrived in 3.4-dev11
with commit 72fd357814 ("MEDIUM: mux-h1: Return an error on h2 upgrade
attempts if not allowed").
Just a few minor user visible issues issues found in mux_h1 and http_htx
(traces, warnings and debug output). This may be backported though isn't
important at all.
This fixes typos and spelling mistakes in the following files:
channel-t.h channel.h filters-t.h http_htx.h htx-t.h tools.h
cfgcond.c channel.c flt_http_comp.c http_ana.c htx.c mqtt.c
mux_h1.c regex.c stats-proxy.c
Some functions such as tcp_parse_tcp_req() are able to emit their own
warnings by relying on warnif_misplaced_*() which directly prints the
warning. However when doing so they still increment the warning counter
which makes cfg_parse_listen() try to emit it, except that what's in the
variable is NULL, so we end up with:
[WARNING] (260) : config : parsing [/etc/haproxy/haproxy.cfg:17] : (null)
Let's just check the errmsg variable before printing the error. If it's
NULL, it's because the message was already printed.
This can be backported to all branches.
This lets lb_ops specify the conditions necessary to bind to this set of
ops. The condition is expressed as a list of mask and match fields on
the algorithm flags. This is then used in proxy_finalize() to locate the
lb_ops corresponding to the current configuration, by iterating over
the list of lb_ops structures. This list is implemented using the same
mechanisms used for configuration keywords: an INITCALL1 macro to a
registration function.
This also moves the lookup and property flags into the lb_ops structure
that were previously applied manually on a case by case basis.
Now the streams-elasticity limit applies to h2 frontend connections.
It allows to reduce the number of advertised streams based on the
number of concurrent connections.
This adds a new tune.streams-elasticity parameter. This parameter
indicates, as a percentage, the average number of streams per connection
at full load. It is used to calculate limits of the number of streams to
advertise on new connections. 0 means that no such limit is set.
When a limit is set, the new function conn_calc_max_streams() determines
the optimal number of streams to allow on a connection. It will assign at
least the ratio of streams left to connections left, and at least a fair
share of what's left times the number of desired streams. It will always
ensure that each connection gets at least 1 stream, and everything beyond
this will be evenly distributed. For now the function is not used.
In order to be able to enforce global streams limitations, we'll first
have to be able to account how many streams we promised to serve via
frontend muxes. We'll always need to support at least one stream, which
is why here we're only counting extra streams beyond the first one. It
also has the benefit of leaving H1 out of this, and save it from updating
a variable. Also in order to avoid an important update cost, we're storing
this value per thread group. For now only H2 is implemented, but QUIC
should follow shortly and should only count bidirectional streams.
In init_srv_requeue(), only attempt to run the tasklet if the server is
actually running, otherwise it will end up being queued a second time,
when the server is actually brought up, and that will lead to a
corrupted mt_list.
This can easily be reproduced by adding a dynamic server, as those start
disabled, and then enabling and disabling it a couple of times.
This should fix github issue #3360.
This should be backported up to 3.2.
expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Health check for server be4/srv4 failed.+reason: Layer4 connection problem.+info: \"Connection refused\".+check duration: [[:digit:]]+ms.+status: 0/1 DOWN."
expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Health check for server be4/srv4 failed.+reason: Layer4 connection problem.+info: \"ECONNREFUSED returned by OS.*\".+check duration: [[:digit:]]+ms.+status: 0/1 DOWN."
