upstream/haproxy
1
0
Fork 0
mirror of https://github.com/haproxy/haproxy.git synced 2026-04-24 07:37:53 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
haproxy/src/h1.c

1269 lines
37 KiB
C
Raw Normal View History

REORG: http: move some very http1-specific parts to h1.{c,h} Certain types and enums are very specific to the HTTP/1 parser, and we'll need to share them with the HTTP/2 to HTTP/1 translation code. Let's move them to h1.c/h1.h. Those with very few occurrences or only used locally were renamed to explicitly mention the relevant HTTP version : enum ht_state -> h1_state. http_msg_state_str -> h1_msg_state_str HTTP_FLG_* -> H1_FLG_* http_char_classes -> h1_char_classes Others like HTTP_IS_*, HTTP_MSG_* are left to be done later.
2017-09-21 03:30:46 -04:00
/*
* HTTP/1 protocol analyzer
*
* Copyright 2000-2017 Willy Tarreau <w@1wt.eu>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
*/
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
#include <ctype.h>
MEDIUM: h1: generate WebSocket key on response if needed Add the Sec-Websocket-Accept header on a websocket handshake response. This header may be missing if a h2 server is used with a h1 client. The response key is calculated following the rfc6455. For this, the handshake request key must be stored in the h1 session, as a new field name ws_key. Note that this is only done if the message has been prealably identified as a Websocket handshake request.
2020-12-11 11:53:03 -05:00
#include <import/sha1.h>
REORG: include: update all files to use haproxy/api.h or api-t.h if needed All files that were including one of the following include files have been updated to only include haproxy/api.h or haproxy/api-t.h once instead: - common/config.h - common/compat.h - common/compiler.h - common/defaults.h - common/initcall.h - common/tools.h The choice is simple: if the file only requires type definitions, it includes api-t.h, otherwise it includes the full api.h. In addition, in these files, explicit includes for inttypes.h and limits.h were dropped since these are now covered by api.h and api-t.h. No other change was performed, given that this patch is large and affects 201 files. At least one (tools.h) was already freestanding and didn't get the new one added.
2020-05-27 06:58:42 -04:00
#include <haproxy/api.h>
MEDIUM: h1: generate WebSocket key on response if needed Add the Sec-Websocket-Accept header on a websocket handshake response. This header may be missing if a h2 server is used with a h1 client. The response key is calculated following the rfc6455. For this, the handshake request key must be stored in the h1 session, as a new field name ws_key. Note that this is only done if the message has been prealably identified as a Websocket handshake request.
2020-12-11 11:53:03 -05:00
#include <haproxy/base64.h>
REORG: include: move common/h1.h to haproxy/h1.h The file was moved as-is. There was a wrong dependency on dynbuf.h instead of buf.h which was addressed. There was no benefit to splitting this between types and functions.
2020-06-02 13:33:08 -04:00
#include <haproxy/h1.h>
REORG: include: split common/http-hdr.h into haproxy/http-hdr{,-t}.h There's only one struct and 2 inline functions. It could have been merged into http.h but that would have added a massive dependency on the hpack parts for nothing, so better keep it this way since hpack is already freestanding and portable.
2020-06-02 13:25:28 -04:00
#include <haproxy/http-hdr.h>
MEDIUM: h1: add a WebSocket key on handshake if needed Add the header Sec-Websocket-Key when generating a h1 handshake websocket without this header. This is the case when doing h2-h1 conversion. The key is randomly generated and base64 encoded. It is stored on the session side to be able to verify response key and reject it if not valid.
2020-12-11 11:53:07 -05:00
#include <haproxy/tools.h>
REORG: http: move some very http1-specific parts to h1.{c,h} Certain types and enums are very specific to the HTTP/1 parser, and we'll need to share them with the HTTP/2 to HTTP/1 translation code. Let's move them to h1.c/h1.h. Those with very few occurrences or only used locally were renamed to explicitly mention the relevant HTTP version : enum ht_state -> h1_state. http_msg_state_str -> h1_msg_state_str HTTP_FLG_* -> H1_FLG_* http_char_classes -> h1_char_classes Others like HTTP_IS_*, HTTP_MSG_* are left to be done later.
2017-09-21 03:30:46 -04:00
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
/* Parse the Content-Length header field of an HTTP/1 request. The function
* checks all possible occurrences of a comma-delimited value, and verifies
* if any of them doesn't match a previous value. It returns <0 if a value
* differs, 0 if the whole header can be dropped (i.e. already known), or >0
* if the value can be indexed (first one). In the last case, the value might
* be adjusted and the caller must only add the updated value.
*/
int h1_parse_cont_len_header(struct h1m *h1m, struct ist *value)
{
char *e, *n;
long long cl;
int not_first = !!(h1m->flags & H1_MF_CLEN);
struct ist word;
BUG/MAJOR: http: reject any empty content-length header value The content-length header parser has its dedicated function, in order to take extreme care about invalid, unparsable, or conflicting values. But there's a corner case in it, by which it stops comparing values when reaching the end of the header. This has for a side effect that an empty value or a value that ends with a comma does not deserve further analysis, and it acts as if the header was absent. While this is not necessarily a problem for the value ending with a comma as it will be cause a header folding and will disappear, it is a problem for the first isolated empty header because this one will not be recontructed when next ones are seen, and will be passed as-is to the backend server. A vulnerable HTTP/1 server hosted behind haproxy that would just use this first value as "0" and ignore the valid one would then not be protected by haproxy and could be attacked this way, taking the payload for an extra request. In field the risk depends on the server. Most commonly used servers already have safe content-length parsers, but users relying on haproxy to protect a known-vulnerable server might be at risk (and the risk of a bug even in a reputable server should never be dismissed). A configuration-based work-around consists in adding the following rule in the frontend, to explicitly reject requests featuring an empty content-length header that would have not be folded into an existing one: http-request deny if { hdr_len(content-length) 0 } The real fix consists in adjusting the parser so that it always expects a value at the beginning of the header or after a comma. It will now reject requests and responses having empty values anywhere in the C-L header. This needs to be backported to all supported versions. Note that the modification was made to functions h1_parse_cont_len_header() and http_parse_cont_len_header(). Prior to 2.8 the latter was in h2_parse_cont_len_header(). One day the two should be refused but the former is also used by Lua. The HTTP messaging reg-tests were completed to test these cases. Thanks to Ben Kallus of Dartmouth College and Narf Industries for reporting this! (this is in GH #2237).
2023-08-09 02:32:48 -04:00
word.ptr = value->ptr;
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
e = value->ptr + value->len;
BUG/MAJOR: http: reject any empty content-length header value The content-length header parser has its dedicated function, in order to take extreme care about invalid, unparsable, or conflicting values. But there's a corner case in it, by which it stops comparing values when reaching the end of the header. This has for a side effect that an empty value or a value that ends with a comma does not deserve further analysis, and it acts as if the header was absent. While this is not necessarily a problem for the value ending with a comma as it will be cause a header folding and will disappear, it is a problem for the first isolated empty header because this one will not be recontructed when next ones are seen, and will be passed as-is to the backend server. A vulnerable HTTP/1 server hosted behind haproxy that would just use this first value as "0" and ignore the valid one would then not be protected by haproxy and could be attacked this way, taking the payload for an extra request. In field the risk depends on the server. Most commonly used servers already have safe content-length parsers, but users relying on haproxy to protect a known-vulnerable server might be at risk (and the risk of a bug even in a reputable server should never be dismissed). A configuration-based work-around consists in adding the following rule in the frontend, to explicitly reject requests featuring an empty content-length header that would have not be folded into an existing one: http-request deny if { hdr_len(content-length) 0 } The real fix consists in adjusting the parser so that it always expects a value at the beginning of the header or after a comma. It will now reject requests and responses having empty values anywhere in the C-L header. This needs to be backported to all supported versions. Note that the modification was made to functions h1_parse_cont_len_header() and http_parse_cont_len_header(). Prior to 2.8 the latter was in h2_parse_cont_len_header(). One day the two should be refused but the former is also used by Lua. The HTTP messaging reg-tests were completed to test these cases. Thanks to Ben Kallus of Dartmouth College and Narf Industries for reporting this! (this is in GH #2237).
2023-08-09 02:32:48 -04:00
while (1) {
if (word.ptr >= e) {
/* empty header or empty value */
goto fail;
}
CLEANUP: assorted typo fixes in the code and comments This is 10th iteration of typo fixes
2020-06-21 12:42:57 -04:00
/* skip leading delimiter and blanks */
BUG/MAJOR: http: reject any empty content-length header value The content-length header parser has its dedicated function, in order to take extreme care about invalid, unparsable, or conflicting values. But there's a corner case in it, by which it stops comparing values when reaching the end of the header. This has for a side effect that an empty value or a value that ends with a comma does not deserve further analysis, and it acts as if the header was absent. While this is not necessarily a problem for the value ending with a comma as it will be cause a header folding and will disappear, it is a problem for the first isolated empty header because this one will not be recontructed when next ones are seen, and will be passed as-is to the backend server. A vulnerable HTTP/1 server hosted behind haproxy that would just use this first value as "0" and ignore the valid one would then not be protected by haproxy and could be attacked this way, taking the payload for an extra request. In field the risk depends on the server. Most commonly used servers already have safe content-length parsers, but users relying on haproxy to protect a known-vulnerable server might be at risk (and the risk of a bug even in a reputable server should never be dismissed). A configuration-based work-around consists in adding the following rule in the frontend, to explicitly reject requests featuring an empty content-length header that would have not be folded into an existing one: http-request deny if { hdr_len(content-length) 0 } The real fix consists in adjusting the parser so that it always expects a value at the beginning of the header or after a comma. It will now reject requests and responses having empty values anywhere in the C-L header. This needs to be backported to all supported versions. Note that the modification was made to functions h1_parse_cont_len_header() and http_parse_cont_len_header(). Prior to 2.8 the latter was in h2_parse_cont_len_header(). One day the two should be refused but the former is also used by Lua. The HTTP messaging reg-tests were completed to test these cases. Thanks to Ben Kallus of Dartmouth College and Narf Industries for reporting this! (this is in GH #2237).
2023-08-09 02:32:48 -04:00
if (unlikely(HTTP_IS_LWS(*word.ptr))) {
word.ptr++;
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
continue;
BUG/MAJOR: http: reject any empty content-length header value The content-length header parser has its dedicated function, in order to take extreme care about invalid, unparsable, or conflicting values. But there's a corner case in it, by which it stops comparing values when reaching the end of the header. This has for a side effect that an empty value or a value that ends with a comma does not deserve further analysis, and it acts as if the header was absent. While this is not necessarily a problem for the value ending with a comma as it will be cause a header folding and will disappear, it is a problem for the first isolated empty header because this one will not be recontructed when next ones are seen, and will be passed as-is to the backend server. A vulnerable HTTP/1 server hosted behind haproxy that would just use this first value as "0" and ignore the valid one would then not be protected by haproxy and could be attacked this way, taking the payload for an extra request. In field the risk depends on the server. Most commonly used servers already have safe content-length parsers, but users relying on haproxy to protect a known-vulnerable server might be at risk (and the risk of a bug even in a reputable server should never be dismissed). A configuration-based work-around consists in adding the following rule in the frontend, to explicitly reject requests featuring an empty content-length header that would have not be folded into an existing one: http-request deny if { hdr_len(content-length) 0 } The real fix consists in adjusting the parser so that it always expects a value at the beginning of the header or after a comma. It will now reject requests and responses having empty values anywhere in the C-L header. This needs to be backported to all supported versions. Note that the modification was made to functions h1_parse_cont_len_header() and http_parse_cont_len_header(). Prior to 2.8 the latter was in h2_parse_cont_len_header(). One day the two should be refused but the former is also used by Lua. The HTTP messaging reg-tests were completed to test these cases. Thanks to Ben Kallus of Dartmouth College and Narf Industries for reporting this! (this is in GH #2237).
2023-08-09 02:32:48 -04:00
}
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
/* digits only now */
for (cl = 0, n = word.ptr; n < e; n++) {
unsigned int c = *n - '0';
if (unlikely(c > 9)) {
/* non-digit */
if (unlikely(n == word.ptr)) // spaces only
goto fail;
break;
}
BUG/MINOR: http: skip leading zeroes in content-length values Ben Kallus also noticed that we preserve leading zeroes on content-length values. While this is totally valid, it would be safer to at least trim them before passing the value, because a bogus server written to parse using "strtol(value, NULL, 0)" could inadvertently take a leading zero as a prefix for an octal value. While there is not much that can be done to protect such servers in general (e.g. lack of check for overflows etc), at least it's quite cheap to make sure the transmitted value is normalized and not taken for an octal one. This is not really a bug, rather a missed opportunity to sanitize the input, but is marked as a bug so that we don't forget to backport it to stable branches. A combined regtest was added to h1or2_to_h1c which already validates end-to-end syntax consistency on aggregate headers.
2023-08-09 05:02:34 -04:00
if (unlikely(!cl && n > word.ptr)) {
/* There was a leading zero before this digit,
* let's trim it.
*/
word.ptr = n;
}
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
if (unlikely(cl > ULLONG_MAX / 10ULL))
goto fail; /* multiply overflow */
cl = cl * 10ULL;
if (unlikely(cl + c < cl))
goto fail; /* addition overflow */
cl = cl + c;
}
/* keep a copy of the exact cleaned value */
word.len = n - word.ptr;
/* skip trailing LWS till next comma or EOL */
for (; n < e; n++) {
if (!HTTP_IS_LWS(*n)) {
if (unlikely(*n != ','))
goto fail;
break;
}
}
/* if duplicate, must be equal */
if (h1m->flags & H1_MF_CLEN && cl != h1m->body_len)
goto fail;
/* OK, store this result as the one to be indexed */
h1m->flags |= H1_MF_CLEN;
h1m->curr_len = h1m->body_len = cl;
*value = word;
BUG/MAJOR: http: reject any empty content-length header value The content-length header parser has its dedicated function, in order to take extreme care about invalid, unparsable, or conflicting values. But there's a corner case in it, by which it stops comparing values when reaching the end of the header. This has for a side effect that an empty value or a value that ends with a comma does not deserve further analysis, and it acts as if the header was absent. While this is not necessarily a problem for the value ending with a comma as it will be cause a header folding and will disappear, it is a problem for the first isolated empty header because this one will not be recontructed when next ones are seen, and will be passed as-is to the backend server. A vulnerable HTTP/1 server hosted behind haproxy that would just use this first value as "0" and ignore the valid one would then not be protected by haproxy and could be attacked this way, taking the payload for an extra request. In field the risk depends on the server. Most commonly used servers already have safe content-length parsers, but users relying on haproxy to protect a known-vulnerable server might be at risk (and the risk of a bug even in a reputable server should never be dismissed). A configuration-based work-around consists in adding the following rule in the frontend, to explicitly reject requests featuring an empty content-length header that would have not be folded into an existing one: http-request deny if { hdr_len(content-length) 0 } The real fix consists in adjusting the parser so that it always expects a value at the beginning of the header or after a comma. It will now reject requests and responses having empty values anywhere in the C-L header. This needs to be backported to all supported versions. Note that the modification was made to functions h1_parse_cont_len_header() and http_parse_cont_len_header(). Prior to 2.8 the latter was in h2_parse_cont_len_header(). One day the two should be refused but the former is also used by Lua. The HTTP messaging reg-tests were completed to test these cases. Thanks to Ben Kallus of Dartmouth College and Narf Industries for reporting this! (this is in GH #2237).
2023-08-09 02:32:48 -04:00
/* Now either n==e and we're done, or n points to the comma,
* and we skip it and continue.
*/
if (n++ == e)
break;
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
word.ptr = n;
}
/* here we've reached the end with a single value or a series of
* identical values, all matching previous series if any. The last
* parsed value was sent back into <value>. We just have to decide
* if this occurrence has to be indexed (it's the first one) or
* silently skipped (it's not the first one)
*/
return !not_first;
fail:
return -1;
}
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
/* Parse the Transfer-Encoding: header field of an HTTP/1 request, looking for
MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice According to the RFC7230, "chunked" encoding must not be applied more than once to a message body. To handle this case, h1_parse_xfer_enc_header() is now responsible to fail when a parsing error is found. It also fails if the "chunked" encoding is not the last one for a request. To help the parsing, two H1 parser flags have been added: H1_MF_TE_CHUNKED and H1_MF_TE_OTHER. These flags are set, respectively, when "chunked" encoding and any other encoding are found. H1_MF_CHNK flag is used when "chunked" encoding is the last one.
2021-09-28 03:36:25 -04:00
* "chunked" encoding to perform some checks (it must be the last encoding for
* the request and must not be performed twice for any message). The
* H1_MF_TE_CHUNKED is set if a valid "chunked" encoding is found. The
* H1_MF_TE_OTHER flag is set if any other encoding is found. The H1_MF_XFER_ENC
* flag is always set. The H1_MF_CHNK is set when "chunked" encoding is the last
* one. Note that transfer codings are case-insensitive (cf RFC7230#4). This
* function returns <0 if a error is found, 0 if the whole header can be dropped
* (not used yet), or >0 if the value can be indexed.
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
*/
MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice According to the RFC7230, "chunked" encoding must not be applied more than once to a message body. To handle this case, h1_parse_xfer_enc_header() is now responsible to fail when a parsing error is found. It also fails if the "chunked" encoding is not the last one for a request. To help the parsing, two H1 parser flags have been added: H1_MF_TE_CHUNKED and H1_MF_TE_OTHER. These flags are set, respectively, when "chunked" encoding and any other encoding are found. H1_MF_CHNK flag is used when "chunked" encoding is the last one.
2021-09-28 03:36:25 -04:00
int h1_parse_xfer_enc_header(struct h1m *h1m, struct ist value)
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
{
char *e, *n;
struct ist word;
h1m->flags |= H1_MF_XFER_ENC;
word.ptr = value.ptr - 1; // -1 for next loop's pre-increment
CLEANUP: Apply ist.cocci Make use of the new rules to use `istend()`.
2021-11-06 10:14:44 -04:00
e = istend(value);
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
while (++word.ptr < e) {
CLEANUP: assorted typo fixes in the code and comments This is 10th iteration of typo fixes
2020-06-21 12:42:57 -04:00
/* skip leading delimiter and blanks */
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
if (HTTP_IS_LWS(*word.ptr))
continue;
n = http_find_hdr_value_end(word.ptr, e); // next comma or end of line
word.len = n - word.ptr;
/* trim trailing blanks */
while (word.len && HTTP_IS_LWS(word.ptr[word.len-1]))
word.len--;
h1m->flags &= ~H1_MF_CHNK;
MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice According to the RFC7230, "chunked" encoding must not be applied more than once to a message body. To handle this case, h1_parse_xfer_enc_header() is now responsible to fail when a parsing error is found. It also fails if the "chunked" encoding is not the last one for a request. To help the parsing, two H1 parser flags have been added: H1_MF_TE_CHUNKED and H1_MF_TE_OTHER. These flags are set, respectively, when "chunked" encoding and any other encoding are found. H1_MF_CHNK flag is used when "chunked" encoding is the last one.
2021-09-28 03:36:25 -04:00
if (isteqi(word, ist("chunked"))) {
if (h1m->flags & H1_MF_TE_CHUNKED) {
/* cf RFC7230#3.3.1 : A sender MUST NOT apply
* chunked more than once to a message body
* (i.e., chunking an already chunked message is
* not allowed)
*/
goto fail;
}
h1m->flags |= (H1_MF_TE_CHUNKED|H1_MF_CHNK);
}
else {
if ((h1m->flags & (H1_MF_RESP|H1_MF_TE_CHUNKED)) == H1_MF_TE_CHUNKED) {
/* cf RFC7230#3.3.1 : If any transfer coding
* other than chunked is applied to a request
* payload body, the sender MUST apply chunked
* as the final transfer coding to ensure that
* the message is properly framed.
*/
goto fail;
}
h1m->flags |= H1_MF_TE_OTHER;
}
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
word.ptr = n;
}
MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice According to the RFC7230, "chunked" encoding must not be applied more than once to a message body. To handle this case, h1_parse_xfer_enc_header() is now responsible to fail when a parsing error is found. It also fails if the "chunked" encoding is not the last one for a request. To help the parsing, two H1 parser flags have been added: H1_MF_TE_CHUNKED and H1_MF_TE_OTHER. These flags are set, respectively, when "chunked" encoding and any other encoding are found. H1_MF_CHNK flag is used when "chunked" encoding is the last one.
2021-09-28 03:36:25 -04:00
return 1;
fail:
return -1;
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
}
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
/* Validate the authority and the host header value for CONNECT method. If there
* is hast header, its value is normalized. 0 is returned on success, -1 if the
* authority is invalid and -2 if the host is invalid.
*/
static int h1_validate_connect_authority(struct ist authority, struct ist *host_hdr)
{
struct ist uri_host, uri_port, host, host_port;
if (!isttest(authority))
goto invalid_authority;
uri_host = authority;
uri_port = http_get_host_port(authority);
MINOR: h1: Consider empty port as invalid in authority for CONNECT For now, this change is useless because http_get_host_port() returns IST_NULL when the port is empty. But this will change. For other methods, empty ports are valid. But not for CONNECT method. To still return a 400-Bad-Request if a CONNECT is performed with an empty port, istlen() is used to test the port, instead of isttest().
2022-11-22 04:27:54 -05:00
if (!istlen(uri_port))
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
goto invalid_authority;
uri_host.len -= (istlen(uri_port) + 1);
if (!host_hdr || !isttest(*host_hdr))
goto end;
/* Get the port of the host header value, if any */
host = *host_hdr;
host_port = http_get_host_port(*host_hdr);
MINOR: h1: Consider empty port as invalid in authority for CONNECT For now, this change is useless because http_get_host_port() returns IST_NULL when the port is empty. But this will change. For other methods, empty ports are valid. But not for CONNECT method. To still return a 400-Bad-Request if a CONNECT is performed with an empty port, istlen() is used to test the port, instead of isttest().
2022-11-22 04:27:54 -05:00
if (isttest(host_port))
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
host.len -= (istlen(host_port) + 1);
MINOR: h1: Consider empty port as invalid in authority for CONNECT For now, this change is useless because http_get_host_port() returns IST_NULL when the port is empty. But this will change. For other methods, empty ports are valid. But not for CONNECT method. To still return a 400-Bad-Request if a CONNECT is performed with an empty port, istlen() is used to test the port, instead of isttest().
2022-11-22 04:27:54 -05:00
if (istlen(host_port)) {
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
if (!isteqi(host, uri_host) || !isteq(host_port, uri_port))
goto invalid_host;
if (http_is_default_port(IST_NULL, uri_port))
*host_hdr = host; /* normalize */
}
else {
if (!http_is_default_port(IST_NULL, uri_port) || !isteqi(host, uri_host))
goto invalid_host;
}
end:
return 0;
invalid_authority:
return -1;
invalid_host:
return -2;
}
BUG/MINOR: h1: Replace authority validation to conform RFC3986 Except for CONNECT method, where a normalization is performed, we expected to have an exact match between the authority and the host header value. However it was too strict. Indeed, default port must be handled and the matching must respect the RFC3986. There is already a scheme based normalizeation performed on the URI later, on the HTX message. And we cannot normalize the URI during H1 parsing to be able to report proper errors on the original raw buffer. And a systematic read-only normalization to validate the authority will consume CPU for only few requests. At the end, we decided to perform extra-checks when the exact match fails. Now, following authority/host are considered as equivalent: http: domain.com <=> domain.com:80 <=> domain.com: https: domain.com <=> domain.com:443 <=> domain.com: This patch depends on: * MINOR: h1: Consider empty port as invalid in authority for CONNECT * MINOR: http: Considere empty ports as valid default ports It is a bug regarding the RFC3986. Technically, I may be backported as far as 2.4. However, this must be discussed first. If it is backported, the commits above must be backported too.
2022-11-22 04:04:16 -05:00
/* Validate the authority and the host header value for non-CONNECT method, when
* an absolute-URI is detected but when it does not exactly match the host
* value. The idea is to detect default port (http or https). authority and host
* are defined here. 0 is returned on success, -1 if the host is does not match
* the authority.
*/
static int h1_validate_mismatch_authority(struct ist scheme, struct ist authority, struct ist host_hdr)
{
struct ist uri_host, uri_port, host, host_port;
if (!isttest(scheme))
goto mismatch;
uri_host = authority;
uri_port = http_get_host_port(authority);
if (isttest(uri_port))
uri_host.len -= (istlen(uri_port) + 1);
host = host_hdr;
host_port = http_get_host_port(host_hdr);
if (isttest(host_port))
host.len -= (istlen(host_port) + 1);
if (!isttest(uri_port) && !isttest(host_port)) {
/* No port on both: we already know the authority does not match
* the host value
*/
goto mismatch;
}
else if (isttest(uri_port) && !http_is_default_port(scheme, uri_port)) {
/* here there is no port for the host value and the port for the
* authority is not the default one
*/
goto mismatch;
}
else if (isttest(host_port) && !http_is_default_port(scheme, host_port)) {
/* here there is no port for the authority and the port for the
* host value is not the default one
*/
goto mismatch;
}
else {
/* the authority or the host value contain a default port and
* there is no port on the other value
*/
if (!isteqi(uri_host, host))
goto mismatch;
}
return 0;
mismatch:
return -1;
}
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
/* Parse the Connection: header of an HTTP/1 request, looking for "close",
* "keep-alive", and "upgrade" values, and updating h1m->flags according to
* what was found there. Note that flags are only added, not removed, so the
* function is safe for being called multiple times if multiple occurrences
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
* are found. If the flag H1_MF_CLEAN_CONN_HDR, the header value is cleaned
* up from "keep-alive" and "close" values. To do so, the header value is
* rewritten in place and its length is updated.
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
*/
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
void h1_parse_connection_header(struct h1m *h1m, struct ist *value)
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
{
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
char *e, *n, *p;
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
struct ist word;
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
word.ptr = value->ptr - 1; // -1 for next loop's pre-increment
p = value->ptr;
e = value->ptr + value->len;
if (h1m->flags & H1_MF_CLEAN_CONN_HDR)
value->len = 0;
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
while (++word.ptr < e) {
CLEANUP: assorted typo fixes in the code and comments This is 10th iteration of typo fixes
2020-06-21 12:42:57 -04:00
/* skip leading delimiter and blanks */
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
if (HTTP_IS_LWS(*word.ptr))
continue;
n = http_find_hdr_value_end(word.ptr, e); // next comma or end of line
word.len = n - word.ptr;
/* trim trailing blanks */
while (word.len && HTTP_IS_LWS(word.ptr[word.len-1]))
word.len--;
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
if (isteqi(word, ist("keep-alive"))) {
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
h1m->flags |= H1_MF_CONN_KAL;
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
if (h1m->flags & H1_MF_CLEAN_CONN_HDR)
goto skip_val;
}
else if (isteqi(word, ist("close"))) {
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
h1m->flags |= H1_MF_CONN_CLO;
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
if (h1m->flags & H1_MF_CLEAN_CONN_HDR)
goto skip_val;
}
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
else if (isteqi(word, ist("upgrade")))
h1m->flags |= H1_MF_CONN_UPG;
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
if (h1m->flags & H1_MF_CLEAN_CONN_HDR) {
if (value->ptr + value->len == p) {
/* no rewrite done till now */
value->len = n - value->ptr;
}
else {
if (value->len)
value->ptr[value->len++] = ',';
istcat(value, word, e - value->ptr);
}
}
skip_val:
word.ptr = p = n;
MINOR: h1: parse the Connection header field The new function h1_parse_connection_header() is called when facing a connection header in the generic parser, and it will set up to 3 bits in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade" tokens was seen.
2018-09-13 08:15:58 -04:00
}
}
MINOR: h1: reject websocket handshake if missing key If a request is identified as a WebSocket handshake, it must contains a websocket key header or else it can be reject, following the rfc6455. A new flag H1_MF_UPG_WEBSOCKET is set on such messages. For the request te be identified as a WebSocket handshake, it must contains the headers: Connection: upgrade Upgrade: websocket This commit is a compagnon of "MEDIUM: h1: generate WebSocket key on response if needed" and "MEDIUM: h1: add a WebSocket key on handshake if needed". Indeed, it ensures that a WebSocket key is added only from a http/2 side and not for a http/1 bogus peer.
2020-12-11 11:53:02 -05:00
/* Parse the Upgrade: header of an HTTP/1 request.
* If "websocket" is found, set H1_MF_UPG_WEBSOCKET flag
*/
void h1_parse_upgrade_header(struct h1m *h1m, struct ist value)
{
char *e, *n;
struct ist word;
h1m->flags &= ~H1_MF_UPG_WEBSOCKET;
word.ptr = value.ptr - 1; // -1 for next loop's pre-increment
CLEANUP: Apply ist.cocci Make use of the new rules to use `istend()`.
2021-11-06 10:14:44 -04:00
e = istend(value);
MINOR: h1: reject websocket handshake if missing key If a request is identified as a WebSocket handshake, it must contains a websocket key header or else it can be reject, following the rfc6455. A new flag H1_MF_UPG_WEBSOCKET is set on such messages. For the request te be identified as a WebSocket handshake, it must contains the headers: Connection: upgrade Upgrade: websocket This commit is a compagnon of "MEDIUM: h1: generate WebSocket key on response if needed" and "MEDIUM: h1: add a WebSocket key on handshake if needed". Indeed, it ensures that a WebSocket key is added only from a http/2 side and not for a http/1 bogus peer.
2020-12-11 11:53:02 -05:00
while (++word.ptr < e) {
/* skip leading delimiter and blanks */
if (HTTP_IS_LWS(*word.ptr))
continue;
n = http_find_hdr_value_end(word.ptr, e); // next comma or end of line
word.len = n - word.ptr;
/* trim trailing blanks */
while (word.len && HTTP_IS_LWS(word.ptr[word.len-1]))
word.len--;
if (isteqi(word, ist("websocket")))
h1m->flags |= H1_MF_UPG_WEBSOCKET;
word.ptr = n;
}
}
REORG: h1: move legacy http functions to http_msg.c Now that h1 and legacy HTTP are two distinct things, there's no need to keep the legacy HTTP parsers in h1.c since they're only used by the legacy code in proto_http.c, and h1.h doesn't need to include hdr_idx anymore. This concerns the following functions : - http_parse_reqline(); - http_parse_stsline(); - http_msg_analyzer(); - http_forward_trailers(); All of these were moved to http_msg.c.
2018-12-11 04:59:20 -05:00
/* Macros used in the HTTP/1 parser, to check for the expected presence of
* certain bytes (ef: LF) or to skip to next byte and yield in case of failure.
*/
/* Expects to find an LF at <ptr>. If not, set <state> to <where> and jump to
* <bad>.
*/
#define EXPECT_LF_HERE(ptr, bad, state, where) \
do { \
if (unlikely(*(ptr) != '\n')) { \
state = (where); \
goto bad; \
} \
} while (0)
/* Increments pointer <ptr>, continues to label <more> if it's still below
* pointer <end>, or goes to <stop> and sets <state> to <where> if the end
* of buffer was reached.
*/
#define EAT_AND_JUMP_OR_RETURN(ptr, end, more, stop, state, where) \
do { \
if (likely(++(ptr) < (end))) \
goto more; \
else { \
state = (where); \
goto stop; \
} \
} while (0)
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* This function parses a contiguous HTTP/1 headers block starting at <start>
* and ending before <stop>, at once, and converts it a list of (name,value)
* pairs representing header fields into the array <hdr> of size <hdr_num>,
* whose last entry will have an empty name and an empty value. If <hdr_num> is
MEDIUM: h1: let the caller pass the initial parser's state This way the caller controls if it's the request or response which has to be used, and it will allow to restart after an incomplete parsing.
2018-09-11 09:33:32 -04:00
* too small to represent the whole message, an error is returned. Some
* protocol elements such as content-length and transfer-encoding will be
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
* parsed and stored into h1m as well. <hdr> may be null, in which case only
* the parsing state will be updated. This may be used to restart the parsing
* where it stopped for example.
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
*
* For now it's limited to the response. If the header block is incomplete,
* 0 is returned, waiting to be called again with more data to try it again.
MEDIUM: h1: let the caller pass the initial parser's state This way the caller controls if it's the request or response which has to be used, and it will allow to restart after an incomplete parsing.
2018-09-11 09:33:32 -04:00
* The caller is responsible for initializing h1m->state to H1_MSG_RPBEFORE,
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
* and h1m->next to zero on the first call, the parser will do the rest. If
* an incomplete message is seen, the caller only needs to present h1m->state
* and h1m->next again, with an empty header list so that the parser can start
* again. In this case, it will detect that it interrupted a previous session
* and will first look for the end of the message before reparsing it again and
* indexing it at the same time. This ensures that incomplete messages fed 1
* character at a time are never processed entirely more than exactly twice,
* and that there is no need to store all the internal state and pre-parsed
* headers or start line between calls.
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
*
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
* A pointer to a start line descriptor may be passed in <slp>, in which case
* the parser will fill it with whatever it found.
*
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
* The code derived from the main HTTP/1 parser above but was simplified and
* optimized to process responses produced or forwarded by haproxy. The caller
* is responsible for ensuring that the message doesn't wrap, and should ensure
* it is complete to avoid having to retry the operation after a failed
* attempt. The message is not supposed to be invalid, which is why a few
* properties such as the character set used in the header field names are not
* checked. In case of an unparsable response message, a negative value will be
* returned with h1m->err_pos and h1m->err_state matching the location and
* state where the error was met. Leading blank likes are tolerated but not
MINOR: h1: make the H1 headers block parser able to parse headers only Currently the H1 headers parser works for either a request or a response because it starts from the start line. It is also able to resume its processing when it was interrupted, but in this case it doesn't update the list. Make it support a new flag, H1_MF_HDRS_ONLY so that the caller can indicate it's only interested in the headers list and not the start line. This will be convenient to parse H1 trailers.
2019-01-04 04:48:03 -05:00
* recommended. If flag H1_MF_HDRS_ONLY is set in h1m->flags, only headers are
* parsed and the start line is skipped. It is not required to set h1m->state
* nor h1m->next in this case.
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
*
* This function returns :
* -1 in case of error. In this case, h1m->err_state is filled (if h1m is
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
* set) with the state the error occurred in and h1m->err_pos with the
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
* the position relative to <start>
* -2 if the output is full (hdr_num reached). err_state and err_pos also
* indicate where it failed.
* 0 in case of missing data.
* > 0 on success, it then corresponds to the number of bytes read since
* <start> so that the caller can go on with the payload.
*/
int h1_headers_to_hdr_list(char *start, const char *stop,
struct http_hdr *hdr, unsigned int hdr_num,
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
struct h1m *h1m, union h1_sl *slp)
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
{
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
enum h1m_state state;
register char *ptr;
register const char *end;
unsigned int hdr_count;
unsigned int skip; /* number of bytes skipped at the beginning */
unsigned int sol; /* start of line */
unsigned int col; /* position of the colon */
unsigned int eol; /* end of line */
unsigned int sov; /* start of value */
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
union h1_sl sl;
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
int skip_update;
int restarting;
MINOR: h1: Reject requests with different occurrences of the header host There is no reason for a client to send several headers host. It even may be considered as a bug. However, it is totally invalid to have different values for those. So now, in such case, an error is triggered during the request parsing. In addition, when several headers host are found with the same value, only the first instance is kept and others are skipped.
2019-10-11 03:01:44 -04:00
int host_idx;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
struct ist n, v; /* header name and value during parsing */
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
skip = 0; // do it only once to keep track of the leading CRLF.
try_again:
hdr_count = sol = col = eol = sov = 0;
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
sl.st.status = 0;
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
skip_update = restarting = 0;
MINOR: h1: Reject requests with different occurrences of the header host There is no reason for a client to send several headers host. It even may be considered as a bug. However, it is totally invalid to have different values for those. So now, in such case, an error is triggered during the request parsing. In addition, when several headers host are found with the same value, only the first instance is kept and others are skipped.
2019-10-11 03:01:44 -04:00
host_idx = -1;
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
MINOR: h1: make the H1 headers block parser able to parse headers only Currently the H1 headers parser works for either a request or a response because it starts from the start line. It is also able to resume its processing when it was interrupted, but in this case it doesn't update the list. Make it support a new flag, H1_MF_HDRS_ONLY so that the caller can indicate it's only interested in the headers list and not the start line. This will be convenient to parse H1 trailers.
2019-01-04 04:48:03 -05:00
if (h1m->flags & H1_MF_HDRS_ONLY) {
state = H1_MSG_HDR_FIRST;
h1m->next = 0;
}
BUG/MEDIUM: h1: Get the h1m state when restarting the headers parsing Since the commit 0f8fb6b7f ("MINOR: h1: make the H1 headers block parser able to parse headers only"), when headers are not received in one time, a parsing error is returned because the local state in the function h1_headers_to_hdr_list() was not initialized with the previous one (in fact, it was not initialized at all). So now, we start the parsing of headers with the state H1_MSG_HDR_FIRST when the flag H1_MF_HDRS_ONLY is set. Otherwise, we always get it from the h1m. This patch must be backported to 1.9.
2019-01-04 10:06:48 -05:00
else {
MINOR: h1: make the H1 headers block parser able to parse headers only Currently the H1 headers parser works for either a request or a response because it starts from the start line. It is also able to resume its processing when it was interrupted, but in this case it doesn't update the list. Make it support a new flag, H1_MF_HDRS_ONLY so that the caller can indicate it's only interested in the headers list and not the start line. This will be convenient to parse H1 trailers.
2019-01-04 04:48:03 -05:00
state = h1m->state;
BUG/MEDIUM: h1: Get the h1m state when restarting the headers parsing Since the commit 0f8fb6b7f ("MINOR: h1: make the H1 headers block parser able to parse headers only"), when headers are not received in one time, a parsing error is returned because the local state in the function h1_headers_to_hdr_list() was not initialized with the previous one (in fact, it was not initialized at all). So now, we start the parsing of headers with the state H1_MSG_HDR_FIRST when the flag H1_MF_HDRS_ONLY is set. Otherwise, we always get it from the h1m. This patch must be backported to 1.9.
2019-01-04 10:06:48 -05:00
if (h1m->state != H1_MSG_RQBEFORE && h1m->state != H1_MSG_RPBEFORE)
restarting = 1;
}
MINOR: h1: make the H1 headers block parser able to parse headers only Currently the H1 headers parser works for either a request or a response because it starts from the start line. It is also able to resume its processing when it was interrupted, but in this case it doesn't update the list. Make it support a new flag, H1_MF_HDRS_ONLY so that the caller can indicate it's only interested in the headers list and not the start line. This will be convenient to parse H1 trailers.
2019-01-04 04:48:03 -05:00
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
ptr = start + h1m->next;
end = stop;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
if (unlikely(ptr >= end))
goto http_msg_ood;
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
/* don't update output if hdr is NULL or if we're restarting */
if (!hdr || restarting)
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
skip_update = 1;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
switch (state) {
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
case H1_MSG_RQBEFORE:
http_msg_rqbefore:
if (likely(HTTP_IS_TOKEN(*ptr))) {
/* we have a start of message, we may have skipped some
* heading CRLF. Skip them now.
*/
skip += ptr - start;
start = ptr;
sol = 0;
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.m.ptr = ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
hdr_count = 0;
state = H1_MSG_RQMETH;
goto http_msg_rqmeth;
}
if (unlikely(!HTTP_IS_CRLF(*ptr))) {
state = H1_MSG_RQBEFORE;
goto http_msg_invalid;
}
if (unlikely(*ptr == '\n'))
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqbefore, http_msg_ood, state, H1_MSG_RQBEFORE);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqbefore_cr, http_msg_ood, state, H1_MSG_RQBEFORE_CR);
/* stop here */
case H1_MSG_RQBEFORE_CR:
http_msg_rqbefore_cr:
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_RQBEFORE_CR);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqbefore, http_msg_ood, state, H1_MSG_RQBEFORE);
/* stop here */
case H1_MSG_RQMETH:
http_msg_rqmeth:
if (likely(HTTP_IS_TOKEN(*ptr)))
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqmeth, http_msg_ood, state, H1_MSG_RQMETH);
if (likely(HTTP_IS_SPHT(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.m.len = ptr - sl.rq.m.ptr;
sl.rq.meth = find_http_meth(start, sl.rq.m.len);
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqmeth_sp, http_msg_ood, state, H1_MSG_RQMETH_SP);
}
if (likely(HTTP_IS_CRLF(*ptr))) {
/* HTTP 0.9 request */
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.m.len = ptr - sl.rq.m.ptr;
sl.rq.meth = find_http_meth(sl.rq.m.ptr, sl.rq.m.len);
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
http_msg_req09_uri:
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.u.ptr = ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
http_msg_req09_uri_e:
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.u.len = ptr - sl.rq.u.ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
http_msg_req09_ver:
CLEANUP: Reapply ist.cocci with `--include-headers-for-types --recursive-includes` Previous uses of `ist.cocci` did not add `--include-headers-for-types` and `--recursive-includes` preventing Coccinelle seeing `struct ist` members of other structs. Reapply the patch with proper flags to further clean up the use of the ist API. The command used was: spatch -sp_file dev/coccinelle/ist.cocci -in_place --include-headers --include-headers-for-types --recursive-includes --dir src/
2022-03-15 08:11:06 -04:00
sl.rq.v = ist2(ptr, 0);
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
goto http_msg_rqline_eol;
}
state = H1_MSG_RQMETH;
goto http_msg_invalid;
case H1_MSG_RQMETH_SP:
http_msg_rqmeth_sp:
if (likely(!HTTP_IS_LWS(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.u.ptr = ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
goto http_msg_rquri;
}
if (likely(HTTP_IS_SPHT(*ptr)))
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqmeth_sp, http_msg_ood, state, H1_MSG_RQMETH_SP);
/* so it's a CR/LF, meaning an HTTP 0.9 request */
goto http_msg_req09_uri;
case H1_MSG_RQURI:
http_msg_rquri:
CLEANUP: http/h1: rely on HA_UNALIGNED_LE instead of checking for CPU families Now that we have flags indicating the CPU's capabilities, better use them instead of missing some updates for new CPU families (ARMv8 was missing there).
2020-02-21 10:31:22 -05:00
#ifdef HA_UNALIGNED_LE
BUG/MINOR: h1: do not accept '#' as part of the URI component Seth Manesse and Paul Plasil reported that the "path" sample fetch function incorrectly accepts '#' as part of the path component. This can in some cases lead to misrouted requests for rules that would apply on the suffix: use_backend static if { path_end .png .jpg .gif .css .js } Note that this behavior can be selectively configured using "normalize-uri fragment-encode" and "normalize-uri fragment-strip". The problem is that while the RFC says that this '#' must never be emitted, as often it doesn't suggest how servers should handle it. A diminishing number of servers still do accept it and trim it silently, while others are rejecting it, as indicated in the conversation below with other implementers: https://lists.w3.org/Archives/Public/ietf-http-wg/2023JulSep/0070.html Looking at logs from publicly exposed servers, such requests appear at a rate of roughly 1 per million and only come from attacks or poorly written web crawlers incorrectly following links found on various pages. Thus it looks like the best solution to this problem is to simply reject such ambiguous requests by default, and include this in the list of controls that can be disabled using "option accept-invalid-http-request". We're already rejecting URIs containing any control char anyway, so we should also reject '#'. In the H1 parser for the H1_MSG_RQURI state, there is an accelerated parser for bytes 0x21..0x7e that has been tightened to 0x24..0x7e (it should not impact perf since 0x21..0x23 are not supposed to appear in a URI anyway). This way '#' falls through the fine-grained filter and we can add the special case for it also conditionned by a check on the proxy's option "accept-invalid-http-request", with no overhead for the vast majority of valid URIs. Here this information is available through h1m->err_pos that's set to -2 when the option is here (so we don't need to change the API to expose the proxy). Example with a trivial GET through netcat: [08/Aug/2023:16:16:52.651] frontend layer1 (#2): invalid request backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:50812 buffer starts at 0 (including 0 out), 16361 free, len 23, wraps at 16336, error at position 7 H1 connection flags 0x00000000, H1 stream flags 0x00000810 H1 msg state MSG_RQURI(4), H1 msg flags 0x00001400 H1 chunk len 0 bytes, H1 body len 0 bytes : 00000 GET /aa#bb HTTP/1.0\r\n 00021 \r\n This should be progressively backported to all stable versions along with the following patch: REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests Similar fixes for h2 and h3 will come in followup patches. Thanks to Seth Manesse and Paul Plasil for reporting this problem with detailed explanations.
2023-08-08 10:17:22 -04:00
/* speedup: skip bytes not between 0x24 and 0x7e inclusive */
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
while (ptr <= end - sizeof(int)) {
BUG/MINOR: h1: do not accept '#' as part of the URI component Seth Manesse and Paul Plasil reported that the "path" sample fetch function incorrectly accepts '#' as part of the path component. This can in some cases lead to misrouted requests for rules that would apply on the suffix: use_backend static if { path_end .png .jpg .gif .css .js } Note that this behavior can be selectively configured using "normalize-uri fragment-encode" and "normalize-uri fragment-strip". The problem is that while the RFC says that this '#' must never be emitted, as often it doesn't suggest how servers should handle it. A diminishing number of servers still do accept it and trim it silently, while others are rejecting it, as indicated in the conversation below with other implementers: https://lists.w3.org/Archives/Public/ietf-http-wg/2023JulSep/0070.html Looking at logs from publicly exposed servers, such requests appear at a rate of roughly 1 per million and only come from attacks or poorly written web crawlers incorrectly following links found on various pages. Thus it looks like the best solution to this problem is to simply reject such ambiguous requests by default, and include this in the list of controls that can be disabled using "option accept-invalid-http-request". We're already rejecting URIs containing any control char anyway, so we should also reject '#'. In the H1 parser for the H1_MSG_RQURI state, there is an accelerated parser for bytes 0x21..0x7e that has been tightened to 0x24..0x7e (it should not impact perf since 0x21..0x23 are not supposed to appear in a URI anyway). This way '#' falls through the fine-grained filter and we can add the special case for it also conditionned by a check on the proxy's option "accept-invalid-http-request", with no overhead for the vast majority of valid URIs. Here this information is available through h1m->err_pos that's set to -2 when the option is here (so we don't need to change the API to expose the proxy). Example with a trivial GET through netcat: [08/Aug/2023:16:16:52.651] frontend layer1 (#2): invalid request backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:50812 buffer starts at 0 (including 0 out), 16361 free, len 23, wraps at 16336, error at position 7 H1 connection flags 0x00000000, H1 stream flags 0x00000810 H1 msg state MSG_RQURI(4), H1 msg flags 0x00001400 H1 chunk len 0 bytes, H1 body len 0 bytes : 00000 GET /aa#bb HTTP/1.0\r\n 00021 \r\n This should be progressively backported to all stable versions along with the following patch: REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests Similar fixes for h2 and h3 will come in followup patches. Thanks to Seth Manesse and Paul Plasil for reporting this problem with detailed explanations.
2023-08-08 10:17:22 -04:00
int x = *(int *)ptr - 0x24242424;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
if (x & 0x80808080)
break;
BUG/MINOR: h1: do not accept '#' as part of the URI component Seth Manesse and Paul Plasil reported that the "path" sample fetch function incorrectly accepts '#' as part of the path component. This can in some cases lead to misrouted requests for rules that would apply on the suffix: use_backend static if { path_end .png .jpg .gif .css .js } Note that this behavior can be selectively configured using "normalize-uri fragment-encode" and "normalize-uri fragment-strip". The problem is that while the RFC says that this '#' must never be emitted, as often it doesn't suggest how servers should handle it. A diminishing number of servers still do accept it and trim it silently, while others are rejecting it, as indicated in the conversation below with other implementers: https://lists.w3.org/Archives/Public/ietf-http-wg/2023JulSep/0070.html Looking at logs from publicly exposed servers, such requests appear at a rate of roughly 1 per million and only come from attacks or poorly written web crawlers incorrectly following links found on various pages. Thus it looks like the best solution to this problem is to simply reject such ambiguous requests by default, and include this in the list of controls that can be disabled using "option accept-invalid-http-request". We're already rejecting URIs containing any control char anyway, so we should also reject '#'. In the H1 parser for the H1_MSG_RQURI state, there is an accelerated parser for bytes 0x21..0x7e that has been tightened to 0x24..0x7e (it should not impact perf since 0x21..0x23 are not supposed to appear in a URI anyway). This way '#' falls through the fine-grained filter and we can add the special case for it also conditionned by a check on the proxy's option "accept-invalid-http-request", with no overhead for the vast majority of valid URIs. Here this information is available through h1m->err_pos that's set to -2 when the option is here (so we don't need to change the API to expose the proxy). Example with a trivial GET through netcat: [08/Aug/2023:16:16:52.651] frontend layer1 (#2): invalid request backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:50812 buffer starts at 0 (including 0 out), 16361 free, len 23, wraps at 16336, error at position 7 H1 connection flags 0x00000000, H1 stream flags 0x00000810 H1 msg state MSG_RQURI(4), H1 msg flags 0x00001400 H1 chunk len 0 bytes, H1 body len 0 bytes : 00000 GET /aa#bb HTTP/1.0\r\n 00021 \r\n This should be progressively backported to all stable versions along with the following patch: REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests Similar fixes for h2 and h3 will come in followup patches. Thanks to Seth Manesse and Paul Plasil for reporting this problem with detailed explanations.
2023-08-08 10:17:22 -04:00
x -= 0x5b5b5b5b;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
if (!(x & 0x80808080))
break;
ptr += sizeof(int);
}
#endif
if (ptr >= end) {
state = H1_MSG_RQURI;
goto http_msg_ood;
}
http_msg_rquri2:
BUG/MINOR: h1: do not accept '#' as part of the URI component Seth Manesse and Paul Plasil reported that the "path" sample fetch function incorrectly accepts '#' as part of the path component. This can in some cases lead to misrouted requests for rules that would apply on the suffix: use_backend static if { path_end .png .jpg .gif .css .js } Note that this behavior can be selectively configured using "normalize-uri fragment-encode" and "normalize-uri fragment-strip". The problem is that while the RFC says that this '#' must never be emitted, as often it doesn't suggest how servers should handle it. A diminishing number of servers still do accept it and trim it silently, while others are rejecting it, as indicated in the conversation below with other implementers: https://lists.w3.org/Archives/Public/ietf-http-wg/2023JulSep/0070.html Looking at logs from publicly exposed servers, such requests appear at a rate of roughly 1 per million and only come from attacks or poorly written web crawlers incorrectly following links found on various pages. Thus it looks like the best solution to this problem is to simply reject such ambiguous requests by default, and include this in the list of controls that can be disabled using "option accept-invalid-http-request". We're already rejecting URIs containing any control char anyway, so we should also reject '#'. In the H1 parser for the H1_MSG_RQURI state, there is an accelerated parser for bytes 0x21..0x7e that has been tightened to 0x24..0x7e (it should not impact perf since 0x21..0x23 are not supposed to appear in a URI anyway). This way '#' falls through the fine-grained filter and we can add the special case for it also conditionned by a check on the proxy's option "accept-invalid-http-request", with no overhead for the vast majority of valid URIs. Here this information is available through h1m->err_pos that's set to -2 when the option is here (so we don't need to change the API to expose the proxy). Example with a trivial GET through netcat: [08/Aug/2023:16:16:52.651] frontend layer1 (#2): invalid request backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:50812 buffer starts at 0 (including 0 out), 16361 free, len 23, wraps at 16336, error at position 7 H1 connection flags 0x00000000, H1 stream flags 0x00000810 H1 msg state MSG_RQURI(4), H1 msg flags 0x00001400 H1 chunk len 0 bytes, H1 body len 0 bytes : 00000 GET /aa#bb HTTP/1.0\r\n 00021 \r\n This should be progressively backported to all stable versions along with the following patch: REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests Similar fixes for h2 and h3 will come in followup patches. Thanks to Seth Manesse and Paul Plasil for reporting this problem with detailed explanations.
2023-08-08 10:17:22 -04:00
if (likely((unsigned char)(*ptr - 33) <= 93)) { /* 33 to 126 included */
if (*ptr == '#') {
if (h1m->err_pos < -1) /* PR_O2_REQBUG_OK not set */
goto invalid_char;
if (h1m->err_pos == -1) /* PR_O2_REQBUG_OK set: just log */
h1m->err_pos = ptr - start + skip;
}
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rquri2, http_msg_ood, state, H1_MSG_RQURI);
BUG/MINOR: h1: do not accept '#' as part of the URI component Seth Manesse and Paul Plasil reported that the "path" sample fetch function incorrectly accepts '#' as part of the path component. This can in some cases lead to misrouted requests for rules that would apply on the suffix: use_backend static if { path_end .png .jpg .gif .css .js } Note that this behavior can be selectively configured using "normalize-uri fragment-encode" and "normalize-uri fragment-strip". The problem is that while the RFC says that this '#' must never be emitted, as often it doesn't suggest how servers should handle it. A diminishing number of servers still do accept it and trim it silently, while others are rejecting it, as indicated in the conversation below with other implementers: https://lists.w3.org/Archives/Public/ietf-http-wg/2023JulSep/0070.html Looking at logs from publicly exposed servers, such requests appear at a rate of roughly 1 per million and only come from attacks or poorly written web crawlers incorrectly following links found on various pages. Thus it looks like the best solution to this problem is to simply reject such ambiguous requests by default, and include this in the list of controls that can be disabled using "option accept-invalid-http-request". We're already rejecting URIs containing any control char anyway, so we should also reject '#'. In the H1 parser for the H1_MSG_RQURI state, there is an accelerated parser for bytes 0x21..0x7e that has been tightened to 0x24..0x7e (it should not impact perf since 0x21..0x23 are not supposed to appear in a URI anyway). This way '#' falls through the fine-grained filter and we can add the special case for it also conditionned by a check on the proxy's option "accept-invalid-http-request", with no overhead for the vast majority of valid URIs. Here this information is available through h1m->err_pos that's set to -2 when the option is here (so we don't need to change the API to expose the proxy). Example with a trivial GET through netcat: [08/Aug/2023:16:16:52.651] frontend layer1 (#2): invalid request backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:50812 buffer starts at 0 (including 0 out), 16361 free, len 23, wraps at 16336, error at position 7 H1 connection flags 0x00000000, H1 stream flags 0x00000810 H1 msg state MSG_RQURI(4), H1 msg flags 0x00001400 H1 chunk len 0 bytes, H1 body len 0 bytes : 00000 GET /aa#bb HTTP/1.0\r\n 00021 \r\n This should be progressively backported to all stable versions along with the following patch: REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests Similar fixes for h2 and h3 will come in followup patches. Thanks to Seth Manesse and Paul Plasil for reporting this problem with detailed explanations.
2023-08-08 10:17:22 -04:00
}
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
if (likely(HTTP_IS_SPHT(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.u.len = ptr - sl.rq.u.ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rquri_sp, http_msg_ood, state, H1_MSG_RQURI_SP);
}
if (likely((unsigned char)*ptr >= 128)) {
/* non-ASCII chars are forbidden unless option
* accept-invalid-http-request is enabled in the frontend.
* In any case, we capture the faulty char.
*/
if (h1m->err_pos < -1)
goto invalid_char;
if (h1m->err_pos == -1)
h1m->err_pos = ptr - start + skip;
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rquri, http_msg_ood, state, H1_MSG_RQURI);
}
if (likely(HTTP_IS_CRLF(*ptr))) {
/* so it's a CR/LF, meaning an HTTP 0.9 request */
goto http_msg_req09_uri_e;
}
/* OK forbidden chars, 0..31 or 127 */
invalid_char:
state = H1_MSG_RQURI;
goto http_msg_invalid;
case H1_MSG_RQURI_SP:
http_msg_rquri_sp:
if (likely(!HTTP_IS_LWS(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.v.ptr = ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
goto http_msg_rqver;
}
if (likely(HTTP_IS_SPHT(*ptr)))
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rquri_sp, http_msg_ood, state, H1_MSG_RQURI_SP);
/* so it's a CR/LF, meaning an HTTP 0.9 request */
goto http_msg_req09_ver;
case H1_MSG_RQVER:
http_msg_rqver:
if (likely(HTTP_IS_VER_TOKEN(*ptr)))
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqver, http_msg_ood, state, H1_MSG_RQVER);
if (likely(HTTP_IS_CRLF(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.rq.v.len = ptr - sl.rq.v.ptr;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
http_msg_rqline_eol:
/* We have seen the end of line. Note that we do not
* necessarily have the \n yet, but at least we know that we
* have EITHER \r OR \n, otherwise the request would not be
* complete. We can then record the request length and return
* to the caller which will be able to register it.
*/
if (likely(!skip_update)) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
if ((sl.rq.v.len == 8) &&
(*(sl.rq.v.ptr + 5) > '1' ||
(*(sl.rq.v.ptr + 5) == '1' && *(sl.rq.v.ptr + 7) >= '1')))
MINOR: h1: report in the h1m struct if the HTTP version is 1.1 or above This will be needed for the mux to know how to process the Connection header, and will save it from having to re-parse the request line since it's captured on the fly.
2018-09-13 05:32:51 -04:00
h1m->flags |= H1_MF_VER_11;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
if (unlikely(hdr_count >= hdr_num)) {
state = H1_MSG_RQVER;
goto http_output_full;
}
MINOR: h1: Add the flag H1_MF_NO_PHDR to not add pseudo-headers during parsing Some pseudo-headers are added during the headers parsing, mainly for the mux H2. With this flag, it is possible to not add them. This avoid some boring filtering in the mux H1.
2018-10-08 09:50:15 -04:00
if (!(h1m->flags & H1_MF_NO_PHDR))
http_set_hdr(&hdr[hdr_count++], ist(":method"), sl.rq.m);
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
if (unlikely(hdr_count >= hdr_num)) {
state = H1_MSG_RQVER;
goto http_output_full;
}
MINOR: h1: Add the flag H1_MF_NO_PHDR to not add pseudo-headers during parsing Some pseudo-headers are added during the headers parsing, mainly for the mux H2. With this flag, it is possible to not add them. This avoid some boring filtering in the mux H1.
2018-10-08 09:50:15 -04:00
if (!(h1m->flags & H1_MF_NO_PHDR))
http_set_hdr(&hdr[hdr_count++], ist(":path"), sl.rq.u);
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
}
sol = ptr - start;
if (likely(*ptr == '\r'))
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rqline_end, http_msg_ood, state, H1_MSG_RQLINE_END);
goto http_msg_rqline_end;
}
/* neither an HTTP_VER token nor a CRLF */
state = H1_MSG_RQVER;
goto http_msg_invalid;
case H1_MSG_RQLINE_END:
http_msg_rqline_end:
/* check for HTTP/0.9 request : no version information
* available. sol must point to the first of CR or LF. However
* since we don't save these elements between calls, if we come
* here from a restart, we don't necessarily know. Thus in this
* case we simply start over.
*/
if (restarting)
goto restart;
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
if (unlikely(sl.rq.v.len == 0))
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
goto http_msg_last_lf;
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_RQLINE_END);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_first, http_msg_ood, state, H1_MSG_HDR_FIRST);
/* stop here */
/*
* Common states below
*/
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPBEFORE:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpbefore:
if (likely(HTTP_IS_TOKEN(*ptr))) {
/* we have a start of message, we may have skipped some
* heading CRLF. Skip them now.
*/
skip += ptr - start;
start = ptr;
sol = 0;
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.v.ptr = ptr;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
hdr_count = 0;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_RPVER;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_rpver;
}
if (unlikely(!HTTP_IS_CRLF(*ptr))) {
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_RPBEFORE;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_invalid;
}
if (unlikely(*ptr == '\n'))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpbefore, http_msg_ood, state, H1_MSG_RPBEFORE);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpbefore_cr, http_msg_ood, state, H1_MSG_RPBEFORE_CR);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* stop here */
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPBEFORE_CR:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpbefore_cr:
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_RPBEFORE_CR);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpbefore, http_msg_ood, state, H1_MSG_RPBEFORE);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* stop here */
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPVER:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpver:
if (likely(HTTP_IS_VER_TOKEN(*ptr)))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpver, http_msg_ood, state, H1_MSG_RPVER);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
if (likely(HTTP_IS_SPHT(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.v.len = ptr - sl.st.v.ptr;
MINOR: h1: report in the h1m struct if the HTTP version is 1.1 or above This will be needed for the mux to know how to process the Connection header, and will save it from having to re-parse the request line since it's captured on the fly.
2018-09-13 05:32:51 -04:00
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
if ((sl.st.v.len == 8) &&
(*(sl.st.v.ptr + 5) > '1' ||
(*(sl.st.v.ptr + 5) == '1' && *(sl.st.v.ptr + 7) >= '1')))
MINOR: h1: report in the h1m struct if the HTTP version is 1.1 or above This will be needed for the mux to know how to process the Connection header, and will save it from having to re-parse the request line since it's captured on the fly.
2018-09-13 05:32:51 -04:00
h1m->flags |= H1_MF_VER_11;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpver_sp, http_msg_ood, state, H1_MSG_RPVER_SP);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_RPVER;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_invalid;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPVER_SP:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpver_sp:
if (likely(!HTTP_IS_LWS(*ptr))) {
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
sl.st.status = 0;
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.c.ptr = ptr;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_rpcode;
}
if (likely(HTTP_IS_SPHT(*ptr)))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpver_sp, http_msg_ood, state, H1_MSG_RPVER_SP);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* so it's a CR/LF, this is invalid */
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_RPVER_SP;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_invalid;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPCODE:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpcode:
BUG/MINOR: h1: the HTTP/1 make status code parser check for digits The H1 parser used by the H2 gateway was a bit lax and could validate non-numbers in the status code. Since it computes the code on the fly it's problematic, as "30:" is read as status code 310. Let's properly check that it's a number now. No backport needed.
2017-11-09 05:15:45 -05:00
if (likely(HTTP_IS_DIGIT(*ptr))) {
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
sl.st.status = sl.st.status * 10 + *ptr - '0';
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpcode, http_msg_ood, state, H1_MSG_RPCODE);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
BUG/MINOR: h1: the HTTP/1 make status code parser check for digits The H1 parser used by the H2 gateway was a bit lax and could validate non-numbers in the status code. Since it computes the code on the fly it's problematic, as "30:" is read as status code 310. Let's properly check that it's a number now. No backport needed.
2017-11-09 05:15:45 -05:00
if (unlikely(!HTTP_IS_LWS(*ptr))) {
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_RPCODE;
BUG/MINOR: h1: the HTTP/1 make status code parser check for digits The H1 parser used by the H2 gateway was a bit lax and could validate non-numbers in the status code. Since it computes the code on the fly it's problematic, as "30:" is read as status code 310. Let's properly check that it's a number now. No backport needed.
2017-11-09 05:15:45 -05:00
goto http_msg_invalid;
}
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
if (likely(HTTP_IS_SPHT(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.c.len = ptr - sl.st.c.ptr;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpcode_sp, http_msg_ood, state, H1_MSG_RPCODE_SP);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
/* so it's a CR/LF, so there is no reason phrase */
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.c.len = ptr - sl.st.c.ptr;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rsp_reason:
CLEANUP: Reapply ist.cocci with `--include-headers-for-types --recursive-includes` Previous uses of `ist.cocci` did not add `--include-headers-for-types` and `--recursive-includes` preventing Coccinelle seeing `struct ist` members of other structs. Reapply the patch with proper flags to further clean up the use of the ist API. The command used was: spatch -sp_file dev/coccinelle/ist.cocci -in_place --include-headers --include-headers-for-types --recursive-includes --dir src/
2022-03-15 08:11:06 -04:00
sl.st.r = ist2(ptr, 0);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_rpline_eol;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPCODE_SP:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpcode_sp:
if (likely(!HTTP_IS_LWS(*ptr))) {
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.r.ptr = ptr;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_rpreason;
}
if (likely(HTTP_IS_SPHT(*ptr)))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpcode_sp, http_msg_ood, state, H1_MSG_RPCODE_SP);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* so it's a CR/LF, so there is no reason phrase */
goto http_msg_rsp_reason;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPREASON:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpreason:
if (likely(!HTTP_IS_CRLF(*ptr)))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpreason, http_msg_ood, state, H1_MSG_RPREASON);
MINOR: h1: Change the union h1_sl to use indirect strings to store infos Instead of using offsets relating to the parsed buffer to store start line infos, we now use indirect strings. So now, these infos remain valid only if the origin buffer remains untouched. But it's not a real problem because this union is used during the parsing and never stored to a later use.
2018-10-08 09:34:02 -04:00
sl.st.r.len = ptr - sl.st.r.ptr;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpline_eol:
/* We have seen the end of line. Note that we do not
* necessarily have the \n yet, but at least we know that we
* have EITHER \r OR \n, otherwise the response would not be
* complete. We can then record the response length and return
* to the caller which will be able to register it.
*/
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
if (likely(!skip_update)) {
if (unlikely(hdr_count >= hdr_num)) {
state = H1_MSG_RPREASON;
goto http_output_full;
}
MINOR: h1: Add the flag H1_MF_NO_PHDR to not add pseudo-headers during parsing Some pseudo-headers are added during the headers parsing, mainly for the mux H2. With this flag, it is possible to not add them. This avoid some boring filtering in the mux H1.
2018-10-08 09:50:15 -04:00
if (!(h1m->flags & H1_MF_NO_PHDR))
http_set_hdr(&hdr[hdr_count++], ist(":status"), sl.st.c);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
sol = ptr - start;
if (likely(*ptr == '\r'))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_rpline_end, http_msg_ood, state, H1_MSG_RPLINE_END);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_rpline_end;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_RPLINE_END:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_rpline_end:
/* sol must point to the first of CR or LF. */
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_RPLINE_END);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_first, http_msg_ood, state, H1_MSG_HDR_FIRST);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* stop here */
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_FIRST:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_first:
sol = ptr - start;
if (likely(!HTTP_IS_CRLF(*ptr))) {
goto http_msg_hdr_name;
}
if (likely(*ptr == '\r'))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_last_lf, http_msg_ood, state, H1_MSG_LAST_LF);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_last_lf;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_NAME:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_name:
/* assumes sol points to the first char */
if (likely(HTTP_IS_TOKEN(*ptr))) {
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
if (!skip_update) {
/* turn it to lower case if needed */
if (isupper((unsigned char)*ptr) && h1m->flags & H1_MF_TOLOWER)
BUILD: tree-wide: cast arguments to tolower/toupper to unsigned char NetBSD apparently uses macros for tolower/toupper and complains about the use of char for array subscripts. Let's properly cast all of them to unsigned char where they are used. This is needed to fix issue #729.
2020-07-05 15:46:32 -04:00
*ptr = tolower((unsigned char)*ptr);
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
}
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_name, http_msg_ood, state, H1_MSG_HDR_NAME);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
if (likely(*ptr == ':')) {
col = ptr - start;
BUG/CRITICAL: http: properly reject empty http header field names The HTTP header parsers surprizingly accepts empty header field names, and this is a leftover from the original code that was agnostic to this. When muxes were introduced, for H2 first, the HPACK decompressor needed to feed headers lists, and since empty header names were strictly forbidden by the protocol, the lists of headers were purposely designed to be terminated by an empty header field name (a principle that is similar to H1's empty line termination). This principle was preserved and generalized to other protocols migrated to muxes (H1/FCGI/H3 etc) without anyone ever noticing that the H1 parser was still able to deliver empty header field names to this list. In addition to this it turns out that the HPACK decompressor, despite a comment in the code, may successfully decompress an empty header field name, and this mistake was propagated to the QPACK decompressor as well. The impact is that an empty header field name may be used to truncate the list of headers and thus make some headers disappear. While for H2/H3 the impact is limited as haproxy sees a request with missing headers, and headers are not used to delimit messages, in the case of HTTP/1, the impact is significant because the presence (and sometimes contents) of certain sensitive headers is detected during the parsing. Thus, some of these headers may be seen, marked as present, their value extracted, but never delivered to upper layers and obviously not forwarded to the other side either. This can have for consequence that certain important header fields such as Connection, Upgrade, Host, Content-length, Transfer-Encoding etc are possibly seen as different between what haproxy uses to parse/forward/route and what is observed in http-request rules and of course, forwarded. One direct consequence is that it is possible to exploit this property in HTTP/1 to make affected versions of haproxy forward more data than is advertised on the other side, and bypass some access controls or routing rules by crafting extraneous requests. Note, however, that responses to such requests will normally not be passed back to the client, but this can still cause some harm. This specific risk can be mostly worked around in configuration using the following rule that will rely on the bug's impact to precisely detect the inconsistency between the known body size and the one expected to be advertised to the server (the rule works from 2.0 to 2.8-dev): http-request deny if { fc_http_major 1 } !{ req.body_size 0 } !{ req.hdr(content-length) -m found } !{ req.hdr(transfer-encoding) -m found } !{ method CONNECT } This will exclusively block such carefully crafted requests delivered over HTTP/1. HTTP/2 and HTTP/3 do not need content-length, and a body that arrives without being announced with a content-length will be forwarded using transfer-encoding, hence will not cause discrepancies. In HAProxy 2.0 in legacy mode ("no option http-use-htx"), this rule will simply have no effect but will not cause trouble either. A clean solution would consist in modifying the loops iterating over these headers lists to check the header name's pointer instead of its length (since both are zero at the end of the list), but this requires to touch tens of places and it's very easy to miss one. Functions such as htx_add_header(), htx_add_trailer(), htx_add_all_headers() would be good starting points for such a possible future change. Instead the current fix focuses on blocking empty headers where they are first inserted, hence in the H1/HPACK/QPACK decoders. One benefit of the current solution (for H1) is that it allows "show errors" to report a precise diagnostic when facing such invalid HTTP/1 requests, with the exact location of the problem and the originating address: $ printf "GET / HTTP/1.1\r\nHost: localhost\r\n:empty header\r\n\r\n" | nc 0 8001 HTTP/1.1 400 Bad request Content-length: 90 Cache-Control: no-cache Connection: close Content-Type: text/html <html><body><h1>400 Bad request</h1> Your browser sent an invalid request. </body></html> $ socat /var/run/haproxy.stat <<< "show errors" Total events captured on [10/Feb/2023:16:29:37.530] : 1 [10/Feb/2023:16:29:34.155] frontend decrypt (#2): invalid request backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:31092 buffer starts at 0 (including 0 out), 16334 free, len 50, wraps at 16336, error at position 33 H1 connection flags 0x00000000, H1 stream flags 0x00000810 H1 msg state MSG_HDR_NAME(17), H1 msg flags 0x00001410 H1 chunk len 0 bytes, H1 body len 0 bytes : 00000 GET / HTTP/1.1\r\n 00016 Host: localhost\r\n 00033 :empty header\r\n 00048 \r\n I want to address sincere and warm thanks for their great work to the team composed of the following security researchers who found the issue together and reported it: Bahruz Jabiyev, Anthony Gavazzi, and Engin Kirda from Northeastern University, Kaan Onarlioglu from Akamai Technologies, Adi Peleg and Harvey Tuch from Google. And kudos to Amaury Denoyelle from HAProxy Technologies for spotting that the HPACK and QPACK decoders would let this pass despite the comment explicitly saying otherwise. This fix must be backported as far as 2.0. The QPACK changes can be dropped before 2.6. In 2.0 there is also the equivalent code for legacy mode, which doesn't suffer from the list truncation, but it would better be fixed regardless. CVE-2023-25725 was assigned to this issue.
2023-02-09 15:36:54 -05:00
if (col <= sol) {
state = H1_MSG_HDR_NAME;
goto http_msg_invalid;
}
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_l1_sp, http_msg_ood, state, H1_MSG_HDR_L1_SP);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
MEDIUM: h1: consider err_pos before deciding to accept a header name or not Till now the H1 parser made for H2 used to be lenient on invalid header field names because they were supposed to be produced by haproxy. Now instead we'll rely on err_pos to know how to act (ie: -2 == must block).
2018-09-12 03:20:40 -04:00
if (likely(h1m->err_pos < -1) || *ptr == '\n') {
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_HDR_NAME;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_invalid;
}
MEDIUM: h1: consider err_pos before deciding to accept a header name or not Till now the H1 parser made for H2 used to be lenient on invalid header field names because they were supposed to be produced by haproxy. Now instead we'll rely on err_pos to know how to act (ie: -2 == must block).
2018-09-12 03:20:40 -04:00
if (h1m->err_pos == -1) /* capture the error pointer */
h1m->err_pos = ptr - start + skip; /* >= 0 now */
/* and we still accept this non-token character */
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_name, http_msg_ood, state, H1_MSG_HDR_NAME);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_L1_SP:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_l1_sp:
/* assumes sol points to the first char */
if (likely(HTTP_IS_SPHT(*ptr)))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_l1_sp, http_msg_ood, state, H1_MSG_HDR_L1_SP);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
/* header value can be basically anything except CR/LF */
sov = ptr - start;
if (likely(!HTTP_IS_CRLF(*ptr))) {
goto http_msg_hdr_val;
}
if (likely(*ptr == '\r'))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_l1_lf, http_msg_ood, state, H1_MSG_HDR_L1_LF);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_hdr_l1_lf;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_L1_LF:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_l1_lf:
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_HDR_L1_LF);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_l1_lws, http_msg_ood, state, H1_MSG_HDR_L1_LWS);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_L1_LWS:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_l1_lws:
if (likely(HTTP_IS_SPHT(*ptr))) {
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
if (!skip_update) {
/* replace HT,CR,LF with spaces */
for (; start + sov < ptr; sov++)
start[sov] = ' ';
}
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_hdr_l1_sp;
}
/* we had a header consisting only in spaces ! */
eol = sov;
goto http_msg_complete_header;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_VAL:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_val:
/* assumes sol points to the first char, and sov
* points to the first character of the value.
*/
/* speedup: we'll skip packs of 4 or 8 bytes not containing bytes 0x0D
* and lower. In fact since most of the time is spent in the loop, we
* also remove the sign bit test so that bytes 0x8e..0x0d break the
* loop, but we don't care since they're very rare in header values.
*/
CLEANUP: http/h1: rely on HA_UNALIGNED_LE instead of checking for CPU families Now that we have flags indicating the CPU's capabilities, better use them instead of missing some updates for new CPU families (ARMv8 was missing there).
2020-02-21 10:31:22 -05:00
#ifdef HA_UNALIGNED_LE64
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
while (ptr <= end - sizeof(long)) {
if ((*(long *)ptr - 0x0e0e0e0e0e0e0e0eULL) & 0x8080808080808080ULL)
goto http_msg_hdr_val2;
ptr += sizeof(long);
}
#endif
CLEANUP: http/h1: rely on HA_UNALIGNED_LE instead of checking for CPU families Now that we have flags indicating the CPU's capabilities, better use them instead of missing some updates for new CPU families (ARMv8 was missing there).
2020-02-21 10:31:22 -05:00
#ifdef HA_UNALIGNED_LE
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
while (ptr <= end - sizeof(int)) {
if ((*(int*)ptr - 0x0e0e0e0e) & 0x80808080)
goto http_msg_hdr_val2;
ptr += sizeof(int);
}
#endif
if (ptr >= end) {
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
state = H1_MSG_HDR_VAL;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_ood;
}
http_msg_hdr_val2:
BUG/MEDIUM: h1: always reject the NUL character in header values Ben Kallus kindly reported that we still hadn't blocked the NUL character from header values as clarified in RFC9110 and that, even though there's no known issure related to this, it may one day be used to construct an attack involving another component. Actually, both Christopher and I sincerely believed we had done it prior to releasing 2.9, shame on us for missing that one and thanks to Ben for the reminder! The change was applied, it was confirmed to properly reject this NUL byte from both header and trailer values, and it's still possible to force it to continue to be supported using the usual pair of unsafe "option accept-invalid-http-{request|response}" for those who would like to keep it for whatever reason that wouldn't make sense. This was tagged medium so that distros also remember to apply it as a preventive measure. It should progressively be backported to all versions down to 2.0.
2024-01-31 09:10:39 -05:00
if (likely(!*ptr)) {
/* RFC9110 clarified that NUL is explicitly forbidden in header values
* (like CR and LF).
*/
if (h1m->err_pos < -1) { /* PR_O2_REQBUG_OK not set */
state = H1_MSG_HDR_VAL;
goto http_msg_invalid;
}
if (h1m->err_pos == -1) /* PR_O2_REQBUG_OK set: just log */
h1m->err_pos = ptr - start + skip;
}
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
if (likely(!HTTP_IS_CRLF(*ptr)))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_val2, http_msg_ood, state, H1_MSG_HDR_VAL);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
eol = ptr - start;
/* Note: we could also copy eol into ->eoh so that we have the
* real header end in case it ends with lots of LWS, but is this
* really needed ?
*/
if (likely(*ptr == '\r'))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_l2_lf, http_msg_ood, state, H1_MSG_HDR_L2_LF);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_hdr_l2_lf;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_L2_LF:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_l2_lf:
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_HDR_L2_LF);
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_hdr_l2_lws, http_msg_ood, state, H1_MSG_HDR_L2_LWS);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_HDR_L2_LWS:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_hdr_l2_lws:
if (unlikely(HTTP_IS_SPHT(*ptr))) {
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
if (!skip_update) {
/* LWS: replace HT,CR,LF with spaces */
for (; start + eol < ptr; eol++)
start[eol] = ' ';
}
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_hdr_val;
}
http_msg_complete_header:
/*
* It was a new header, so the last one is finished. Assumes
* <sol> points to the first char of the name, <col> to the
* colon, <sov> points to the first character of the value and
* <eol> to the first CR or LF so we know how the line ends. We
* will trim spaces around the value. It's possible to do it by
* adjusting <eol> and <sov> which are no more used after this.
* We can add the header field to the list.
*/
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
if (likely(!skip_update)) {
while (sov < eol && HTTP_IS_LWS(start[sov]))
sov++;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
while (eol - 1 > sov && HTTP_IS_LWS(start[eol - 1]))
eol--;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
n = ist2(start + sol, col - sol);
v = ist2(start + sov, eol - sov);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
do {
int ret;
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
if (unlikely(hdr_count >= hdr_num)) {
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
state = H1_MSG_HDR_L2_LWS;
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
goto http_output_full;
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
}
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
if (isteqi(n, ist("transfer-encoding"))) {
MINOR: h1: Change T-E header parsing to fail if chunked encoding is found twice According to the RFC7230, "chunked" encoding must not be applied more than once to a message body. To handle this case, h1_parse_xfer_enc_header() is now responsible to fail when a parsing error is found. It also fails if the "chunked" encoding is not the last one for a request. To help the parsing, two H1 parser flags have been added: H1_MF_TE_CHUNKED and H1_MF_TE_OTHER. These flags are set, respectively, when "chunked" encoding and any other encoding are found. H1_MF_CHNK flag is used when "chunked" encoding is the last one.
2021-09-28 03:36:25 -04:00
ret = h1_parse_xfer_enc_header(h1m, v);
if (ret < 0) {
state = H1_MSG_HDR_L2_LWS;
ptr = v.ptr; /* Set ptr on the error */
goto http_msg_invalid;
}
else if (ret == 0) {
/* skip it */
break;
}
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
}
else if (isteqi(n, ist("content-length"))) {
ret = h1_parse_cont_len_header(h1m, &v);
if (ret < 0) {
state = H1_MSG_HDR_L2_LWS;
BUG/MINOR: h1: Report the right error position when a header value is invalid During H1 messages parsing, when the parser has finished to parse a full header line, some tests are performed on its value, depending on its name, to be sure it is valid. The content-length is checked and converted in integer and the host header is also checked. If an error occurred during this step, the error position must point on the header value. But from the parser point of view, we are already on the start of the next header. Thus the effective reported position in the error capture is the beginning of the unparsed header line. It is a bit confusing when we try to figure out why a message is rejected. Now, the parser state is updated to point on the invalid value. This way, the error position really points on the right position. This patch must be backported as far as 1.9.
2020-01-06 07:41:01 -05:00
ptr = v.ptr; /* Set ptr on the error */
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
goto http_msg_invalid;
}
else if (ret == 0) {
/* skip it */
break;
}
}
else if (isteqi(n, ist("connection"))) {
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
h1_parse_connection_header(h1m, &v);
if (!v.len) {
/* skip it */
break;
MINOR: h1: Reject requests with different occurrences of the header host There is no reason for a client to send several headers host. It even may be considered as a bug. However, it is totally invalid to have different values for those. So now, in such case, an error is triggered during the request parsing. In addition, when several headers host are found with the same value, only the first instance is kept and others are skipped.
2019-10-11 03:01:44 -04:00
}
}
MINOR: h1: reject websocket handshake if missing key If a request is identified as a WebSocket handshake, it must contains a websocket key header or else it can be reject, following the rfc6455. A new flag H1_MF_UPG_WEBSOCKET is set on such messages. For the request te be identified as a WebSocket handshake, it must contains the headers: Connection: upgrade Upgrade: websocket This commit is a compagnon of "MEDIUM: h1: generate WebSocket key on response if needed" and "MEDIUM: h1: add a WebSocket key on handshake if needed". Indeed, it ensures that a WebSocket key is added only from a http/2 side and not for a http/1 bogus peer.
2020-12-11 11:53:02 -05:00
else if (isteqi(n, ist("upgrade"))) {
h1_parse_upgrade_header(h1m, v);
}
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
else if (!(h1m->flags & H1_MF_RESP) && isteqi(n, ist("host"))) {
if (host_idx == -1)
MINOR: h1: Reject requests with different occurrences of the header host There is no reason for a client to send several headers host. It even may be considered as a bug. However, it is totally invalid to have different values for those. So now, in such case, an error is triggered during the request parsing. In addition, when several headers host are found with the same value, only the first instance is kept and others are skipped.
2019-10-11 03:01:44 -04:00
host_idx = hdr_count;
else {
if (!isteqi(v, hdr[host_idx].v)) {
state = H1_MSG_HDR_L2_LWS;
BUG/MINOR: h1: Report the right error position when a header value is invalid During H1 messages parsing, when the parser has finished to parse a full header line, some tests are performed on its value, depending on its name, to be sure it is valid. The content-length is checked and converted in integer and the host header is also checked. If an error occurred during this step, the error position must point on the header value. But from the parser point of view, we are already on the start of the next header. Thus the effective reported position in the error capture is the beginning of the unparsed header line. It is a bit confusing when we try to figure out why a message is rejected. Now, the parser state is updated to point on the invalid value. This way, the error position really points on the right position. This patch must be backported as far as 1.9.
2020-01-06 07:41:01 -05:00
ptr = v.ptr; /* Set ptr on the error */
MINOR: h1: Reject requests with different occurrences of the header host There is no reason for a client to send several headers host. It even may be considered as a bug. However, it is totally invalid to have different values for those. So now, in such case, an error is triggered during the request parsing. In addition, when several headers host are found with the same value, only the first instance is kept and others are skipped.
2019-10-11 03:01:44 -04:00
goto http_msg_invalid;
}
/* if the same host, skip it */
break;
MEDIUM: h1: Add an option to sanitize connection headers during parsing The flag H1_MF_CLEAN_CONN_HDR has been added to let the H1 parser sanitize connection headers. It means it will remove all "close" and "keep-alive" values during the parsing. One noticeable effect is that connection headers may be unfolded. In practice, this is not a problem because it is not frequent to have multiple values for the connection headers. If this flag is set, during the parsing The function h1_parse_next_connection_header() is called in a loop instead of h1_parse_conection_header(). No need to backport this patch
2019-03-29 10:03:13 -04:00
}
MEDIUM: h1: deduplicate the content-length header Just like we used to do in proto_http, we now check that each and every occurrence of the content-length header field and each of its values are exactly identical, and we normalize the header to return the last value of the first header with spaces trimmed.
2018-09-14 11:11:33 -04:00
}
MINOR: h1: add headers to the list after controls, not before This will ease removal/skipping of duplicates such as content-length.
2018-09-14 10:28:15 -04:00
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
http_set_hdr(&hdr[hdr_count++], n, v);
} while (0);
}
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
sol = ptr - start;
BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates must be skipped until the end of the message is found. Then the parsing is restarted from the beginning. But not all updates were skipped, leading to invalid rewritting or segfault. No backport is needed.
2018-09-19 08:01:04 -04:00
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
if (likely(!HTTP_IS_CRLF(*ptr)))
goto http_msg_hdr_name;
if (likely(*ptr == '\r'))
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EAT_AND_JUMP_OR_RETURN(ptr, end, http_msg_last_lf, http_msg_ood, state, H1_MSG_LAST_LF);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
goto http_msg_last_lf;
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
case H1_MSG_LAST_LF:
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_last_lf:
REORG: h1: create a new h1m_state This is the *parsing* state of an HTTP/1 message. Currently the h1_state is composite as it's made both of parsing and control (100SENT, BODY, DONE, TUNNEL, ENDING etc). The purpose here is to have a purely H1 state that can be used by H1 parsers. For now it's equivalent to h1_state.
2018-09-11 05:45:04 -04:00
EXPECT_LF_HERE(ptr, http_msg_invalid, state, H1_MSG_LAST_LF);
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
ptr++;
/* <ptr> now points to the first byte of payload. If needed sol
* still points to the first of either CR or LF of the empty
* line ending the headers block.
*/
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
if (likely(!skip_update)) {
if (unlikely(hdr_count >= hdr_num)) {
state = H1_MSG_LAST_LF;
goto http_output_full;
}
MINOR: h1: Add EOH marker during headers parsing When headers parsing ends, a pseudo header with an empty name and an empty value is added to the array of parsed headers to mark its end. It is convenient to loop on this array, but not really useful if we want remove the last header or add a new one, because we don't really know where is the last CRLF (the empty line ending the headers block). So now, instead the name of this pseudo header points on this last CRLF. Its length is still 0 and its value is still empty, so loops on the array remains unchanged.
2018-09-25 07:59:46 -04:00
http_set_hdr(&hdr[hdr_count++], ist2(start+sol, 0), ist(""));
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
MEDIUM: h1: remove the useless H1_MSG_BODY state This state was only a delimiter between headers and body but it now causes more harm than good because it requires someone to change it. Since the H1 parser knows if we're in DATA or CHUNK_SIZE, simply let it set the right next state so that h1m->state constantly matches what is expected afterwards.
2018-09-12 11:25:32 -04:00
/* reaching here we've parsed the whole message. We may detect
* that we were already continuing an interrupted parsing pass
* so we were silently looking for the end of message not
* updating anything before deciding to parse it fully at once.
* It's guaranteed that we won't match this test twice in a row
* since restarting will turn zero.
*/
if (restarting)
goto restart;
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
if (!(h1m->flags & (H1_MF_HDRS_ONLY|H1_MF_RESP))) {
struct http_uri_parser parser = http_uri_parser_init(sl.rq.u);
BUG/MINOR: h1: Replace authority validation to conform RFC3986 Except for CONNECT method, where a normalization is performed, we expected to have an exact match between the authority and the host header value. However it was too strict. Indeed, default port must be handled and the matching must respect the RFC3986. There is already a scheme based normalizeation performed on the URI later, on the HTX message. And we cannot normalize the URI during H1 parsing to be able to report proper errors on the original raw buffer. And a systematic read-only normalization to validate the authority will consume CPU for only few requests. At the end, we decided to perform extra-checks when the exact match fails. Now, following authority/host are considered as equivalent: http: domain.com <=> domain.com:80 <=> domain.com: https: domain.com <=> domain.com:443 <=> domain.com: This patch depends on: * MINOR: h1: Consider empty port as invalid in authority for CONNECT * MINOR: http: Considere empty ports as valid default ports It is a bug regarding the RFC3986. Technically, I may be backported as far as 2.4. However, this must be discussed first. If it is backported, the commits above must be backported too.
2022-11-22 04:04:16 -05:00
struct ist scheme, authority;
int ret;
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
BUG/MINOR: h1: Replace authority validation to conform RFC3986 Except for CONNECT method, where a normalization is performed, we expected to have an exact match between the authority and the host header value. However it was too strict. Indeed, default port must be handled and the matching must respect the RFC3986. There is already a scheme based normalizeation performed on the URI later, on the HTX message. And we cannot normalize the URI during H1 parsing to be able to report proper errors on the original raw buffer. And a systematic read-only normalization to validate the authority will consume CPU for only few requests. At the end, we decided to perform extra-checks when the exact match fails. Now, following authority/host are considered as equivalent: http: domain.com <=> domain.com:80 <=> domain.com: https: domain.com <=> domain.com:443 <=> domain.com: This patch depends on: * MINOR: h1: Consider empty port as invalid in authority for CONNECT * MINOR: http: Considere empty ports as valid default ports It is a bug regarding the RFC3986. Technically, I may be backported as far as 2.4. However, this must be discussed first. If it is backported, the commits above must be backported too.
2022-11-22 04:04:16 -05:00
scheme = http_parse_scheme(&parser);
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
authority = http_parse_authority(&parser, 1);
if (sl.rq.meth == HTTP_METH_CONNECT) {
struct ist *host = ((host_idx != -1) ? &hdr[host_idx].v : NULL);
ret = h1_validate_connect_authority(authority, host);
if (ret < 0) {
if (h1m->err_pos < -1) {
state = H1_MSG_LAST_LF;
BUILD: h1: silence an initiialized warning with gcc-4.7 and -Os Building h1.c with gcc-4.7 -Os produces the following warning: src/h1.c: In function 'h1_headers_to_hdr_list': src/h1.c:1101:36: warning: 'ptr' may be used uninitialized in this function [-Wmaybe-uninitialized] In fact ptr may be taken from sl.rq.u.ptr which is only initialized after passing through the relevant states, but gcc doesn't know which states are visited. Adding an ALREADY_CHECKED() statement there is sufficient to shut it up and doesn't affect the emitted code. This may be backported to stable versions to make sure that builds on older distros and systems is clean.
2022-10-04 02:02:03 -04:00
/* WT: gcc seems to see a path where sl.rq.u.ptr was used
* uninitialized, but it doesn't know that the function is
* called with initial states making this impossible.
*/
ALREADY_CHECKED(sl.rq.u.ptr);
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
ptr = ((ret == -1) ? sl.rq.u.ptr : host->ptr); /* Set ptr on the error */
goto http_msg_invalid;
}
if (h1m->err_pos == -1) /* capture the error pointer */
h1m->err_pos = ((ret == -1) ? sl.rq.u.ptr : host->ptr) - start + skip; /* >= 0 now */
}
}
else if (host_idx != -1 && istlen(authority)) {
struct ist host = hdr[host_idx].v;
/* For non-CONNECT method, the authority must match the host header value */
if (!isteqi(authority, host)) {
BUG/MINOR: h1: Replace authority validation to conform RFC3986 Except for CONNECT method, where a normalization is performed, we expected to have an exact match between the authority and the host header value. However it was too strict. Indeed, default port must be handled and the matching must respect the RFC3986. There is already a scheme based normalizeation performed on the URI later, on the HTX message. And we cannot normalize the URI during H1 parsing to be able to report proper errors on the original raw buffer. And a systematic read-only normalization to validate the authority will consume CPU for only few requests. At the end, we decided to perform extra-checks when the exact match fails. Now, following authority/host are considered as equivalent: http: domain.com <=> domain.com:80 <=> domain.com: https: domain.com <=> domain.com:443 <=> domain.com: This patch depends on: * MINOR: h1: Consider empty port as invalid in authority for CONNECT * MINOR: http: Considere empty ports as valid default ports It is a bug regarding the RFC3986. Technically, I may be backported as far as 2.4. However, this must be discussed first. If it is backported, the commits above must be backported too.
2022-11-22 04:04:16 -05:00
ret = h1_validate_mismatch_authority(scheme, authority, host);
if (ret < 0) {
if (h1m->err_pos < -1) {
state = H1_MSG_LAST_LF;
ptr = host.ptr; /* Set ptr on the error */
goto http_msg_invalid;
}
if (h1m->err_pos == -1) /* capture the error pointer */
h1m->err_pos = v.ptr - start + skip; /* >= 0 now */
BUG/MEDIUM: h1: Improve authority validation for CONNCET request From time to time, users complain to get 400-Bad-request responses for totally valid CONNECT requests. After analysis, it is due to the H1 parser performs an exact match between the authority and the host header value. For non-CONNECT requests, it is valid. But for CONNECT requests the authority must contain a port while it is often omitted from the host header value (for default ports). So, to be sure to not reject valid CONNECT requests, a basic authority validation is now performed during the message parsing. In addition, the host header value is normalized. It means the default port is removed if possible. This patch should solve the issue #1761. It must be backported to 2.6 and probably as far as 2.4.
2022-07-05 08:50:17 -04:00
}
}
}
}
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
state = H1_MSG_DATA;
if (h1m->flags & H1_MF_XFER_ENC) {
if (h1m->flags & H1_MF_CLEN) {
MEDIUM: h1: Force close mode for invalid uses of T-E header Transfer-Encoding header is not supported in HTTP/1.0. However, softwares dealing with HTTP/1.0 and HTTP/1.1 messages may accept it and transfer it. When a Content-Length header is also provided, it must be ignored. Unfortunately, this may lead to vulnerabilities (request smuggling or response splitting) if an intermediary is only implementing HTTP/1.0. Because it may ignore Transfer-Encoding header and only handle Content-Length one. To avoid any security issues, when Transfer-Encoding and Content-Length headers are found in a message, the close mode is forced. The same is performed for HTTP/1.0 message with a Transfer-Encoding header only. This change is conform to what it is described in the last HTTP/1.1 draft. See also httpwg/http-core#879. Note that Content-Length header is also removed from any incoming messages if a Transfer-Encoding header is found. However it is not true (not yet) for responses generated by HAProxy.
2021-09-27 03:47:03 -04:00
/* T-E + C-L: force close and remove C-L */
h1m->flags |= H1_MF_CONN_CLO;
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
h1m->flags &= ~H1_MF_CLEN;
BUG/MEDIUM: h1: Ignore C-L value in the H1 parser if T-E is also set In fact, during the parsing there is already a test to remove the Content-Length header if a Transfer-Encoding one is found. However, in the parser, the content-length value was still used to set the body length (the final one and the remaining one). This value is thus also used to set the extra field in the HTX message and is then used during the sending stage to announce the chunk size. So, Content-Length header value must be ignored by the H1 parser to properly reformat the message when it is sent. This patch must be backported as far as 2.6. Lower versions don"t handle this case.
2023-09-27 09:21:28 -04:00
h1m->curr_len = h1m->body_len = 0;
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
hdr_count = http_del_hdr(hdr, ist("content-length"));
}
MEDIUM: h1: Force close mode for invalid uses of T-E header Transfer-Encoding header is not supported in HTTP/1.0. However, softwares dealing with HTTP/1.0 and HTTP/1.1 messages may accept it and transfer it. When a Content-Length header is also provided, it must be ignored. Unfortunately, this may lead to vulnerabilities (request smuggling or response splitting) if an intermediary is only implementing HTTP/1.0. Because it may ignore Transfer-Encoding header and only handle Content-Length one. To avoid any security issues, when Transfer-Encoding and Content-Length headers are found in a message, the close mode is forced. The same is performed for HTTP/1.0 message with a Transfer-Encoding header only. This change is conform to what it is described in the last HTTP/1.1 draft. See also httpwg/http-core#879. Note that Content-Length header is also removed from any incoming messages if a Transfer-Encoding header is found. However it is not true (not yet) for responses generated by HAProxy.
2021-09-27 03:47:03 -04:00
else if (!(h1m->flags & H1_MF_VER_11)) {
/* T-E + HTTP/1.0: force close */
h1m->flags |= H1_MF_CONN_CLO;
}
MEDIUM: h1: better handle transfer-encoding vs content-length The transfer-encoding header processing was a bit lenient in this part because it was made to read messages already validated by haproxy. We absolutely need to reinstate the strict processing defined in RFC7230 as is currently being done in proto_http.c. That is, transfer-encoding presence alone is enough to cancel content-length, and must be terminated by the "chunked" token, except in the response where we can fall back to the close mode if it's not last. For this we now use a specific parsing function which updates the flags and we introduce a new flag H1_MF_XFER_ENC indicating that the transfer-encoding header is present. Last, if such a header is found, we delete all content-length header fields found in the message.
2018-09-14 10:34:47 -04:00
if (h1m->flags & H1_MF_CHNK)
state = H1_MSG_CHUNK_SIZE;
else if (!(h1m->flags & H1_MF_RESP)) {
/* cf RFC7230#3.3.3 : transfer-encoding in
* request without chunked encoding is invalid.
*/
goto http_msg_invalid;
}
}
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
break;
default:
/* impossible states */
goto http_msg_invalid;
}
MEDIUM: h1: remove the useless H1_MSG_BODY state This state was only a delimiter between headers and body but it now causes more harm than good because it requires someone to change it. Since the H1 parser knows if we're in DATA or CHUNK_SIZE, simply let it set the right next state so that h1m->state constantly matches what is expected afterwards.
2018-09-12 11:25:32 -04:00
/* Now we've left the headers state and are either in H1_MSG_DATA or
* H1_MSG_CHUNK_SIZE.
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
*/
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
if (slp && !skip_update)
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
*slp = sl;
MEDIUM: h1: let the caller pass the initial parser's state This way the caller controls if it's the request or response which has to be used, and it will allow to restart after an incomplete parsing.
2018-09-11 09:33:32 -04:00
h1m->state = state;
h1m->next = ptr - start + skip;
return h1m->next;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
http_msg_ood:
/* out of data at <ptr> during state <state> */
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
if (slp && !skip_update)
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
*slp = sl;
MEDIUM: h1: let the caller pass the initial parser's state This way the caller controls if it's the request or response which has to be used, and it will allow to restart after an incomplete parsing.
2018-09-11 09:33:32 -04:00
h1m->state = state;
h1m->next = ptr - start + skip;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
return 0;
http_msg_invalid:
/* invalid message, error at <ptr> */
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
if (slp && !skip_update)
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
*slp = sl;
MEDIUM: h1: let the caller pass the initial parser's state This way the caller controls if it's the request or response which has to be used, and it will allow to restart after an incomplete parsing.
2018-09-11 09:33:32 -04:00
h1m->err_state = h1m->state = state;
h1m->err_pos = h1m->next = ptr - start + skip;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
return -1;
http_output_full:
/* no more room to store the current header, error at <ptr> */
MINOR: h1: make the message parser support a null <hdr> argument This will allow some iterative calls to be made on incomplete messages without having to store all the headers.
2018-09-11 10:04:48 -04:00
if (slp && !skip_update)
MEDIUM: h1: make the parser support a pointer to a start line This will allow the parser to fill some extra fields like the method or status without having to store them permanently in the HTTP message. At this point however the parser cannot restart from an interrupted read.
2018-09-11 09:34:50 -04:00
*slp = sl;
MEDIUM: h1: let the caller pass the initial parser's state This way the caller controls if it's the request or response which has to be used, and it will allow to restart after an incomplete parsing.
2018-09-11 09:33:32 -04:00
h1m->err_state = h1m->state = state;
h1m->err_pos = h1m->next = ptr - start + skip;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
return -2;
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
restart:
BUG/MEDIUM: h1: Properly reset h1m flags when headers parsing is restarted If H1 headers are not fully received at once, the parsing is restarted a last time when all headers are finally received. When this happens, the h1m flags are sanitized to remove all value set during parsing. But some flags where erroneously preserved. Among others, H1_MF_TE_CHUNKED flag was not removed, what could lead to parsing error. To fix the bug and make things easy, a mask has been added with all flags that must be preserved. It will be more stable. This mask is used to sanitize h1m flags. This patch should fix the issue #1469. It must be backported to 2.5.
2021-12-01 12:01:48 -05:00
h1m->flags &= H1_MF_RESTART_MASK;
BUG/MINOR: h1: Properly reset h1m when parsing is restarted Otherwise some processing may be performed twice. For instance, if the header "Content-Length" is parsed on the first pass, when the parsing is restarted, we skip it because we think another header with the same value was already seen. In fact, it is currently the only existing bug that can be encountered. But it is safer to reset all the h1m on restart to avoid any future bugs. This patch must be backported to 2.0 and 1.9
2019-09-03 10:05:31 -04:00
h1m->curr_len = h1m->body_len = h1m->next = 0;
MEDIUM: h1: implement the request parser as well The original H1 request parsing code was reintroduced into the generic H1 parser so that it can be used regardless of the direction. If the parser is interrupted and restarts, it makes use of the H1_MF_RESP flag to decide whether to re-parse a request or a response. While parsing the request, the method is decoded and set into the start line structure.
2018-09-11 11:57:05 -04:00
if (h1m->flags & H1_MF_RESP)
h1m->state = H1_MSG_RPBEFORE;
else
h1m->state = H1_MSG_RQBEFORE;
MEDIUM: h1: support partial message parsing While it was not needed in the H2 mux which was reading full H1 messages from the channel, it is mandatory for the H1 mux reading contents from outside to be able to restart on a message. The problem is that the headers are indexed on the fly, and it's not fun to have to store everything between calls. The solution here is to complete the first pass doing a partial restart, and only once the end of message was found, to start over it again at once, filling entries. This way there is a bounded number of passes on the contents and no need to store an intermediary result anymore. Later this principle could even be used to decide to completely drop an output buffer to save memory.
2018-09-11 10:20:30 -04:00
goto try_again;
MEDIUM: h1: reimplement the http/1 response parser for the gateway The HTTP/2->HTTP/1 gateway will need to process HTTP/1 responses. We cannot sanely rely on the HTTP/1 txn to parse a response because : 1) responses generated by haproxy such as error messages, redirects, stats or Lua are neither parsed nor indexed ; this could be addressed over the long term but will take time. 2) the http txn is useless to parse the body : the states present there are only meaningful to received bytes (ie next bytes to parse) and not at all to sent bytes. Thus chunks cannot be followed at all. Even when implementing this later, it's unsure whether it will be possible when dealing with compression. So using the HTTP txn is now out of the equation and the only remaining solution is to call an HTTP/1 message parser. We already have one, it was slightly modified to avoid keeping states by benefitting from the fact that the response was produced by haproxy and this is entirely available. It assumes the following rules are true, or that incuring an extra cost to work around them is acceptable : - the response buffer is read-write and supports modifications in place - headers sent through / by haproxy are not folded. Folding is still implemented by replacing CR/LF/tabs/spaces with spaces if encountered - HTTP/0.9 responses are never sent by haproxy and have never been supported at all - haproxy will not send partial responses, the whole headers block will be sent at once ; this means that we don't need to keep expensive states and can afford to restart the parsing from the beginning when facing a partial response ; - response is contiguous (does not wrap). This was already the case with the original parser and ensures we can safely dereference all fields with (ptr,len) The parser replaces all of the http_msg fields that were necessary with local variables. The parser is not called on an http_msg but on a string with a start and an end. The HTTP/1 states were reused for ease of use, though the request-specific ones have not been implemented for now. The error position and error state are supported and optional ; these ones may be used later for bug hunting. The parser issues the list of all the headers into a caller-allocated array of struct ist. The content-length/transfer-encoding header are checked and the relevant info fed the h1 message state (flags + body_len).
2017-07-26 03:07:47 -04:00
}
MEDIUM: h1: add a WebSocket key on handshake if needed Add the header Sec-Websocket-Key when generating a h1 handshake websocket without this header. This is the case when doing h2-h1 conversion. The key is randomly generated and base64 encoded. It is stored on the session side to be able to verify response key and reject it if not valid.
2020-12-11 11:53:07 -05:00
/* Generate a random key for a WebSocket Handshake in respect with rfc6455
* The key is 128-bits long encoded as a base64 string in <key_out> parameter
* (25 bytes long).
*/
void h1_generate_random_ws_input_key(char key_out[25])
{
/* generate a random websocket key */
const uint64_t rand1 = ha_random64(), rand2 = ha_random64();
char key[16];
memcpy(key, &rand1, 8);
memcpy(&key[8], &rand2, 8);
a2base64(key, 16, key_out, 25);
}
MEDIUM: h1: generate WebSocket key on response if needed Add the Sec-Websocket-Accept header on a websocket handshake response. This header may be missing if a h2 server is used with a h1 client. The response key is calculated following the rfc6455. For this, the handshake request key must be stored in the h1 session, as a new field name ws_key. Note that this is only done if the message has been prealably identified as a Websocket handshake request.
2020-12-11 11:53:03 -05:00
#define H1_WS_KEY_SUFFIX_GUID "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
/*
* Calculate the WebSocket handshake response key from <key_in>. Following the
* rfc6455, <key_in> must be 24 bytes longs. The result is stored in <key_out>
* as a 29 bytes long string.
*/
void h1_calculate_ws_output_key(const char *key, char *result)
{
blk_SHA_CTX sha1_ctx;
char hash_in[60], hash_out[20];
/* concatenate the key with a fixed suffix */
memcpy(hash_in, key, 24);
memcpy(&hash_in[24], H1_WS_KEY_SUFFIX_GUID, 36);
/* sha1 the result */
blk_SHA1_Init(&sha1_ctx);
blk_SHA1_Update(&sha1_ctx, hash_in, 60);
blk_SHA1_Final((unsigned char *)hash_out, &sha1_ctx);
/* encode in base64 the hash */
a2base64(hash_out, 20, result, 29);
}
Reference in a new issue Copy permalink