upstream/haproxy
1
0
Fork 0
mirror of https://github.com/haproxy/haproxy.git synced 2026-04-21 06:06:59 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
haproxy/src/http_client.c

1039 lines
26 KiB
C
Raw Normal View History

MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
/*
* HTTP Client
*
* Copyright (C) 2021 HAProxy Technologies, William Lallemand <wlallemand@haproxy.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
* This file implements an HTTP Client API.
*
*/
MINOR: httpclient: cleanup the include files Include the correct .h files in http_client.c and http_client.h. The api.h is needed in http_client.c and http_client-t.h is now include directly from http_client.h
2021-08-20 08:25:15 -04:00
#include <haproxy/api.h>
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
#include <haproxy/applet.h>
#include <haproxy/cli.h>
#include <haproxy/dynbuf.h>
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
#include <haproxy/cfgparse.h>
#include <haproxy/connection.h>
#include <haproxy/global.h>
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
#include <haproxy/istbuf.h>
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
#include <haproxy/h1_htx.h>
#include <haproxy/http.h>
#include <haproxy/http_client.h>
#include <haproxy/http_htx.h>
#include <haproxy/htx.h>
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
#include <haproxy/log.h>
#include <haproxy/proxy.h>
MINOR: httpclient: cleanup the include files Include the correct .h files in http_client.c and http_client.h. The api.h is needed in http_client.c and http_client-t.h is now include directly from http_client.h
2021-08-20 08:25:15 -04:00
#include <haproxy/server.h>
BUILD: httpclient: include missing ssl_sock-t It's needed for SSL_SOCK_VERIFY_NONE.
2021-10-06 05:28:24 -04:00
#include <haproxy/ssl_sock-t.h>
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
#include <haproxy/stream_interface.h>
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
#include <haproxy/tools.h>
#include <string.h>
static struct proxy *httpclient_proxy;
static struct server *httpclient_srv_raw;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#ifdef USE_OPENSSL
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
static struct server *httpclient_srv_ssl;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#endif
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
static struct applet httpclient_applet;
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
/* --- This part of the file implement an HTTP client over the CLI ---
* The functions will be starting by "hc_cli" for "httpclient cli"
*/
/* What kind of data we need to read */
#define HC_CLI_F_RES_STLINE 0x01
#define HC_CLI_F_RES_HDR 0x02
#define HC_CLI_F_RES_BODY 0x04
#define HC_CLI_F_RES_END 0x08
/* These are the callback used by the HTTP Client when it needs to notify new
* data, we only sets a flag in the IO handler */
void hc_cli_res_stline_cb(struct httpclient *hc)
{
struct appctx *appctx = hc->caller;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
if (!appctx)
return;
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
appctx->ctx.cli.i0 |= HC_CLI_F_RES_STLINE;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
appctx_wakeup(appctx);
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
}
void hc_cli_res_headers_cb(struct httpclient *hc)
{
struct appctx *appctx = hc->caller;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
if (!appctx)
return;
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
appctx->ctx.cli.i0 |= HC_CLI_F_RES_HDR;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
appctx_wakeup(appctx);
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
}
void hc_cli_res_body_cb(struct httpclient *hc)
{
struct appctx *appctx = hc->caller;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
if (!appctx)
return;
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
appctx->ctx.cli.i0 |= HC_CLI_F_RES_BODY;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
appctx_wakeup(appctx);
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
}
void hc_cli_res_end_cb(struct httpclient *hc)
{
struct appctx *appctx = hc->caller;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
if (!appctx)
return;
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
appctx->ctx.cli.i0 |= HC_CLI_F_RES_END;
BUG/MINOR: httpclient/cli: change the appctx test in the callbacks The callbacks of the CLI httpclient are testing the appctx pointer before doing the appctx_wakeup but was dereferencing the appctx pointer before.
2021-08-20 05:35:29 -04:00
appctx_wakeup(appctx);
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
}
/*
* Parse an httpclient keyword on the cli:
* httpclient <ID> <method> <URI>
*/
static int hc_cli_parse(char **args, char *payload, struct appctx *appctx, void *private)
{
struct httpclient *hc;
char *err = NULL;
enum http_meth_t meth;
char *meth_str;
struct ist uri;
MINOR: httpclient: support payload within a buffer httpclient_req_gen() takes a payload argument which can be use to put a payload in the request. This payload can only fit a request buffer. This payload can also be specified by the "body" named parameter within the lua. httpclient. It is also used within the CLI httpclient when specified as a CLI payload with "<<".
2021-10-25 13:48:37 -04:00
struct ist body = IST_NULL;
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
if (!cli_has_level(appctx, ACCESS_LVL_ADMIN))
return 1;
if (!*args[1] || !*args[2]) {
memprintf(&err, ": not enough parameters");
goto err;
}
meth_str = args[1];
uri = ist(args[2]);
MINOR: httpclient: support payload within a buffer httpclient_req_gen() takes a payload argument which can be use to put a payload in the request. This payload can only fit a request buffer. This payload can also be specified by the "body" named parameter within the lua. httpclient. It is also used within the CLI httpclient when specified as a CLI payload with "<<".
2021-10-25 13:48:37 -04:00
if (payload)
body = ist(payload);
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
meth = find_http_meth(meth_str, strlen(meth_str));
hc = httpclient_new(appctx, meth, uri);
if (!hc) {
goto err;
}
/* update the httpclient callbacks */
hc->ops.res_stline = hc_cli_res_stline_cb;
hc->ops.res_headers = hc_cli_res_headers_cb;
hc->ops.res_payload = hc_cli_res_body_cb;
hc->ops.res_end = hc_cli_res_end_cb;
appctx->ctx.cli.p0 = hc; /* store the httpclient ptr in the applet */
appctx->ctx.cli.i0 = 0;
BUG/MINOR: httpclient: set default Accept and User-Agent headers Some servers require at least an Accept and a User-Agent header in the request. This patch sets some default value. Must be backported in 2.5.
2022-01-14 08:10:33 -05:00
if (httpclient_req_gen(hc, hc->req.url, hc->req.meth, NULL, body) != ERR_NONE)
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
goto err;
if (!httpclient_start(hc))
goto err;
return 0;
err:
memprintf(&err, "Can't start the HTTP client%s.\n", err ? err : "");
return cli_err(appctx, err);
}
/* This function dumps the content of the httpclient receive buffer
* on the CLI output
*
* Return 1 when the processing is finished
* return 0 if it needs to be called again
*/
static int hc_cli_io_handler(struct appctx *appctx)
{
struct stream_interface *si = appctx->owner;
struct buffer *trash = alloc_trash_chunk();
struct httpclient *hc = appctx->ctx.cli.p0;
struct http_hdr *hdrs, *hdr;
if (!trash)
goto out;
if (appctx->ctx.cli.i0 & HC_CLI_F_RES_STLINE) {
BUG/MINOR: httpclient/cli: display junk characters in vsn ist are not ended by '\0', leading to junk characters being displayed when using %s for printing the HTTP start line. Fix the issue by replacing %s by %.*s + istlen. Must be backported in 2.5.
2022-02-16 05:37:02 -05:00
chunk_appendf(trash, "%.*s %d %.*s\n", (unsigned int)istlen(hc->res.vsn), istptr(hc->res.vsn),
hc->res.status, (unsigned int)istlen(hc->res.reason), istptr(hc->res.reason));
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
if (ci_putchk(si_ic(si), trash) == -1)
si_rx_room_blk(si);
appctx->ctx.cli.i0 &= ~HC_CLI_F_RES_STLINE;
goto out;
}
if (appctx->ctx.cli.i0 & HC_CLI_F_RES_HDR) {
hdrs = hc->res.hdrs;
for (hdr = hdrs; isttest(hdr->v); hdr++) {
if (!h1_format_htx_hdr(hdr->n, hdr->v, trash))
goto out;
}
if (!chunk_memcat(trash, "\r\n", 2))
goto out;
if (ci_putchk(si_ic(si), trash) == -1)
si_rx_room_blk(si);
appctx->ctx.cli.i0 &= ~HC_CLI_F_RES_HDR;
goto out;
}
if (appctx->ctx.cli.i0 & HC_CLI_F_RES_BODY) {
int ret;
ret = httpclient_res_xfer(hc, &si_ic(si)->buf);
channel_add_input(si_ic(si), ret); /* forward what we put in the buffer channel */
MINOR: httpclient: httpclient_data() returns the available data httpclient_data() returns the available data in the httpclient.
2021-09-21 04:45:34 -04:00
if (!httpclient_data(hc)) {/* remove the flag if the buffer was emptied */
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
appctx->ctx.cli.i0 &= ~HC_CLI_F_RES_BODY;
}
goto out;
}
/* we must close only if F_END is the last flag */
if (appctx->ctx.cli.i0 == HC_CLI_F_RES_END) {
si_shutw(si);
si_shutr(si);
appctx->ctx.cli.i0 &= ~HC_CLI_F_RES_END;
goto out;
}
out:
/* we didn't clear every flags, we should come back to finish things */
if (appctx->ctx.cli.i0)
si_rx_room_blk(si);
free_trash_chunk(trash);
return 0;
}
static void hc_cli_release(struct appctx *appctx)
{
struct httpclient *hc = appctx->ctx.cli.p0;
/* Everything possible was printed on the CLI, we can destroy the client */
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
httpclient_stop_and_destroy(hc);
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
return;
}
/* register cli keywords */
static struct cli_kw_list cli_kws = {{ },{
MINOR: httpclient/cli: access should be only done from expert mode Only enable the usage of the CLI HTTP client in expert mode.
2021-10-19 04:58:30 -04:00
{ { "httpclient", NULL }, "httpclient <method> <URI> : launch an HTTP request", hc_cli_parse, hc_cli_io_handler, hc_cli_release, NULL, ACCESS_EXPERT},
MINOR: httpclient/cli: implement a simple client over the CLI This commit implements an HTTP Client over the CLI, this was made as working example for the HTTP Client API. It usable over the CLI by specifying a method and an URL: echo "httpclient GET http://127.0.0.1:8000/demo.file" | socat /tmp/haproxy.sock - Only IP addresses are accessibles since the API does not allow to resolve addresses yet.
2021-08-18 10:46:21 -04:00
{ { NULL }, NULL, NULL, NULL }
}};
INITCALL1(STG_REGISTER, cli_register_kw, &cli_kws);
/* --- This part of the file implements the actual HTTP client API --- */
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/*
* Generate a simple request and fill the httpclient request buffer with it.
* The request contains a request line generated from the absolute <url> and
* <meth> as well as list of headers <hdrs>.
*
* If the buffer was filled correctly the function returns 0, if not it returns
* an error_code but there is no guarantee that the buffer wasn't modified.
*/
MINOR: httpclient: support payload within a buffer httpclient_req_gen() takes a payload argument which can be use to put a payload in the request. This payload can only fit a request buffer. This payload can also be specified by the "body" named parameter within the lua. httpclient. It is also used within the CLI httpclient when specified as a CLI payload with "<<".
2021-10-25 13:48:37 -04:00
int httpclient_req_gen(struct httpclient *hc, const struct ist url, enum http_meth_t meth, const struct http_hdr *hdrs, const struct ist payload)
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
{
struct htx_sl *sl;
struct htx *htx;
int err_code = 0;
struct ist meth_ist, vsn;
MINOR: httpclient: support payload within a buffer httpclient_req_gen() takes a payload argument which can be use to put a payload in the request. This payload can only fit a request buffer. This payload can also be specified by the "body" named parameter within the lua. httpclient. It is also used within the CLI httpclient when specified as a CLI payload with "<<".
2021-10-25 13:48:37 -04:00
unsigned int flags = HTX_SL_F_VER_11 | HTX_SL_F_NORMALIZED_URI | HTX_SL_F_HAS_SCHM;
BUG/MINOR: httpclient: allow to replace the host header This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
2021-11-24 09:38:17 -05:00
int i;
BUG/MINOR: httpclient: set default Accept and User-Agent headers Some servers require at least an Accept and a User-Agent header in the request. This patch sets some default value. Must be backported in 2.5.
2022-01-14 08:10:33 -05:00
int foundhost = 0, foundaccept = 0, foundua = 0;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
if (!b_alloc(&hc->req.buf))
goto error;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
if (meth >= HTTP_METH_OTHER)
goto error;
meth_ist = http_known_methods[meth];
vsn = ist("HTTP/1.1");
htx = htx_from_buf(&hc->req.buf);
if (!htx)
goto error;
BUG/MINOR: httpclient: don't send an empty body Forbid the httpclient to send an empty chunked client when there is no data to send. It does happen when doing a simple GET too. Must be backported in 2.5.
2022-01-14 08:08:34 -05:00
if (!hc->ops.req_payload && !isttest(payload))
flags |= HTX_SL_F_BODYLESS;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
sl = htx_add_stline(htx, HTX_BLK_REQ_SL, flags, meth_ist, url, vsn);
if (!sl) {
goto error;
}
sl->info.req.meth = meth;
BUG/MINOR: httpclient: allow to replace the host header This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
2021-11-24 09:38:17 -05:00
for (i = 0; hdrs && hdrs[i].n.len; i++) {
/* Don't check the value length because a header value may be empty */
if (isttest(hdrs[i].v) == 0)
continue;
if (isteqi(hdrs[i].n, ist("host")))
foundhost = 1;
BUG/MINOR: httpclient: set default Accept and User-Agent headers Some servers require at least an Accept and a User-Agent header in the request. This patch sets some default value. Must be backported in 2.5.
2022-01-14 08:10:33 -05:00
else if (isteqi(hdrs[i].n, ist("accept")))
foundaccept = 1;
else if (isteqi(hdrs[i].n, ist("user-agent")))
foundua = 1;
BUG/MINOR: httpclient: allow to replace the host header This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
2021-11-24 09:38:17 -05:00
if (!htx_add_header(htx, hdrs[i].n, hdrs[i].v))
goto error;
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
BUG/MINOR: httpclient: allow to replace the host header This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
2021-11-24 09:38:17 -05:00
if (!foundhost) {
/* Add Host Header from URL */
if (!htx_add_header(htx, ist("Host"), ist("h")))
MINOR: httpclient: add the EOH when no headers where provided httpclient_req_gen() now adds the end of headers block when no header was provided, which avoid adding it manually.
2021-09-20 10:19:15 -04:00
goto error;
BUG/MINOR: httpclient: allow to replace the host header This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
2021-11-24 09:38:17 -05:00
if (!http_update_host(htx, sl, url))
MINOR: httpclient: add the EOH when no headers where provided httpclient_req_gen() now adds the end of headers block when no header was provided, which avoid adding it manually.
2021-09-20 10:19:15 -04:00
goto error;
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
BUG/MINOR: httpclient: set default Accept and User-Agent headers Some servers require at least an Accept and a User-Agent header in the request. This patch sets some default value. Must be backported in 2.5.
2022-01-14 08:10:33 -05:00
if (!foundaccept) {
if (!htx_add_header(htx, ist("Accept"), ist("*/*")))
goto error;
}
if (!foundua) {
if (!htx_add_header(htx, ist("User-Agent"), ist(HTTPCLIENT_USERAGENT)))
goto error;
}
BUG/MINOR: httpclient: allow to replace the host header This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
2021-11-24 09:38:17 -05:00
if (!htx_add_endof(htx, HTX_BLK_EOH))
goto error;
MINOR: httpclient: support payload within a buffer httpclient_req_gen() takes a payload argument which can be use to put a payload in the request. This payload can only fit a request buffer. This payload can also be specified by the "body" named parameter within the lua. httpclient. It is also used within the CLI httpclient when specified as a CLI payload with "<<".
2021-10-25 13:48:37 -04:00
if (isttest(payload)) {
/* add the payload if it can feat in the buffer, no need to set
* the Content-Length, the data will be sent chunked */
if (!htx_add_data_atonce(htx, payload))
goto error;
}
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
/* If req.payload was set, does not set the end of stream which *MUST*
* be set in the callback */
if (!hc->ops.req_payload)
htx->flags |= HTX_FL_EOM;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
htx_to_buf(htx, &hc->req.buf);
return 0;
error:
err_code |= ERR_ALERT | ERR_ABORT;
return err_code;
}
/*
* transfer the response to the destination buffer and wakeup the HTTP client
* applet so it could fill again its buffer.
*
CLEANUP: assorted typo fixes in the code and comments This is 27th iteration of typo fixes
2021-10-15 07:18:21 -04:00
* Return the number of bytes transferred.
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
*/
int httpclient_res_xfer(struct httpclient *hc, struct buffer *dst)
{
int ret;
MINOR: httpclient: Don't limit data transfer to 1024 bytes For debug purpose, no more 1024 bytes were copied at a time. But there is no reason to keep this limitation. Thus, it is removed. This patch may be backported to 2.5.
2022-01-12 08:46:03 -05:00
ret = b_force_xfer(dst, &hc->res.buf, b_data(&hc->res.buf));
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/* call the client once we consumed all data */
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
if (!b_data(&hc->res.buf)) {
b_free(&hc->res.buf);
if (hc->appctx)
appctx_wakeup(hc->appctx);
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
return ret;
}
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
/*
* Transfer raw HTTP payload from src, and insert it into HTX format in the
* httpclient.
*
* Must be used to transfer the request body.
* Then wakeup the httpclient so it can transfer it.
*
* <end> tries to add the ending data flag if it succeed to copy all data.
*
* Return the number of bytes copied from src.
*/
int httpclient_req_xfer(struct httpclient *hc, struct ist src, int end)
{
int ret = 0;
struct htx *htx;
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
if (!b_alloc(&hc->req.buf))
goto error;
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
htx = htx_from_buf(&hc->req.buf);
if (!htx)
goto error;
if (hc->appctx)
appctx_wakeup(hc->appctx);
ret += htx_add_data(htx, src);
/* if we copied all the data and the end flag is set */
if ((istlen(src) == ret) && end) {
htx->flags |= HTX_FL_EOM;
}
htx_to_buf(htx, &hc->req.buf);
error:
return ret;
}
MINOR: httpclient: sets an alternative destination httpclient_set_dst() allows to set an alternative destination address using HAProxy addres format. This will ignore the address within the URL.
2022-02-17 13:10:55 -05:00
/*
* Sets a destination for the httpclient from an HAProxy addr format
* This will prevent to determine the destination from the URL
* Return 0 in case of success or -1 otherwise.
*/
int httpclient_set_dst(struct httpclient *hc, const char *dst)
{
struct sockaddr_storage *sk;
char *errmsg = NULL;
sockaddr_free(&hc->dst);
/* 'sk' is statically allocated (no need to be freed). */
sk = str2sa_range(dst, NULL, NULL, NULL, NULL, NULL,
&errmsg, NULL, NULL,
PA_O_PORT_OK | PA_O_STREAM | PA_O_XPRT | PA_O_CONNECT);
if (!sk) {
ha_alert("httpclient: Failed to parse destination address in %s\n", errmsg);
free(errmsg);
return -1;
}
if (!sockaddr_alloc(&hc->dst, sk, sizeof(*sk))) {
ha_alert("httpclient: Failed to allocate sockaddr in %s:%d.\n", __FUNCTION__, __LINE__);
return -1;
}
return 0;
}
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/*
* Start the HTTP client
* Create the appctx, session, stream and wakeup the applet
*
* FIXME: It also fill the sockaddr with the IP address, but currently only IP
* in the URL are supported, it lacks a resolver.
*
* Return the <appctx> or NULL if it failed
*/
struct appctx *httpclient_start(struct httpclient *hc)
{
struct applet *applet = &httpclient_applet;
struct appctx *appctx;
struct session *sess;
struct stream *s;
int len;
MINOR: httpclient: sets an alternative destination httpclient_set_dst() allows to set an alternative destination address using HAProxy addres format. This will ignore the address within the URL.
2022-02-17 13:10:55 -05:00
struct sockaddr_storage ss_url;
struct sockaddr_storage* ss_dst;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
struct split_url out;
BUG/MINOR: httpclient: reinit flags in httpclient_start() When starting for the 2nd time a request from the same httpclient *hc context, the flags are not reinitialized and the httpclient will stop after the first call to the IO handler, because the END flag is always present. This patch also add a test before httpclient_start() to ensure we don't start a client already started. Must be backported in 2.5.
2022-02-17 06:52:09 -05:00
/* if the client was started and not ended, an applet is already
* running, we shouldn't try anything */
if (httpclient_started(hc) && !httpclient_ended(hc))
return NULL;
hc->flags = 0;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/* parse URI and fill sockaddr_storage */
MINOR: httpclient: sets an alternative destination httpclient_set_dst() allows to set an alternative destination address using HAProxy addres format. This will ignore the address within the URL.
2022-02-17 13:10:55 -05:00
len = url2sa(istptr(hc->req.url), istlen(hc->req.url), &ss_url, &out);
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
if (len == -1) {
BUG/MEDIUM: httpclient: replace ist0 by istptr ASAN reported a buffer overflow in the httpclient. This overflow is the consequence of ist0() which is incorrect here. Replace all occurences of ist0() by istptr() which is more appropried here since all ist in the httpclient were created from strings.
2021-09-26 12:12:43 -04:00
ha_alert("httpclient: cannot parse uri '%s'.\n", istptr(hc->req.url));
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
goto out;
}
/* The HTTP client will be created in the same thread as the caller,
* avoiding threading issues */
MINOR: applet: remove the thread mask from appctx_new() appctx_new() is exclusively called with tid_bit and it only uses the mask to pass it to the accompanying task. There is no point requiring the caller to know about a mask there, nor is there any point in creating an applet outside of the context of its own thread anyway. Let's drop this and pass tid_bit to task_new() directly.
2021-09-13 04:07:38 -04:00
appctx = appctx_new(applet);
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
if (!appctx)
goto out;
sess = session_new(httpclient_proxy, NULL, &appctx->obj_type);
if (!sess) {
ha_alert("httpclient: out of memory in %s:%d.\n", __FUNCTION__, __LINE__);
goto out_free_appctx;
}
BUG/MEDIUM: httpclient: Xfer the request when the stream is created Since the HTTP legacy mode was removed, it is unexpected to create an HTTP stream without a valid request. Thanks to this change, the wait_for_request analyzer was significatly simplified. And it is possible because HTTP multiplexers already take care to have a valid request to create a stream. But it means that any HTTP applet on the client side must do the same. The httpclient client is one of them. And it is not a problem because the request is generated before starting the applet. We must just take care to set the right state. For now it works "by chance", because the applet seems to be scheduled before the stream itself. But if this change, this will lead to crash because the stream expects to have a request when wait_for_request analyzer. This patch should be backported to 2.5.
2022-01-12 09:27:41 -05:00
if ((s = stream_new(sess, &appctx->obj_type, &hc->req.buf)) == NULL) {
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
ha_alert("httpclient: Failed to initialize stream %s:%d.\n", __FUNCTION__, __LINE__);
goto out_free_appctx;
}
MINOR: httpclient: sets an alternative destination httpclient_set_dst() allows to set an alternative destination address using HAProxy addres format. This will ignore the address within the URL.
2022-02-17 13:10:55 -05:00
/* if httpclient_set_dst() was used, sets the alternative address */
if (hc->dst)
ss_dst = hc->dst;
else
ss_dst = &ss_url;
if (!sockaddr_alloc(&s->si[1].dst, ss_dst, sizeof(*hc->dst))) {
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
ha_alert("httpclient: Failed to initialize stream in %s:%d.\n", __FUNCTION__, __LINE__);
goto out_free_stream;
}
/* choose the SSL server or not */
switch (out.scheme) {
case SCH_HTTP:
s->target = &httpclient_srv_raw->obj_type;
break;
case SCH_HTTPS:
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#ifdef USE_OPENSSL
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
s->target = &httpclient_srv_ssl->obj_type;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#else
ha_alert("httpclient: OpenSSL is not available %s:%d.\n", __FUNCTION__, __LINE__);
goto out_free_stream;
#endif
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
break;
}
s->flags |= SF_ASSIGNED|SF_ADDR_SET;
s->si[1].flags |= SI_FL_NOLINGER;
s->res.flags |= CF_READ_DONTWAIT;
/* applet is waiting for data */
si_cant_get(&s->si[0]);
appctx_wakeup(appctx);
task_wakeup(s->task, TASK_WOKEN_INIT);
hc->appctx = appctx;
MINOR: httpclient: test if started during stop_and_destroy() If the httpclient was never started, it is safe to destroy completely the httpclient.
2021-09-28 06:15:37 -04:00
hc->flags |= HTTPCLIENT_FS_STARTED;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
appctx->ctx.httpclient.ptr = hc;
BUG/MEDIUM: httpclient: Xfer the request when the stream is created Since the HTTP legacy mode was removed, it is unexpected to create an HTTP stream without a valid request. Thanks to this change, the wait_for_request analyzer was significatly simplified. And it is possible because HTTP multiplexers already take care to have a valid request to create a stream. But it means that any HTTP applet on the client side must do the same. The httpclient client is one of them. And it is not a problem because the request is generated before starting the applet. We must just take care to set the right state. For now it works "by chance", because the applet seems to be scheduled before the stream itself. But if this change, this will lead to crash because the stream expects to have a request when wait_for_request analyzer. This patch should be backported to 2.5.
2022-01-12 09:27:41 -05:00
/* The request was transferred when the stream was created. So switch
* directly to REQ_BODY or RES_STLINE state
*/
appctx->st0 = (hc->ops.req_payload ? HTTPCLIENT_S_REQ_BODY : HTTPCLIENT_S_RES_STLINE);
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
return appctx;
out_free_stream:
LIST_DELETE(&s->list);
pool_free(pool_head_stream, s);
out_free_sess:
session_free(sess);
out_free_appctx:
appctx_free(appctx);
out:
return NULL;
}
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
/*
* This function tries to destroy the httpclient if it wasn't running.
* If it was running, stop the client and ask it to autodestroy itself.
*
CLEANUP: assorted typo fixes in the code and comments This is 27th iteration of typo fixes
2021-10-15 07:18:21 -04:00
* Once this function is used, all pointer sto the client must be removed
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
*
*/
void httpclient_stop_and_destroy(struct httpclient *hc)
{
MINOR: httpclient: test if started during stop_and_destroy() If the httpclient was never started, it is safe to destroy completely the httpclient.
2021-09-28 06:15:37 -04:00
/* The httpclient was already stopped or never started, we can safely destroy it */
if (hc->flags & HTTPCLIENT_FS_ENDED || !(hc->flags & HTTPCLIENT_FS_STARTED)) {
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
httpclient_destroy(hc);
} else {
/* if the client wasn't stopped, ask for a stop and destroy */
hc->flags |= (HTTPCLIENT_FA_AUTOKILL | HTTPCLIENT_FA_STOP);
if (hc->appctx)
appctx_wakeup(hc->appctx);
}
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/* Free the httpclient */
void httpclient_destroy(struct httpclient *hc)
{
MINOR: httpclient: destroy() must free the headers and the ists httpclient_destroy() must free all the ist in the httpclient structure, the URL in the request, the vsn and reason in the response. It also must free the list of headers of the response.
2021-09-27 09:17:47 -04:00
struct http_hdr *hdrs;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
if (!hc)
return;
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
MINOR: httpclient: destroy checks if a client was started but not stopped During httpclient_destroy, add a condition in the BUG_ON which checks that the client was started before it has ended. A httpclient structure could have been created without being started.
2021-10-05 09:50:45 -04:00
/* we should never destroy a client which was started but not stopped */
BUG_ON(httpclient_started(hc) && !httpclient_ended(hc));
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
MINOR: httpclient: destroy() must free the headers and the ists httpclient_destroy() must free all the ist in the httpclient structure, the URL in the request, the vsn and reason in the response. It also must free the list of headers of the response.
2021-09-27 09:17:47 -04:00
/* request */
istfree(&hc->req.url);
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
b_free(&hc->req.buf);
MINOR: httpclient: destroy() must free the headers and the ists httpclient_destroy() must free all the ist in the httpclient structure, the URL in the request, the vsn and reason in the response. It also must free the list of headers of the response.
2021-09-27 09:17:47 -04:00
/* response */
istfree(&hc->res.vsn);
istfree(&hc->res.reason);
hdrs = hc->res.hdrs;
while (hdrs && isttest(hdrs->n)) {
istfree(&hdrs->n);
istfree(&hdrs->v);
hdrs++;
}
ha_free(&hc->res.hdrs);
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
b_free(&hc->res.buf);
MINOR: httpclient: sets an alternative destination httpclient_set_dst() allows to set an alternative destination address using HAProxy addres format. This will ignore the address within the URL.
2022-02-17 13:10:55 -05:00
sockaddr_free(&hc->dst);
MINOR: httpclient: destroy() must free the headers and the ists httpclient_destroy() must free all the ist in the httpclient structure, the URL in the request, the vsn and reason in the response. It also must free the list of headers of the response.
2021-09-27 09:17:47 -04:00
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
free(hc);
return;
}
/* Allocate an httpclient and its buffers
* Return NULL on failure */
struct httpclient *httpclient_new(void *caller, enum http_meth_t meth, struct ist url)
{
struct httpclient *hc;
hc = calloc(1, sizeof(*hc));
if (!hc)
goto err;
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
hc->req.buf = BUF_NULL;
hc->res.buf = BUF_NULL;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
hc->caller = caller;
BUG/MEDIUM: httpclient/cli: free of unallocated hc->req.uri httpclient_new() sets the hc->req.uri ist without duplicating its memory, which is a problem since the string in the ist could be inaccessible at some point. The API was made to use a ist which was allocated dynamically, but httpclient_new() didn't do that, which result in a crash when calling istfree(). This patch fixes the problem by doing an istdup() Fix issue #1452.
2021-11-10 10:57:25 -05:00
hc->req.url = istdup(url);
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
hc->req.meth = meth;
return hc;
err:
httpclient_destroy(hc);
return NULL;
}
static void httpclient_applet_io_handler(struct appctx *appctx)
{
struct httpclient *hc = appctx->ctx.httpclient.ptr;
struct stream_interface *si = appctx->owner;
struct stream *s = si_strm(si);
struct channel *req = &s->req;
struct channel *res = &s->res;
struct htx_blk *blk = NULL;
struct htx *htx;
BUG/MINOR: httpclient: fix uninitialized sl variable Reported by coverity in ticket #1355 CID 1461505: Memory - illegal accesses (UNINIT) Using uninitialized value "sl". Fix the problem by initializing sl to NULL.
2021-08-20 05:24:13 -04:00
struct htx_sl *sl = NULL;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
int32_t pos;
uint32_t hdr_num;
BUG/MINOR: httpclient/lua: rcv freeze when no request payload This patch fixes the receive part of the lua httpclient when no payload was sent. The lua task was not awoken once it jumped into hlua_httpclient_rcv_yield(), which caused the lua client to freeze. It works with a payload because the payload push is doing the wakeup. A change in the state machine of the IO handler is also require to achieve correctly the change from the REQ state to the RES state, it has to detect if there is the right EOM flag in the request.
2021-11-04 04:45:58 -04:00
int ret;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
while (1) {
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
/* required to stop */
if (hc->flags & HTTPCLIENT_FA_STOP)
goto end;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
switch(appctx->st0) {
case HTTPCLIENT_S_REQ:
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
/* we know that the buffer is empty here, since
* it's the first call, we can freely copy the
* request from the httpclient buffer */
BUG/MINOR: httpclient/lua: rcv freeze when no request payload This patch fixes the receive part of the lua httpclient when no payload was sent. The lua task was not awoken once it jumped into hlua_httpclient_rcv_yield(), which caused the lua client to freeze. It works with a payload because the payload push is doing the wakeup. A change in the state machine of the IO handler is also require to achieve correctly the change from the REQ state to the RES state, it has to detect if there is the right EOM flag in the request.
2021-11-04 04:45:58 -04:00
ret = b_xfer(&req->buf, &hc->req.buf, b_data(&hc->req.buf));
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
if (!ret)
goto more;
BUG/MINOR: httpclient/lua: rcv freeze when no request payload This patch fixes the receive part of the lua httpclient when no payload was sent. The lua task was not awoken once it jumped into hlua_httpclient_rcv_yield(), which caused the lua client to freeze. It works with a payload because the payload push is doing the wakeup. A change in the state machine of the IO handler is also require to achieve correctly the change from the REQ state to the RES state, it has to detect if there is the right EOM flag in the request.
2021-11-04 04:45:58 -04:00
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
if (!b_data(&hc->req.buf))
b_free(&hc->req.buf);
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
htx = htx_from_buf(&req->buf);
BUG/MINOR: httpclient/lua: rcv freeze when no request payload This patch fixes the receive part of the lua httpclient when no payload was sent. The lua task was not awoken once it jumped into hlua_httpclient_rcv_yield(), which caused the lua client to freeze. It works with a payload because the payload push is doing the wakeup. A change in the state machine of the IO handler is also require to achieve correctly the change from the REQ state to the RES state, it has to detect if there is the right EOM flag in the request.
2021-11-04 04:45:58 -04:00
if (!htx)
goto more;
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
channel_add_input(req, htx->data);
BUG/MINOR: httpclient/lua: rcv freeze when no request payload This patch fixes the receive part of the lua httpclient when no payload was sent. The lua task was not awoken once it jumped into hlua_httpclient_rcv_yield(), which caused the lua client to freeze. It works with a payload because the payload push is doing the wakeup. A change in the state machine of the IO handler is also require to achieve correctly the change from the REQ state to the RES state, it has to detect if there is the right EOM flag in the request.
2021-11-04 04:45:58 -04:00
if (htx->flags & HTX_FL_EOM) /* check if a body need to be added */
appctx->st0 = HTTPCLIENT_S_RES_STLINE;
else
appctx->st0 = HTTPCLIENT_S_REQ_BODY;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
goto more; /* we need to leave the IO handler once we wrote the request */
break;
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
case HTTPCLIENT_S_REQ_BODY:
/* call the payload callback */
{
if (hc->ops.req_payload) {
/* call the request callback */
hc->ops.req_payload(hc);
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
/* check if the request buffer is empty */
htx = htx_from_buf(&req->buf);
if (!htx_is_empty(htx))
goto more;
/* Here htx_to_buf() will set buffer data to 0 because
* the HTX is empty, and allow us to do an xfer.
*/
htx_to_buf(htx, &req->buf);
ret = b_xfer(&req->buf, &hc->req.buf, b_data(&hc->req.buf));
if (!ret)
goto more;
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
if (!b_data(&hc->req.buf))
b_free(&hc->req.buf);
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
htx = htx_from_buf(&req->buf);
if (!htx)
goto more;
channel_add_input(req, htx->data);
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
}
BUG/MEDIUM: httpclient: channel_add_input() must use htx->data The httpclient uses channel_add_input() to notify the channel layer that it must forward some data. This function was used with b_data(&req->buf) which ask to send the size of a buffer (because of the HTX metadata which fill the buffer completely). This is wrong and will have the consequence of trying to send data that doesn't exist, letting HAProxy looping at 100% CPU. When using htx channel_add_input() must be used with the size of the htx payload, and not the size of a buffer. When sending the request payload it also need to sets the buffer size to 0, which is achieved with a htx_to_buf() when the htx payload is empty.
2021-11-08 10:55:14 -05:00
htx = htx_from_buf(&req->buf);
MINOR: httpclient: request streaming with a callback This patch add a way to handle HTTP requests streaming using a callback. The end of the data must be specified by using the "end" parameter in httpclient_req_xfer().
2021-10-28 09:34:26 -04:00
if (!htx)
goto more;
/* if the request contains the HTX_FL_EOM, we finished the request part. */
if (htx->flags & HTX_FL_EOM)
appctx->st0 = HTTPCLIENT_S_RES_STLINE;
goto more; /* we need to leave the IO handler once we wrote the request */
}
break;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
case HTTPCLIENT_S_RES_STLINE:
/* copy the start line in the hc structure,then remove the htx block */
if (!b_data(&res->buf))
goto more;
htx = htxbuf(&res->buf);
if (!htx)
goto more;
blk = htx_get_first_blk(htx);
if (blk && (htx_get_blk_type(blk) == HTX_BLK_RES_SL))
sl = htx_get_blk_ptr(htx, blk);
if (!sl || (!(sl->flags & HTX_SL_F_IS_RESP)))
goto more;
/* copy the status line in the httpclient */
hc->res.status = sl->info.res.status;
hc->res.vsn = istdup(htx_sl_res_vsn(sl));
hc->res.reason = istdup(htx_sl_res_reason(sl));
co_htx_remove_blk(res, htx, blk);
/* caller callback */
if (hc->ops.res_stline)
hc->ops.res_stline(hc);
/* if there is no HTX data anymore and the EOM flag is
* set, leave (no body) */
if (htx_is_empty(htx) && htx->flags & HTX_FL_EOM)
appctx->st0 = HTTPCLIENT_S_RES_END;
else
appctx->st0 = HTTPCLIENT_S_RES_HDR;
break;
case HTTPCLIENT_S_RES_HDR:
/* first copy the headers in a local hdrs
* structure, once we the total numbers of the
* header we allocate the right size and copy
* them. The htx block of the headers are
* removed each time one is read */
{
struct http_hdr hdrs[global.tune.max_http_hdr];
if (!b_data(&res->buf))
goto more;
htx = htxbuf(&res->buf);
if (!htx)
goto more;
hdr_num = 0;
for (pos = htx_get_first(htx); pos != -1; pos = htx_get_next(htx, pos)) {
struct htx_blk *blk = htx_get_blk(htx, pos);
enum htx_blk_type type = htx_get_blk_type(blk);
if (type == HTX_BLK_EOH) {
hdrs[hdr_num].n = IST_NULL;
hdrs[hdr_num].v = IST_NULL;
co_htx_remove_blk(res, htx, blk);
break;
}
if (type != HTX_BLK_HDR)
continue;
hdrs[hdr_num].n = istdup(htx_get_blk_name(htx, blk));
hdrs[hdr_num].v = istdup(htx_get_blk_value(htx, blk));
if (!isttest(hdrs[hdr_num].v) || !isttest(hdrs[hdr_num].n))
goto end;
co_htx_remove_blk(res, htx, blk);
hdr_num++;
}
BUG/MINOR: httpclient: check if hdr_num is not 0 Check if hdr_num is not 0 before allocating or copying the headers to the hc->hdrs space.
2021-08-20 05:59:49 -04:00
if (hdr_num) {
/* alloc and copy the headers in the httpclient struct */
hc->res.hdrs = calloc((hdr_num + 1), sizeof(*hc->res.hdrs));
if (!hc->res.hdrs)
goto end;
memcpy(hc->res.hdrs, hdrs, sizeof(struct http_hdr) * (hdr_num + 1));
/* caller callback */
if (hc->ops.res_headers)
hc->ops.res_headers(hc);
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/* if there is no HTX data anymore and the EOM flag is
* set, leave (no body) */
MINOR: httpclient: httpclient_ended() returns 1 if the client ended httpclient_ended() returns 1 if there is no more data to collect, because the client received everything or the connection ended.
2021-09-21 04:58:10 -04:00
if (htx_is_empty(htx) && htx->flags & HTX_FL_EOM) {
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
appctx->st0 = HTTPCLIENT_S_RES_END;
MINOR: httpclient: httpclient_ended() returns 1 if the client ended httpclient_ended() returns 1 if there is no more data to collect, because the client received everything or the connection ended.
2021-09-21 04:58:10 -04:00
} else {
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
appctx->st0 = HTTPCLIENT_S_RES_BODY;
MINOR: httpclient: httpclient_ended() returns 1 if the client ended httpclient_ended() returns 1 if there is no more data to collect, because the client received everything or the connection ended.
2021-09-21 04:58:10 -04:00
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
}
break;
case HTTPCLIENT_S_RES_BODY:
/*
* The IO handler removes the htx blocks in the response buffer and
* push them in the hc->res.buf buffer in a raw format.
*/
htx = htxbuf(&res->buf);
if (!htx || htx_is_empty(htx))
goto more;
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
if (!b_alloc(&hc->res.buf))
goto more;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
if (b_full(&hc->res.buf))
BUG/MINOR: httpclient: Revisit HC request and response buffers allocation For now, these buffers are allocated when the httpclient is created and freed when it is released. Usually, we try to avoid to keep buffer allocated if it is not required. Empty buffers should be released ASAP. Apart for that, there is no issue with the response side because a copy is always performed. However, for the request side, a swap with the channel's buffer is always performed. And there is no guarantee the channel's buffer is allocated. Thus, after the swap, the httpclient can retrieve a null buffer. In practice, this never happens. But this may change. And it will be required for a futur fix. So, now, we systematically take care to have an allocated buffer when we want to write in it. And it is released as soon as it becomes empty. This patch should be backported to 2.5.
2022-01-12 05:14:08 -05:00
goto process_data;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/* decapsule the htx data to raw data */
for (pos = htx_get_first(htx); pos != -1; pos = htx_get_next(htx, pos)) {
enum htx_blk_type type;
blk = htx_get_blk(htx, pos);
type = htx_get_blk_type(blk);
if (type == HTX_BLK_DATA) {
struct ist v = htx_get_blk_value(htx, blk);
if ((b_room(&hc->res.buf) < v.len) )
goto process_data;
__b_putblk(&hc->res.buf, v.ptr, v.len);
co_htx_remove_blk(res, htx, blk);
/* the data must be processed by the caller in the receive phase */
if (hc->ops.res_payload)
hc->ops.res_payload(hc);
} else {
/* remove any block which is not a data block */
co_htx_remove_blk(res, htx, blk);
}
}
/* if not finished, should be called again */
if (!(htx->flags & HTX_FL_EOM))
goto more;
/* end of message, we should quit */
appctx->st0 = HTTPCLIENT_S_RES_END;
break;
case HTTPCLIENT_S_RES_END:
goto end;
break;
}
}
process_data:
si_rx_chan_rdy(si);
return;
more:
/* There was not enough data in the response channel */
si_rx_room_blk(si);
if (appctx->st0 == HTTPCLIENT_S_RES_END)
goto end;
/* The state machine tries to handle as much data as possible, if there
* isn't any data to handle and a shutdown is detected, let's stop
* everything */
if ((req->flags & (CF_SHUTR|CF_SHUTR_NOW)) ||
(res->flags & (CF_SHUTW|CF_SHUTW_NOW))) {
goto end;
}
return;
end:
if (hc->ops.res_end)
hc->ops.res_end(hc);
si_shutw(si);
si_shutr(si);
return;
}
static void httpclient_applet_release(struct appctx *appctx)
{
struct httpclient *hc = appctx->ctx.httpclient.ptr;
MINOR: httpclient: httpclient_ended() returns 1 if the client ended httpclient_ended() returns 1 if there is no more data to collect, because the client received everything or the connection ended.
2021-09-21 04:58:10 -04:00
/* mark the httpclient as ended */
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
hc->flags |= HTTPCLIENT_FS_ENDED;
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
/* the applet is leaving, remove the ptr so we don't try to call it
* again from the caller */
hc->appctx = NULL;
MINOR: httpclient: stop_and_destroy() ask the applet to autokill httpclient_stop_and_destroy() tries to destroy the httpclient structure if the client was stopped. In the case the client wasn't stopped, it ask the client to stop itself and to destroy the httpclient structure itself during the release of the applet.
2021-09-28 05:00:43 -04:00
/* destroy the httpclient when set to autotokill */
if (hc->flags & HTTPCLIENT_FA_AUTOKILL) {
httpclient_destroy(hc);
}
MINOR: httpclient: implement a simple HTTP Client API This commit implements a very simple HTTP Client API. A client can be operated by several functions: - httpclient_new(), httpclient_destroy(): create and destroy the struct httpclient instance. - httpclient_req_gen(): generate a complete HTX request using the the absolute URL, the method and a list of headers. This request is complete and sets the HTX End of Message flag. This is limited to small request we don't need a body. - httpclient_start() fill a sockaddr storage with a IP extracted from the URL (it cannot resolve an fqdm for now), start the applet. It also stores the ptr of the caller which could be an appctx or something else. - hc->ops contains a list of callbacks used by the HTTPClient, they should be filled manually after an httpclient_new(): * res_stline(): the client received a start line, its content will be stored in hc->res.vsn, hc->res.status, hc->res.reason * res_headers(): the client received headers, they are stored in hc->res.hdrs. * res_payload(): the client received some payload data, they are stored in the hc->res.buf buffer and could be extracted with the httpclient_res_xfer() function, which takes a destination buffer as a parameter * res_end(): this callback is called once we finished to receive the response.
2021-08-13 10:05:53 -04:00
return;
}
/* HTTP client applet */
static struct applet httpclient_applet = {
.obj_type = OBJ_TYPE_APPLET,
.name = "<HTTPCLIENT>",
.fct = httpclient_applet_io_handler,
.release = httpclient_applet_release,
};
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
/*
* Initialize the proxy for the HTTP client with 2 servers, one for raw HTTP,
* the other for HTTPS.
*/
static int httpclient_init()
{
int err_code = 0;
char *errmsg = NULL;
httpclient_proxy = alloc_new_proxy("<HTTPCLIENT>", PR_CAP_LISTEN|PR_CAP_INT, &errmsg);
if (!httpclient_proxy) {
err_code |= ERR_ALERT | ERR_FATAL;
goto err;
}
BUG/MINOR: http_client: make sure to preset the proxy's default settings Proxies must call proxy_preset_defaults() to initialize their settings that are usually learned from defaults sections (e.g. connection retries, pool purge delay etc). At the moment there was likely no impact, but not doing so could cause trouble soon when using the client more extensively or when new defaults are introduced and failed to be initialized. No backport is needed.
2021-08-20 04:23:12 -04:00
proxy_preset_defaults(httpclient_proxy);
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
httpclient_proxy->options2 |= PR_O2_INDEPSTR;
httpclient_proxy->mode = PR_MODE_HTTP;
httpclient_proxy->maxconn = 0;
httpclient_proxy->accept = NULL;
httpclient_proxy->timeout.client = TICK_ETERNITY;
/* The HTTP Client use the "option httplog" with the global log server */
httpclient_proxy->conf.logformat_string = default_http_log_format;
httpclient_proxy->http_needed = 1;
/* clear HTTP server */
httpclient_srv_raw = new_server(httpclient_proxy);
if (!httpclient_srv_raw) {
err_code |= ERR_ALERT | ERR_FATAL;
memprintf(&errmsg, "out of memory.");
goto err;
}
httpclient_srv_raw->iweight = 0;
httpclient_srv_raw->uweight = 0;
httpclient_srv_raw->xprt = xprt_get(XPRT_RAW);
httpclient_srv_raw->id = strdup("<HTTPCLIENT>");
if (!httpclient_srv_raw->id)
goto err;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#ifdef USE_OPENSSL
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
/* SSL HTTP server */
httpclient_srv_ssl = new_server(httpclient_proxy);
if (!httpclient_srv_ssl) {
memprintf(&errmsg, "out of memory.");
err_code |= ERR_ALERT | ERR_FATAL;
goto err;
}
httpclient_srv_ssl->iweight = 0;
httpclient_srv_ssl->uweight = 0;
httpclient_srv_ssl->xprt = xprt_get(XPRT_SSL);
httpclient_srv_ssl->use_ssl = 1;
MINOR: httpclient: add the server to the proxy Add the raw and ssl server to the proxy list so they can be freed during the deinit() of HAProxy. As a side effect the 2 servers need to have a different ID so the SSL one was renamed "<HTTPSCLIENT>".
2021-08-24 11:18:13 -04:00
httpclient_srv_ssl->id = strdup("<HTTPSCLIENT>");
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
if (!httpclient_srv_ssl->id)
goto err;
MINOR: httpclient: set verify none on the https server There is currently no way to specify the CA to verify from the httpclient API. Sets the verify to none so we can still do https request.
2021-08-24 11:15:58 -04:00
httpclient_srv_ssl->ssl_ctx.verify = SSL_SOCK_VERIFY_NONE;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#endif
MINOR: httpclient: set verify none on the https server There is currently no way to specify the CA to verify from the httpclient API. Sets the verify to none so we can still do https request.
2021-08-24 11:15:58 -04:00
CLEANUP: assorted typo fixes in the code and comments This is 27th iteration of typo fixes
2021-10-15 07:18:21 -04:00
/* add the proxy in the proxy list only if everything is successful */
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
httpclient_proxy->next = proxies_list;
proxies_list = httpclient_proxy;
MINOR: httpclient: add the server to the proxy Add the raw and ssl server to the proxy list so they can be freed during the deinit() of HAProxy. As a side effect the 2 servers need to have a different ID so the SSL one was renamed "<HTTPSCLIENT>".
2021-08-24 11:18:13 -04:00
/* link the 2 servers in the proxy */
httpclient_srv_raw->next = httpclient_proxy->srv;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
httpclient_proxy->srv = httpclient_srv_raw;
#ifdef USE_OPENSSL
httpclient_srv_ssl->next = httpclient_proxy->srv;
MINOR: httpclient: add the server to the proxy Add the raw and ssl server to the proxy list so they can be freed during the deinit() of HAProxy. As a side effect the 2 servers need to have a different ID so the SSL one was renamed "<HTTPSCLIENT>".
2021-08-24 11:18:13 -04:00
httpclient_proxy->srv = httpclient_srv_ssl;
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#endif
MINOR: httpclient: add the server to the proxy Add the raw and ssl server to the proxy list so they can be freed during the deinit() of HAProxy. As a side effect the 2 servers need to have a different ID so the SSL one was renamed "<HTTPSCLIENT>".
2021-08-24 11:18:13 -04:00
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
return 0;
err:
ha_alert("httpclient: cannot initialize.\n");
free(errmsg);
MEDIUM: server: extend refcount for all servers In a future patch, it will be possible to remove at runtime every servers, both static and dynamic. This requires to extend the server refcount for all instances. First, refcount manipulation functions have been renamed to better express the API usage. * srv_refcount_use -> srv_take The refcount is always initialize to 1 on the server creation in new_server. It's also incremented for each check/agent configured on a server instance. * free_server -> srv_drop This decrements the refcount and if null, the server is freed, so code calling it must not use the server reference after it. As a bonus, this function now returns the next server instance. This is useful when calling on the server loop without having to save the next pointer before each invocation. In these functions, remove the checks that prevent refcount on non-dynamic servers. Each reference to "dynamic" in variable/function naming have been eliminated as well.
2021-08-25 09:34:53 -04:00
srv_drop(httpclient_srv_raw);
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#ifdef USE_OPENSSL
MEDIUM: server: extend refcount for all servers In a future patch, it will be possible to remove at runtime every servers, both static and dynamic. This requires to extend the server refcount for all instances. First, refcount manipulation functions have been renamed to better express the API usage. * srv_refcount_use -> srv_take The refcount is always initialize to 1 on the server creation in new_server. It's also incremented for each check/agent configured on a server instance. * free_server -> srv_drop This decrements the refcount and if null, the server is freed, so code calling it must not use the server reference after it. As a bonus, this function now returns the next server instance. This is useful when calling on the server loop without having to save the next pointer before each invocation. In these functions, remove the checks that prevent refcount on non-dynamic servers. Each reference to "dynamic" in variable/function naming have been eliminated as well.
2021-08-25 09:34:53 -04:00
srv_drop(httpclient_srv_ssl);
BUILD: httpclient: fix build without OpenSSL Add some defines around the ssl server so we can build without OpenSSL.
2021-08-24 12:33:28 -04:00
#endif
MINOR: httpclient: initialize the proxy Initialize a proxy which contain a server for the raw HTTP, and another one for the HTTPS. This proxy will use the global server log definition and the 'option httplog' directive. This proxy is internal and will only be used for the HTTP Client API.
2021-08-13 08:47:57 -04:00
free_proxy(httpclient_proxy);
return err_code;
}
static int httpclient_cfg_postparser()
{
struct logsrv *logsrv;
struct proxy *curproxy = httpclient_proxy;
/* copy logs from "global" log list */
list_for_each_entry(logsrv, &global.logsrvs, list) {
struct logsrv *node = malloc(sizeof(*node));
if (!node) {
ha_alert("httpclient: cannot allocate memory.\n");
goto err;
}
memcpy(node, logsrv, sizeof(*node));
LIST_INIT(&node->list);
LIST_APPEND(&curproxy->logsrvs, &node->list);
}
if (curproxy->conf.logformat_string) {
char *err = NULL;
curproxy->conf.args.ctx = ARGC_LOG;
if (!parse_logformat_string(curproxy->conf.logformat_string, curproxy, &curproxy->logformat,
LOG_OPT_MANDATORY|LOG_OPT_MERGE_SPACES,
SMP_VAL_FE_LOG_END, &err)) {
ha_alert("httpclient: failed to parse log-format : %s.\n", err);
free(err);
goto err;
}
curproxy->conf.args.file = NULL;
curproxy->conf.args.line = 0;
}
return 0;
err:
return 1;
}
/* initialize the proxy and servers for the HTTP client */
INITCALL0(STG_REGISTER, httpclient_init);
REGISTER_CONFIG_POSTPARSER("httpclient", httpclient_cfg_postparser);
Reference in a new issue Copy permalink