Commit graph

4169 commits

Author SHA1 Message Date
Thierry FOURNIER
5554e2983d BUG/MINOR: lua: last log character truncated.
The send_log function needs a final \n.

This bug is repported by Michael Ezzell.

Minor bug: when writing to syslog from Lua scripts, the last character from
each log entry is truncated.

core.Alert("this is truncated");

Sep  7 15:07:56 localhost haproxy[7055]: this is truncate

This issue appears to be related to the fact that send_log() (in src/log.c)
is expecting a newline at the end of the message's format string:

/*
 * This function adds a header to the message and sends the syslog message
 * using a printf format string. It expects an LF-terminated message.
 */
void send_log(struct proxy *p, int level, const char *format, ...)

I believe the fix would be in in src/hlua.c at line 760
<http://git.haproxy.org/?p=haproxy.git;a=blob;f=src/hlua.c;h=1e4d47c31e66c16c837ff2aa5ef577f6cafdc7e7;hb=316e3196285b89a917c7d84794ced59a6a5b4eba#l760>,
where this...

   send_log(px, level, "%s", trash.str);

...should be adding a newline into the format string to accommodate what
the code expects.

    send_log(px, level, "%s\n", trash.str);

This change provides what seems to be the correct behavior:

Sep  7 15:08:30 localhost haproxy[7150]: this is truncated

All other uses of send_log() in hlua.c have a trailing dot "." in the
message that is masking the truncation issue because the output message
stops on a clean word boundary.  I suspect these would also benefit from
"\n" appended to their format strings as well, since this appears to be the
pattern seen throughout the rest of the code base.

Reported-by: Michael Ezzell <michael@ezzell.net>
2015-09-09 22:12:27 +02:00
Willy Tarreau
07101d5a16 BUG/MEDIUM: dns: use the correct server hostname when resolving
The server's host name picked for resolution was incorrect, it did not
skip the address family specifier, did not resolve environment variables,
and messed up with the optional trailing colon.

Instead, let's get the fqdn returned by str2sa_range() and use that
exclusively.
2015-09-08 16:16:35 +02:00
Willy Tarreau
9f69f46d1f BUG/MINOR: tools: make str2sa_range() report unresolvable addresses
If an environment variable is used in an address, and is not set, it's
silently considered as ":" or "0.0.0.0:0" which is not correct as it
can hide environment issues and lead to unexpected behaviours. Let's
report this case when it happens.

This fix should be backported to 1.5.
2015-09-08 16:01:25 +02:00
Willy Tarreau
72b8c1f0aa MEDIUM: tools: make str2sa_range() optionally return the FQDN
The function does a bunch of things among which resolving environment
variables, skipping address family specifiers and trimming port ranges.
It is the only one which sees the complete host name before trying to
resolve it. The DNS resolving code needs to know the original hostname,
so we modify this function to optionally provide it to the caller.

Note that the function itself doesn't know if the host part was a host
or an address, but str2ip() knows that and can be asked not to try to
resolve. So we first try to parse the address without resolving and
try again with resolving enabled. This way we know if the address is
explicit or needs some kind of resolution.
2015-09-08 15:50:19 +02:00
Baptiste Assmann
90447582d7 MINOR: DNS client query type failover management
In the first version of the DNS resolver, HAProxy sends an ANY query
type and in case of issue fails over to the type pointed by the
directive in 'resolve-prefer'.
This patch allows the following new failover management:
1. default query type is still ANY
2. if response is truncated or in error because ANY is not supported by
   the server, then a fail over to a new query type is performed. The
   new query type is the one pointed by the directive 'resolve-prefer'.
3. if no response or still some errors occurs, then a query type fail over
   is performed to the remaining IP address family.
2015-09-08 15:04:17 +02:00
Baptiste Assmann
3440f0da2a MEDIUM: dns: handling of truncated response
First dns client implementation simply ignored most of DNS response
flags.
This patch changes the way the flags are parsed, using bit masks and
also take care of truncated responses.
Such response are reported to the above layer which can handle it
properly.
2015-09-08 14:59:49 +02:00
Baptiste Assmann
0df5d9669a MINOR: dns: New DNS response analysis code: DNS_RESP_TRUNCATED
This patch introduces a new internal response state about the analysis
of a DNS response received by a server.
It is dedicated to report to above layer that the response is
'truncated'.
2015-09-08 14:58:07 +02:00
Baptiste Assmann
6cdea9359b MINOR: dns: dns_nameserver structure update: new counter for truncated response
This patch updates the dns_nameserver structure to integrate a counter
dedicated to 'truncated' response sent by servers.
Such response are important to track, since HAProxy is supposed to
replay its request.
2015-09-08 14:57:28 +02:00
Baptiste Assmann
01daef3162 MINOR: dns: coding style update
No affectation in a if condition.
2015-09-08 10:52:09 +02:00
Baptiste Assmann
11c4e4eefb BUG/MAJOR: dns: dns client resolution infinite loop
Under certain circonstance (a configuration with many servers relying on
DNS resolution and one of them triggering the replay of a request
because of a timeout or invalid response to an ANY query), HAProxy could
end up in an infinite loop over the currently supposed running DNS
queries.

This was caused because the FIFO list of running queries was improperly
updated in snr_resolution_error_cb. The head of the list was removed
instead of the resolution in error, when moving the resolution to the
end of the list.

In the mean time, a LIST_DEL statement is removed since useless. This
action is already performed by the dns_reset_resolution function.
2015-09-08 10:51:50 +02:00
Baptiste Assmann
f0d9370f6b BUG/MEDIUM: dns: DNS resolution doesn't start
Patch f046f11561 introduced a regression:
DNS resolution doesn't start anymore, while it was supposed to make it
start with first health check.

Current patch fix this issue by triggering a new DNS resolution if the
last_resolution time is not set.
2015-09-08 10:51:22 +02:00
Willy Tarreau
9c03b33329 BUG/MAJOR: http: don't call http_send_name_header() after an error
A crash was reported when using the "famous" http-send-name-header
directive. This time it's a bit tricky, it requires a certain number of
conditions to be met including maxconn on a server, queuing, timeout in
the queue and cookie-based persistence.

The problem is that in stream.c, before calling http_send_name_header(),
we check a number of conditions to know if we have to replace the header
name. But prior to reaching this place, it's possible for
sess_update_stream_int() to fail and change the stream-int's state to
SI_ST_CLO, send an error 503 to the client, and flush all buffers. But
http_send_name_header() can only be called with valid buffer contents
matching the http_msg's description. So when it rewinds the stream to
modify the header, buf->o becomes negative by the size of the incoming
request and is used as the argument to memmove() which basically
displaces 4GB of memory off a few bytes to write the new name, resulting
in a core and a core file that's really not fun to play with.

The solution obviously consists in refraining from calling this nasty
function when the stream interface is already closed.

This bug also affects 1.5 and possibly 1.4, so the fix must be backported
there.
2015-09-07 19:41:42 +02:00
Thierry FOURNIER
316e319628 BUG/MEDIUM: lua: outgoing connection was broken since 1.6-dev2 (bis)
See commit id bdc97a8795

Michael Ezzell reported that the following Lua code fails in
dev4 when the TCP is not established immediately (due to a little
bit of latency):

   function tricky_socket()
        local sock = core.tcp();
        sock:settimeout(3);
        core.log(core.alert,"calling connect()\n");
        local connected, con_err = sock:connect("x.x.x.x",80);
        core.log(core.alert,"returned from connect()\n");
        if con_err ~= nil then
          core.log(core.alert,"connect() failed with error: '" .. con_err .. "'\n");
        end

The problem is that the flags who want to wake up the applet are
resetted before each applet call, so the applet must set again the
flags if the connection is not established.
2015-09-06 08:22:49 +02:00
Willy Tarreau
b7ce424be2 BUG/MINOR: http: remove stupid HTTP_METH_NONE entry
When converting the "method" fetch to a string, we used to get an empty
string if the first character was not an upper case. This was caused by
the lookup function which returns HTTP_METH_NONE when a lookup is not
possible, and this method being mapped to an empty string in the array.

This is a totally stupid mechanism, there's no reason for having the
result depend on the first char. In fact the message parser already
checks that the syntax matches an HTTP token so we can only land there
with a valid token, hence only HTTP_METH_OTHER should be returned.

This fix should be backported to all actively supported branches.
2015-09-03 17:15:21 +02:00
Thierry FOURNIER
42148735bc MEDIUM: actions: remove ACTION_STOP
Before this patch, two type of custom actions exists: ACT_ACTION_CONT and
ACT_ACTION_STOP. ACT_ACTION_CONT is a non terminal action and ACT_ACTION_STOP is
a terminal action.

Note that ACT_ACTION_STOP is not used in HAProxy.

This patch remove this behavior. Only type type of custom action exists, and it
is called ACT_CUSTOM. Now, the custion action can return a code indicating the
required behavior. ACT_RET_CONT wants that HAProxy continue the current rule
list evaluation, and ACT_RET_STOP wants that HAPRoxy stops the the current rule
list evaluation.
2015-09-02 18:36:38 +02:00
Willy Tarreau
bd99d5818d BUG/MAJOR: http: don't manipulate the server connection if it's killed
Jesse Hathaway reported a crash that Cyril Bont diagnosed as being
caused by the manipulation of srv_conn after setting it to NULL. This
happens in http-server-close mode when the server returns either a 401
or a 407, because the connection was previously closed then it's being
assigned the CO_FL_PRIVATE flag.

This bug only affects 1.6-dev as it was introduced by connection reuse code
with commit 387ebf8 ("MINOR: connection: add a new flag CO_FL_PRIVATE").
2015-09-02 10:52:05 +02:00
Baptiste Assmann
f046f11561 BUG/MEDIUM: dns: wrong first time DNS resolution
First DNS resolution is supposed to be triggered by first health check,
which is not the case with current code.
This patch fixes this behavior by setting the
resolution->last_resolution time to 0 instead of now_ms when parsing
server's configuration at startup.
2015-08-28 17:23:04 +02:00
Willy Tarreau
630ef4585a BUG/MEDIUM: lua: fix a segfault in txn:done() if called twice
When called from an http ruleset, txn:done() can still crash the process
because it closes the stream without consuming pending data resulting in
the transaction's buffer representation to differ from the real buffer.

This patch also adjusts the transaction's state to indicate that it's
closed to be consistent with what's already done in redirect rules.
2015-08-28 10:28:24 +02:00
Willy Tarreau
a678b43119 CLEANUP: lua: fix some indent issues
Just the result from a few copy-pastes with different tab sizes.
2015-08-28 10:16:23 +02:00
Thierry FOURNIER
e1587b3314 BUG/MEDIUM: lua: cannot process more Lua hooks after a "done()" function call
When the Lua execution flow endswith the command done (core.done or txn.done())
an error is detourned, and the stack is no longer usable. This patch juste
reinitilize the stack if this case is detected.
2015-08-28 10:12:49 +02:00
Willy Tarreau
0458b08a5a BUG/MEDIUM: lua: txn:done() still causes a segfault in TCP mode
We must not dereference s->txn to get the channel, as it doesn't
exist in TCP mode.
2015-08-28 09:40:04 +02:00
Thierry FOURNIER
4bb375ca18 MEDIUM: lua: turns txn:close into txn:done
The function txn:close() must be terminal because it demands the session
destruction. This patch renames this function to "done()" to be much
clearer about the fact that it is a final operation.
2015-08-27 14:33:52 +02:00
Thierry FOURNIER
35d70efc33 MINOR: http: Action for manipulating the returned status code.
This patch is inspired by Bowen Ni's proposal and it is based on his first
implementation:

   With Lua integration in HAProxy 1.6, one can change the request method,
   path, uri, header, response header etc except response line.
   I'd like to contribute the following methods to allow modification of the
   response line.

   [...]

   There are two new keywords in 'http-response' that allows you to rewrite
   them in the native HAProxy config. There are also two new APIs in Lua that
   allows you to do the same rewriting in your Lua script.

   Example:
   Use it in HAProxy config:
   *http-response set-code 404*
   Or use it in Lua script:
   *txn.http:res_set_reason("Redirect")*

I dont take the full patch because the manipulation of the "reason" is useless.
standard reason are associated with each returned code, and unknown code can
take generic reason.

So, this patch can set the status code, and the reason is automatically adapted.
2015-08-27 14:29:44 +02:00
Thierry FOURNIER
3f4bc65a22 DOC: fix "http_action_set_req_line()" comments
Bowen repports errors about http_action_set_req_line() comments.
Some other errors appears from the patches about "actions" reorganisation.
2015-08-27 11:31:19 +02:00
Thierry FOURNIER
93405e1fde BUG/MINOR: lua: in some case a sample may remain undefined
When we transform a top stack entry in sample, the empty stack case
is not handled. This patch fix this behavior.
2015-08-27 11:31:02 +02:00
Thierry FOURNIER
0a99b89531 MINOR: lua: add core.done() function
This function immediately give back the control to HAProxy core.
2015-08-27 11:27:29 +02:00
Thierry FOURNIER
bc965348d7 DOC: ssl: missing LF
An error message miss LF
2015-08-27 11:24:23 +02:00
Thierry FOURNIER
10ec214f41 BUG/MEDIUM: lua: the lua fucntion Channel:close() causes a segfault
The function dont remove remaineing analysers and dont update response
channel timeout.

The fix is a copy of the behavior of the functions http_apply_redirect_rule()
and stream_int_retnclose().
2015-08-25 18:24:11 +02:00
Willy Tarreau
bdc97a8795 BUG/MEDIUM: lua: outgoing connection was broken since 1.6-dev2
Tsvetan Tsvetanov reported that the following Lua code fails in
dev2 and dev3 :

	function hello(txn)
	    local request_msg = txn.req:dup()
	    local tsm_sock = core.tcp()
	    tsm_sock:connect("127.0.0.1", 7777)
	    local res = tsm_sock:send(request_msg)
	    local response = tsm_sock:receive('*l')
	    txn.res:send(response)
	    txn:close()
	end

Thierry diagnosed that it was caused by commit 563cc37 ("MAJOR: stream:
use a regular ->update for all stream interfaces"). It broke lua's
ability to establish outgoing connections.

The reason is that the applet used to be notified about established
connections just after the stream analyser loop, and that's not the
case anymore. In peers, this issue didn't happen because peers use
a handshake so after sending data, the response is received and wakes
the applet up again. Here we have to indicate that we want to send or
receive data, this will cause the notification to happen as soon as
the connection is ready. This is similar to pretending that we're
working on a full buffer after all. In theory subscribing for reads
is not needed, but it's added here for completeness.

Reported-By: Tsvetan Tsvetanov <cpi.cecko@gmail.com>
2015-08-25 16:58:00 +02:00
Pieter Baauw
8669e971a3 MINOR cfgparse: Correct the mailer warning text to show the right names to the user 2015-08-24 21:59:39 +02:00
Emeric Brun
b157d73beb BUG/MAJOR: peers: fix current table pointer not re-initialized on session release.
This bug causes malfunctions after re-connect. For instance the re-sync fails.
2015-08-21 14:24:32 +02:00
Emeric Brun
e1ab808ff8 BUG/MEDIUM: peers: fix wrong message id on stick table updates acknowledgement.
The table definition message id was used instead of the update acknowledgement id.

This bug causes a malformated message and a protocol error and breaks the
connection.

After that, the updates remain unacknowledged.
2015-08-21 14:24:17 +02:00
Willy Tarreau
29fbe51490 MAJOR: tproxy: remove support for cttproxy
This was the first transparent proxy technology supported by haproxy
circa 2005 but it was obsoleted in 2007 by Tproxy 4.0 which removed a
lot of the earlier versions' shortcomings and was finally merged into
the kernel. Since nobody has been using cttproxy for many years now
and nobody has even just tried to compile the files, it's time to
remove it. The doc was updated as well.
2015-08-20 19:35:14 +02:00
Thierry FOURNIER
e0627bdae0 MINOR: stick-tables: Add GPC0 actions
This patch adds access to GPC0 through http and tcp actions
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
236657b5e0 MINOR: stick-tables: Add GPT0 access
This patch adds acces to GPT0. The access can be done with http and
tcp actions, and through a converter.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
3cf1111eb3 MEDIUM: stick-tables: Add GPT0 in the stick tables
This patch adds GPT0 (General Purpose Tag 0). It is useful for storing
a simple state.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
7e25df3f17 MAJOR: stick-tables: remove key storage from the key struct
Now, the key struct only points to the storage provided by the
sample as input.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
5d24ebc3d7 MEDIUM: stick-tables: use the sample type names
This patch removes the special stick tables types names and
use the standard sample type names. This avoid the maintainance
of two types and remove the switch/case for matching a sample
type for each stick table type.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
bc8c404449 MAJOR: stick-tables: use sample types in place of dedicated types
This patch is the first step for sample integration. Actually
the stick tables uses her own data type, and some converters
must be called to convert sample type to stick-tables types.

This patch removes the stick-table types and replace it by
the sample types. This prevent:
 - Maintenance of two types of converters
 - reduce the code using the samples converters
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
4dc15d1a2d MINOR: actions: Remove wrappers
Now the prototype for each action from each section are the same, and
a discriminant for determining for each section we are called are added.
So, this patch removes the wrappers for the action functions called from
more than one section.

This patch removes 132 lines of useless code.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
afa80496db MEDIUM: actions: Normalize the return code of the configuration parsers
This patch normalize the return code of the configuration parsers. Before
these changes, the tcp action parser returned -1 if fail and 0 for the
succes. The http action returned 0 if fail and 1 if succes.

The normalisation does:
 - ACT_RET_PRS_OK for succes
 - ACT_RET_PRS_ERR for failure
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
322a124867 MINOR: actions: mutualise the action keyword lookup
Each (http|tcp)-(request|response) action use the same method
for looking up the action keyword during the cofiguration parsing.

This patch mutualize the code.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
c2bb050f7f MINOR: proto_tcp: proto_tcp.h is now useles
After removing the keyword register from types/proto_tcp.h, the header
file remains empty. This patch remove it
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
36481b8667 MEDIUM: actions: Merge (http|tcp)-(request|reponse) keywords structs
This patch merges the conguration keyword struct. Each declared configuration
keyword struct are similar with the others. This patch simplify the code.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
24ff6c6fce MEDIUM: actions: Add standard return code for the action API
Action function can return 3 status:
 - error if the action encounter fatal error (like out of memory)
 - yield if the action must terminate his work later
 - continue in other cases
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
0ea5c7fafa MINOR: actions: change actions names
For performances considerations, some actions are not processed by remote
function. They are directly processed by the function. Some of these actions
does the same things but for different processing part (request / response).

This patch give the same name for the same actions, and change the normalization
of the other actions names.

This patch is ONLY a rename, it doesn't modify the code.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
91f6ba0f2c MINOR: actions: Declare all the embedded actions in the same header file
This patch group the action name in one file. Some action are called
many times and need an action embedded in the action caller. The main
goal is to have only one header file grouping all definitions.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
22e49011b1 MINOR: actions: remove the mark indicating the last entry in enum
This mark permit to detect if the action tag is over the allowed range.
 - Normally, this case doesn't appear
 - If it appears, it is processed by ded fault case of the switch
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
5563e4b469 MINOR: actions: add "from" information
This struct member is used to specify who is the rule caller. It permits
to use one function for differents callers.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
d0d65aeab6 MEDIUM: capture: Move the capture configuration storage in the union
This patch moves the capture configuration struct (capture_prm) in the main
"arg" union. This reduce the size of the struct.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
5ec63e008d MEDIUM: track-sc: Move the track-sc configuration storage in the union
This patch moves the track-sc configuration struct (track_ctr_prm) in the main
"arg" union. This reduce the size od the struct.
2015-08-20 17:13:47 +02:00
Thierry FOURNIER
e209797ef0 MINOR: proto_http: replace generic opaque types by real used types in "http_capture" by id
This patch removes the generic opaque type for storing the configuration of the
action "http_capture" by id.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
32b15003fe MINOR: proto_http: replace generic opaque types by real used types in "http_capture"
This patch removes the generic opaque type for storing the configuration of the
action "http_capture"".
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
8855a92d8c MINOR: proto_http: replace generic opaque types by real used types for the actions on thr request line
This patch removes the generic opaque type for storing the configuration of the
action "set-method", "set-path", "set-query" and "set-uri".
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
a002dc9df8 MINOR: proto_http: use an "expr" type in place of generic opaque type.
This patch removes the generic opaque type for storing the configuration of the
acion "set-src" (HTTP_REQ_ACT_SET_SRC), and use the dedicated type "struct expr"
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
f8c1dcea01 MINOR: vars: use the vars types as argument in place of opaque type
The (http|tcp)-(request|response) action rules use common
opaque type. For the HAProxy embbedded feature, types are know,
it better to add this types in the action union and use it.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
231ef1d99c MINOR: lua: use the hlua_rule type in place of opaque type
The (http|tcp)-(request|response) action rules use common
opaque type. For the HAProxy embbedded feature, types are know,
it better to add this types in the action union and use it.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
7677f400f5 MINOR: actions: Remove the data opaque pointer
This patch removes the "data" opaque pointer and replace it by the generic
opaque pointer "p[0]".
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
a28a9429b2 MEDIUM: actions: Merge (http|tcp)-(request|reponse) action structs
This patch is the first of a serie which merge all the action structs. The
function "tcp-request content", "tcp-response-content", "http-request" and
"http-response" have the same values and the same process for some defined
actions, but the struct and the prototype of the declared function are
different.

This patch try to unify all of these entries.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
b2f8f087fc MINOR: map: The map can return IPv4 and IPv6
The maps can store both IPv4 and IPv6 as sample.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
7d4335c935 MEDIUM: pattern/map: Maps can returns various types
A map can store and return various types as output. The only one example is the
IPv4 and IPv6 types. The previous patch remove the type from the sample storage
struct and use the conoverter output type, expecting that all entries of the
map have the same type.

This will be wrong when the maps will support both IPv4 and IPv6 as output.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
5cc18d46f3 MINOR: samples: data assignation simplification
With the difference between the "struct sample" data and the
"struct sample_storage" data, it was not possible to write
data = data, and we did a memcpy. This patch remove some of
these memcpy.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
2046c46468 MEDIUM: 51degrees: Adapt the 51Degrees library
I can't test this patch because the avalaible 51degrees library is
"51Degrees-C-3.1.5.2" and HAProxy obviously build with another version
(some defines and symbols disappear).

This patch is provided as-is in best effort.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
136f9d34a9 MINOR: samples: rename union from "data" to "u"
The union name "data" is a little bit heavy while we read the source
code because we can read "data.data.sint". The rename from "data" to "u"
makes the read easiest like "data.u.sint".
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
8c542cac07 MEDIUM: samples: Use the "struct sample_data" in the "struct sample"
This patch remove the struct information stored both in the struct
sample_data and in the striuct sample. Now, only thestruct sample_data
contains data, and the struct sample use the struct sample_data for storing
his own data.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
503bb09873 MINOR: samples: rename some struct member from "smp" to "data"
This members contains data and not sample.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
12ba0c29d6 MINOR: samples: rename a struct from sample_storage to sample_data
This a first step of sample reorganization.
2015-08-20 17:13:46 +02:00
Pradeep Jindal
cc79b003cb BUG/MINOR: ssl: TLS Ticket Key rotation broken via socket command
It seems haproxy was doing wrong pointer arithmetic to update the ticket
ring correctly.
2015-08-20 16:23:22 +02:00
Thierry FOURNIER
c7d7607351 BUG/MEDIUM: lua: bad error processing
When the lua actions (http|tcp)-(request|rersponse) fails a yield is
returned. this patch fix the behavior and returns a continue.
2015-08-19 10:54:06 +02:00
Willy Tarreau
0f4eadd483 BUG/MEDIUM: counters: ensure that src_{inc,clr}_gpc0 creates a missing entry
During 1.5-dev20 there was some code refactoring to make the src_* fetch
function use the same code as sc_*. Unfortunately this introduced a
regression where src_* doesn't create an entry anymore if it does not
exist in the table. The reason is that smp_fetch_sc_stkctr() only calls
stktable_lookup_key() while src_inc_*/src_clr_* used to make use of
stktable_update_key() which additionally create the entry if it does
not exist.

There's no point modifying the common function for these two exceptions,
so instead we now have a function dedicated to the creation of this entry
for src_* only. It is called when the entry didn't exist, so that requires
minimal modifications to existing code.

Thanks to Thierry Fournier for helping diagnose the issue.

This fix must be backported to 1.5.
2015-08-18 17:15:20 +02:00
Thierry FOURNIER
827752e7ec BUG/MEDIUM: stick-tables: refcount error after copying SC for the session to the stream
Each copyied SC counter will be used and released by the stream, so we
need to increment the refcount.
2015-08-18 13:39:49 +02:00
Thierry FOURNIER
6f82b6dc35 BUG/MEDIUM: vars: segfault during the configuration parsing
This bug is introduced by the patch 48a9cd104d
"MINOR: vars: reduce the code size of some wrappers". A dereferencement was
removed.
2015-08-17 18:48:18 +02:00
Baptiste Assmann
93c20623db MINOR: server SRV_ADMF_CMAINT flag doesn't imply SRV_ADMF_FMAINT
The newly created server flag SRV_ADMF_CMAINT means that the server is
in 'disabled' mode because of configuration statement 'disabled'.
The flag SRV_ADMF_FMAINT should not be set anymore in such case and is
reserved only when the server is Forced in maintenance mode from the
stats socket.
2015-08-17 15:42:07 +02:00
Thierry FOURNIER
c8fdb983c5 BUG/MEDIUM: stream: The stream doen't inherit SC from the session
During the processing of tcp-request connection, the stream doesn't exists, so the
stick counters are stored in the session. When the stream is created it must
inherit from the session sc.

This patch fix this behavior.

[WT: this is specific to 1.6, no backport needed]
2015-08-16 23:55:32 +02:00
Thierry FOURNIER
03d0e45dbb MEDIUM: cli: rely on the map's output type instead of the sample type
Next patch will remove sample_storage->type, and the only user is the
"show map" feature on the CLI which can use the map's output type instead.
Let's do that first.
2015-08-11 14:14:11 +02:00
Thierry FOURNIER
a123ad886a MINOR: sample/proto_tcp: export "smp_fetch_src"
This patch exports the sample fetch "smp_fetch_src()".
2015-08-11 14:14:11 +02:00
Thierry FOURNIER
cc4d1716a2 MINOR: sample: Add ipv6 to ipv4 and sint to ipv6 casts
The RFC4291 says that when the IPv6 adress have the followin form:
0000::ffff:a.b.c.d, if can be converted to an IPv4 adress. This patch
enable this conversion in casts.

As the sint can be casted as ipv4, and ipv4 can be casted as ipv6, we
can directly cast sint as ipv6 using the RFC4291.
2015-08-11 14:14:10 +02:00
Thierry FOURNIER
48a9cd104d MINOR: vars: reduce the code size of some wrappers
Some function are just a wrappers. This patch reduce the size of
this wrapper for improving the readability. One check is moved
from the wrapper to the main function, and some middle vars are
removed.
2015-08-11 14:14:10 +02:00
Thierry FOURNIER
422a3af4ce MINOR: proto_tcp: add session in the action prototype
Some actions require the "struct session" while the "struct stream" is not
avalaible. This patch adds a pointer to the session.
2015-08-11 14:08:29 +02:00
Thierry FOURNIER
c89f4f5305 BUG/MINOR: proto_tcp: custom action continue is ignored
The custom action is ignored by 'tcp-request connection'. This patch
fix this behavior and take in account the value of the flag 'action'.
2015-08-11 13:45:46 +02:00
Thierry FOURNIER
a6b6343cff CLEANUP: http/tcp actions: remove the scope member
The scope member is not used. This patch removes this entry.
2015-08-11 13:44:53 +02:00
Thierry FOURNIER
9b49f589ed CLEANUP: proto_http: remove useless initialisation
This initialisation of the opaque array is useless.
2015-08-11 13:44:51 +02:00
Willy Tarreau
f3934b8c6d MEDIUM: config: emit a warning on a frontend without listener
Commit c6678e2 ("MEDIUM: config: authorize frontend and listen without bind")
completely removed the test for bind lines in frontends in order to make it
easier for automated tools to generate configs (eg: replacing a bind with
another one passing via a temporary config without any bind line). The
problem is that some common mistakes are totally hidden now. For example,
this apparently valid entry is silently ignored :

       listen 1.2.3.4:8000
           server s1 127.0.0.1:8000

Hint: 1.2.3.4:8000 is mistakenly the proxy name here.

Thus instead we now emit a warning to indicate that a frontend was found
with no listener. This should be backported to 1.5 to help spot abnormal
configurations.
2015-08-11 12:17:41 +02:00
Willy Tarreau
9496552e7d CLEANUP: appsession: remove appsession.c and sessionhash.c
Now there's no more code using appsessions we can remove them.
2015-08-10 19:17:47 +02:00
Willy Tarreau
6db62c58d6 CLEANUP: config: remove appsession initialization
Now it asks to check the documentation.
2015-08-10 19:16:18 +02:00
Willy Tarreau
53a09d520e MAJOR: http: remove references to appsession
appsessions started to be deprecated with the introduction of stick
tables, and the latter are much more powerful and flexible, and in
addition they are replicated between nodes and maintained across
reloads. Let's now remove appsession completely.
2015-08-10 19:16:18 +02:00
Thierry FOURNIER
61e96c68b9 BUG/MEDIUM: lua: timeout error with converters, wrapper and actions.
test conf:

   global
      tune.lua.session-timeout 0
      lua-load lol.lua
      debug
      maxconn 4096

   listen test
	   bind 0.0.0.0:10010
		mode tcp
		tcp-request content lua act_test
		balance roundrobin
		server test 127.0.0.1:3304

lua test:

   function act_test(txn)
      while true do
         core.Alert("TEST")
      end
   end

The function "act_test()" is not executed because a zero timeout is not
considered as TICK_ETERNITY, but is considered as 0.

This path fix this behavior. This is the same problem than the bugfix
685c014e99.
2015-08-10 06:18:30 +02:00
Willy Tarreau
f2ee0162c3 MINOR: init: indicate to check 'bind' lines when no listeners were found.
It used to still mention '<listen>'.
2015-08-09 11:01:51 +02:00
Camilo Lopez
685c014e99 BUG/MEDIUM: lua: Lua tasks fail to start.
I've been trying out 1.6 dev3 with lua support, and trying to start
lua tasks seems to not be working.

Using this configuration

global
  lua-load /lua/lol.lua
  debug
  maxconn 4096

backend shard_b
  server db01 mysql_shard_b:3306

backend shard_a
  server db01 mysql_shard_a:3306

listen mysql-cluster
  bind 0.0.0.0:8001
  mode tcp
  balance roundrobin
  use_backend shard_b

And this lua function

  core.register_task(function()
    while true do
      core.Alert("LOLOLOLOLOL")
    end
  end)

I'd always get a timeout error starting the registered function.

The problem lies as far as I can tell in the fact that is possible for
now_ms to not change (is this maybe a problem on my config/system?)
until the expiration check happens, in the resume function that
actually kickstarts the lua task, making HAProxy think that expiration
time for the task is up, if I understand correctly tasks are meant to
never really timeout.
2015-08-09 10:22:34 +02:00
Andrew Hayworth
e63ac871f8 MINOR: log: Add log-format variable %HQ, to log HTTP query strings
Since sample fetches are not always available in the response phase,
this patch implements %HQ such that:

  GET /foo?bar=baz HTTP/1.0

...would be logged as:

  ?bar=baz
2015-08-09 10:16:49 +02:00
Baptiste Assmann
9f5ada32e4 MINOR: server: add new SRV_ADMF_CMAINT flag
The purpose of SRV_ADMF_CMAINT flag is to keep in mind the server was
forced to maintenance status because of the configuration file.
2015-08-08 18:18:17 +02:00
Baptiste Assmann
2359ff1de2 BUG/MEDIUM: DNS resolution response parsing broken
In some cases, parsing of the DNS response is broken and the response is
considered as invalid, despite being valid.

The current patch fixes this issue. It's a temporary solution until I
rework the response parsing to store the response buffer into a real DNS
packet structure.
2015-08-08 18:14:20 +02:00
Baptiste Assmann
37bb372ea2 MINOR: DNS counters: increment valid counter
Valid counter was never incremented.
Now it is.
2015-08-08 18:13:59 +02:00
Dragan Dosen
9373fc5811 MINOR: 51d: unable to start haproxy without "51degrees-data-file"
This patch adds a few checks on "global._51degrees.data_file_path" and allows
haproxy to start even when the pattern or trie data file is not specified.

If the "51d" converter is used, a new function "_51d_conv_check" will check
"global._51degrees.data_file_path" and displays a warning if necessary.

In src/haproxy.c, the global 51Degrees "cache_size" has moved outside of the
FIFTYONEDEGREES_H_PATTERN_INCLUDED ifdef block.
2015-08-08 12:15:55 +02:00
Willy Tarreau
449d74a906 MEDIUM: backend: add the "http-reuse aggressive" strategy
This strategy is less extreme than "always", it only dispatches first
requests to validated reused connections, and moves a connection from
the idle list to the safe list once it has seen a second request, thus
proving that it could be reused.
2015-08-06 16:29:01 +02:00
Willy Tarreau
7017cb040c MINOR: server: add a list of safe, already reused idle connections
These ones are considered safe as they have already been reused.
They will be useful in "aggressive" and "always" http-reuse modes
in order to place the first request of a connection with the least
risk.
2015-08-06 16:29:01 +02:00
Willy Tarreau
161d45ffc7 MEDIUM: backend: implement "http-reuse safe"
The "safe" mode consists in picking existing connections only when
processing a request that's not the first one from a connection. This
ensures that in case where the server finally times out and closes, the
client can decide to replay idempotent requests.
2015-08-06 11:50:53 +02:00
Willy Tarreau
efb90f9dd3 MAJOR: backend: improve the connection reuse mechanism
Now instead of closing the existing connection attached to the
stream interface, we first check if the one we pick was attached to
another stream interface, in which case the connections are swapped
if possible (eg: if the current connection is not private). That way
the previous connection remains attached to an existing session and
significantly increases the chances of being reused.
2015-08-06 11:41:06 +02:00
Willy Tarreau
8dff998b91 MAJOR: backend: initial work towards connection reuse
In connect_server(), if we don't have a connection attached to the
stream-int, we first look into the server's idle_conns list and we
pick the first one there, we detach it from its owner if it had one.
If we used to have a connection, we close it.

This mechanism works well but doesn't scale : as servers increase,
the likeliness that the connection attached to the stream interface
doesn't match the server and gets closed increases.
2015-08-06 11:34:21 +02:00
Willy Tarreau
d8fecee291 MINOR: config: add new setting "http-reuse"
For now it only supports "never", meaning that we never want to reuse a
shared connection, and "always", meaning that we can use any connection
that was not marked private. When "never" is set, this also implies that
no idle connection may become a shared one.
2015-08-06 11:14:20 +02:00
Willy Tarreau
387ebf84dd MINOR: connection: add a new flag CO_FL_PRIVATE
This flag is set on an outgoing connection when this connection gets
some properties that must not be shared with other connections, such
as dynamic transparent source binding, SNI or a proxy protocol header,
or an authentication challenge from the server. This will be needed
later to implement connection reuse.
2015-08-06 11:14:17 +02:00
Willy Tarreau
173a1c6b43 MINOR: server: add a list of already used idle connections
There's a difference with the other idle conns in that these new
ones have already been used and may be reused by other streams.
2015-08-06 11:13:47 +02:00
Willy Tarreau
4320eaac62 MINOR: stream-int: make si_idle_conn() only accept valid connections
This function is now dedicated to idle connections only, which means
that it must not be used without any endpoint nor anything not a
connection. The connection remains attached to the stream interface.
2015-08-06 11:11:10 +02:00
Willy Tarreau
323a2d925c MEDIUM: stream-int: queue idle connections at the server
Now we get a per-server list of all idle connections. That way we'll
be able to reclaim them upon shortage later.
2015-08-06 11:06:25 +02:00
Willy Tarreau
600802aef0 MINOR: server: add a list of private idle connections
For now it's not populated but we have the list entry. It will carry
all idle connections that sessions don't want to share. They may be
used later to reclaim connections upon socket shortage for example.
2015-08-06 10:59:08 +02:00
Willy Tarreau
973a54235f MEDIUM: stream-int: simplify si_alloc_conn()
Since we now always call this function with the reuse parameter cleared,
let's simplify the function's logic as it cannot return the existing
connection anymore. The savings on this inline function are appreciable
(240 bytes) :

$ size haproxy.old haproxy.new
   text    data     bss     dec     hex filename
1020383   40816   36928 1098127  10c18f haproxy.old
1020143   40816   36928 1097887  10c09f haproxy.new
2015-08-05 21:51:09 +02:00
Willy Tarreau
c12b5e663d MEDIUM: backend: don't call si_alloc_conn() when we reuse a valid connection
connect_server() already does most of the check that is done again in
si_alloc_conn(), so let's simply reuse the existing connection instead
of calling the function again. It will also simplify the connection
reuse.

Indeed, for reuse to be set, it also requires srv_conn to be valid. In the
end, the only situation where we have to release the existing connection
and allocate a new one is when reuse == 0.
2015-08-05 21:42:12 +02:00
Willy Tarreau
7b00492ce3 CLEANUP: backend: factor out objt_server() in connect_server()
objt_server() is called multiple times at various places while some
places already make use of srv for this. Let's move the call at the
top of the function and use it all over the place.
2015-08-05 10:12:47 +02:00
Remi Gacogne
7fb9de25e2 MINOR: stream: initialize the current_rule field to NULL on stream init
Currently it is possible for the current_rule field to be evaluated before
being set, leading to valgrind complaining:

==16783== Conditional jump or move depends on uninitialised value(s)
==16783==    at 0x44E662: http_res_get_intercept_rule (proto_http.c:3730)
==16783==    by 0x44E662: http_process_res_common (proto_http.c:6528)
==16783==    by 0x4797B7: process_stream (stream.c:1851)
==16783==    by 0x414634: process_runnable_tasks (task.c:238)
==16783==    by 0x40B02F: run_poll_loop (haproxy.c:1528)
==16783==    by 0x407F25: main (haproxy.c:1887)

This was introduced by commit 152b81e7b2.
2015-07-26 21:50:50 +02:00
Jrmie Courrges-Anglas
17c3f6284c BUILD: ssl: Allow building against libssl without SSLv3.
If SSLv3 is explicitely requested but not available, warn the user and
bail out.
2015-07-26 08:11:57 +02:00
Pieter Baauw
ed35c371dc BUG/MEDIUM: mailer: DATA part must be terminated with <CRLF>.<CRLF>
The dot is send in the wrong place.
As defined in https://www.ietf.org/rfc/rfc2821.txt 'the character sequence "<CRLF>.<CRLF>" ends the mail text'
2015-07-22 22:39:39 +02:00
Willy Tarreau
d69d6f3678 BUG/MAJOR: dns: fix the length of the string to be copied
Jan A. Bruder reported that some very specific hostnames on server
lines were causing haproxy to crash on startup. Given that hist
backtrace showed some heap corruption, it was obvious there was an
overflow somewhere. The bug in fact is a typo in dns_str_to_dn_label()
which mistakenly copies one extra byte from the host name into the
output value, thus effectively corrupting the structure.

The bug triggers while parsing the next server of similar length
after the corruption, which generally triggers at config time but
could theorically crash at any moment during runtime depending on
what malloc sizes are needed next. This is why it's tagged major.

No backport is needed, this bug was introduced in 1.6-dev2.
2015-07-22 16:53:22 +02:00
Willy Tarreau
2100b49122 CLEANUP/MINOR: dns: dns_str_to_dn_label() only needs a const char
The string is an input, let's constify it.
2015-07-22 16:42:43 +02:00
Thierry FOURNIER
5d86fae234 MEDIUM: vars/sample: operators can use variables as parameter
This patch allow the existing operators to take a variable as parameter.
This is useful to add the content of two variables. This patch modify
the behavior of operators.
2015-07-22 00:48:24 +02:00
Thierry FOURNIER
fd77e05f65 MINOR: vars: returns variable content
This patch copy the content of a variable in a sample. The function
returns 0 if the variable is not found.
2015-07-22 00:48:24 +02:00
Thierry FOURNIER
00c005c726 MEDIUM: sample: switch to saturated arithmetic
This patch check calculus for overflow and returns capped values.
This permits to protect against integer overflow in certain operations
involving ratios, percentages, limits or anything. That can sometimes
be critically important with some operations (eg: content-length < X).
2015-07-22 00:48:24 +02:00
Thierry FOURNIER
bf65cd4d77 MAJOR: arg: converts uint and sint in sint
This patch removes the 32 bits unsigned integer and the 32 bit signed
integer. It replaces these types by a unique type 64 bit signed.
2015-07-22 00:48:23 +02:00
Thierry FOURNIER
07ee64ef4d MAJOR: sample: converts uint and sint in 64 bits signed integer
This patch removes the 32 bits unsigned integer and the 32 bit signed
integer. It replaces these types by a unique type 64 bit signed.

This makes easy the usage of integer and clarify signed and unsigned use.
With the previous version, signed and unsigned are used ones in place of
others, and sometimes the converter loose the sign. For example, divisions
are processed with "unsigned", if one entry is negative, the result is
wrong.

Note that the integer pattern matching and dotted version pattern matching
are already working with signed 64 bits integer values.

There is one user-visible change : the "uint()" and "sint()" sample fetch
functions which used to return a constant integer have been replaced with
a new more natural, unified "int()" function. These functions were only
introduced in the latest 1.6-dev2 so there's no impact on regular
deployments.
2015-07-22 00:48:23 +02:00
Thierry FOURNIER
763a5d85f7 MINOR: standard: add 64 bits conversion functions
This patch adds 3 functions for 64 bit integer conversion.

 * lltoa_r : converts signed 64 bit integer to string
 * read_uint64 : converts from string to signed 64 bits integer with capping
 * read_int64 : converts from string to unsigned 64 bits integer with capping
2015-07-21 23:27:10 +02:00
Baptiste Assmann
19a106d24a MINOR: server: server_find functions: id, name, best_match
This patch introduces three new functions which can be used to find a
server in a farm using different server information:
- server unique id (srv->puid)
- server name
- find best match using either name or unique id

When performing best matching, the following applies:
 - use the server name first (if provided)
 - use the server id if provided
 in any case, the function can update the caller about mismatches
 encountered.
2015-07-21 23:24:16 +02:00
Baptiste Assmann
7cc419ae1d MINOR: server: new server flag: SRV_F_FORCED_ID
This flag aims at reporting whether the server unique id (srv->puid) has
been forced by the administrator in HAProxy's configuration.
If not set, it means HAProxy has generated automatically the server's
unique id.
2015-07-21 23:24:16 +02:00
Baptiste Assmann
8a027ccb38 MINOR: proxy: bit field for proxy_find_best_match diff status
function proxy_find_best_match can update the caller by updating an int
provided in argument.
For now, proxy_find_best_match hardcode bit values 0x01, 0x02 and 0x04,
which is not understandable when reading a code exploiting them.

This patch defines 3 macros with a more explicit wording, so further
reading of a code exploiting the magic bit values will be understandable
more easily.
2015-07-21 23:24:16 +02:00
Thierry FOURNIER
fac9ccfb70 BUG/MINOR: http/sample: gmtime/localtime can fail
The man said that gmtime() and localtime() can return a NULL value.
This is not tested. It appears that all the values of a 32 bit integer
are valid, but it is better to check the return of these functions.

However, if the integer move from 32 bits to 64 bits, some 64 values
can be unsupported.
2015-07-20 12:21:35 +02:00
Nenad Merdanovic
8a39a1fcc3 BUG/MINOR: payload: Add volatile flag to smp_fetch_req_ssl_ec_ext
This bug was introduced in 5fc7d7e. No backport to 1.5 needed.

Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-07-20 10:11:44 +02:00
Willy Tarreau
7a08d3b2d7 CLEANUP: stream-int: remove stream_int_unregister_handler() and si_detach()
The former was not used anymore and the latter was only used by the former.
They were only aliases to other existing functions anyway.
2015-07-19 18:48:20 +02:00
Willy Tarreau
a9ff5e64c1 CLEANUP: stream-int: fix a few outdated comments about stream_int_register_handler()
They were not updated after the infrastructure change.
2015-07-19 18:46:30 +02:00
Willy Tarreau
069b0c8e6f CLEANUP: stream: remove a useless call to si_detach()
It's pointless to call si_detach() after si_reset() since it does
nothing due to si->end being NULL.
2015-07-19 17:57:02 +02:00
Baptiste Assmann
4c5490ab9c BUG/MINOR: dns: wrong time unit for some DNS default parameters
Madison May reported that the timeout applied by the default
configuration is inproperly set up.
This patch fix this:
 - hold valid default to 10s
 - timeout retry default to 1s
2015-07-18 08:22:01 +02:00
Thierry FOURNIER
29176f3ebb BUG/MINOR: lua: type error in the arguments wrapper
The expected arguments time and size are returned as int. This patch
fix this bug and returns the expected type.
2015-07-10 16:29:31 +02:00
Willy Tarreau
732eac41f4 MEDIUM: ssl: add sni support on the server lines
The new "sni" server directive takes a sample fetch expression and
uses its return value as a hostname sent as the TLS SNI extension.
A typical use case consists in forwarding the front connection's SNI
value to the server in a bridged HTTPS forwarder :

   sni ssl_fc_sni
2015-07-10 11:43:15 +02:00
Willy Tarreau
630764188b MINOR: ssl: provide ia function to set the SNI extension on a connection
ssl_sock_set_servername() is used to set the SNI hostname on an
outgoing connection. This function comes from code originally
provided by Christopher Faulet of Qualys.
2015-07-10 11:37:29 +02:00
Willy Tarreau
42529c38ac MINOR: stream: maintain consistence between channel_forward and HTTP forward
When the HTTP forwarder is used, it resets msg->sov so that we know that
the parsing pointer has advanced by exactly (msg->eoh + msg->eol - msg->sov)
bytes which may have to be rewound in case we want to perform an HTTP fetch
after forwarding has started (eg: upon connect).

But when the backend is in TCP mode, there may be no HTTP forwarding
analyser installed, still we may want to perform these HTTP fetches in
case we have already ensured at the TCP layer that we have a properly
parsed HTTP transaction.

In order to solve this, we reset msg->sov before doing a channel_forward()
so that we can still compute http_rewind() on the pending data. That ensures
the buffer is always rewindable even in mixed TCP+HTTP mode.
2015-07-10 11:37:29 +02:00
Willy Tarreau
28d976d5ee MINOR: args: add new context for servers
We'll have to support fetch expressions and args on server lines for
"usesrc", "usedst", "sni", etc...
2015-07-09 11:39:33 +02:00
Willy Tarreau
53e1a6d317 BUG/MINOR: log: missing some ARGC_* entries in fmt_directives()
ARGC_CAP was not added to fmt_directives() which is used to format
error messages when failing to parse log format expressions. The
whole switch/case has been reorganized to match the declaration
order making it easier to spot missing values. The default is not
the "log" directive anymore but "undefined" asking to report the
bug.

Backport to 1.5 is not strictly needed but is desirable at least
for code sanity.
2015-07-09 11:20:00 +02:00
Nenad Merdanovic
5fc7d7e8ce MINOR: Add sample fetch to detect Supported Elliptic Curves Extension
Clients that support ECC cipher suites SHOULD send the specified extension
within the SSL ClientHello message according to RFC4492, section 5.1. We
can use this extension to chain-proxy requests so that, on the same IP
address, a ECC compatible clients gets an EC certificate and a non-ECC
compatible client gets a regular RSA certificate. The main advantage of this
approach compared to the one presented by Dave Zhu on the mailing list
is that we can make it work with OpenSSL versions before 1.0.2.

Example:
frontend ssl-relay
        mode tcp
        bind 0.0.0.0:443
        use_backend ssl-ecc if { req.ssl_ec_ext 1 }
        default_backend ssl-rsa

backend ssl-ecc
        mode tcp
        server ecc unix@/var/run/haproxy_ssl_ecc.sock send-proxy-v2 check

backend ssl-rsa
        mode tcp
        server rsa unix@/var/run/haproxy_ssl_rsa.sock send-proxy-v2 check

listen  all-ssl
        bind unix@/var/run/haproxy_ssl_ecc.sock accept-proxy ssl crt /usr/local/haproxy/ecc.foo.com.pem user nobody
        bind unix@/var/run/haproxy_ssl_rsa.sock accept-proxy ssl crt /usr/local/haproxy/www.foo.com.pem user nobody

Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-07-09 09:26:59 +02:00
Willy Tarreau
fc017fec48 CLEANUP: ssl: make ssl_sock_generate_certificate() use ssl_sock_generated_cert_serial()
It saves from open-coding the hashing method. Instead all serial
generation is done in a single function.
2015-07-07 18:09:34 +02:00
Willy Tarreau
646b864fe5 CLEANUP: ssl: make ssl_sock_generated_cert_serial() take a const
It doesn't modify the input.
2015-07-07 18:09:15 +02:00
Willy Tarreau
f67214554c MINOR: ssl: make self-generated certs also work with raw IPv6 addresses
The current method of retrieving the incoming connection's destination
address to hash it is not compatible with IPv6 nor the proxy protocol
because it directly tries to get an IPv4 address from the socket. Instead
we must ask the connection. This is only used when no SNI is provided.
2015-07-07 18:04:38 +02:00
Dragan Dosen
96a0be78ed BUG/MEDIUM: 51d: possible incorrect operations on smp->data.str.str
In src/51d.c, the function _51d_conv(), a final '\0' is added into
smp->data.str.str, which can cause a problem if the SMP_F_CONST flag is
set in smp->flags or if smp->data.str.size is not available.

This patch adds a check on smp->flags and smp->data.str.size, and copies
the smp->data.str.str to another buffer by using smp_dup(). If necessary,
the "const" flag is set after device detection. Also, this patch removes
the unnecessary call to chunk_reset() on temp argument.
2015-07-07 17:19:33 +02:00
Adis Nezirovic
2fbcafc9ce MEDIUM: http: Add new 'set-src' option to http-request
This option enables overriding source IP address in a HTTP request. It is
useful when we want to set custom source IP (e.g. front proxy rewrites address,
but provides the correct one in headers) or we wan't to mask source IP address
for privacy or compliance.

It acts on any expression which produces correct IP address.
2015-07-06 16:17:28 +02:00
Adis Nezirovic
79beb248b9 CLEANUP: sample: generalize sample_fetch_string() as sample_fetch_as_type()
This modification makes possible to use sample_fetch_string() in more places,
where we might need to fetch sample values which are not plain strings. This
way we don't need to fetch string, and convert it into another type afterwards.

When using aliased types, the caller should explicitly check which exact type
was returned (e.g. SMP_T_IPV4 or SMP_T_IPV6 for SMP_T_ADDR).

All usages of sample_fetch_string() are converted to use new function.
2015-07-06 16:17:25 +02:00
Willy Tarreau
4e5d58e517 MINOR: stats: improve compression stats reporting
Compression stats were not easy to read and could be confusing because
the saving ratio could be taken for global savings while it was only
relative to compressible input. Let's make that a bit clearer using
the new tooltips with a bit more details and also report the effective
ratio over all output bytes.
2015-07-04 14:35:15 +02:00
Willy Tarreau
27f78241e6 BUG/MAJOR: tcp: tcp rulesets were still broken
Commit cc87a11 ("MEDIUM: tcp: add register keyword system.") broke the
TCP ruleset by merging custom rules and accept. It was fixed a first time
by commit e91ffd0 ("BUG/MAJOR: tcp: only call registered actions when
they're registered") but the accept action still didn't work anymore
and was causing the matching rule to simply be ignored.

Since the code introduced a very fragile behaviour by not even mentionning
that accept and custom were silently merged, let's fix this once for all by
adding an explicit check for the accept action. Nevertheless, as previously
mentionned, the action should be changed so that custom is the only action
and the continue vs break indication directly comes from the callee.

No backport is needed, this bug only affects 1.6-dev.
2015-07-04 11:36:30 +02:00
KOVACS Krisztian
7209c204bd BUG/MAJOR: connection: fix TLV offset calculation for proxy protocol v2 parsing
Until now, the code assumed that it can get the offset to the first TLV
header just by subtracting the length of the TLV part from the length of
the complete buffer. However, if the buffer contains actual data after
the header, this computation is flawed and leads to haproxy trying to
parse TLV headers from the proxied data.

This change fixes this by making sure that the offset to the first TLV
header is calculated based from the start of the buffer -- simply by
adding the size of the proxy protocol v2 header plus the address
family-dependent size of the address information block.
2015-07-03 17:05:20 +02:00
Willy Tarreau
27187ab56a BUG/MAJOR: buffers: make the buffer_slow_realign() function respect output data
The function buffer_slow_realign() was initially designed for requests
only and did not consider pending outgoing data. This causes a problem
when called on responses where data remain in the buffer, which may
happen with pipelined requests when the client is slow to read data.

The user-visible effect is that if less than <maxrewrite> bytes are
present in the buffer from a previous response and these bytes cross
the <maxrewrite> boundary close to the end of the buffer, then a new
response will cause a realign and will destroy these pending data and
move the pointer to what's believed to contain pending output data.
Thus the client receives the crap that lies in the buffer instead of
the original output bytes.

This new implementation now properly realigns everything including the
outgoing data which are moved to the end of the buffer while the input
data are moved to the beginning.

This implementation still uses a buffer-to-buffer copy which is not
optimal in terms of performance and which should be replaced by a
buffer switch later.

Prior to this patch, the following script would return different hashes
on each round when run from a 100 Mbps-connected machine :

  i=0
  while usleep 100000; do
    echo round $((i++))
    set -- $(nc6 0 8001 < 1kreq5k.txt | grep -v '^[0-9A-Z]' | md5sum)
    if [ "$1" != "3861afbb6566cd48740ce01edc426020" ]; then echo $1;break;fi
  done

The file contains 1000 times this request with "Connection: close" on the
last one :

  GET /?s=5k&R=1 HTTP/1.1

The config is very simple :

  global
        tune.bufsize 16384
        tune.maxrewrite 8192

  defaults
        mode http
        timeout client 10s
        timeout server 5s
        timeout connect 3s

  listen px
        bind :8001
        option http-server-close
        server s1 127.0.0.1:8000

And httpterm-1.7.2 is used as the server on port 8000.

After the fix, 1 million requests were sent and all returned the same
contents.

Many thanks to Charlie Smurthwaite of atechmedia.com for his precious
help on this issue, which would not have been diagnosed without his
very detailed traces and numerous tests.

The patch must be backported to 1.5 which is where the bug was introduced.
2015-07-02 15:27:24 +02:00
David Carlier
e6c3941668 BUILD/MINOR: tools: rename popcount to my_popcountl
This is in order to avoid conflicting with NetBSD popcount* functions
since 6.x release, the final l to mentions the argument is a long like
NetBSD does.

This patch could be backported to 1.5 to fix the build issue there as well.
2015-07-02 11:32:25 +02:00
Dragan Dosen
105c8e6368 MEDIUM: 51d: add LRU-based cache on User-Agent string detection
This cache is used by 51d converter. The input User-Agent string, the
converter args and a random seed are used as a hashing key. The cached
entries contains a pointer to the resulting string for specific
User-Agent string detection.

The cache size can be tuned using 51degrees-cache-size parameter.
2015-06-30 10:43:03 +02:00
Dragan Dosen
93b38d9191 MEDIUM: 51Degrees code refactoring and cleanup
Moved 51Degrees code from src/haproxy.c, src/sample.c and src/cfgparse.c
into a separate files src/51d.c and include/import/51d.h.

Added two new functions init_51degrees() and deinit_51degrees(), updated
Makefile and other code reorganizations related to 51Degrees.
2015-06-30 10:43:03 +02:00
Willy Tarreau
e44136fe69 BUG/MEDIUM: vars: do not freeze the connection when the expression cannot be fetched
Commit 4834bc7 ("MEDIUM: vars: adds support of variables") brought a bug.
Setting a variable from an expression that doesn't resolve infinitely
blocks the processing.

The internal actions API must be changed to let the caller pass the various
flags regarding the state of the analysis (SMP_OPT_FINAL).

For now we only fix the issue by making the action_store() function always
return 1 to prevent any blocking.

No backport is needed.
2015-06-23 15:17:33 +02:00
Willy Tarreau
ebcd4844e8 MEDIUM: vars: move the session variables to the session, not the stream
It's important that the session-wide variables are in the session and not
in the stream.
2015-06-19 11:59:02 +02:00
Willy Tarreau
7233098da1 MINOR: vars: make the accounting not depend on the stream
We'll need to move the session variables to the session. For this, the
accounting must not depend on the stream. Instead we pass the pointers
to the different lists.
2015-06-19 11:21:56 +02:00
Willy Tarreau
57b8a53f03 BUG/MEDIUM: lru: fix possible memory leak when ->free() is used
Commit 7810ad7 ("BUG/MAJOR: lru: fix unconditional call to free due to
unexpected semi-colon") was not enough, it happens that the free() is
not performed at the right place because if the evicted node is recycled,
we must also release its data before it gets overwritten.

No backport is needed.
2015-06-17 20:33:30 +02:00
Willy Tarreau
b7636d1a10 BUG/MEDIUM: logs: fix improper systematic use of quotes with a few tags
Dmitry Sivachenko reported the following build warning using Clang, which
is a real bug :

  src/log.c:1538:22: warning: use of logical '&&' with constant operand
        [-Wconstant-logical-operand]
                                  if (tmp->options && LOG_OPT_QUOTE)
                                                   ^  ~~~~~~~~~~~~~
The effect is that recent log tags related to HTTP method, path, uri,
query have a bug making them always use quotes.

This bug was introduced in 1.6-dev2 with commit 0ebc55f ("MEDIUM: logs:
Add HTTP request-line log format directives"), so no backport is needed.
2015-06-17 19:58:02 +02:00
Willy Tarreau
7810ad7d59 BUG/MAJOR: lru: fix unconditional call to free due to unexpected semi-colon
Dmitry Sivachenko reported the following build warning using Clang, which
is a real bug :

  src/lru.c:133:32: warning: if statement has empty body [-Wempty-body]
                                  if (old->data && old->free);
                                                             ^
It results in calling old->free(old->data) even when old->free is NULL,
hence crashing on cached patterns.

The same bug appears a few lines below in lru64_destroy() :

  src/lru.c:195:33: warning: if statement has empty body [-Wempty-body]
                          if (elem->data && elem->free);
                                                       ^
Both were introduced in 1.6-dev2 with commit f90ac55 ("MINOR: lru: Add the
possibility to free data when an item is removed"), so no backport is needed.
2015-06-17 19:55:32 +02:00
Willy Tarreau
666f504906 BUILD/MINOR: stats: fix build warning due to condition always true
Dmitry Sivachenko reported the following harmless build warning using Clang :

  src/dumpstats.c:5196:48: warning: address of array 'strm_li(sess)->proto->name'
        will always evaluate to 'true' [-Wpointer-bool-conversion]
    ...strm_li(sess) && strm_li(sess)->proto->name ? strm_li(sess)->proto->nam...
                     ~~ ~~~~~~~~~~~~~~~~~~~~~~^~~~
proto->name cannot be null here as it's the protocol name which is stored
directly in the structure.

The same case is present in 1.5 though the code changed.
2015-06-17 19:49:52 +02:00
Willy Tarreau
22b0a68120 BUILD/MINOR: lua: fix a harmless build warning
Dmitry Sivachenko reported the following build warning using Clang,
though it's harmless :

  src/hlua.c:1911:13: warning: variable '_socket_info_expanded_form' is not needed
        and will not be emitted [-Wunneeded-internal-declaration]
  static char _socket_info_expanded_form[] = SOCKET_INFO_EXPANDED_FORM;
              ^
Indeed, the variable is not used except to compute a sizeof which is
taken from the string it is initialized from. It probably is a leftover
after various code refactorings. Let's get rid of it now since it's not
used anymore.

No backport is needed.
2015-06-17 19:46:16 +02:00
Willy Tarreau
745d412758 BUG/MINOR: ssl: fix smp_fetch_ssl_fc_session_id
Dmitry Sivachenko reported the following build warning using Clang
which is a real bug :

src/ssl_sock.c:4104:44: warning: address of 'smp->data.str.len' will always
      evaluate to 'true' [-Wpointer-bool-conversion]
        if (!smp->data.str.str || !&smp->data.str.len)

The impact is very low however, it will return an empty session_id
instead of no session id when none is found.

The fix should be backported to 1.5.
2015-06-17 19:46:14 +02:00
Willy Tarreau
ad90f0d1aa [RELEASE] Released version 1.6-dev2
Released version 1.6-dev2 with the following main changes :
    - BUG/MINOR: ssl: Display correct filename in error message
    - MEDIUM: logs: Add HTTP request-line log format directives
    - BUG/MEDIUM: check: tcpcheck regression introduced by e16c1b3f
    - BUG/MINOR: check: fix tcpcheck error message
    - MINOR: use an int instead of calling tcpcheck_get_step_id
    - MINOR: tcpcheck_rule structure update
    - MINOR: include comment in tcpcheck error log
    - DOC: tcpcheck comment documentation
    - MEDIUM: server: add support for changing a server's address
    - MEDIUM: server: change server ip address from stats socket
    - MEDIUM: protocol: add minimalist UDP protocol client
    - MEDIUM: dns: implement a DNS resolver
    - MAJOR: server: add DNS-based server name resolution
    - DOC: server name resolution + proto DNS
    - MINOR: dns: add DNS statistics
    - MEDIUM: http: configurable http result codes for http-request deny
    - BUILD: Compile clean when debug options defined
    - MINOR: lru: Add the possibility to free data when an item is removed
    - MINOR: lru: Add lru64_lookup function
    - MEDIUM: ssl: Add options to forge SSL certificates
    - MINOR: ssl: Export functions to manipulate generated certificates
    - MEDIUM: config: add DeviceAtlas global keywords
    - MEDIUM: global: add the DeviceAtlas required elements to struct global
    - MEDIUM: sample: add the da-csv converter
    - MEDIUM: init: DeviceAtlas initialization
    - BUILD: Makefile: add options to build with DeviceAtlas
    - DOC: README: explain how to build with DeviceAtlas
    - BUG/MEDIUM: http: fix the url_param fetch
    - BUG/MEDIUM: init: segfault if global._51d_property_names is not initialized
    - MAJOR: peers: peers protocol version 2.0
    - MINOR: peers: avoid re-scheduling of pending stick-table's updates still not pushed.
    - MEDIUM: peers: re-schedule stick-table's entry for sync when data is modified.
    - MEDIUM: peers: support of any stick-table data-types for sync
    - BUG/MAJOR: sample: regression on sample cast to stick table types.
    - CLEANUP: deinit: remove codes for cleaning p->block_rules
    - DOC: Fix L4TOUT typo in documentation
    - DOC: set-log-level in Logging section preamble
    - BUG/MEDIUM: compat: fix segfault on FreeBSD
    - MEDIUM: check: include server address and port in the send-state header
    - MEDIUM: backend: Allow redispatch on retry intervals
    - MINOR: Add TLS ticket keys reference and use it in the listener struct
    - MEDIUM: Add support for updating TLS ticket keys via socket
    - DOC: Document new socket commands "show tls-keys" and "set ssl tls-key"
    - MINOR: Add sample fetch which identifies if the SSL session has been resumed
    - DOC: Update doc about weight, act and bck fields in the statistics
    - BUG/MEDIUM: ssl: fix tune.ssl.default-dh-param value being overwritten
    - MINOR: ssl: add a destructor to free allocated SSL ressources
    - MEDIUM: ssl: add the possibility to use a global DH parameters file
    - MEDIUM: ssl: replace standards DH groups with custom ones
    - MEDIUM: stats: Add enum srv_stats_state
    - MEDIUM: stats: Separate server state and colour in stats
    - MEDIUM: stats: Only report drain state in stats if server has SRV_ADMF_DRAIN set
    - MEDIUM: stats: Differentiate between DRAIN and DRAIN (agent)
    - MEDIUM: Lower priority of email alerts for log-health-checks messages
    - MEDIUM: Send email alerts when servers are marked as UP or enter the drain state
    - MEDIUM: Document when email-alerts are sent
    - BUG/MEDIUM: lua: bad argument number in analyser and in error message
    - MEDIUM: lua: automatically converts strings in proxy, tables, server and ip
    - BUG/MINOR: utf8: remove compilator warning
    - MEDIUM: map: uses HAProxy facilities to store default value
    - BUG/MINOR: lua: error in detection of mandatory arguments
    - BUG/MINOR: lua: set current proxy as default value if it is possible
    - BUG/MEDIUM: http: the action set-{method|path|query|uri} doesn't run.
    - BUG/MEDIUM: lua: undetected infinite loop
    - BUG/MAJOR: http: don't read past buffer's end in http_replace_value
    - BUG/MEDIUM: http: the function "(req|res)-replace-value" doesn't respect the HTTP syntax
    - MEDIUM/CLEANUP: http: rewrite and lighten http_transform_header() prototype
    - BUILD: lua: it miss the '-ldl' directive
    - MEDIUM: http: allows 'R' and 'S' in the protocol alphabet
    - MINOR: http: split the function http_action_set_req_line() in two parts
    - MINOR: http: split http_transform_header() function in two parts.
    - MINOR: http: export function inet_set_tos()
    - MINOR: lua: txn: add function set_(loglevel|tos|mark)
    - MINOR: lua: create and register HTTP class
    - DOC: lua: fix some typos
    - MINOR: lua: add log functions
    - BUG/MINOR: lua: Fix SSL initialisation
    - DOC: lua: some fixes
    - MINOR: lua: (req|res)_get_headers return more than one header value
    - MINOR: lua: map system integration in Lua
    - BUG/MEDIUM: http: functions set-{path,query,method,uri} breaks the HTTP parser
    - MINOR: sample: add url_dec converter
    - MEDIUM: sample: fill the struct sample with the session, proxy and stream pointers
    - MEDIUM: sample change the prototype of sample-fetches and converters functions
    - MINOR: sample: fill the struct sample with the options.
    - MEDIUM: sample: change the prototype of sample-fetches functions
    - MINOR: http: split the url_param in two parts
    - CLEANUP: http: bad indentation
    - MINOR: http: add body_param fetch
    - MEDIUM: http: url-encoded parsing function can run throught wrapped buffer
    - DOC: http: req.body_param documentation
    - MINOR: proxy: custom capture declaration
    - MINOR: capture: add two "capture" converters
    - MEDIUM: capture: Allow capture with slot identifier
    - MINOR: http: add array of generic pointers in http_res_rules
    - MEDIUM: capture: adds http-response capture
    - MINOR: common: escape CSV strings
    - MEDIUM: stats: escape some strings in the CSV dump
    - MINOR: tcp: add custom actions that can continue tcp-(request|response) processing
    - MINOR: lua: Lua tcp action are not final action
    - DOC: lua: schematics about lua socket organization
    - BUG/MINOR: debug: display (null) in place of "meth"
    - DOC: mention the "lua action" in documentation
    - MINOR: standard: add function that converts signed int to a string
    - BUG/MINOR: sample: wrong conversion of signed values
    - MEDIUM: sample: Add type any
    - MINOR: debug: add a special converter which display its input sample content.
    - MINOR: tcp: increase the opaque data array
    - MINOR: tcp/http/conf: extends the keyword registration options
    - MINOR: build: fix build dependency
    - MEDIUM: vars: adds support of variables
    - MINOR: vars: adds get and set functions
    - MINOR: lua: Variable access
    - MINOR: samples: add samples which returns constants
    - BUG/MINOR: vars/compil: fix some warnings
    - BUILD: add 51degrees options to makefile.
    - MINOR: global: add several 51Degrees members to global
    - MINOR: config: add 51Degrees config parsing.
    - MINOR: init: add 51Degrees initialisation code
    - MEDIUM: sample: add fiftyone_degrees converter.
    - MEDIUM: deinit: add cleanup for 51Degrees to deinit
    - MEDIUM: sample: add trie support to 51Degrees
    - DOC: add 51Degrees notes to configuration.txt.
    - DOC: add build indications for 51Degrees to README.
    - MEDIUM: cfgparse: introduce weak and strong quoting
    - BUG/MEDIUM: cfgparse: incorrect memmove in quotes management
    - MINOR: cfgparse: remove line size limitation
    - MEDIUM: cfgparse: expand environment variables
    - BUG/MINOR: cfgparse: fix typo in 'option httplog' error message
    - BUG/MEDIUM: cfgparse: segfault when userlist is misused
    - CLEANUP: cfgparse: remove reference to 'ruleset' section
    - MEDIUM: cfgparse: check section maximum number of arguments
    - MEDIUM: cfgparse: max arguments check in the global section
    - MEDIUM: cfgparse: check max arguments in the proxies sections
    - CLEANUP: stream-int: remove a redundant clearing of the linger_risk flag
    - MINOR: connection: make conn_sock_shutw() actually perform the shutdown() call
    - MINOR: stream-int: use conn_sock_shutw() to shutdown a connection
    - MINOR: connection: perform the call to xprt->shutw() in conn_data_shutw()
    - MEDIUM: stream-int: replace xprt->shutw calls with conn_data_shutw()
    - MINOR: checks: use conn_data_shutw_hard() instead of call via xprt
    - MINOR: connection: implement conn_sock_send()
    - MEDIUM: stream-int: make conn_si_send_proxy() use conn_sock_send()
    - MEDIUM: connection: make conn_drain() perform more controls
    - REORG: connection: move conn_drain() to connection.c and rename it
    - CLEANUP: stream-int: remove inclusion of fd.h that is not used anymore
    - MEDIUM: channel: don't always set CF_WAKE_WRITE on bi_put*
    - CLEANUP: lua: don't use si_ic/si_oc on known stream-ints
    - BUG/MEDIUM: peers: correctly configure the client timeout
    - MINOR: peers: centralize configuration of the peers frontend
    - MINOR: proxy: store the default target into the frontend's configuration
    - MEDIUM: stats: use frontend_accept() as the accept function
    - MEDIUM: peers: use frontend_accept() instead of peer_accept()
    - CLEANUP: listeners: remove unused timeout
    - MEDIUM: listener: store the default target per listener
    - BUILD: fix automatic inclusion of libdl.
    - MEDIUM: lua: implement a simple memory allocator
    - MEDIUM: compression: postpone buffer adjustments after compression
    - MEDIUM: compression: don't send leading zeroes with chunk size
    - BUG/MINOR: compression: consider the expansion factor in init
    - MINOR: http: check the algo name "identity" instead of the function pointer
    - CLEANUP: compression: statify all algo-specific functions
    - MEDIUM: compression: add a distinction between UA- and config- algorithms
    - MEDIUM: compression: add new "raw-deflate" compression algorithm
    - MEDIUM: compression: split deflate_flush() into flush and finish
    - CLEANUP: compression: remove unused reset functions
    - MAJOR: compression: integrate support for libslz
    - BUG/MEDIUM: http: hdr_cnt would not count any header when called without name
    - BUG/MAJOR: http: null-terminate the http actions keywords list
    - CLEANUP: lua: remove the unused hlua_sleep memory pool
    - BUG/MAJOR: lua: use correct object size when initializing a new converter
    - CLEANUP: lua: remove hard-coded sizeof() in object creations and mallocs
    - CLEANUP: lua: fix confusing local variable naming in hlua_txn_new()
    - CLEANUP: hlua: stop using variable name "s" alternately for hlua_txn and hlua_smp
    - CLEANUP: lua: get rid of the last "*ht" for struct hlua_txn.
    - CLEANUP: lua: rename last occurrences of "*s" to "*htxn" for hlua_txn
    - CLEANUP: lua: rename variable "sc" for struct hlua_smp
    - CLEANUP: lua: get rid of the last two "*hs" for hlua_smp
    - REORG/MAJOR: session: rename the "session" entity to "stream"
    - REORG/MEDIUM: stream: rename stream flags from SN_* to SF_*
    - MINOR: session: start to reintroduce struct session
    - MEDIUM: stream: allocate the session when a stream is created
    - MEDIUM: stream: move the listener's pointer to the session
    - MEDIUM: stream: move the frontend's pointer to the session
    - MINOR: session: add a pointer to the session's origin
    - MEDIUM: session: use the pointer to the origin instead of s->si[0].end
    - CLEANUP: sample: remove useless tests in fetch functions for l4 != NULL
    - MEDIUM: http: move header captures from http_txn to struct stream
    - MINOR: http: create a dedicated pool for http_txn
    - MAJOR: http: move http_txn out of struct stream
    - MAJOR: sample: don't pass l7 anymore to sample fetch functions
    - CLEANUP: lua: remove unused hlua_smp->l7 and hlua_txn->l7
    - MEDIUM: http: remove the now useless http_txn from {req/res} rules
    - CLEANUP: lua: don't pass http_txn anymore to hlua_request_act_wrapper()
    - MAJOR: sample: pass a pointer to the session to each sample fetch function
    - MINOR: stream: provide a few helpers to retrieve frontend, listener and origin
    - CLEANUP: stream: don't set ->target to the incoming connection anymore
    - MINOR: stream: move session initialization before the stream's
    - MINOR: session: store the session's accept date
    - MINOR: session: don't rely on s->logs.logwait in embryonic sessions
    - MINOR: session: implement session_free() and use it everywhere
    - MINOR: session: add stick counters to the struct session
    - REORG: stktable: move the stkctr_* functions from stream to sticktable
    - MEDIUM: streams: support looking up stkctr in the session
    - MEDIUM: session: update the session's stick counters upon session_free()
    - MEDIUM: proto_tcp: track the session's counters in the connection ruleset
    - MAJOR: tcp: make tcp_exec_req_rules() only rely on the session
    - MEDIUM: stream: don't call stream_store_counters() in kill_mini_session() nor session_accept()
    - MEDIUM: stream: move all the session-specific stuff of stream_accept() earlier
    - MAJOR: stream: don't initialize the stream anymore in stream_accept
    - MEDIUM: session: remove the task pointer from the session
    - REORG: session: move the session parts out of stream.c
    - MINOR: stream-int: make appctx_new() take the applet in argument
    - MEDIUM: peers: move the appctx initialization earlier
    - MINOR: session: introduce session_new()
    - MINOR: session: make use of session_new() when creating a new session
    - MINOR: peers: make use of session_new() when creating a new session
    - MEDIUM: peers: initialize the task before the stream
    - MINOR: session: set the CO_FL_CONNECTED flag on the connection once ready
    - CLEANUP: stream.c: do not re-attach the connection to the stream
    - MEDIUM: stream: isolate connection-specific initialization code
    - MEDIUM: stream: also accept appctx as origin in stream_accept_session()
    - MEDIUM: peers: make use of stream_accept_session()
    - MEDIUM: frontend: make ->accept only return +/-1
    - MEDIUM: stream: return the stream upon accept()
    - MEDIUM: frontend: move some stream initialisation to stream_new()
    - MEDIUM: frontend: move the fd-specific settings to session_accept_fd()
    - MEDIUM: frontend: don't restrict frontend_accept() to connections anymore
    - MEDIUM: frontend: move some remaining stream settings to stream_new()
    - CLEANUP: frontend: remove one useless local variable
    - MEDIUM: stream: don't rely on the session's listener anymore in stream_new()
    - MEDIUM: lua: make use of stream_new() to create an outgoing connection
    - MINOR: lua: minor cleanup in hlua_socket_new()
    - MINOR: lua: no need for setting timeouts / conn_retries in hlua_socket_new()
    - MINOR: peers: no need for setting timeouts / conn_retries in peer_session_create()
    - CLEANUP: stream-int: swap stream-int and appctx declarations
    - CLEANUP: namespaces: fix protection against multiple inclusions
    - MINOR: session: maintain the session count stats in the session, not the stream
    - MEDIUM: session: adjust the connection flags before stream_new()
    - MINOR: stream: pass the pointer to the origin explicitly to stream_new()
    - CLEANUP: poll: move the conditions for waiting out of the poll functions
    - BUG/MEDIUM: listener: don't report an error when resuming unbound listeners
    - BUG/MEDIUM: init: don't limit cpu-map to the first 32 processes only
    - BUG/MAJOR: tcp/http: fix current_rule assignment when restarting over a ruleset
    - BUG/MEDIUM: stream-int: always reset si->ops when si->end is nullified
    - DOC: update the entities diagrams
    - BUG/MEDIUM: http: properly retrieve the front connection
    - MINOR: applet: add a new "owner" pointer in the appctx
    - MEDIUM: applet: make the applet not depend on a stream interface anymore
    - REORG: applet: move the applet definitions out of stream_interface
    - CLEANUP: applet: rename struct si_applet to applet
    - REORG: stream-int: create si_applet_ops dedicated to applets
    - MEDIUM: applet: add basic support for an applet run queue
    - MEDIUM: applet: implement a run queue for active appctx
    - MEDIUM: stream-int: add a new function si_applet_done()
    - MAJOR: applet: now call si_applet_done() instead of si_update() in I/O handlers
    - MAJOR: stream: use a regular ->update for all stream interfaces
    - MEDIUM: dumpstats: don't unregister the applet anymore
    - MEDIUM: applet: centralize the call to si_applet_done() in the I/O handler
    - MAJOR: stream: do not allocate request buffers anymore when the left side is an applet
    - MINOR: stream-int: add two flags to indicate an applet's wishes regarding I/O
    - MEDIUM: applet: make the applets only use si_applet_{cant|want|stop}_{get|put}
    - MEDIUM: stream-int: pause the appctx if the task is woken up
    - BUG/MAJOR: tcp: only call registered actions when they're registered
    - BUG/MEDIUM: peers: fix applet scheduling
    - BUG/MEDIUM: peers: recent applet changes broke peers updates scheduling
    - MINOR: tools: provide an rdtsc() function for time comparisons
    - IMPORT: lru: import simple ebtree-based LRU functions
    - IMPORT: hash: import xxhash-r39
    - MEDIUM: pattern: add a revision to all pattern expressions
    - MAJOR: pattern: add LRU-based cache on pattern matching
    - BUG/MEDIUM: http: remove content-length from chunked messages
    - DOC: http: update the comments about the rules for determining transfer-length
    - BUG/MEDIUM: http: do not restrict parsing of transfer-encoding to HTTP/1.1
    - BUG/MEDIUM: http: incorrect transfer-coding in the request is a bad request
    - BUG/MEDIUM: http: remove content-length form responses with bad transfer-encoding
    - MEDIUM: http: restrict the HTTP version token to 1 digit as per RFC7230
    - MEDIUM: http: disable support for HTTP/0.9 by default
    - MEDIUM: http: add option-ignore-probes to get rid of the floods of 408
    - BUG/MINOR: config: clear proxy->table.peers.p for disabled proxies
    - MEDIUM: init: don't stop proxies in parent process when exiting
    - MINOR: stick-table: don't attach to peers in stopped state
    - MEDIUM: config: initialize stick-tables after peers, not before
    - MEDIUM: peers: add the ability to disable a peers section
    - MINOR: peers: store the pointer to the signal handler
    - MEDIUM: peers: unregister peers that were never started
    - MEDIUM: config: propagate the table's process list to the peers sections
    - MEDIUM: init: stop any peers section not bound to the correct process
    - MEDIUM: config: validate that peers sections are bound to exactly one process
    - MAJOR: peers: allow peers section to be used with nbproc > 1
    - DOC: relax the peers restriction to single-process
    - DOC: document option http-ignore-probes
    - DOC: fix the comments about the meaning of msg->sol in HTTP
    - BUG/MEDIUM: http: wait for the exact amount of body bytes in wait_for_request_body
    - BUG/MAJOR: http: prevent risk of reading past end with balance url_param
    - MEDIUM: stream: move HTTP request body analyser before process_common
    - MEDIUM: http: add a new option http-buffer-request
    - MEDIUM: http: provide 3 fetches for the body
    - DOC: update the doc on the proxy protocol
    - BUILD: pattern: fix build warnings introduced in the LRU cache
    - BUG/MEDIUM: stats: properly initialize the scope before dumping stats
    - CLEANUP: config: fix misleading information in error message.
    - MINOR: config: report the number of processes using a peers section in the error case
    - BUG/MEDIUM: config: properly compute the default number of processes for a proxy
    - MEDIUM: http: add new "capture" action for http-request
    - BUG/MEDIUM: http: fix the http-request capture parser
    - BUG/MEDIUM: http: don't forward client shutdown without NOLINGER except for tunnels
    - BUILD/MINOR: ssl: fix build failure introduced by recent patch
    - BUG/MAJOR: check: fix breakage of inverted tcp-check rules
    - CLEANUP: checks: fix double usage of cur / current_step in tcp-checks
    - BUG/MEDIUM: checks: do not dereference head of a tcp-check at the end
    - CLEANUP: checks: simplify the loop processing of tcp-checks
    - BUG/MAJOR: checks: always check for end of list before proceeding
    - BUG/MEDIUM: checks: do not dereference a list as a tcpcheck struct
    - BUG/MAJOR: checks: break infinite loops when tcp-checks starts with comment
    - MEDIUM: http: make url_param iterate over multiple occurrences
    - BUG/MEDIUM: peers: apply a random reconnection timeout
    - MEDIUM: config: reject invalid config with name duplicates
    - MEDIUM: config: reject conflicts in table names
    - CLEANUP: proxy: make the proxy lookup functions more user-friendly
    - MINOR: proxy: simply ignore duplicates in proxy name lookups
    - MINOR: config: don't open-code proxy name lookups
    - MEDIUM: config: clarify the conflicting modes detection for backend rules
    - CLEANUP: proxy: remove now unused function findproxy_mode()
    - MEDIUM: stick-table: remove the now duplicate find_stktable() function
    - MAJOR: config: remove the deprecated reqsetbe / reqisetbe actions
    - MINOR: proxy: add a new function proxy_find_by_id()
    - MINOR: proxy: add a flag to memorize that the proxy's ID was forced
    - MEDIUM: proxy: add a new proxy_find_best_match() function
    - CLEANUP: http: explicitly reference request in http_apply_redirect_rules()
    - MINOR: http: prepare support for parsing redirect actions on responses
    - MEDIUM: http: implement http-response redirect rules
    - MEDIUM: http: no need to close the request on redirect if data was parsed
    - BUG/MEDIUM: http: fix body processing for the stats applet
    - BUG/MINOR: da: fix log-level comparison to emove annoying warning
    - CLEANUP: global: remove one ifdef USE_DEVICEATLAS
    - CLEANUP: da: move the converter registration to da.c
    - CLEANUP: da: register the config keywords in da.c
    - CLEANUP: adjust the envelope name in da.h to reflect the file name
    - CLEANUP: da: remove ifdef USE_DEVICEATLAS from da.c
    - BUILD: make 51D easier to build by defaulting to 51DEGREES_SRC
    - BUILD: fix build warning when not using 51degrees
    - BUILD: make DeviceAtlas easier to build by defaulting to DEVICEATLAS_SRC
    - BUILD: ssl: fix recent build breakage on older SSL libs
2015-06-17 15:53:25 +02:00
Willy Tarreau
c8ad3beded BUILD: ssl: fix recent build breakage on older SSL libs
Commit 31af49d ("MEDIUM: ssl: Add options to forge SSL certificates")
introduced some dependencies on SSL_CTRL_SET_TLSEXT_HOSTNAME for which
a few checks were missing, breaking the build on openssl 0.9.8.
2015-06-17 15:50:40 +02:00
Thierry FOURNIER
0b243fd63b BUG/MINOR: vars/compil: fix some warnings
A switch case doesn't have default entry, and the compilator sends
a warning about uninitilized var.

   warning: 'vars' may be used uninitialized in this function [-Wmaybe-uninitialized]
2015-06-17 10:42:45 +02:00
Emeric Brun
31c56530b8 BUG/MAJOR: sample: regression on sample cast to stick table types.
This regression was introduce by commit
9c627e84b2 (MEDIUM: sample: Add type any)

New sample type 'any' was not handled in the matrix used to cast
to stick-tables types.
2015-06-16 18:34:50 +02:00
Emeric Brun
9490095abb MEDIUM: peers: support of any stick-table data-types for sync
It is possible to propagate entries of any data-types in stick-tables between
several haproxy instances over TCP connections in a multi-master fashion. Each
instance pushes its local updates and insertions to remote peers. The pushed
values overwrite remote ones without aggregation. Interrupted exchanges are
automatically detected and recovered from the last known point.
2015-06-16 16:11:59 +02:00
Emeric Brun
57056f0347 MEDIUM: peers: re-schedule stick-table's entry for sync when data is modified.
This was correctly done for data of type 'serverid' but it is now
necessary for all stick-tables data-types if we want to perform their
sync.
2015-06-16 16:11:36 +02:00
Emeric Brun
aaf5860fd6 MINOR: peers: avoid re-scheduling of pending stick-table's updates still not pushed. 2015-06-16 16:11:12 +02:00
Thierry FOURNIER
cc103299c7 MINOR: samples: add samples which returns constants
This patch adds sample which returns constants values. This is useful
for intialising variables.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
053ba8adfd MINOR: lua: Variable access
This patch adds two Lua function for using HAPRoxy's
vraibles. The function are stored in the TXN class,
and her name is "set_var" and "get_var".
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
c365d99a07 MINOR: vars: adds get and set functions
This patch adds two functions used for variable acces using the
variable full name. If the variable doesn't exists in the variable
pool name, it is created.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
4834bc773c MEDIUM: vars: adds support of variables
This patch adds support of variables during the processing of each stream. The
variables scope can be set as 'session', 'transaction', 'request' or 'response'.
The variable type is the type returned by the assignment expression. The type
can change while the processing.

The allocated memory can be controlled for each scope and each request, and for
the global process.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
0e11863a6f MINOR: tcp/http/conf: extends the keyword registration options
This patch permits to register a new keyword with the keyword "tcp-request content"
'tcp-request connection", tcp-response content", http-request" and "http-response"
which is identified only by matching the start of the keyword.

for example, we register the keyword "set-var" with the option "match_pfx"
and the configuration keyword "set-var(var_name)" matchs this entry.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
fbdb77582d MINOR: tcp: increase the opaque data array
This patch increase the opaque data array for the tcp_rules.
It is used by the "store" action (next commited) which deal
with variables.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
9687c77c91 MINOR: debug: add a special converter which display its input sample content.
This converter displays its input sample type and content. It is useful
for debugging some complex configurations.
2015-06-13 23:01:36 +02:00
Thierry FOURNIER
9c627e84b2 MEDIUM: sample: Add type any
This type is used to accept any type of sample as input, and prevent
any automatic "cast". It runs like the type "ADDR" which accept the
type "IPV4" and "IPV6".
2015-06-13 22:59:14 +02:00
Thierry FOURNIER
0f811440d5 BUG/MINOR: sample: wrong conversion of signed values
The signed values are casted as unsigned before conversion. This patch
use the good converters according with the sample type.

Note: it depends on previous patch to parse signed ints.
2015-06-13 22:59:14 +02:00
Thierry FOURNIER
1480bd8dd2 MINOR: standard: add function that converts signed int to a string
This function is the same as "ultoa_r", but it takes a signed value
as input.
2015-06-13 22:59:14 +02:00
Baptiste Assmann
3863f97349 MINOR: dns: add DNS statistics
add a new command on the stats socket to print a DNS resolvers section
(including per server) statistics: "show stats resolvers <id>"
2015-06-13 22:07:35 +02:00
Baptiste Assmann
a68ca96375 MAJOR: server: add DNS-based server name resolution
Relies on the DNS protocol freshly implemented in HAProxy.
It performs a server IP addr resolution based on a server hostname.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
325137d603 MEDIUM: dns: implement a DNS resolver
Implementation of a DNS client in HAProxy to perform name resolution to
IP addresses.

It relies on the freshly created UDP client to perform the DNS
resolution. For now, all UDP socket calls are performed in the
DNS layer, but this might change later when the protocols are
extended to be more suited to datagram mode.

A new section called 'resolvers' is introduced thanks to this patch. It
is used to describe DNS servers IP address and also many parameters.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
5d4e4f7a57 MEDIUM: protocol: add minimalist UDP protocol client
Basic introduction of a UDP layer in HAProxy. It can be used as a
client only and manages UDP exchanges with servers.

It can't be used to load-balance UDP protocols, but only used by
internal features such as DNS resolution.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
3d8f831f13 MEDIUM: server: change server ip address from stats socket
New command available on the stats socket to change a server addr using
the command "set server <backend>/<server> addr <ip4|ip6>"
2015-06-13 22:07:35 +02:00
Baptiste Assmann
14e4014a48 MEDIUM: server: add support for changing a server's address
Ability to change a server IP address during HAProxy run time.
For now this is provided via function update_server_addr() which
currently is not called.

A log is emitted on each change. For now we do it inconditionally,
but later we'll want to do it only on certain circumstances, which
explains why the logging block is enclosed in if(1).
2015-06-13 22:07:35 +02:00
Christopher Faulet
3054880e09 MINOR: ssl: Export functions to manipulate generated certificates
Following functions are now available in the SSL public API:

  * ssl_sock_create_cert
  * ssl_sock_get_generated_cert
  * ssl_sock_set_generated_cert
  * ssl_sock_generated_cert_serial

These functions could be used to create a certificate by hand, set it in the
cache used to store generated certificates and retrieve it. Here is an example
(pseudo code):

  X509        *cacert     = ...;
  EVP_PKEY    *capkey     = ...;
  char        *servername = ...;
  unsigned int serial;

  serial = ssl_sock_generated_cert_serial(servername, strlen(servername));
  if (!ssl_sock_get_generated_cert(serial, cacert)) {
      SSL_CTX *ctx = ssl_sock_create_cert(servername, serial, cacert, capkey);
      ssl_sock_set_generated_cert(ctx, serial, cacert);
  }
2015-06-12 18:06:59 +02:00
Christopher Faulet
31af49d62b MEDIUM: ssl: Add options to forge SSL certificates
With this patch, it is possible to configure HAProxy to forge the SSL
certificate sent to a client using the SNI servername. We do it in the SNI
callback.

To enable this feature, you must pass following BIND options:

 * ca-sign-file <FILE> : This is the PEM file containing the CA certitifacte and
   the CA private key to create and sign server's certificates.

 * (optionally) ca-sign-pass <PASS>: This is the CA private key passphrase, if
   any.

 * generate-certificates: Enable the dynamic generation of certificates for a
   listener.

Because generating certificates is expensive, there is a LRU cache to store
them. Its size can be customized by setting the global parameter
'tune.ssl.ssl-ctx-cache-size'.
2015-06-12 18:06:59 +02:00
Christopher Faulet
92939d20fa MINOR: lru: Add lru64_lookup function
It lookup a key in a LRU cache for use with specified domain and revision. It
differs from lru64_get as it does not create missing keys. The function returns
NULL if an error or a cache miss occurs.
2015-06-12 18:06:59 +02:00
Christopher Faulet
f90ac55d9e MINOR: lru: Add the possibility to free data when an item is removed
Now, When a item is committed in an LRU tree, you can define a function to free
data owned by this item. This function will be called when the item is removed
from the LRU tree or when the tree is destroyed..
2015-06-12 18:06:59 +02:00
Dragan Dosen
acd6b1f767 BUG/MEDIUM: init: segfault if global._51d_property_names is not initialized
When using the "51d" converter without specifying the list of 51Degrees
properties to detect (see parameter "51degrees-property-name-list"), the
"global._51d_property_names" could be left uninitialized which will lead to
segfault during init.
2015-06-11 21:47:03 +02:00
Godbach
28b48ccbc8 CLEANUP: deinit: remove codes for cleaning p->block_rules
Since all rules listed in p->block_rules have been moved to the beginning of
the http-request rules in check_config_validity(), there is no need to clean
p->block_rules in deinit().

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2015-06-09 15:35:29 +02:00
Thierry FOURNIER
4c2479e1c4 BUG/MINOR: debug: display (null) in place of "meth"
The array which contains names of types, miss the METH entry.

[wt: should be backported to 1.5 as well]
2015-06-09 10:58:14 +02:00
Willy Tarreau
a9f4702390 BUILD: fix build warning when not using 51degrees
An ifdef was missing to avoid declaring these variables :

src/haproxy.c: In function 'deinit':
src/haproxy.c:1253:47: warning: unused variable '_51d_prop_nameb' [-Wunused-variable]
src/haproxy.c:1253:30: warning: unused variable '_51d_prop_name' [-Wunused-variable]
2015-06-02 19:30:59 +02:00
Thomas Holmes
4d441a759c MEDIUM: sample: add trie support to 51Degrees
Trie or pattern algorithm is used depending on what 51Degrees source
files are provided to MAKE.
2015-06-02 19:30:53 +02:00
Thomas Holmes
5ed33eed53 MEDIUM: deinit: add cleanup for 51Degrees to deinit 2015-06-02 14:00:25 +02:00
Thomas Holmes
951d44d24d MEDIUM: sample: add fiftyone_degrees converter.
It takes up to 5 string arguments that are to be 51Degrees property names.
It will then create a chunk with values detected based on the request header
supplied (this should be the User-Agent).
2015-06-02 14:00:25 +02:00
Thomas Holmes
6abded4f8d MINOR: init: add 51Degrees initialisation code
This creates a dataset using the file given in global._51d_data_file_path.
2015-06-02 14:00:25 +02:00
Thomas Holmes
0809f26869 MINOR: config: add 51Degrees config parsing. 2015-06-02 14:00:25 +02:00
Willy Tarreau
25eda1e182 CLEANUP: da: remove ifdef USE_DEVICEATLAS from da.c
This file is only built when the macro is defined, there's no point
having it ifdefed here.
2015-06-02 13:42:12 +02:00
Willy Tarreau
0d74f77ee7 CLEANUP: da: register the config keywords in da.c
No need to export the registration to haproxy.c, it can be done
locally.
2015-06-02 13:42:12 +02:00
Willy Tarreau
f63386ad27 CLEANUP: da: move the converter registration to da.c
There's no reason to put it into sample.c, it's better to register it
locally in da.c, it removes a number of ifdefs and exports.
2015-06-02 13:42:12 +02:00
Willy Tarreau
6bd42e7f41 BUG/MINOR: da: fix log-level comparison to emove annoying warning
When haproxy is run on the foreground with DeviceAtlas enabled, one
line of warning is seen for every test because the comparison is always
true even when loglevel is zero :

willy@wtap:haproxy$ ./haproxy  -db -f test-da.cfg
[WARNING] 151/150831 (25506) : deviceatlas : final memory image 7148029 bytes.
Deviceatlas module loaded.
[WARNING] 151/150832 (25506) : deviceatlas : .
[WARNING] 151/150833 (25506) : deviceatlas : .
[WARNING] 151/150833 (25506) : deviceatlas : .
^C

Don't emit a warning when loglevel is null.
2015-06-02 13:42:12 +02:00
David Carlier
4a02641a86 MEDIUM: init: DeviceAtlas initialization
This diff initialises few DeviceAtlas struct fields member with
their inherent default values.
Furthermore, the specific DeviceAtlas configuration keywords are
registered and the module is initialised and all necessary
resources are freed during the deinit phase.
2015-06-02 13:24:50 +02:00
David Carlier
4542b10ae1 MEDIUM: sample: add the da-csv converter
This diff declares the deviceatlas module and can accept up to 5
property names for the API lookup.

[wt: this should probably be moved to its own file using the keyword
      registration mechanism]
2015-06-02 13:24:50 +02:00
David Carlier
8167f30661 MEDIUM: config: add DeviceAtlas global keywords
This diff is for the DeviceAtlas convertor.

This patch adds the following converters :
  deviceatlas-json-file
  deviceatlas-log-level
  deviceatlas-property-separator

First, the configuration keywords handling (only the log
level configuration part does not end the haproxy process
if it is wrongly set, it fallbacks to the default level).
Furthermore, init, deinit phases and the API lookup phase,
the da_haproxy function which is fed by the input provided
and set all necessary properties chosen via the configuration
to the output, separated by the separator.
2015-06-02 13:24:44 +02:00
Remi Gacogne
d3a341a96f MEDIUM: ssl: replace standards DH groups with custom ones
It is likely that powerful adversaries have been pre-computing the
standardized DH groups, because being widely used have made them
valuable targets. While users are advised to generate their own
DH parameters, replace the ones we ship by values been randomly
generated for this product only.

[wt: replaced dh1024_p, dh2048_p, and dh4096_p with locally-generated
 ones as recommended by Rmi]
2015-05-31 22:41:28 +02:00
Remi Gacogne
47783ef05b MEDIUM: ssl: add the possibility to use a global DH parameters file
This patch adds the ssl-dh-param-file global setting. It sets the
default DH parameters that will be used during the SSL/TLS handshake when
ephemeral Diffie-Hellman (DHE) key exchange is used, for all "bind" lines
which do not explicitely define theirs.
2015-05-31 22:02:00 +02:00
Thierry FOURNIER
79318d79ba MINOR: lua: Lua tcp action are not final action
Actually, the registered lua actions with "tcp-request lua" and
"tcp-response lua" are final actions. This patch change the action
type type and permit to continue the evaluation of tcp-* processing
after the evaluation of the lua actions.
2015-05-29 17:49:50 +02:00
Thierry FOURNIER
561a0f989d MINOR: tcp: add custom actions that can continue tcp-(request|response) processing
Actually, the tcp-request and tcp-response custom ation are always final
actions. This patch create a new type of action that can permit to
continue the evaluation of tcp-request and tcp-response processing.
2015-05-29 17:49:48 +02:00
Emeric Brun
b3971ab062 MAJOR: peers: peers protocol version 2.0
This patch does'nt add any new feature: the functional behavior
is the same than version 1.0.

Technical differences:

In this version all updates on different stick tables are
multiplexed on the same tcp session. There is only one established
tcp session per peer whereas in first version there was one established
tcp session per peer and per stick table.

Messages format was reviewed to be more evolutive and to support
further types of data exchange such as SSL sessions or other sticktable's
data types (currently only the sticktable's server id is supported).
2015-05-29 15:50:33 +02:00
Willy Tarreau
b8cdf52da0 BUG/MEDIUM: http: fix body processing for the stats applet
Commit 9fbe18e ("MEDIUM: http: add a new option http-buffer-request")
introduced a regression due to a misplaced check causing the admin
mode of the HTTP stats not to work anymore.

This patch tried to ensure that when we need a request body for the
stats applet, and we have already waited for this body, we don't wait
for it again, but the condition was applied too early causing a
disabling of the entire processing the body, and based on the wrong
HTTP state (MSG_BODY) resulting in the test never matching.

Thanks to Chad Lavoie for reporting the problem.

This bug is 1.6-only, no backport is needed.
2015-05-29 01:12:38 +02:00
William Lallemand
df1425ad45 MEDIUM: cfgparse: check max arguments in the proxies sections
Add checks on the maximum number of arguments in proxies sections.
2015-05-28 18:43:03 +02:00
William Lallemand
1a748aed07 MEDIUM: cfgparse: max arguments check in the global section
Most of the keywords in the global section does not check the maximum
number of arguments. This leds sometines to unused and wrong arguments
in the configuration file. This patch add a maximum argument test in
many keywords of this section.
2015-05-28 18:43:03 +02:00
William Lallemand
6e62fb6405 MEDIUM: cfgparse: check section maximum number of arguments
This patch checks the number of arguments of the keywords:
'global', 'defaults', 'listen', 'backend', 'frontend', 'peers' and
'userlist'

The 'global' section does not take any arguments.

Proxy sections does not support bind address as argument anymore.  Those
sections supports only an <id> argument.

The 'defaults' section didn't had any check on its arguments. It takes
an optional <name> argument.

'peers' section takes a <peersect> argument.

'userlist' section takes a <listname> argument.
2015-05-28 18:43:03 +02:00
William Lallemand
5109719e77 CLEANUP: cfgparse: remove reference to 'ruleset' section
The 'ruleset' section was never implemented. This patch remove
references and tests about this keyword.
2015-05-28 18:43:03 +02:00
William Lallemand
4ac9f54612 BUG/MEDIUM: cfgparse: segfault when userlist is misused
If the 'userlist' keyword parsing returns an error and no userlist were
previously created. The parsing of 'user' and 'group' leads to NULL
derefence.

The userlist pointer is now tested to prevent this issue.
2015-05-28 18:43:03 +02:00
William Lallemand
77063bc0c6 BUG/MINOR: cfgparse: fix typo in 'option httplog' error message
The error message was displaying the wrong argument when 'option
httplog' took a wrong argument.
2015-05-28 18:43:03 +02:00
Remi Gacogne
d3a23c3eb8 MINOR: ssl: add a destructor to free allocated SSL ressources
Using valgrind or another memory leak tracking tool is easier
when the memory internally allocated by OpenSSL is cleanly released
at shutdown.
2015-05-28 18:03:29 +02:00
Remi Gacogne
4f902b8832 BUG/MEDIUM: ssl: fix tune.ssl.default-dh-param value being overwritten
Herv Commowick reported that the logic used to avoid complaining about
ssl-default-dh-param not being set when static DH params are present
in the certificate file was clearly wrong when more than one sni_ctx
is used.
This patch stores whether static DH params are being used for each
SSL_CTX individually, and does not overwrite the value of
tune.ssl.default-dh-param.
2015-05-28 18:03:27 +02:00
Thierry FOURNIER
be1ccaea92 MEDIUM: stats: escape some strings in the CSV dump
Some strings which must be dumped in the CSV output can contain one of
the following chars : <,>, <">, or CR/LF. This patch escapes these
strings if the case is encountered.
2015-05-28 17:49:50 +02:00
Thierry FOURNIER
ddea626de4 MINOR: common: escape CSV strings
This function checks a string for using it in a CSV output format. If
the string contains one of the following four char <">, <,>, CR or LF,
the string is encapsulated between <"> and the <"> are escaped by a <"">
sequence.

The rounding by <"> is optionnal. It can be canceled, forced or the
function choose automatically the right way.
2015-05-28 17:47:19 +02:00
Willy Tarreau
2de8a50918 MEDIUM: http: no need to close the request on redirect if data was parsed
There are two reasons for not keeping the client connection alive upon a
redirect :
  - save the client from uploading all data
  - avoid keeping a connection alive if the redirect goes to another domain

The first case should consider an exception when all the data from the
client have been read already. This specifically happens on response
redirects after a POST to a server. This is an easy situation to detect.

It could later be improved to cover the cases where option
http-buffer-request is used.
2015-05-28 17:45:43 +02:00
Willy Tarreau
51d861a44f MEDIUM: http: implement http-response redirect rules
Sometimes it's problematic not to have "http-response redirect" rules,
for example to perform a browser-based redirect based on certain server
conditions (eg: match of a header).

This patch adds "http-response redirect location <fmt>" which gives
enough flexibility for most imaginable operations. The connection to
the server is closed when this is performed so that we don't risk to
forward any pending data from the server.

Any pending response data are trimmed so that we don't risk to
forward anything pending to the client. It's harmless to also do that
for requests so we don't need to consider the direction.
2015-05-28 17:45:43 +02:00
Willy Tarreau
be4653b6d4 MINOR: http: prepare support for parsing redirect actions on responses
In order to support http-response redirect, the parsing needs to be
adapted a little bit to only support the "location" type, and to
adjust the log-format parser so that it knows the direction of the
sample fetch calls.
2015-05-28 17:43:11 +02:00
Willy Tarreau
b329a312e3 CLEANUP: http: explicitly reference request in http_apply_redirect_rules()
This function was made to perform a redirect on requests only, it was
using a message or txn->req in an inconsistent way and did not consider
the possibility that it could be used for the other direction. Let's
clean it up to have both a request and a response messages.
2015-05-28 17:42:16 +02:00
Thierry FOURNIER
e80fadaaca MEDIUM: capture: adds http-response capture
This patch adds a http response capture keyword with the same behavior
as the previous patch called "MEDIUM: capture: Allow capture with slot
identifier".
2015-05-28 13:51:00 +02:00
Thierry FOURNIER
82bf70dff4 MEDIUM: capture: Allow capture with slot identifier
This patch modifies the current http-request capture function
and adds a new keyword "id" that permits to identify a capture slot.
If the identified doesn't exists, the action fails silently.

Note that this patch removs an unused list initilisation, which seems
to be inherited from a copy/paste. It's harmless and does not need to
be backported.

   LIST_INIT((struct list *)&rule->arg.act.p[0]);
2015-05-28 13:50:29 +02:00
Thierry FOURNIER
35ab27561e MINOR: capture: add two "capture" converters
This patch adds "capture-req" and "capture-res". These two converters
capture their entry in the allocated slot given in argument and pass
the input on the output.
2015-05-28 13:50:29 +02:00
Thierry FOURNIER
a0a1b75560 MINOR: proxy: custom capture declaration
This patch adds a new keyword called "declare". This keyword
allow to declare some capture slots in requests and response.
It is useful for sharing capture between frontend and backends.
2015-05-28 13:50:28 +02:00
Willy Tarreau
eb3e34854d MEDIUM: proxy: add a new proxy_find_best_match() function
This function tries to spot a proxy by its name, ID and type, and
in case some elements don't match, it tries to determine which ones
could be ignored and reports which ones were ignored so that the
caller can decide whether or not it wants to pick this proxy. This
will be used for maintaining the status across reloads where the
config might have changed a bit.
2015-05-27 16:51:28 +02:00
Willy Tarreau
0d1fdf7df5 MINOR: proxy: add a flag to memorize that the proxy's ID was forced
This will be used to know if proxy's ID should be considered when names
mismatch upon check status reload.
2015-05-27 16:51:28 +02:00
Willy Tarreau
3c56a7d94f MINOR: proxy: add a new function proxy_find_by_id()
It does the same as the other one except that it only focuses on the
numeric ID and the capabilities. It's used by proxy_find_by_name()
for numeric names.
2015-05-27 16:49:44 +02:00
Willy Tarreau
98d0485a90 MAJOR: config: remove the deprecated reqsetbe / reqisetbe actions
These ones were already obsoleted in 1.4, marked for removal in 1.5,
and not documented anymore. They used to emit warnings, and do still
require quite some code to stay in place. Let's remove them now.
2015-05-26 12:18:29 +02:00
Willy Tarreau
e2dc1fa8ca MEDIUM: stick-table: remove the now duplicate find_stktable() function
Since proxy_tbl_by_name() already does the same job, let's not keep
duplicate functions and use this one only.
2015-05-26 12:08:07 +02:00
Willy Tarreau
63d38fda4a CLEANUP: proxy: remove now unused function findproxy_mode()
It's not used anymore.
2015-05-26 12:05:53 +02:00
Willy Tarreau
afb3992d35 MEDIUM: config: clarify the conflicting modes detection for backend rules
We don't use findproxy_mode() anymore so we can check the conflicting
modes and report the anomalies accordingly with line numbers and more
explicit details.
2015-05-26 12:04:09 +02:00
Willy Tarreau
8f50b68879 MINOR: config: don't open-code proxy name lookups
We can now safely use the standard functions to detect proxy name
duplicates.
2015-05-26 11:45:02 +02:00
Willy Tarreau
c739aa85e8 MINOR: proxy: simply ignore duplicates in proxy name lookups
Now that we can't have duplicate proxies with similar capabilities, we
can remove some painful check. The first one is the check that made the
lookup function return NULL when a duplicate is found, as it prevented
it from being used in the config parser to detect duplicates.
2015-05-26 11:35:41 +02:00
Willy Tarreau
9e0bb1013e CLEANUP: proxy: make the proxy lookup functions more user-friendly
First, findproxy() was renamed proxy_find_by_name() so that its explicit
that a name is required for the lookup. Second, we give this function
the ability to search for tables if needed. Third we now provide inline
wrappers to pass the appropriate PR_CAP_* flags and to explicitly look
up a frontend, backend or table.
2015-05-26 11:24:42 +02:00
Willy Tarreau
e45288c0ca MEDIUM: config: reject conflicts in table names
A nasty situation happens when two tables have the same name. Since it
is possible to declare a table in a frontend and another one in a backend,
this situation may happen and result in a random behaviour each time a
table is designated in a "stick" or "track" rule. Let's make sure this
is properly detected and stopped. Such a config will now report :

[ALERT] 145/104933 (31571) : parsing [prx.cfg:36] : stick-table name 't' conflicts with table declared in frontend 't' at prx.cfg:30.
[ALERT] 145/104933 (31571) : Error(s) found in configuration file : prx.cfg
[ALERT] 145/104933 (31571) : Fatal errors found in configuration.
2015-05-26 10:49:46 +02:00
Willy Tarreau
911fa2eb8e MEDIUM: config: reject invalid config with name duplicates
Since 1.4 we used to emit a warning when two frontends or two backends
had the same name. In 1.5 we added the same warning for two peers sections.
In 1.6 we added the same warning for two mailers sections. It's about time
to reject such invalid configurations, the impact they have on the code
complexity is huge and it is becoming a real obstacle to some improvements
such as restoring servers check status across reloads.

Now these errors are reported as fatal errors and will need to be fixed.
Anyway, till now there was no guarantee that what was written was working
as expected since the behaviour is not defined (eg: use_backend with a
name used by two backends leads to undefined behaviour).

Example of output :

[ALERT] 145/104759 (31564) : Parsing [prx.cfg:12]: mailers section 'm' has the same name as another mailers section declared at prx.cfg:10.
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:16]: peers section 'p' has the same name as another peers section declared at prx.cfg:14.
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:21]: frontend 'f' has the same name as another frontend declared at prx.cfg:18.
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:27]: backend 'b' has the same name as another backend declared at prx.cfg:24.
[ALERT] 145/104759 (31564) : Error(s) found in configuration file : prx.cfg
[ALERT] 145/104759 (31564) : Fatal errors found in configuration.
2015-05-26 10:48:17 +02:00
Dragan Dosen
26f77e534c BUG/MEDIUM: http: fix the url_param fetch
The "name" and "name_len" arguments in function "smp_fetch_url_param"
could be left uninitialized for subsequent calls.

[wt: no backport needed, this is an 1.6 regression introduced by
 commit 4fdc74c ("MINOR: http: split the url_param in two parts") ]
2015-05-25 19:01:39 +02:00
Joseph Lynch
726ab7145c MEDIUM: backend: Allow redispatch on retry intervals
For backend load balancing it sometimes makes sense to redispatch rather
than retrying against the same server. For example, when machines or routers
fail you may not want to waste time retrying against a dead server and
would instead prefer to immediately redispatch against other servers.

This patch allows backend sections to specify that they want to
redispatch on a particular interval. If the interval N is positive the
redispatch occurs on every Nth retry, and if the interval N is negative then
the redispatch occurs on the Nth retry prior to the last retry (-1 is the
default and maintains backwards compatibility). In low latency environments
tuning this setting can save a few hundred milliseconds when backends fail.
2015-05-22 07:07:40 +02:00
Thierry FOURNIER
8be451c52a MEDIUM: http: url-encoded parsing function can run throught wrapped buffer
The functions smp_fetch_param(), find_next_url_param() and
find_url_param_pos() can look for argument in 2 chunks and not only
one.
2015-05-20 16:05:38 +02:00
Thierry FOURNIER
e28c49975a MINOR: http: add body_param fetch
This fetch returns one body param or the list of each body param.
This first version runs only with one chunk.
2015-05-20 15:56:23 +02:00
Thierry FOURNIER
0948d41a12 CLEANUP: http: bad indentation
Some function argument uses space in place of tabulation
for the indentation.
2015-05-20 15:56:23 +02:00
Thierry FOURNIER
4fdc74c22c MINOR: http: split the url_param in two parts
This patch is the part of the body_param fetch. The goal is to have
generic url-encoded parser which can used for parsing the query string
and the body.
2015-05-20 15:56:23 +02:00
Willy Tarreau
b4e34da692 BUG/MEDIUM: peers: apply a random reconnection timeout
Commit 9ff95bb ("BUG/MEDIUM: peers: correctly configure the client timeout")
uncovered an old bug in the peers : upon disconnect, we reconnect immediately.
This sometimes results in both ends to do the same thing in parallel causing
a loop of connect/accept/close/close that can last several seconds. The risk
of occurrence of the trouble increases with latency, and is emphasized by the
fact that idle connections are now frequently recycled (after 5s of idle).

In order to avoid this we must apply a random delay before reconnecting.
Fortunately the mechanism already supports a reconnect delay, so here we
compute the random timeout when killing a session. The delay is 50ms plus
a random between 0 and 2 seconds. Ideally an exponential back-off would
be preferred but it's preferable to keep the fix simple.

This bug was reported by Marco Corte.

This fix must be backported to 1.5 since the fix above was backported into
1.5.12.
2015-05-20 10:49:07 +02:00
Willy Tarreau
1ede1daab6 MEDIUM: http: make url_param iterate over multiple occurrences
There are some situations hwere it's desirable to scan multiple occurrences
of a same parameter name in the query string. This change ensures this can
work, even with an empty name which will then iterate over all parameters.
2015-05-19 13:16:07 +02:00
Nenad Merdanovic
26ea822190 MINOR: Add sample fetch which identifies if the SSL session has been resumed
Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-05-18 07:07:53 +02:00
Nenad Merdanovic
200b0facde MEDIUM: Add support for updating TLS ticket keys via socket
Until now, HAproxy needed to be restarted to change the TLS ticket
keys. With this patch, the TLS keys can be updated on a per-file
basis using the admin socket. Two new socket commands have been
introduced: "show tls-keys" and "set ssl tls-keys".

Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-05-16 11:28:04 +02:00
Nenad Merdanovic
146defaff4 MINOR: Add TLS ticket keys reference and use it in the listener struct
Within the listener struct we need to use a reference to the TLS
ticket keys which binds the actual keys with the filename. This will
make it possible to update the keys through the socket

Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-05-16 11:28:04 +02:00
Willy Tarreau
449f952cb3 BUG/MAJOR: checks: break infinite loops when tcp-checks starts with comment
If a tcp-check sequence starts with "comment", then the action is not
matched in the while() loop and the pointer doesn't advance so we face
an endless loop. It is normally detected early except in the case where
very slow checks are performed causing it to trigger after the admin stops
watching.

This bug is 1.6-only and very recent so it didn't have the time to affect
anyone.
2015-05-13 15:39:48 +02:00
Willy Tarreau
5581c27b57 BUG/MEDIUM: checks: do not dereference a list as a tcpcheck struct
The method used to skip to next rule in the list is wrong, it assumes
that the list element starts at the same offset as the rule. It happens
to be true on most architectures since the list is the first element for
now but it's definitely wrong. Now the code doesn't crash anymore when
the struct list is moved anywhere else in the struct tcpcheck_rule.

This fix must be backported to 1.5.
2015-05-13 15:31:34 +02:00
Willy Tarreau
f2c87353a7 BUG/MAJOR: checks: always check for end of list before proceeding
This is the most important fix of this series. There's a risk of endless
loop and crashes caused by the fact that we go past the head of the list
when skipping to next rule, without checking if it's still a valid element.
Most of the time, the ->action field is checked, which points to the proxy's
check_req pointer (generally NULL), meaning the element is confused with a
TCPCHK_ACT_SEND action.

The situation was accidently made worse with the addition of tcp-check
comment since it also skips list elements. However, since the action that
makes it go forward is TCPCHK_ACT_COMMENT (3), there's little chance to
see this as a valid pointer, except on 64-bit machines where it can match
the end of a check_req string pointer.

This fix heavily depends on previous cleanup and both must be backported
to 1.5 where the bug is present.
2015-05-13 15:31:34 +02:00