Commit graph

2997 commits

Author SHA1 Message Date
Willy Tarreau
073edf3311 BUILD: ssl: previous patch failed
This is a minor error, s/SMP_T_CSTR/SMP_T_STR + SMP_F_CONST.
2014-04-09 15:41:52 +02:00
David S
c1ad52e8f7 MINOR: ssl: add ssl_fc_unique_id to fetch TLS Unique ID
The TLS unique id, or unique channel binding, is a byte string that can be
pulled from a TLS connection and it is unique to that connection. It is
defined in RFC 5929 section 3.  The value is used by various upper layer
protocols as part of an extra layer of security.  For example XMPP
(RFC 6120) and EST (RFC 7030).

Add the ssl_fc_unique_id keyword and corresponding sample fetch method.
Value is retrieved from OpenSSL and base64 encoded as described in RFC
5929 section 3.
2014-04-09 13:48:33 +02:00
Willy Tarreau
6a0b6bd648 BUG/MAJOR: counters: check for null-deref when looking up an alternate table
Constructions such as sc0_get_gpc0(foo) allow to look up the same key as
the current key but in an alternate table. A check was missing to ensure
we already have a key, resulting in a crash if this lookup is performed
before the associated track-sc rule.

This bug was reported on the mailing list by Neil@iamafreeman and
narrowed down further by Lukas Tribus and Thierry Fournier.

This bug was introduced in 1.5-dev20 by commit "0f791d4 MEDIUM: counters:
support looking up a key in an alternate table".
2014-04-09 13:32:11 +02:00
Apollon Oikonomopoulos
25a15227f5 BUG/MINOR: reject malformed HTTP/0.9 requests
RFC 1945 (§4.1) defines an HTTP/0.9 request ("Simple-Request") as:

  Simple-Request  = "GET" SP Request-URI CRLF

HAProxy tries to automatically upgrade HTTP/0.9 requests to
to HTTP/1.0, by appending "HTTP/1.0" to the request and setting the
Request-URI to "/" if it was not present. The latter however is
RFC-incompatible, as HTTP/0.9 requests must already have a Request-URI
according to the definition above. Additionally,
http_upgrade_v09_to_v10() does not check whether the request method is
indeed GET (the mandatory method for HTTP/0.9).

As a result, any single- or double-word request line is regarded as a
valid HTTP request. We fix this by failing in http_upgrade_v09_to_v10()
if the request method is not GET or the request URI is not present.
2014-04-06 07:53:07 +02:00
Willy Tarreau
272adea423 REORG: cfgparse: move server keyword parsing to server.c
The cfgparse.c file becomes huge, and a large part of it comes from the
server keyword parser. Since the configuration is a bit more modular now,
move this parser to server.c.

This patch also moves the check of the "server" keyword earlier in the
supported keywords list, resulting in a slightly faster config parsing
for configs with large numbers of servers (about 10%).

No functional change was made, only the code was moved.
2014-03-31 10:42:03 +02:00
Bertrand Jacquin
702d44f2ff MEDIUM: proxy: support use_backend with dynamic names
We have a use case where we look up a customer ID in an HTTP header
and direct it to the corresponding server. This can easily be done
using ACLs and use_backend rules, but the configuration becomes
painful to maintain when the number of customers grows to a few
tens or even a several hundreds.

We realized it would be nice if we could make the use_backend
resolve its name at run time instead of config parsing time, and
use a similar expression as http-request add-header to decide on
the proper backend to use. This permits the use of prefixes or
even complex names in backend expressions. If no name matches,
then the default backend is used. Doing so allowed us to get rid
of all the use_backend rules.

Since there are some config checks on the use_backend rules to see
if the referenced backend exists, we want to keep them to detect
config errors in normal config. So this patch does not modify the
default behaviour and proceeds this way :

  - if the backend name in the use_backend directive parses as a log
    format rule, it's used as-is and is resolved at run time ;

  - otherwise it's a static name which must be valid at config time.

There was the possibility of doing this with the use-server directive
instead of use_backend, but it seems like use_backend is more suited
to this task, as it can be used for other purposes. For example, it
becomes easy to serve a customer-specific proxy.pac file based on the
customer ID by abusing the errorfile primitive :

     use_backend bk_cust_%[hdr(X-Cust-Id)] if { hdr(X-Cust-Id) -m found }
     default_backend bk_err_404

     backend bk_cust_1
         errorfile 200 /etc/haproxy/static/proxy.pac.cust1

Signed-off-by: Bertrand Jacquin <bjacquin@exosec.fr>
2014-03-31 10:18:30 +02:00
Thierry FOURNIER
fa45f1d06c MEDIUM: config: Dynamic sections.
This patch permit to register new sections in the haproxy's
configuration file. This run like all the "keyword" registration, it is
used during the haproxy initialization, typically with the
"__attribute__((constructor))" functions.
2014-03-31 09:56:40 +02:00
Thierry FOURNIER
9f95e4084c MINOR: standard: Add ipv6 support in the function url2sa().
The function url2sa() converts faster url like http://<ip>:<port> in a
struct sockaddr_storage. This patch add:
 - the https support
 - permit to return the length parsed
 - support IPv6
 - support DNS synchronous resolution only during start of haproxy.

The faster IPv4 convertion way is keeped. IPv6 is slower, because I use
the standard IPv6 parser function.
2014-03-31 09:54:44 +02:00
Thierry FOURNIER
46006bde3c MINOR: pattern: Add function to prune and reload pattern list.
This function it is used for dynamically update all the patterns
attached to one file. This function is atomic. All parsing or indexation
failures are reported in the haproxy logs.
2014-03-28 13:23:07 +01:00
Thierry FOURNIER
a47a94fb13 MINOR: session: don't always assume there's a listener
For outgoing connections initiated from an applet, there might not be
any listener. It's the case with peers, which resort to a hack consisting
in making the session's listener point to the peer. This listener is only
used for statistics now so it's much easier to check for its presence now.
2014-03-28 13:16:32 +01:00
Thierry FOURNIER
f7e04e9c2f MINOR: cli: remove information about acl or map owner.
The string <display> contain all the owners of the displayed pattern.
The message " - also used by an ACL/map" is useless.
2014-03-28 13:14:00 +01:00
Thierry FOURNIER
65ce6133ba MINOR: pattern/cli: Update used terms in documentation and cli
This patch replace the word <name> by the word <file>. This word defines
the (string) returned by show "map/acl". This patch also update
documentation to explain how is composed the map or acl identifier.
2014-03-28 13:13:26 +01:00
Willy Tarreau
0e9b1b4d1f MEDIUM: compression: consider the "q=" attribute in Accept-Encoding
Till now we didn't consider "q=". It's problematic because the first
effect is that compression tokens were not even matched if it was
present.

It is important to parse it correctly because we still want to allow
a user-agent to send "q=0" to explicitly disable a compressor, or to
specify its preferences.

Now, q-values are respected in order of precedence, and when several
q-values are equal, the first occurrence is used.
2014-03-19 12:12:01 +01:00
Thierry FOURNIER
e5978bfc25 BUG/MEDIUM: acl: boolean only matches were broken by recent changes
The ACL changes made in the last patchset force the execution
of each pattern matching function. The function pat_match_nothing
was not provided to be excuted, it was just used as a flag that
was checked by the ACL execution code. Now this function is
executed and always returns false.

This patch makes it work as expected. Now, it returns the boolean
status of the received sample just as was done previously in the
ACL code.

This bug is a part of the patchset just merged. It does not need
to be backported.
2014-03-17 20:12:40 +01:00
Thierry FOURNIER
c5a4e98639 MEDIUM: acl: Change the acl register struct
This patch replace a lot of pointeur by pattern matching identifier. If
the declared ACL use all the predefined pattern matching functions, the
register function gets the functions provided by "pattern.c" and
identified by the PAT_LATCH_*.

In the case of the acl uses his own functions, they can be declared, and
the acl registration doesn't change it.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
b7729c96a4 MINOR: pattern: forbid dns resolutions
This patch adds the flags "-n" on the acl parser. the flag "-n" forbif
the DNS resolutions. The maps have always the dns resolutions disabled.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
66eb9bf691 MINOR: pattern: Remove the flag "PAT_F_FROM_FILE".
This flag is no longer used. The last place using this, are the display
of the result of pattern matching in the cli command "get map" or "get
acl".

The first parameter of this command is the reference of the file used to
perform the lookup.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
fc7ac7b89c MINOR: standard: Disable ip resolution during the runtime
The function str2net runs DNS resolution if valid ip cannot be parsed.
The DNS function used is the standard function of the libc and it
performs asynchronous request.

The asynchronous request is not compatible with the haproxy
archictecture.

str2net() is used during the runtime throught the "socket".

This patch remove the DNS resolution during the runtime.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
94580c9f52 MINOR: dumpstat/conf: display all the configuration lines that using pattern reference 2014-03-17 18:06:08 +01:00
Thierry FOURNIER
eeaa951726 MINOR: configuration: File and line propagation
This patch permits to communicate file and line of the
configuration file at the configuration parser.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
31db4ae7b4 MINOR: pattern: index duplicates
The indexation functions now accept duplicates. This way it is possible
to always have some consistency between lists and trees. The "add" command
will always add regardless of any previous existence. The new entry will
not be used because both trees and list retrieve keys in insertion order.
Thus the "add" operation will always succeed (as long as there is enough
memory).
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
64c585ffeb MINOR: cli: Block the usage of the command "acl add" in many cases.
If acl is shared with a map, the "add acl" command must be blocked
because it not take a sample on his parameters. The absense of this
parameter can cause error with corresponding maps.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
0b6d15fdc8 MINOR: regex: The pointer regstr in the struc regex is no longer used.
The pointer <regstr> is only used to compare and identify the original
regex string with the patterns. Now the patterns have a reference map
containing this original string. It is useless to store this value two
times.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
149e0fe44f MINOR: pattern: The function pat_ref_set() have now atomic behavior
Before this patch, this function try to add values in best effort. If
the parsing iof the value fail, the operation continue until the end.

Now, this function stop on the first error and left the pattern in
coherant state.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
364cfdff7a MEDIUM: dumpstats: Display error message during add of values.
This patch adds new display type. This display returns allocated string,
when the string is flush into buffers, it is freed. This permit to
return the content of "memprintf(err, ...)" messages.

The pat_ref_add functions has changed to return error.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
9860c41258 MINOR: acl/pattern: Acl "-M" option force to load file as map file with two columns 2014-03-17 18:06:08 +01:00
Thierry FOURNIER
c0bd9100eb MINOR: pattern: Check if the file reference is not used with acl and map
The format of the acl file are not the same than the format of the map
files. In some case, the same file can be used, but this is ambiguous
for the user because the patterns are not the expected.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
39bef456fb MINOR: pattern/map/acl: Centralization of the file parsers
The acl and map function do the same work with the file parsing. This
patch merge these code in only one.

Note that the function map_read_entries_from_file() in the file "map.c"
is moved to the the function pat_ref_read_from_file_smp() in the file
"pattern.c". The code of this function is not modified, only the the
name and the arguments order has changed.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
9356c68727 MEDIUM: dumpstats/pattern: display and use each pointer of each pattern dumped
Each pattern displayed is associated to the value of his pattern
reference. This value can be used for deleting the entry. It is useful
with complex regex: the users are not forced to write the regex with all
the amiguous chars and escaped chars on the CLI.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
e369ca2e66 MEDIUM: pattern_find_smp: functions find_smp uses the pat_ref_elt to find the element to be removed
The find_smp search the smp using the value of the pat_ref_elt pointer.

The pat_find_smp_* are no longer used. The function pattern_find_smp()
known all pattern indexation, and can be found
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
7acca4b269 MEDIUM: pattern: delete() function uses the pat_ref_elt to find the element to be removed
All the pattern delete function can use her reference to the original
"struct pat_ref_elt" to find the element to be remove. The functions
pat_del_list_str() and pat_del_meth() were deleted because after
applying this modification, they have the same code than pat_del_list_ptr().
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
86db66a7c8 MINOR: dumpstats: display the reference for th key/pattern and value.
Before this patch, the "get map/acl" function try to convert and display
the sample. This behavior is not efficient because some type like the
regex cannot be reversed and displayed as string.

This patch display the original stored reference.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
6bb53ff164 MINOR: pattern: Each pattern expression element store the reference struct.
Now, each pattern entry known the original "struct pat_ref_elt" from
that was built. This patch permit to delete each pattern entry without
confusion. After this patch, each reference can use his pointer to be
targeted.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
0d6ba513a5 MINOR: pattern: store configuration reference for each acl or map pattern.
This patch permit to add reference for each pattern reference. This is
useful to identify the acl listed.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d25c842326 MINOR: pattern: Merge function pattern_add() with pat_ref_push().
The function Pattern_add() is only used by pat_ref_push(). This patch
remove the function pattern_add() and merge his code in the function
pat_ref_push().
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
aa222aadb6 MINOR: pattern: The function "pattern_register()" is no longer used.
Remove the function "pattern_register()" and its prototype because it
is no longer used.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
3534d881bc MINOR: pattern/acl: Each pattern of each acl can be load with specified id
This patch adds -u option on the acl pattern loading to force a unique ID
value.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
af5a29d5f8 MINOR: pattern: Each pattern is identified by unique id.
The pattern reference are stored with two identifiers: the unique_id and
the reference.

The reference identify a file. Each file with the same name point to the
same reference. We can register many times one file. If the file is
modified, all his dependencies are also modified. The reference can be
used with map or acl.

The unique_id identify inline acl. The unique id is unique for each acl.
You cannot force the same id in the configuration file, because this
repport an error.

The format of the acl and map listing through the "socket" has changed
for displaying these new ids.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
5d34408785 MEDIUM: pattern: The expected type is stored in the pattern head, and conversion is executed once.
This patch extract the expect_type variable from the "struct pattern" to
"struct pattern_head". This variable is set during the declaration of
ACL and MAP. With this change, the function "pat_parse_len()" become
useless and can be replaced by "pat_parse_int()".

Implicit ACLs by default rely on the fetch's output type, so let's simply do
the same for all other ones. It has been verified that they all match.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
c5959fd5d4 MEDIUM: pattern: merge same pattern
Sometimes the same pattern file is used with the same index, parse and
parse_smp functions. If this two condition are true, these two pattern
are identical and the same struct can be used.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1e00d3853b MAJOR: pattern/map: Extends the map edition system in the patterns
This patch add the following socket command line options:

  show acl [<id>]
  clear acl <id>
  get acl <id> <pattern>
  del acl <id> <pattern>
  add acl <id> <pattern>

The system used for maps is backported in the pattern functions.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
888863534c MINOR: map/pattern: The sample parser is stored in the pattern
We cannot separe the pattern and the value. Now, the patern known the
value and the pattern is able to parsehis associated sample staroage.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
3667e514d9 MEDIUM: pattern/dumpstats: The function pattern_lookup() is no longer used
This function are used in dumpstats. Now this function is replaced by
delete and find_smp function pointer.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
55d0b10f06 MEDIUM: pattern: add sample lookup function.
Some functions needs to change the sample associated to pattern. This
new pointer permit to return the a pointer to the sample pointer. The
caller can use or change the value.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
6f7203d673 MEDIUM: pattern: add prune function
This path add specific pointer to each expression to point on prune
function. Now, each pattern expression embed his own prune function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b113650e54 MEDIUM: pattern: add delete functions
This commit adds a delete function for patterns. It looks up all
instances of the pattern to delete and deletes them all. The fetch
keyword declarations have been extended to point to the appropriate
delete function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
33a7433ac9 MEDIUM: pattern: Index IPv6 addresses in a tree.
This commit adds second tree node in the pattern struct and use it to
index IPv6 addresses. This commit report feature used in the list. If
IPv4 not match the tree, try to convert the IPv4 address in IPv6 with
prefixing the IPv4 address by "::ffff", after this operation, the match
function try lookup in the IPv6 tree. If the IPv6 sample dont match the
IPv6 tree, try to convert the IPv6 addresses prefixed by "2002:IPv4",
"::ffff:IPv4" and "::0000:IPv4" in IPv4 address. after this operation,
the match function try lookup in the IPv4 tree.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
5338eea8eb MEDIUM: pattern: The match function browse itself the list or the tree.
The match function known the format of the pattern. The pattern can be
stored in a list or in a tree. The pattern matching function use itself
the good entry point and indexation type.

Each pattern matching function return the struct pattern that match. If
the flag "fill" is set, the struct pattern is filled, otherwise the
content of this struct must not be used.

With this feature, the general pattern matching function cannot have
exceptions for building the "struct pattern".
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b4be12952a MINOR: map/dumpstats: The cli cmd "get map ..." display the "int" format.
The "get map ..." command display the integer range that match the
request.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b990384c9a MINOR: dumpstats: change the "get map" display management
The original get map display function set the comma separator after each
word displayed. This is not efficient because we cannot knew if the
displayed word is the last.

This new system set the comma separator before the displayed word, and
independant "\n" is set a the end of the function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d572343d51 MINOR: dumpstats: change map inline sentences
The errors and messages used in the cli are sometimes impossibkle to
understand. This patch changes all these messages.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1794fdf37e MEDIUM: pattern: The function pattern_exec_match() returns "struct pattern" if the patten match.
Before this commit, the pattern_exec_match() function returns the
associate sample, the associate struct pattern or the associate struct
pattern_tree. This is complex to use, because we can check the type of
information returned.

Now the function return always a "struct pattern". If <fill> is not set,
only the value of the pointer can be used as boolean (NULL or other). If
<fill> is set, you can use the <smp> pointer and the pattern
information.

If information must be duplicated, it is stored in trash buffer.
Otherwise, the pattern can point on existing strings.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1432a0c5db MINOR: dumpstats: Group map inline help 2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d437314979 MEDIUM: sample/http_proto: Add new type called method
The method are actuelly stored using two types. Integer if the method is
known and string if the method is not known. The fetch is declared as
UINT, but in some case it can provides STR.

This patch create new type called METH. This type contain interge for
known method and string for the other methods. It can be used with
automatic converters.

The pattern matching can expect method.

During the free or prune function, http_meth pettern is freed. This
patch initialise the freed pointer to NULL.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
7654c9ff44 MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags
The operations applied on types SMP_T_CSTR and SMP_T_STR are the same,
but the check code and the declarations are double, because it must
declare action for SMP_T_C* and SMP_T_*. The declared actions and checks
are the same. this complexify the code. Only the "conv" functions can
change from "C*" to "*"

Now, if a function needs to modify input string, it can call the new
function smp_dup(). This one duplicate data in a trash buffer.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b050463375 MINOR: standard: Add function for converting cidr to network mask. 2014-03-17 18:06:07 +01:00
Thierry FOURNIER
0e9af55700 MINOR: sample: dont call the sample cast function "c_none"
If the cast function to execute is c_none, dont execute it and return
true. The function c_none, do nothing. This save a call.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
e1bcac5b8f MINOR: pattern: Rename "pat_idx_elt" to "pattern_tree"
This is just for having coherent struct names.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
edc15c3a35 MEDIUM: pattern: The parse functions just return "struct pattern" without memory allocation
The pattern parse functions put the parsed result in a "struct pattern"
without memory allocation. If the pattern must reference the input data
without changes, the pattern point to the parsed string. If buffers are
needed to store translated data, it use th trash buffer. The indexation
function that allocate the memory later if it is needed.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b9b08460a2 MEDIUM: pattern: add indexation function.
Before this patch, the indexation function check the declared patttern
matching function and index the data according with this function. This
is not useful to add some indexation mode.

This commit adds dedicated indexation function. Each struct pattern is
associated with one indexation function. This function permit to index
data according with the type of pattern and with the type of match.
2014-03-17 18:06:06 +01:00
Willy Tarreau
1cf8f08c17 MINOR: sample: move smp_to_type to sample.c
This way it can be exported and reused anywhere else to report type names.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
3ead5b93c6 MINOR: pattern: separe list element from the data part.
This commit separes the "struct list" used for the chain the "struct
pattern" which contain the pattern data. Later, this change will permit
to manipulate lists ans trees with the same "struct pattern".
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
972028fa67 MEDIUM: pattern: Change the prototype of the function pattern_register().
Each pattern parser take only one string. This change is reported to the
function prototype of the function "pattern_register()". Now, it is
called with just one string and no need to browse the array of args.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
580c32cb3a MEDIUM: pattern: The pattern parser no more uses <opaque> and just takes one string.
After the previous patches, the "pat_parse_strcat()" function disappear,
and the "pat_parse_int()" and "pat_parse_dotted_ver()" functions dont
use anymore the "opaque" argument, and take only one string on his
input.

So, after this patch, each pattern parser no longer use the opaque
variable and take only one string as input. This patch change the
prototype of the pattern parsing functions.

Now, the "char **args" is replaced by a "char *arg", the "int *opaque"
is removed and these functions return 1 in succes case, and 0 if fail.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
511e9475f2 MEDIUM: acl/pattern: standardisation "of pat_parse_int()" and "pat_parse_dotted_ver()"
The goal of these patch is to simplify the prototype of
"pat_pattern_*()" functions. I want to replace the argument "char
**args" by a simple "char *arg" and remove the "opaque" argument.

"pat_parse_int()" and "pat_parse_dotted_ver()" are the unique pattern
parser using the "opaque" argument and using more than one string
argument of the char **args. These specificities are only used with ACL.
Other systems using this pattern parser (MAP and CLI) just use one
string for describing a range.

This two functions can read a range, but the min and the max must y
specified. This patch extends the syntax to describe a range with
implicit min and max. This is used for operators like "lt", "le", "gt",
and "ge". the syntax is the following:

   ":x" -> no min to "x"
   "x:" -> "x" to no max

This patch moves the parsing of the comparison operator from the
functions "pat_parse_int()" and "pat_parse_dotted_ver()" to the acl
parser. The acl parser read the operator and the values and build a
volatile string readable by the functions "pat_parse_int()" and
"pat_parse_dotted_ver()". The transformation is done with these rules:

If the parser is "pat_parse_int()":

   "eq x" -> "x"
   "le x" -> ":x"
   "lt x" -> ":y" (with y = x - 1)
   "ge x" -> "x:"
   "gt x" -> "y:" (with y = x + 1)

If the parser is "pat_parse_dotted_ver()":

   "eq x.y" -> "x.y"
   "le x.y" -> ":x.y"
   "lt x.y" -> ":w.z" (with w.z = x.y - 1)
   "ge x.y" -> "x.y:"
   "gt x.y" -> "w.z:" (with w.z = x.y + 1)

Note that, if "y" is not present, assume that is "0".

Now "pat_parse_int()" and "pat_parse_dotted_ver()" accept only one
pattern and the variable "opaque" is no longer used. The prototype of
the pattern parsers can be changed.
2014-03-17 18:06:06 +01:00
Thierry FOURNIER
9eec0a646b MAJOR: auth: Change the internal authentication system.
This patch remove the limit of 32 groups. It also permit to use standard
"pat_parse_str()" function in place of "pat_parse_strcat()". The
"pat_parse_strcat()" is no longer used and its removed. Before this
patch, the groups are stored in a bitfield, now they are stored in a
list of strings. The matching is slower, but the number of groups is
low and generally the list of allowed groups is short.

The fetch function "smp_fetch_http_auth_grp()" used with the name
"http_auth_group" return valid username. It can be used as string for
displaying the username or with the acl "http_auth_group" for checking
the group of the user.

Maybe the names of the ACL and fetch methods are no longer suitable, but
I keep the current names for conserving the compatibility with existing
configurations.

The function "userlist_postinit()" is created from verification code
stored in the big function "check_config_validity()". The code is
adapted to the new authentication storage system and it is moved in the
"src/auth.c" file. This function is used to check the validity of the
users declared in groups and to check the validity of groups declared
on the "user" entries.

This resolve function is executed before the check of all proxy because
many acl needs solved users and groups.
2014-03-17 18:06:06 +01:00
Willy Tarreau
2049092f26 CLEANUP: acl: remove obsolete test in parse_acl_expr()
The ACL keyword returned by find_acl_kw() is checked for having a
valid ->parse() function. This dates back 2007 when ACLs were reworked
in order to differenciate old and new keywords. This check is
inappropriate and confusing since all keywords have a parser now.
2014-03-17 18:04:27 +01:00
Thierry FOURNIER
e87cac16cc MEDIUM: sample: change the behavior of the bin2str cast
The bin2str cast gives the hexadecimal representation of the binary
content when it is used as string. This was inherited from the
stick-table casts without realizing that it was a mistake. Indeed,
it breaks string processing on binary contents, preventing any _reg,
_beg, etc from working.

For example, with an HTTP GET request, the fetch "req.payload(0,3)"
returns the 3 bytes "G", "E", and "T" in binary. If this fetch is
used with regex, it is automatically converted to "474554" and the
regex is applied on this string, so it never matches.

This commit changes the cast so that bin2str does not convert the
contents anymore, and returns a string type. The contents can thus
be matched as is, and the NULL character continues to mark the end
of the string to avoid any issue with some string-based functions.

This commit could almost have been marked as a bug fix since it
does what the doc says.

Note that in case someone would rely on the hex encoding, then the
same behaviour could be achieved by appending ",hex" after the sample
fetch function (brought by previous patch).
2014-03-17 17:31:46 +01:00
Thierry FOURNIER
2f49d6d17b MINOR: sample: add hex converter
This new filter converts BIN type to its hexadecimal
representation in STR type. It is used to keep the
compatibility with the original bin2str cast.

It will be useful when bin2str changes to copy the
string as-is without encoding anymore.
2014-03-17 16:39:18 +01:00
Thierry FOURNIER
d048d8b891 BUG/MINOR: http: fix encoding of samples used in http headers
The binary samples are sometimes copied as is into http headers.
A sample can contain bytes unallowed by the http rfc concerning
header content, for example if it was extracted from binary data.
The resulting http request can thus be invalid.

This issue does not yet happen because haproxy currently (mistakenly)
hex-encodes binary data, so it is not really possible to retrieve
invalid HTTP chars.

The solution consists in hex-encoding all non-printable chars prefixed
by a '%' sign.

No backport is needed since existing code is not affected yet.
2014-03-17 16:39:03 +01:00
Thierry FOURNIER
e059ec9393 MINOR: standard: add function "encode_chunk"
This function has the same behavior as encode_string(), except it
takes a "struct chunk" instead of a "char *" on input.
2014-03-17 16:38:56 +01:00
Willy Tarreau
7cbc915a1d MEDIUM: config: faster lookup for duplicated proxy name
cfg_parse_listen() currently checks for duplicated proxy names.
Now that we have a tree for this, we can use it.

The config load time was further reduced by 1.6, which is now
about 4.5 times faster than what it was without the trees.

In fact it was the last CPU-intensive processing involving proxy
names. Now the only remaining point is the automatic fullconn
computation which can be bypassed by having a fullconn in the
defaults section, reducing the load time by another 10x.
2014-03-15 08:21:42 +01:00
Willy Tarreau
991610d33f MEDIUM: stick-table: lookup table names using trees.
This will speed up config parsing time when many stick-table references
are used, as well as stick-table manipulations from the CLI.
2014-03-15 08:03:57 +01:00
Willy Tarreau
7ecc420bbc MEDIUM: proxy: make get_backend_server() use findproxy() to lookup proxies
This one is used by stats to look up proxy and server names in server state
manipulation. Get rid of the linear search.
2014-03-15 07:57:11 +01:00
Willy Tarreau
cfd837fb6c MEDIUM: proxy: make findproxy() use trees to look up proxies
Both proxy IDs and names are now looked up from the trees.
2014-03-15 07:50:05 +01:00
Willy Tarreau
f79d950163 MEDIUM: proxy: create a tree to store proxies by name
Large configurations can take time to parse when thousands of backends
are in use. Let's store all the proxies in trees.

findproxy_mode() has been modified to use the tree for lookups, which
has divided the parsing time by about 2.5. But many lookups are still
present at many places and need to be dealt with.
2014-03-15 07:48:35 +01:00
Willy Tarreau
7519560767 MINOR: http: release compression context only in http_end_txn()
Currently there are two places where the compression context is released,
one in session_free() and another one in http_end_txn_clean_session().
Both of them call http_end_txn(), either directly or via http_reset_txn(),
and this function is made for this exact purpose. So let's centralize the
call there instead.
2014-03-14 19:26:20 +01:00
Willy Tarreau
80a92c02f4 BUG/MEDIUM: http: don't start to forward request data before the connect
Currently, "balance url_param check_post" randomly works. If the client
sends chunked data and there's another chunk after the one containing the
data, http_request_forward_body() will advance msg->sov and move the start
of data to the beginning of the last chunk, and get_server_ph_post() will
not find the data.

In order to avoid this, we add an HTTP_MSGF_WAIT_CONN flag whose goal is
to prevent the forwarding code from parsing until the connection is
confirmed, so that we're certain not to fail on a redispatch. Note that
we need to force channel_auto_connect() since the output buffer is empty
and a previous analyser might have stopped auto-connect.

The flag is currently set whenever some L7 POST analysis is needed for a
connect() so that it correctly addresses all corner cases involving a
possible rewind of the buffer, waiting for a better fix.

Note that this has been broken for a very long time. Even all 1.4 versions
seem broken but differently, with ->sov pointing to the end of the arguments.
So the fix should be considered for backporting to all stable releases,
possibly including 1.3 which works differently.
2014-03-14 12:22:56 +01:00
Thierry FOURNIER
da5d4a5560 BUG/MINOR: log: The log of quotted capture header has been terminated by 2 quotes.
Julien Vehent repport that the log format '%{+Q}hr' display the value
termnated by two chars '"' like this: '"value""'. This patch just remove
the second quote.

This bug is old but 1.5-specific but users of older 1.5 versions may be
interested in a backport.
2014-03-14 11:55:54 +01:00
Thierry FOURNIER
338a031e29 BUG/MEDIUM: map: The map parser includes blank lines.
The parser check the end line comparing to the null character.
In fact, the end of line can be also '\r' or '\n'.

The effect is that empty lines are loaded and indexed in maps.

The bug was introduced by commit d5f624dd ("MEDIUM: sample:
add the "map" converter") in 1.5-dev20. No backport is needed.
2014-03-12 19:32:18 +01:00
Willy Tarreau
4a4e6bca60 BUG/MEDIUM: compression: fix the output type of the compressor name
smp_fetch_res_comp_algo() returns the name of the compression algorithm
in use. The output type is set to SMP_T_STR instead of SMP_T_CSTR, which
causes any transformation to be operated without a cast. Fortunately,
the current converters do not overwrite a zero-sized area, so the result
is an empty string. Fix this to have SMP_T_CSTR instead so that the cast
is always performed using a copy before any transformation is done.
2014-03-11 16:23:05 +01:00
Joshua M. Clulow
0724903143 BUG/MINOR: raw_sock: also consider ENOTCONN in addition to EAGAIN for recv()
I was testing haproxy-1.5-dev22 on SmartOS (an illumos-based system)
and ran into a problem.  There's a small window after non-blocking
connect() is called, but before the TCP connection is established,
where recv() may return ENOTCONN.  On Linux, the behaviour here seems
to be always to return EAGAIN.  The fix is relatively trivial, and
appears to make haproxy work reliably on current SmartOS (see patch
below).  It's possible that other UNIX platforms exhibit this
behaviour as well.

Note: the equivalent was already done for send() in commit 0ea0cf6
("BUG: raw_sock: also consider ENOTCONN in addition to EAGAIN").
Both patches should be backported to 1.4.
2014-03-04 07:27:18 +01:00
Lukas Tribus
7640e72a31 MINOR: set IP_FREEBIND on IPv6 sockets in transparent mode
Lets set IP_FREEBIND on IPv6 sockets as well, this works since Linux 3.3
and doesn't require CAP_NET_ADMIN privileges (IPV6_TRANSPARENT does).

This allows unprivileged users to bind to non-local IPv6 addresses, which
can be useful when setting up the listening sockets or when connecting
to backend servers with a specific, non-local source IPv6 address (at that
point we usually dropped root privileges already).
2014-03-03 21:31:10 +01:00
Willy Tarreau
583021306b BUG/MINOR: config: fix a crash on startup when a disabled backend references a peer
Disabled backends don't have their symbols resolved. We must not initialize
their peers section since they're not valid and instead still contain the
section's name.

There are other places where such unions are still in use, and other similar
errors might still happen. Ideally we should get rid of all of them in the
quite sensible config stage.
2014-02-24 20:59:47 +01:00
Willy Tarreau
38d25d2a19 MEDIUM: acl: fix pattern type for payload / payload_lv
Since commit 0ce3aa0c ("MEDIUM: acl: implement payload and payload_lv"),
the payload and payload_lv ACL patterns were declared as strings because
at this date there was no support for binary patterns. At this time, these
ACLs were not reliably usable due to the binary-to-string cast involved,
and because it was not possible to specify the direction of the match.

Since recent evolutions, the new fetch methods "req.payload" and
"res.payload" have leveraged the ambiguity and were of type "binary",
with an implicit ACL mapping of the same type. The doc also states
that "payload" is an alias for "req.payload" etc... while these two
don't share the same type.

Better fix this mess before it's too late. "payload" and "payload_lv"
return a binary content, so their ACLs must by default use a binary
pattern. That way they behave like their "req." and "res." sisters.

This change might break some configs making use of these, but there's
almost a zero probability that anyone managed to use them to match
exact strings, so in practice the change should be safe.
2014-02-24 19:01:37 +01:00
Willy Tarreau
36346247ac BUG/MEDIUM: http: continue to emit 503 on keep-alive to different server
Finn Arne Gangstad reported that commit 6b726adb35 ("MEDIUM: http: do
not report connection errors for second and further requests") breaks
support for serving static files by abusing the errorfile 503 statement.

Indeed, a second request over a connection sent to any server or backend
returning 503 would silently be dropped.

The proper solution consists in adding a flag on the session indicating
that the server connection was reused, and to only avoid the error code
in this case.
2014-02-24 18:26:30 +01:00
Willy Tarreau
2481d167ef BUG/MEDIUM: backend: prefer-last-server breaks redispatch
Since 1.5-dev20, we have a working server-side keep-alive and an option
"prefer-last-server" to indicate that we explicitly want to reuse the
same server as the last one. Unfortunately this breaks the redispatch
feature because assign_server() insists on reusing the same server as
the first one attempted even if the connection failed to establish.

A simple solution consists in only considering the last connection if
it was connected. Otherwise there is no reason for being interested in
reusing the same server.
2014-02-24 13:21:32 +01:00
Willy Tarreau
abd03df9de BUG/MINOR: config: server on-marked-* statement is ignored in default-server
Commits e0d1bfb ("[MINOR] Allow shutdown of sessions when a server
becomes unavailable") and eb2c24a ("MINOR: checks: add on-marked-up
option") mentionned that the directive was supported in default-server
but while it can be stated there, it's ignored because the config value
is not copied from the default server upon creation of a new server.
Moving the statement to the "server" lines works fine though. Thanks
to Baptiste Assmann for reporting and diagnosing this bug.

These features were introduced in 1.5-dev6 and 1.5-dev10 respectively,
so no backport is needed.
2014-02-18 10:39:38 +01:00
Willy Tarreau
518ceddebe BUG/MEDIUM: ssl: always send a full buffer after EAGAIN
Igor Chan reported a very interesting bug which was triggered by the
recent dynamic size change in SSL.

The OpenSSL API refuses to send less data than any failed previous
attempt. So what's happening is that if an SSL_write() in streaming
mode sends 5kB of data and the openssl layer cannot send them all,
it returns SSL_ERROR_WANT_WRITE, which haproxy reacts to by enabling
polling on the file descriptor. In the mean time, haproxy may detect
that the buffer was almost full and will disable streaming mode. Upon
write notification, it will try to send again, but less data this
time (limited to tune.ssl_max_record). OpenSSL disagrees with this
and returns a generic error SSL_ERROR_SSL.

The solution which was found consists in adding a flag to the SSL
context to remind that we must not shrink writes after a failed
attempt. Thus, if EAGAIN is encountered, the next send() will not
be limited in order to retry the same size as before.
2014-02-17 16:02:01 +01:00
Dirkjan Bussink
48f1c4e3ad MEDIUM: ssl: Use ALPN support as it will be available in OpenSSL 1.0.2
The current ALPN support is based on custom OpenSSL patches. These are
however not the same as what has landed on OpenSSL:

http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=6f017a8f9db3a79f3a3406cf8d493ccd346db691

This patch change the code so it supports ALPN as it will be part of
OpenSSL.
2014-02-16 19:49:51 +01:00
Willy Tarreau
07ba08b57c BUG/MINOR: ssl: fix syntax in config error message
Some error messages about server lines had a confusing '|' instead
of '[' to delimit the config file name.
2014-02-16 19:22:08 +01:00
Willy Tarreau
5498472ec0 BUG/MEDIUM: config: immediately abort if peers section has no name
Cyril Bonté reported that despite commit 0dbbf317 which attempted
to fix the crash when a peers section has no name, we still get a
segfault after the error message when parsing the peers. The reason
is that the returned error code is ERR_FATAL and not ERR_ABORT, so
the parsing continues while the section was not initialized.

This is 1.5-specific, no backport is needed.
2014-02-16 08:20:13 +01:00
Cyril Bonté
9a60ff9cb6 BUG/MEDIUM: peers: fix key consistency for integer stick tables
Peers with integer stick tables are breaking the keys received. This is due to
the fact that the sender converts the key with htonl() but the receiver doesn't
convert the value back to its original format.

Peers appeared in haproxy-1.5, no backport is needed.
2014-02-16 08:14:16 +01:00
Willy Tarreau
84310e2e73 MINOR: sample: add a rand() sample fetch to return a sample.
Sometimes it can be useful to generate a random value, at least
for debugging purposes, but also to take routing decisions or to
pass such a value to a backend server.
2014-02-14 11:59:04 +01:00
Willy Tarreau
610f04bbf6 MINOR: config: add global directives to set default SSL ciphers
The ability to globally override the default client and server cipher
suites has been requested multiple times since the introduction of SSL.
This commit adds two new keywords to the global section for this :
  - ssl-default-bind-ciphers
  - ssl-default-server-ciphers

It is still possible to preset them at build time by setting the macros
LISTEN_DEFAULT_CIPHERS and CONNECT_DEFAULT_CIPHERS.
2014-02-13 11:36:41 +01:00
Willy Tarreau
7e3127391f MINOR: config: make the stream interface idle timer user-configurable
The new tune.idletimer value allows one to set a different value for
idle stream detection. The default value remains set to one second.
It is possible to disable it using zero, and to change the default
value at build time using DEFAULT_IDLE_TIMER.
2014-02-12 16:36:12 +01:00
Willy Tarreau
03effc15d6 MINOR: ssl: add DEFAULT_SSL_MAX_RECORD to set the record size at build time
For some deployments it may help to have tune.ssl.maxrecord set to a
more efficient one at build time. This build setting allows this.
2014-02-12 14:55:41 +01:00
Willy Tarreau
c5890e66cd MEDIUM: stream-int: automatically disable CF_STREAMER flags after idle
Disabling the streamer flags after an idle period will help TCP proxies
to better adapt to the streams they're forwarding, especially with SSL
where this will allow the SSL sender to use smaller records. This is
typically used to optimally relay HTTP and derivatives such as SPDY or
HTTP/2 in pure TCP mode when haproxy is used as an SSL offloader.

This idea was first proposed by Ilya Grigorik on the haproxy mailing
list, and his tests seem to confirm the improvement :

  https://www.mail-archive.com/haproxy@formilux.org/msg12576.html
2014-02-12 11:46:03 +01:00
Willy Tarreau
852745d548 BUG/MAJOR: check: fix memory leak in "tcp-check connect" over SSL
tcp-check must not reinitialize the SSL stack upon each check!
It's done once after the config parsing and leaks memory and eats
performance when done upon every check.

This bug was introduced in 1.5-dev22, no backport is needed.
2014-02-11 17:55:04 +01:00
Willy Tarreau
f522f3d291 BUG/MEDIUM: stats: the "lastsess" field must appear last in the CSV.
It happens that latest change broke some monitoring tools which expect the
field to be found at the same position as indicated in the doc. Let's move
it to the last column instead.
2014-02-10 22:22:49 +01:00