upstream/haproxy
1
0
Fork 0
mirror of https://github.com/haproxy/haproxy.git synced 2026-03-29 05:43:58 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
haproxy/src/sample.c

1690 lines
54 KiB
C
Raw Normal View History

[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*
REORG: rename "pattern" files They're now called "sample" everywhere to match their description.
2012-04-27 15:52:18 -04:00
* Sample management functions.
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
*
* Copyright 2009-2010 EXCELIANCE, Emeric Brun <ebrun@exceliance.fr>
REORG: rename "pattern" files They're now called "sample" everywhere to match their description.
2012-04-27 15:52:18 -04:00
* Copyright (C) 2012 Willy Tarreau <w@1wt.eu>
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
*/
MINOR: sample: add "json" converter This converter escapes string to use it as json/ascii escaped string. It can read UTF-8 with differents behavior on errors and encode it in json/ascii. json([<input-code>]) Escapes the input string and produces an ASCII ouput string ready to use as a JSON string. The converter tries to decode the input string according to the <input-code> parameter. It can be "ascii", "utf8", "utf8s", "utf8"" or "utf8ps". The "ascii" decoder never fails. The "utf8" decoder detects 3 types of errors: - bad UTF-8 sequence (lone continuation byte, bad number of continuation bytes, ...) - invalid range (the decoded value is within a UTF-8 prohibited range), - code overlong (the value is encoded with more bytes than necessary). The UTF-8 JSON encoding can produce a "too long value" error when the UTF-8 character is greater than 0xffff because the JSON string escape specification only authorizes 4 hex digits for the value encoding. The UTF-8 decoder exists in 4 variants designated by a combination of two suffix letters : "p" for "permissive" and "s" for "silently ignore". The behaviors of the decoders are : - "ascii" : never fails ; - "utf8" : fails on any detected errors ; - "utf8s" : never fails, but removes characters corresponding to errors ; - "utf8p" : accepts and fixes the overlong errors, but fails on any other error ; - "utf8ps" : never fails, accepts and fixes the overlong errors, but removes characters corresponding to the other errors. This converter is particularly useful for building properly escaped JSON for logging to servers which consume JSON-formated traffic logs. Example: capture request header user-agent len 150 capture request header Host len 15 log-format {"ip":"%[src]","user-agent":"%[capture.req.hdr(1),json]"} Input request from client 127.0.0.1: GET / HTTP/1.0 User-Agent: Very "Ugly" UA 1/2 Output log: {"ip":"127.0.0.1","user-agent":"Very \"Ugly\" UA 1\/2"}
2014-08-12 04:20:47 -04:00
#include <ctype.h>
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
#include <string.h>
#include <arpa/inet.h>
CLEANUP: includes: fix includes for a number of users of fd.h It appears that fd.h includes a number of unneeded files and was included from standard.h, and as such served as an intermediary to provide almost everything to everyone. By removing its useless includes, a long dependency chain broke but could easily be fixed.
2012-09-02 16:34:23 -04:00
#include <stdio.h>
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: remove remains of BUFSIZE in HTTP auth and sample conversions Sample conversions rely on two alternative buffers which were previously allocated as static bufs of size BUFSIZE. Now they're initialized to the global buffer size. It was the same for HTTP authentication. Note that it seems that none of them was prone to any mistake when dealing with the buffer size, but better stay on the safe side by maintaining the old assumption that a trash buffer is always "large enough".
2012-10-29 15:44:36 -04:00
#include <types/global.h>
REORG: buffers: split buffers into chunk,buffer,channel Many parts of the channel definition still make use of the "buffer" word.
2012-08-24 13:22:53 -04:00
#include <common/chunk.h>
MINOR: sample: add new converters to hash input From time to time it's useful to hash input data (scramble input, or reduce the space needed in a stick table). This patch provides 3 simple converters allowing use of the available hash functions to hash input data. The output is an unsigned integer which can be passed into a header, a log or used as an index for a stick table. One nice usage is to scramble source IP addresses before logging when there are requirements to hide them.
2014-07-15 14:15:37 -04:00
#include <common/hash.h>
REORG: rename "pattern" files They're now called "sample" everywhere to match their description.
2012-04-27 15:52:18 -04:00
#include <common/standard.h>
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
#include <common/uri_auth.h>
MINOR: ssl: adds sample converter base64 for binary type. The new converter encode binary type sample to base64 string. i.e. : ssl_c_serial,base64
2014-04-30 12:21:37 -04:00
#include <common/base64.h>
REORG: rename "pattern" files They're now called "sample" everywhere to match their description.
2012-04-27 15:52:18 -04:00
MEDIUM: pattern: use the standard arg parser We don't need the pattern-specific args parsers anymore, make use of the common parser instead. We still need to improve this by adding a validation function to report abnormal argument values or combinations. We don't report precise parsing errors yet but this was not previously done either.
2012-04-20 08:45:49 -04:00
#include <proto/arg.h>
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
#include <proto/auth.h>
#include <proto/log.h>
MEDIUM: sample/http_proto: Add new type called method The method are actuelly stored using two types. Integer if the method is known and string if the method is not known. The fetch is declared as UINT, but in some case it can provides STR. This patch create new type called METH. This type contain interge for known method and string for the other methods. It can be used with automatic converters. The pattern matching can expect method. During the free or prune function, http_meth pettern is freed. This patch initialise the freed pointer to NULL.
2013-12-16 19:10:10 -05:00
#include <proto/proto_http.h>
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
#include <proto/proxy.h>
REORG: rename "pattern" files They're now called "sample" everywhere to match their description.
2012-04-27 15:52:18 -04:00
#include <proto/sample.h>
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
#include <proto/stick_table.h>
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MINOR: sample: move smp_to_type to sample.c This way it can be exported and reused anywhere else to report type names.
2014-02-07 06:14:54 -05:00
/* sample type names */
const char *smp_to_type[SMP_TYPES] = {
[SMP_T_BOOL] = "bool",
[SMP_T_UINT] = "uint",
[SMP_T_SINT] = "sint",
[SMP_T_ADDR] = "addr",
[SMP_T_IPV4] = "ipv4",
[SMP_T_IPV6] = "ipv6",
[SMP_T_STR] = "str",
[SMP_T_BIN] = "bin",
};
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* static sample used in sample_process() when <p> is NULL */
MINOR: pattern: replace struct pattern with struct sample This change is pretty minor. Struct pattern is only used for pattern_process() now so changing it to use the common type is quite obvious. It's worth noting that the last argument of pattern_process() is never used so the function is self-sufficient. Note that pattern_process() does not initialize the pattern at all before calling fetch->process(), and that minimal initialization will be required when we later change the argument for the sample.
2012-04-23 15:35:11 -04:00
static struct sample temp_smp;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* list head of all known sample fetch keywords */
static struct sample_fetch_kw_list sample_fetches = {
.list = LIST_HEAD_INIT(sample_fetches.list)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
};
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* list head of all known sample format conversion keywords */
static struct sample_conv_kw_list sample_convs = {
.list = LIST_HEAD_INIT(sample_convs.list)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
};
MEDIUM: samples: use new flags to describe compatibility between fetches and their usages Samples fetches were relying on two flags SMP_CAP_REQ/SMP_CAP_RES to describe whether they were compatible with requests rules or with response rules. This was never reliable because we need a finer granularity (eg: an HTTP request method needs to parse an HTTP request, and is available past this point). Some fetches are also dependant on the context (eg: "hdr" uses request or response depending where it's involved, causing some abiguity). In order to solve this, we need to precisely indicate in fetches what they use, and their users will have to compare with what they have. So now we have a bunch of bits indicating where the sample is fetched in the processing chain, with a few variants indicating for some of them if it is permanent or volatile (eg: an HTTP status is stored into the transaction so it is permanent, despite being caught in the response contents). The fetches also have a second mask indicating their validity domain. This one is computed from a conversion table at registration time, so there is no need for doing it by hand. This validity domain consists in a bitmask with one bit set for each usage point in the processing chain. Some provisions were made for upcoming controls such as connection-based TCP rules which apply on top of the connection layer but before instantiating the session. Then everywhere a fetch is used, the bit for the control point is checked in the fetch's validity domain, and it becomes possible to finely ensure that a fetch will work or not. Note that we need these two separate bitfields because some fetches are usable both in request and response (eg: "hdr", "payload"). So the keyword will have a "use" field made of a combination of several SMP_USE_* values, which will be converted into a wider list of SMP_VAL_* flags. The knowledge of permanent vs dynamic information has disappeared for now, as it was never used. Later we'll probably reintroduce it differently when dealing with variables. Its only use at the moment could have been to avoid caching a dynamic rate measurement, but nothing is cached as of now.
2013-01-07 09:42:20 -05:00
const unsigned int fetch_cap[SMP_SRC_ENTRIES] = {
[SMP_SRC_INTRN] = (SMP_VAL_FE_CON_ACC | SMP_VAL_FE_SES_ACC | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_LISTN] = (SMP_VAL_FE_CON_ACC | SMP_VAL_FE_SES_ACC | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_FTEND] = (SMP_VAL_FE_CON_ACC | SMP_VAL_FE_SES_ACC | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_L4CLI] = (SMP_VAL_FE_CON_ACC | SMP_VAL_FE_SES_ACC | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_L5CLI] = (SMP_VAL___________ | SMP_VAL_FE_SES_ACC | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_TRACK] = (SMP_VAL_FE_CON_ACC | SMP_VAL_FE_SES_ACC | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_L6REQ] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________),
[SMP_SRC_HRQHV] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________),
[SMP_SRC_HRQHP] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_FE_REQ_CNT |
SMP_VAL_FE_HRQ_HDR | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_HRQBO] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL_FE_HRQ_BDY | SMP_VAL_FE_SET_BCK |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________),
[SMP_SRC_BKEND] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL_BE_REQ_CNT | SMP_VAL_BE_HRQ_HDR | SMP_VAL_BE_HRQ_BDY |
SMP_VAL_BE_SET_SRV | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_SERVR] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL_BE_SRV_CON | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_L4SRV] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_L5SRV] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_L6RES] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL___________),
[SMP_SRC_HRSHV] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL___________),
[SMP_SRC_HRSHP] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL_BE_RES_CNT |
SMP_VAL_BE_HRS_HDR | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL_FE_LOG_END),
[SMP_SRC_HRSBO] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL_BE_HRS_BDY | SMP_VAL_BE_STO_RUL |
SMP_VAL_FE_RES_CNT | SMP_VAL_FE_HRS_HDR | SMP_VAL_FE_HRS_BDY |
SMP_VAL___________),
[SMP_SRC_RQFIN] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL_FE_LOG_END),
[SMP_SRC_RSFIN] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL_FE_LOG_END),
[SMP_SRC_TXFIN] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL_FE_LOG_END),
[SMP_SRC_SSFIN] = (SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL___________ | SMP_VAL___________ | SMP_VAL___________ |
SMP_VAL_FE_LOG_END),
};
static const char *fetch_src_names[SMP_SRC_ENTRIES] = {
[SMP_SRC_INTRN] = "internal state",
[SMP_SRC_LISTN] = "listener",
[SMP_SRC_FTEND] = "frontend",
[SMP_SRC_L4CLI] = "client address",
[SMP_SRC_L5CLI] = "client-side connection",
[SMP_SRC_TRACK] = "track counters",
[SMP_SRC_L6REQ] = "request buffer",
[SMP_SRC_HRQHV] = "HTTP request headers",
[SMP_SRC_HRQHP] = "HTTP request",
[SMP_SRC_HRQBO] = "HTTP request body",
[SMP_SRC_BKEND] = "backend",
[SMP_SRC_SERVR] = "server",
[SMP_SRC_L4SRV] = "server address",
[SMP_SRC_L5SRV] = "server-side connection",
[SMP_SRC_L6RES] = "response buffer",
[SMP_SRC_HRSHV] = "HTTP response headers",
[SMP_SRC_HRSHP] = "HTTP response",
[SMP_SRC_HRSBO] = "HTTP response body",
[SMP_SRC_RQFIN] = "request buffer statistics",
[SMP_SRC_RSFIN] = "response buffer statistics",
[SMP_SRC_TXFIN] = "transaction statistics",
[SMP_SRC_SSFIN] = "session statistics",
};
MINOR: sample: provide a function to report the name of a sample check point We need to put names on places where samples are used in order to emit warnings and errors. Let's do that now.
2013-03-25 09:52:41 -04:00
static const char *fetch_ckp_names[SMP_CKP_ENTRIES] = {
[SMP_CKP_FE_CON_ACC] = "frontend tcp-request connection rule",
[SMP_CKP_FE_SES_ACC] = "frontend tcp-request session rule",
[SMP_CKP_FE_REQ_CNT] = "frontend tcp-request content rule",
[SMP_CKP_FE_HRQ_HDR] = "frontend http-request header rule",
[SMP_CKP_FE_HRQ_BDY] = "frontend http-request body rule",
[SMP_CKP_FE_SET_BCK] = "frontend use-backend rule",
[SMP_CKP_BE_REQ_CNT] = "backend tcp-request content rule",
[SMP_CKP_BE_HRQ_HDR] = "backend http-request header rule",
[SMP_CKP_BE_HRQ_BDY] = "backend http-request body rule",
[SMP_CKP_BE_SET_SRV] = "backend use-server, balance or stick-match rule",
[SMP_CKP_BE_SRV_CON] = "server source selection",
[SMP_CKP_BE_RES_CNT] = "backend tcp-response content rule",
[SMP_CKP_BE_HRS_HDR] = "backend http-response header rule",
[SMP_CKP_BE_HRS_BDY] = "backend http-response body rule",
[SMP_CKP_BE_STO_RUL] = "backend stick-store rule",
[SMP_CKP_FE_RES_CNT] = "frontend tcp-response content rule",
[SMP_CKP_FE_HRS_HDR] = "frontend http-response header rule",
[SMP_CKP_FE_HRS_BDY] = "frontend http-response body rule",
[SMP_CKP_FE_LOG_END] = "logs",
};
MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()' into the ACL keyword If the acl keyword is a "fetch", the dedicated parsing function "sample_parse_expr()" is used. Otherwise, the acl parsing function "parse_acl_expr()" is extended to understand the syntax of a series of converters placed after the "fetch" keyword. Before this patch, each acl uses a "struct sample_fetch" and executes it with the "<fetch>->process()" function. Now, the dedicated function "sample_process()" is called. These syntax are now avalaible: acl bad req.hdr(host),lower -m str www http-request redirect prefix /go-away if bad acl bad hdr_beg(host),lower www http-request redirect prefix /go-away if bad
2013-11-21 04:50:10 -05:00
/* This function returns the type of the data returned by the sample_expr.
* It assumes that the <expr> and all of its converters are properly
* initialized.
*/
inline
int smp_expr_output_type(struct sample_expr *expr)
{
struct sample_conv_expr *smp_expr;
if (!LIST_ISEMPTY(&expr->conv_exprs)) {
smp_expr = LIST_PREV(&expr->conv_exprs, struct sample_conv_expr *, list);
return smp_expr->conv->out_type;
}
return expr->fetch->out_type;
}
MEDIUM: samples: use new flags to describe compatibility between fetches and their usages Samples fetches were relying on two flags SMP_CAP_REQ/SMP_CAP_RES to describe whether they were compatible with requests rules or with response rules. This was never reliable because we need a finer granularity (eg: an HTTP request method needs to parse an HTTP request, and is available past this point). Some fetches are also dependant on the context (eg: "hdr" uses request or response depending where it's involved, causing some abiguity). In order to solve this, we need to precisely indicate in fetches what they use, and their users will have to compare with what they have. So now we have a bunch of bits indicating where the sample is fetched in the processing chain, with a few variants indicating for some of them if it is permanent or volatile (eg: an HTTP status is stored into the transaction so it is permanent, despite being caught in the response contents). The fetches also have a second mask indicating their validity domain. This one is computed from a conversion table at registration time, so there is no need for doing it by hand. This validity domain consists in a bitmask with one bit set for each usage point in the processing chain. Some provisions were made for upcoming controls such as connection-based TCP rules which apply on top of the connection layer but before instantiating the session. Then everywhere a fetch is used, the bit for the control point is checked in the fetch's validity domain, and it becomes possible to finely ensure that a fetch will work or not. Note that we need these two separate bitfields because some fetches are usable both in request and response (eg: "hdr", "payload"). So the keyword will have a "use" field made of a combination of several SMP_USE_* values, which will be converted into a wider list of SMP_VAL_* flags. The knowledge of permanent vs dynamic information has disappeared for now, as it was never used. Later we'll probably reintroduce it differently when dealing with variables. Its only use at the moment could have been to avoid caching a dynamic rate measurement, but nothing is cached as of now.
2013-01-07 09:42:20 -05:00
/* fill the trash with a comma-delimited list of source names for the <use> bit
* field which must be composed of a non-null set of SMP_USE_* flags. The return
* value is the pointer to the string in the trash buffer.
*/
const char *sample_src_names(unsigned int use)
{
int bit;
trash.len = 0;
trash.str[0] = '\0';
for (bit = 0; bit < SMP_SRC_ENTRIES; bit++) {
if (!(use & ~((1 << bit) - 1)))
break; /* no more bits */
if (!(use & (1 << bit)))
continue; /* bit not set */
trash.len += snprintf(trash.str + trash.len, trash.size - trash.len, "%s%s",
(use & ((1 << bit) - 1)) ? "," : "",
fetch_src_names[bit]);
}
return trash.str;
}
MINOR: sample: provide a function to report the name of a sample check point We need to put names on places where samples are used in order to emit warnings and errors. Let's do that now.
2013-03-25 09:52:41 -04:00
/* return a pointer to the correct sample checkpoint name, or "unknown" when
* the flags are invalid. Only the lowest bit is used, higher bits are ignored
* if set.
*/
const char *sample_ckp_names(unsigned int use)
{
int bit;
for (bit = 0; bit < SMP_CKP_ENTRIES; bit++)
if (use & (1 << bit))
return fetch_ckp_names[bit];
return "unknown sample check place, please report this bug";
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*
MEDIUM: samples: use new flags to describe compatibility between fetches and their usages Samples fetches were relying on two flags SMP_CAP_REQ/SMP_CAP_RES to describe whether they were compatible with requests rules or with response rules. This was never reliable because we need a finer granularity (eg: an HTTP request method needs to parse an HTTP request, and is available past this point). Some fetches are also dependant on the context (eg: "hdr" uses request or response depending where it's involved, causing some abiguity). In order to solve this, we need to precisely indicate in fetches what they use, and their users will have to compare with what they have. So now we have a bunch of bits indicating where the sample is fetched in the processing chain, with a few variants indicating for some of them if it is permanent or volatile (eg: an HTTP status is stored into the transaction so it is permanent, despite being caught in the response contents). The fetches also have a second mask indicating their validity domain. This one is computed from a conversion table at registration time, so there is no need for doing it by hand. This validity domain consists in a bitmask with one bit set for each usage point in the processing chain. Some provisions were made for upcoming controls such as connection-based TCP rules which apply on top of the connection layer but before instantiating the session. Then everywhere a fetch is used, the bit for the control point is checked in the fetch's validity domain, and it becomes possible to finely ensure that a fetch will work or not. Note that we need these two separate bitfields because some fetches are usable both in request and response (eg: "hdr", "payload"). So the keyword will have a "use" field made of a combination of several SMP_USE_* values, which will be converted into a wider list of SMP_VAL_* flags. The knowledge of permanent vs dynamic information has disappeared for now, as it was never used. Later we'll probably reintroduce it differently when dealing with variables. Its only use at the moment could have been to avoid caching a dynamic rate measurement, but nothing is cached as of now.
2013-01-07 09:42:20 -05:00
* Registers the sample fetch keyword list <kwl> as a list of valid keywords
* for next parsing sessions. The fetch keywords capabilities are also computed
* from their ->use field.
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
*/
MEDIUM: samples: use new flags to describe compatibility between fetches and their usages Samples fetches were relying on two flags SMP_CAP_REQ/SMP_CAP_RES to describe whether they were compatible with requests rules or with response rules. This was never reliable because we need a finer granularity (eg: an HTTP request method needs to parse an HTTP request, and is available past this point). Some fetches are also dependant on the context (eg: "hdr" uses request or response depending where it's involved, causing some abiguity). In order to solve this, we need to precisely indicate in fetches what they use, and their users will have to compare with what they have. So now we have a bunch of bits indicating where the sample is fetched in the processing chain, with a few variants indicating for some of them if it is permanent or volatile (eg: an HTTP status is stored into the transaction so it is permanent, despite being caught in the response contents). The fetches also have a second mask indicating their validity domain. This one is computed from a conversion table at registration time, so there is no need for doing it by hand. This validity domain consists in a bitmask with one bit set for each usage point in the processing chain. Some provisions were made for upcoming controls such as connection-based TCP rules which apply on top of the connection layer but before instantiating the session. Then everywhere a fetch is used, the bit for the control point is checked in the fetch's validity domain, and it becomes possible to finely ensure that a fetch will work or not. Note that we need these two separate bitfields because some fetches are usable both in request and response (eg: "hdr", "payload"). So the keyword will have a "use" field made of a combination of several SMP_USE_* values, which will be converted into a wider list of SMP_VAL_* flags. The knowledge of permanent vs dynamic information has disappeared for now, as it was never used. Later we'll probably reintroduce it differently when dealing with variables. Its only use at the moment could have been to avoid caching a dynamic rate measurement, but nothing is cached as of now.
2013-01-07 09:42:20 -05:00
void sample_register_fetches(struct sample_fetch_kw_list *kwl)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MEDIUM: samples: use new flags to describe compatibility between fetches and their usages Samples fetches were relying on two flags SMP_CAP_REQ/SMP_CAP_RES to describe whether they were compatible with requests rules or with response rules. This was never reliable because we need a finer granularity (eg: an HTTP request method needs to parse an HTTP request, and is available past this point). Some fetches are also dependant on the context (eg: "hdr" uses request or response depending where it's involved, causing some abiguity). In order to solve this, we need to precisely indicate in fetches what they use, and their users will have to compare with what they have. So now we have a bunch of bits indicating where the sample is fetched in the processing chain, with a few variants indicating for some of them if it is permanent or volatile (eg: an HTTP status is stored into the transaction so it is permanent, despite being caught in the response contents). The fetches also have a second mask indicating their validity domain. This one is computed from a conversion table at registration time, so there is no need for doing it by hand. This validity domain consists in a bitmask with one bit set for each usage point in the processing chain. Some provisions were made for upcoming controls such as connection-based TCP rules which apply on top of the connection layer but before instantiating the session. Then everywhere a fetch is used, the bit for the control point is checked in the fetch's validity domain, and it becomes possible to finely ensure that a fetch will work or not. Note that we need these two separate bitfields because some fetches are usable both in request and response (eg: "hdr", "payload"). So the keyword will have a "use" field made of a combination of several SMP_USE_* values, which will be converted into a wider list of SMP_VAL_* flags. The knowledge of permanent vs dynamic information has disappeared for now, as it was never used. Later we'll probably reintroduce it differently when dealing with variables. Its only use at the moment could have been to avoid caching a dynamic rate measurement, but nothing is cached as of now.
2013-01-07 09:42:20 -05:00
struct sample_fetch *sf;
int bit;
for (sf = kwl->kw; sf->kw != NULL; sf++) {
for (bit = 0; bit < SMP_SRC_ENTRIES; bit++)
if (sf->use & (1 << bit))
sf->val |= fetch_cap[bit];
}
LIST_ADDQ(&sample_fetches.list, &kwl->list);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
/*
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Registers the sample format coverstion keyword list <pckl> as a list of valid keywords for next
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
* parsing sessions.
*/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
void sample_register_convs(struct sample_conv_kw_list *pckl)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
LIST_ADDQ(&sample_convs.list, &pckl->list);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
/*
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Returns the pointer on sample fetch keyword structure identified by
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
* string of <len> in buffer <kw>.
*
*/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_fetch *find_sample_fetch(const char *kw, int len)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
int index;
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_fetch_kw_list *kwl;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
list_for_each_entry(kwl, &sample_fetches.list, list) {
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
for (index = 0; kwl->kw[index].kw != NULL; index++) {
if (strncmp(kwl->kw[index].kw, kw, len) == 0 &&
kwl->kw[index].kw[len] == '\0')
return &kwl->kw[index];
}
}
return NULL;
}
/*
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Returns the pointer on sample format conversion keyword structure identified by
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
* string of <len> in buffer <kw>.
*
*/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_conv *find_sample_conv(const char *kw, int len)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
int index;
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_conv_kw_list *kwl;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
list_for_each_entry(kwl, &sample_convs.list, list) {
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
for (index = 0; kwl->kw[index].kw != NULL; index++) {
if (strncmp(kwl->kw[index].kw, kw, len) == 0 &&
kwl->kw[index].kw[len] == '\0')
return &kwl->kw[index];
}
}
return NULL;
}
/******************************************************************/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* Sample casts functions */
MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description We need the pattern fetchers and converters to correctly set the output type so that they can be used by ACL fetchers. By using the sample type instead of the keyword type, we also open the possibility to create some multi-type pattern fetch methods later (eg: "src" being v4/v6). Right now the type in the keyword is used to validate the configuration.
2012-04-23 16:38:26 -04:00
/* Note: these functions do *NOT* set the output type on the */
/* sample, the caller is responsible for doing this on return. */
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/******************************************************************/
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_ip2int(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.uint = ntohl(smp->data.ipv4.s_addr);
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_UINT;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_ip2str(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MINOR: chunks: centralize the trash chunk allocation At the moment, we need trash chunks almost everywhere and the only correctly implemented one is in the sample code. Let's move this to the chunks so that all other places can use this allocator. Additionally, the get_trash_chunk() function now really returns two different chunks. Previously it used to always overwrite the same chunk and point it to a different buffer, which was a bit tricky because it's not obvious that two consecutive results do alias each other.
2012-12-23 14:22:19 -05:00
struct chunk *trash = get_trash_chunk();
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
if (!inet_ntop(AF_INET, (void *)&smp->data.ipv4, trash->str, trash->size))
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 0;
trash->len = strlen(trash->str);
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.str = *trash;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_STR;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_ip2ipv6(struct sample *smp)
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
{
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
v4tov6(&smp->data.ipv6, &smp->data.ipv4);
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_IPV6;
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_ipv62str(struct sample *smp)
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
{
MINOR: chunks: centralize the trash chunk allocation At the moment, we need trash chunks almost everywhere and the only correctly implemented one is in the sample code. Let's move this to the chunks so that all other places can use this allocator. Additionally, the get_trash_chunk() function now really returns two different chunks. Previously it used to always overwrite the same chunk and point it to a different buffer, which was a bit tricky because it's not obvious that two consecutive results do alias each other.
2012-12-23 14:22:19 -05:00
struct chunk *trash = get_trash_chunk();
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
if (!inet_ntop(AF_INET6, (void *)&smp->data.ipv6, trash->str, trash->size))
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
return 0;
trash->len = strlen(trash->str);
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.str = *trash;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_STR;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
return 1;
}
/*
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_ipv62ip(struct sample *smp)
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
{
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
return v6tov4(&smp->data.ipv4, &smp->data.ipv6);
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
}
*/
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_int2ip(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.ipv4.s_addr = htonl(smp->data.uint);
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_IPV4;
return 1;
}
static int c_str2addr(struct sample *smp)
{
if (!buf2ip(smp->data.str.str, smp->data.str.len, &smp->data.ipv4)) {
BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end Applying inet_pton() to input contents is not reliable because the function requires a zero-terminated string. While inet_pton() will stop when contents do not match an IPv6 address anymore, it could theorically read past the end of a buffer if the data to be converted was at the end of a buffer (this cannot happen right now thanks to the reserve at the end of the buffer). At least the conversion does not work. Fix this by using buf2ip6() instead, which copies the string into a padded aread. This bug came with recent commit b805f71 (MEDIUM: sample: let the cast functions set their output type), no backport is needed.
2013-12-11 06:38:57 -05:00
if (!buf2ip6(smp->data.str.str, smp->data.str.len, &smp->data.ipv6))
return 0;
smp->type = SMP_T_IPV6;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end Applying inet_pton() to input contents is not reliable because the function requires a zero-terminated string. While inet_pton() will stop when contents do not match an IPv6 address anymore, it could theorically read past the end of a buffer if the data to be converted was at the end of a buffer (this cannot happen right now thanks to the reserve at the end of the buffer). At least the conversion does not work. Fix this by using buf2ip6() instead, which copies the string into a padded aread. This bug came with recent commit b805f71 (MEDIUM: sample: let the cast functions set their output type), no backport is needed.
2013-12-11 06:38:57 -05:00
return 1;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
}
smp->type = SMP_T_IPV4;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_str2ip(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
if (!buf2ip(smp->data.str.str, smp->data.str.len, &smp->data.ipv4))
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 0;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_IPV4;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_str2ipv6(struct sample *smp)
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
{
BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end Applying inet_pton() to input contents is not reliable because the function requires a zero-terminated string. While inet_pton() will stop when contents do not match an IPv6 address anymore, it could theorically read past the end of a buffer if the data to be converted was at the end of a buffer (this cannot happen right now thanks to the reserve at the end of the buffer). At least the conversion does not work. Fix this by using buf2ip6() instead, which copies the string into a padded aread. This bug came with recent commit b805f71 (MEDIUM: sample: let the cast functions set their output type), no backport is needed.
2013-12-11 06:38:57 -05:00
if (!buf2ip6(smp->data.str.str, smp->data.str.len, &smp->data.ipv6))
return 0;
smp->type = SMP_T_IPV6;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end Applying inet_pton() to input contents is not reliable because the function requires a zero-terminated string. While inet_pton() will stop when contents do not match an IPv6 address anymore, it could theorically read past the end of a buffer if the data to be converted was at the end of a buffer (this cannot happen right now thanks to the reserve at the end of the buffer). At least the conversion does not work. Fix this by using buf2ip6() instead, which copies the string into a padded aread. This bug came with recent commit b805f71 (MEDIUM: sample: let the cast functions set their output type), no backport is needed.
2013-12-11 06:38:57 -05:00
return 1;
[MEDIUM] IPv6 support for stick-tables Since IPv6 is a different type than IPv4, the pattern fetch functions src6 and dst6 were added. IPv6 stick-tables can also fetch IPv4 addresses with src and dst. In this case, the IPv4 addresses are mapped to their IPv6 counterpart, according to RFC 4291.
2011-03-24 06:09:31 -04:00
}
MEDIUM: sample: change the behavior of the bin2str cast The bin2str cast gives the hexadecimal representation of the binary content when it is used as string. This was inherited from the stick-table casts without realizing that it was a mistake. Indeed, it breaks string processing on binary contents, preventing any _reg, _beg, etc from working. For example, with an HTTP GET request, the fetch "req.payload(0,3)" returns the 3 bytes "G", "E", and "T" in binary. If this fetch is used with regex, it is automatically converted to "474554" and the regex is applied on this string, so it never matches. This commit changes the cast so that bin2str does not convert the contents anymore, and returns a string type. The contents can thus be matched as is, and the NULL character continues to mark the end of the string to avoid any issue with some string-based functions. This commit could almost have been marked as a bug fix since it does what the doc says. Note that in case someone would rely on the hex encoding, then the same behaviour could be achieved by appending ",hex" after the sample fetch function (brought by previous patch).
2014-03-12 10:07:59 -04:00
/* The sample is always copied into a new one so that smp->size is always
* valid. The NULL char always enforces the end of string if it is met.
*/
MINOR: sample: manage binary to string type convertion in stick-table and samples. Binary type is converted to a null terminated hexa string.
2012-10-17 07:36:06 -04:00
static int c_bin2str(struct sample *smp)
{
MINOR: chunks: centralize the trash chunk allocation At the moment, we need trash chunks almost everywhere and the only correctly implemented one is in the sample code. Let's move this to the chunks so that all other places can use this allocator. Additionally, the get_trash_chunk() function now really returns two different chunks. Previously it used to always overwrite the same chunk and point it to a different buffer, which was a bit tricky because it's not obvious that two consecutive results do alias each other.
2012-12-23 14:22:19 -05:00
struct chunk *trash = get_trash_chunk();
MINOR: sample: manage binary to string type convertion in stick-table and samples. Binary type is converted to a null terminated hexa string.
2012-10-17 07:36:06 -04:00
unsigned char c;
int ptr = 0;
MEDIUM: sample: change the behavior of the bin2str cast The bin2str cast gives the hexadecimal representation of the binary content when it is used as string. This was inherited from the stick-table casts without realizing that it was a mistake. Indeed, it breaks string processing on binary contents, preventing any _reg, _beg, etc from working. For example, with an HTTP GET request, the fetch "req.payload(0,3)" returns the 3 bytes "G", "E", and "T" in binary. If this fetch is used with regex, it is automatically converted to "474554" and the regex is applied on this string, so it never matches. This commit changes the cast so that bin2str does not convert the contents anymore, and returns a string type. The contents can thus be matched as is, and the NULL character continues to mark the end of the string to avoid any issue with some string-based functions. This commit could almost have been marked as a bug fix since it does what the doc says. Note that in case someone would rely on the hex encoding, then the same behaviour could be achieved by appending ",hex" after the sample fetch function (brought by previous patch).
2014-03-12 10:07:59 -04:00
while (ptr < smp->data.str.len) {
c = smp->data.str.str[ptr];
if (!c)
break;
trash->str[ptr] = c;
ptr++;
MINOR: sample: manage binary to string type convertion in stick-table and samples. Binary type is converted to a null terminated hexa string.
2012-10-17 07:36:06 -04:00
}
MEDIUM: sample: change the behavior of the bin2str cast The bin2str cast gives the hexadecimal representation of the binary content when it is used as string. This was inherited from the stick-table casts without realizing that it was a mistake. Indeed, it breaks string processing on binary contents, preventing any _reg, _beg, etc from working. For example, with an HTTP GET request, the fetch "req.payload(0,3)" returns the 3 bytes "G", "E", and "T" in binary. If this fetch is used with regex, it is automatically converted to "474554" and the regex is applied on this string, so it never matches. This commit changes the cast so that bin2str does not convert the contents anymore, and returns a string type. The contents can thus be matched as is, and the NULL character continues to mark the end of the string to avoid any issue with some string-based functions. This commit could almost have been marked as a bug fix since it does what the doc says. Note that in case someone would rely on the hex encoding, then the same behaviour could be achieved by appending ",hex" after the sample fetch function (brought by previous patch).
2014-03-12 10:07:59 -04:00
trash->len = ptr;
trash->str[ptr] = 0;
MINOR: sample: manage binary to string type convertion in stick-table and samples. Binary type is converted to a null terminated hexa string.
2012-10-17 07:36:06 -04:00
smp->data.str = *trash;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_STR;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
MINOR: sample: manage binary to string type convertion in stick-table and samples. Binary type is converted to a null terminated hexa string.
2012-10-17 07:36:06 -04:00
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_int2str(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MINOR: chunks: centralize the trash chunk allocation At the moment, we need trash chunks almost everywhere and the only correctly implemented one is in the sample code. Let's move this to the chunks so that all other places can use this allocator. Additionally, the get_trash_chunk() function now really returns two different chunks. Previously it used to always overwrite the same chunk and point it to a different buffer, which was a bit tricky because it's not obvious that two consecutive results do alias each other.
2012-12-23 14:22:19 -05:00
struct chunk *trash = get_trash_chunk();
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
char *pos;
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
pos = ultoa_r(smp->data.uint, trash->str, trash->size);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
if (!pos)
return 0;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
trash->size = trash->size - (pos - trash->str);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
trash->str = pos;
trash->len = strlen(pos);
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.str = *trash;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_STR;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 1;
}
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
/* This function duplicates data and removes the flag "const". */
int smp_dup(struct sample *smp)
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
{
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
struct chunk *trash;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
/* If the const flag is not set, we don't need to duplicate the
* pattern as it can be modified in place.
*/
if (!(smp->flags & SMP_F_CONST))
return 1;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
switch (smp->type) {
case SMP_T_BOOL:
case SMP_T_UINT:
case SMP_T_SINT:
case SMP_T_ADDR:
case SMP_T_IPV4:
case SMP_T_IPV6:
/* These type are not const. */
break;
case SMP_T_STR:
case SMP_T_BIN:
/* Duplicate data. */
trash = get_trash_chunk();
trash->len = smp->data.str.len < trash->size ? smp->data.str.len : trash->size;
memcpy(trash->str, smp->data.str.str, trash->len);
smp->data.str = *trash;
break;
default:
/* Other cases are unexpected. */
return 0;
}
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
/* remove const flag */
smp->flags &= ~SMP_F_CONST;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
return 1;
}
MINOR: sample: dont call the sample cast function "c_none" If the cast function to execute is c_none, dont execute it and return true. The function c_none, do nothing. This save a call.
2013-12-14 08:55:04 -05:00
int c_none(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
return 1;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
static int c_str2int(struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
int i;
uint32_t ret = 0;
BUG/MINOR: sample: The c_str2int converter does not fail if the entry is not an integer If the string not start with a number, the converter fails. In other, it converts a maximum of characters to a number and stop to the first character that not match a number.
2014-01-27 12:20:48 -05:00
if (smp->data.str.len == 0)
return 0;
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
for (i = 0; i < smp->data.str.len; i++) {
uint32_t val = smp->data.str.str[i] - '0';
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
BUG/MINOR: sample: The c_str2int converter does not fail if the entry is not an integer If the string not start with a number, the converter fails. In other, it converts a maximum of characters to a number and stop to the first character that not match a number.
2014-01-27 12:20:48 -05:00
if (val > 9) {
if (i == 0)
return 0;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
break;
BUG/MINOR: sample: The c_str2int converter does not fail if the entry is not an integer If the string not start with a number, the converter fails. In other, it converts a maximum of characters to a number and stop to the first character that not match a number.
2014-01-27 12:20:48 -05:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
ret = ret * 10 + val;
}
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.uint = ret;
MEDIUM: sample: let the cast functions set their output type This patch allows each sample cast function to specify the sample output type. The goal is to be able to emit an output type IPv4 or IPv6 depending on what is found in the input if the next converter is able to process them both. The patch also adds a new pseudo type called "ADDR". This type is an alias for IPV4 and IPV6 which is only used as an input type by converters who want to express their compatibility with both address formats. It may not be emitted. The goal is to unify as much as possible the processing of IPv4 and IPv6 in order not to add extra keywords for the maps which act as converters, but will match samples like ACLs do with their patterns.
2013-11-26 14:47:54 -05:00
smp->type = SMP_T_UINT;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return 1;
}
MEDIUM: sample/http_proto: Add new type called method The method are actuelly stored using two types. Integer if the method is known and string if the method is not known. The fetch is declared as UINT, but in some case it can provides STR. This patch create new type called METH. This type contain interge for known method and string for the other methods. It can be used with automatic converters. The pattern matching can expect method. During the free or prune function, http_meth pettern is freed. This patch initialise the freed pointer to NULL.
2013-12-16 19:10:10 -05:00
static int c_str2meth(struct sample *smp)
{
enum http_meth_t meth;
int len;
meth = find_http_meth(smp->data.str.str, smp->data.str.len);
if (meth == HTTP_METH_OTHER) {
len = smp->data.str.len;
smp->data.meth.str.str = smp->data.str.str;
smp->data.meth.str.len = len;
}
else
smp->flags &= ~SMP_F_CONST;
smp->data.meth.meth = meth;
smp->type = SMP_T_METH;
return 1;
}
static int c_meth2str(struct sample *smp)
{
int len;
enum http_meth_t meth;
if (smp->data.meth.meth == HTTP_METH_OTHER) {
/* The method is unknown. Copy the original pointer. */
len = smp->data.meth.str.len;
smp->data.str.str = smp->data.meth.str.str;
smp->data.str.len = len;
smp->type = SMP_T_STR;
}
else if (smp->data.meth.meth < HTTP_METH_OTHER) {
/* The method is known, copy the pointer containing the string. */
meth = smp->data.meth.meth;
smp->data.str.str = http_known_methods[meth].name;
smp->data.str.len = http_known_methods[meth].len;
smp->flags |= SMP_F_CONST;
smp->type = SMP_T_STR;
}
else {
/* Unknown method */
return 0;
}
return 1;
}
MINOR: sample: allow IP address to cast to binary IP addresses are a perfect example of fixed size data which we could cast to binary, still it was not allowed by lack of cast function, eventhough the opposite was allowed in ACLs. Make that possible both in sample expressions and in stick tables.
2014-07-15 15:03:26 -04:00
static int c_addr2bin(struct sample *smp)
{
struct chunk *chk = get_trash_chunk();
if (smp->type == SMP_T_IPV4) {
chk->len = 4;
memcpy(chk->str, &smp->data.ipv4, chk->len);
}
else if (smp->type == SMP_T_IPV6) {
chk->len = 16;
memcpy(chk->str, &smp->data.ipv6, chk->len);
}
else
return 0;
smp->data.str = *chk;
smp->type = SMP_T_BIN;
return 1;
}
MINOR: sample: allow integers to cast to binary Doing so finally allows to apply the hex converter to integers as well. Note that all integers are represented in 32-bit, big endian so that their conversion remains human readable and portable. A later improvement to the hex converter could be to make it trim leading zeroes, and/or to only report a number of least significant bytes.
2014-07-15 15:19:08 -04:00
static int c_int2bin(struct sample *smp)
{
struct chunk *chk = get_trash_chunk();
*(unsigned int *)chk->str = htonl(smp->data.uint);
chk->len = 4;
smp->data.str = *chk;
smp->type = SMP_T_BIN;
return 1;
}
MINOR: sample: allow IP address to cast to binary IP addresses are a perfect example of fixed size data which we could cast to binary, still it was not allowed by lack of cast function, eventhough the opposite was allowed in ACLs. Make that possible both in sample expressions and in stick tables.
2014-07-15 15:03:26 -04:00
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*****************************************************************/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* Sample casts matrix: */
/* sample_casts[from type][to type] */
/* NULL pointer used for impossible sample casts */
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*****************************************************************/
MINOR: sample: export sample_casts just export the sample cast matrix "sample_casts" to prepare the generic sample conversion parser.
2013-11-21 04:53:12 -05:00
sample_cast_fct sample_casts[SMP_TYPES][SMP_TYPES] = {
MEDIUM: sample/http_proto: Add new type called method The method are actuelly stored using two types. Integer if the method is known and string if the method is not known. The fetch is declared as UINT, but in some case it can provides STR. This patch create new type called METH. This type contain interge for known method and string for the other methods. It can be used with automatic converters. The pattern matching can expect method. During the free or prune function, http_meth pettern is freed. This patch initialise the freed pointer to NULL.
2013-12-16 19:10:10 -05:00
/* to: BOOL UINT SINT ADDR IPV4 IPV6 STR BIN METH */
/* from: BOOL */ { c_none, c_none, c_none, NULL, NULL, NULL, c_int2str, NULL, NULL, },
MINOR: sample: allow integers to cast to binary Doing so finally allows to apply the hex converter to integers as well. Note that all integers are represented in 32-bit, big endian so that their conversion remains human readable and portable. A later improvement to the hex converter could be to make it trim leading zeroes, and/or to only report a number of least significant bytes.
2014-07-15 15:19:08 -04:00
/* UINT */ { c_none, c_none, c_none, c_int2ip, c_int2ip, NULL, c_int2str, c_int2bin, NULL, },
/* SINT */ { c_none, c_none, c_none, c_int2ip, c_int2ip, NULL, c_int2str, c_int2bin, NULL, },
MEDIUM: sample/http_proto: Add new type called method The method are actuelly stored using two types. Integer if the method is known and string if the method is not known. The fetch is declared as UINT, but in some case it can provides STR. This patch create new type called METH. This type contain interge for known method and string for the other methods. It can be used with automatic converters. The pattern matching can expect method. During the free or prune function, http_meth pettern is freed. This patch initialise the freed pointer to NULL.
2013-12-16 19:10:10 -05:00
/* ADDR */ { NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, },
MINOR: sample: allow IP address to cast to binary IP addresses are a perfect example of fixed size data which we could cast to binary, still it was not allowed by lack of cast function, eventhough the opposite was allowed in ACLs. Make that possible both in sample expressions and in stick tables.
2014-07-15 15:03:26 -04:00
/* IPV4 */ { NULL, c_ip2int, c_ip2int, c_none, c_none, c_ip2ipv6, c_ip2str, c_addr2bin, NULL, },
/* IPV6 */ { NULL, NULL, NULL, c_none, NULL, c_none, c_ipv62str, c_addr2bin, NULL, },
MEDIUM: sample/http_proto: Add new type called method The method are actuelly stored using two types. Integer if the method is known and string if the method is not known. The fetch is declared as UINT, but in some case it can provides STR. This patch create new type called METH. This type contain interge for known method and string for the other methods. It can be used with automatic converters. The pattern matching can expect method. During the free or prune function, http_meth pettern is freed. This patch initialise the freed pointer to NULL.
2013-12-16 19:10:10 -05:00
/* STR */ { c_str2int, c_str2int, c_str2int, c_str2addr, c_str2ip, c_str2ipv6, c_none, c_none, c_str2meth, },
/* BIN */ { NULL, NULL, NULL, NULL, NULL, NULL, c_bin2str, c_none, c_str2meth, },
/* METH */ { NULL, NULL, NULL, NULL, NULL, NULL, c_meth2str, c_meth2str, c_none, },
[CLEANUP] stick_table: move pattern to key functions to stick_table.c pattern.c depended on stick_table while in fact it should be the opposite. So we move from pattern.c everything related to stick_tables and invert the dependency. That way the code becomes more logical and intuitive.
2010-06-06 07:22:23 -04:00
};
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Parse a sample expression configuration:
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
* fetch keyword followed by format conversion keywords.
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Returns a pointer on allocated sample expression structure.
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
* The caller must have set al->ctx.
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
*/
MINOR: configuration: File and line propagation This patch permits to communicate file and line of the configuration file at the configuration parser.
2014-02-11 08:00:19 -05:00
struct sample_expr *sample_parse_expr(char **str, int *idx, const char *file, int line, char **err_msg, struct arg_list *al)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
const char *begw; /* beginning of word */
const char *endw; /* end of word */
const char *endt; /* end of term */
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_expr *expr;
struct sample_fetch *fetch;
struct sample_conv *conv;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
unsigned long prev_type;
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
char *fkw = NULL;
char *ckw = NULL;
BUG/MEDIUM: sample: simplify and fix the argument parsing Some errors may be reported about missing mandatory arguments when some sample fetch arguments are marked as mandatory and implicit (eg: proxy names such as in table_cnt or be_conn). In practice the argument parser already handles all the situations very well, it's just that the sample fetch parser want to go beyond its role and starts some controls that it should not do. Simply removing these useless controls lets make_arg_list() create the correct argument types when such types are encountered. This regression was introduced by the recent use of sample_parse_expr() in ACLs which makes use of its own argument parser, while previously the arguments were parsed in the ACL function itself. No backport is needed.
2013-12-12 18:40:11 -05:00
int err_arg;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
begw = str[*idx];
for (endw = begw; *endw && *endw != '(' && *endw != ','; endw++);
if (endw == begw) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "missing fetch method");
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
/* keep a copy of the current fetch keyword for error reporting */
fkw = my_strndup(begw, endw - begw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
fetch = find_sample_fetch(begw, endw - begw);
if (!fetch) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "unknown fetch method '%s'", fkw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
endt = endw;
if (*endt == '(') {
BUG/MEDIUM: sample: simplify and fix the argument parsing Some errors may be reported about missing mandatory arguments when some sample fetch arguments are marked as mandatory and implicit (eg: proxy names such as in table_cnt or be_conn). In practice the argument parser already handles all the situations very well, it's just that the sample fetch parser want to go beyond its role and starts some controls that it should not do. Simply removing these useless controls lets make_arg_list() create the correct argument types when such types are encountered. This regression was introduced by the recent use of sample_parse_expr() in ACLs which makes use of its own argument parser, while previously the arguments were parsed in the ACL function itself. No backport is needed.
2013-12-12 18:40:11 -05:00
/* look for the end of this term and skip the opening parenthesis */
endt = ++endw;
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
while (*endt && *endt != ')')
endt++;
if (*endt != ')') {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "missing closing ')' after arguments to fetch keyword '%s'", fkw);
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
/* At this point, we have :
* - begw : beginning of the keyword
BUG/MEDIUM: sample: simplify and fix the argument parsing Some errors may be reported about missing mandatory arguments when some sample fetch arguments are marked as mandatory and implicit (eg: proxy names such as in table_cnt or be_conn). In practice the argument parser already handles all the situations very well, it's just that the sample fetch parser want to go beyond its role and starts some controls that it should not do. Simply removing these useless controls lets make_arg_list() create the correct argument types when such types are encountered. This regression was introduced by the recent use of sample_parse_expr() in ACLs which makes use of its own argument parser, while previously the arguments were parsed in the ACL function itself. No backport is needed.
2013-12-12 18:40:11 -05:00
* - endw : end of the keyword, first character not part of keyword
* nor the opening parenthesis (so first character of args
* if present).
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
* - endt : end of the term (=endw or last parenthesis if args are present)
*/
if (fetch->out_type >= SMP_TYPES) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "returns type of fetch method '%s' is unknown", fkw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
prev_type = fetch->out_type;
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
expr = calloc(1, sizeof(struct sample_expr));
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
if (!expr)
goto out_error;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
LIST_INIT(&(expr->conv_exprs));
expr->fetch = fetch;
MEDIUM: sample: pass an empty list instead of a null for fetch args ACL and sample fetches use args list and it is really not convenient to check for null args everywhere. Now for empty args we pass a constant list of end of lists. It will allow us to remove many useless checks.
2012-10-19 13:49:09 -04:00
expr->arg_p = empty_arg_list;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
BUG/MEDIUM: sample: simplify and fix the argument parsing Some errors may be reported about missing mandatory arguments when some sample fetch arguments are marked as mandatory and implicit (eg: proxy names such as in table_cnt or be_conn). In practice the argument parser already handles all the situations very well, it's just that the sample fetch parser want to go beyond its role and starts some controls that it should not do. Simply removing these useless controls lets make_arg_list() create the correct argument types when such types are encountered. This regression was introduced by the recent use of sample_parse_expr() in ACLs which makes use of its own argument parser, while previously the arguments were parsed in the ACL function itself. No backport is needed.
2013-12-12 18:40:11 -05:00
/* Note that we call the argument parser even with an empty string,
* this allows it to automatically create entries for mandatory
* implicit arguments (eg: local proxy name).
*/
al->kw = expr->fetch->kw;
al->conv = NULL;
if (make_arg_list(endw, endt - endw, fetch->arg_mask, &expr->arg_p, err_msg, NULL, &err_arg, al) < 0) {
memprintf(err_msg, "fetch method '%s' : %s", fkw, *err_msg);
goto out_error;
}
MEDIUM: sample: pass an empty list instead of a null for fetch args ACL and sample fetches use args list and it is really not convenient to check for null args everywhere. Now for empty args we pass a constant list of end of lists. It will allow us to remove many useless checks.
2012-10-19 13:49:09 -04:00
BUG/MEDIUM: sample: simplify and fix the argument parsing Some errors may be reported about missing mandatory arguments when some sample fetch arguments are marked as mandatory and implicit (eg: proxy names such as in table_cnt or be_conn). In practice the argument parser already handles all the situations very well, it's just that the sample fetch parser want to go beyond its role and starts some controls that it should not do. Simply removing these useless controls lets make_arg_list() create the correct argument types when such types are encountered. This regression was introduced by the recent use of sample_parse_expr() in ACLs which makes use of its own argument parser, while previously the arguments were parsed in the ACL function itself. No backport is needed.
2013-12-12 18:40:11 -05:00
if (!expr->arg_p) {
expr->arg_p = empty_arg_list;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
BUG/MEDIUM: sample: simplify and fix the argument parsing Some errors may be reported about missing mandatory arguments when some sample fetch arguments are marked as mandatory and implicit (eg: proxy names such as in table_cnt or be_conn). In practice the argument parser already handles all the situations very well, it's just that the sample fetch parser want to go beyond its role and starts some controls that it should not do. Simply removing these useless controls lets make_arg_list() create the correct argument types when such types are encountered. This regression was introduced by the recent use of sample_parse_expr() in ACLs which makes use of its own argument parser, while previously the arguments were parsed in the ACL function itself. No backport is needed.
2013-12-12 18:40:11 -05:00
else if (fetch->val_args && !fetch->val_args(expr->arg_p, err_msg)) {
memprintf(err_msg, "invalid args in fetch method '%s' : %s", fkw, *err_msg);
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
goto out_error;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
/* Now process the converters if any. We have two supported syntaxes
* for the converters, which can be combined :
* - comma-delimited list of converters just after the keyword and args ;
* - one converter per keyword
* The combination allows to have each keyword being a comma-delimited
* series of converters.
*
* We want to process the former first, then the latter. For this we start
* from the beginning of the supposed place in the exiting conv chain, which
* starts at the last comma (endt).
*/
while (1) {
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_conv_expr *conv_expr;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
if (*endt == ')') /* skip last closing parenthesis */
endt++;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
if (*endt && *endt != ',') {
if (ckw)
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "missing comma after conv keyword '%s'", ckw);
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
else
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "missing comma after fetch keyword '%s'", fkw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
while (*endt == ',') /* then trailing commas */
endt++;
begw = endt; /* start of conv keyword */
if (!*begw) {
/* none ? skip to next string */
(*idx)++;
begw = str[*idx];
if (!begw || !*begw)
break;
}
for (endw = begw; *endw && *endw != '(' && *endw != ','; endw++);
free(ckw);
ckw = my_strndup(begw, endw - begw);
conv = find_sample_conv(begw, endw - begw);
if (!conv) {
/* we found an isolated keyword that we don't know, it's not ours */
if (begw == str[*idx])
break;
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "unknown conv method '%s'", ckw);
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
goto out_error;
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
endt = endw;
if (*endt == '(') {
/* look for the end of this term */
while (*endt && *endt != ')')
endt++;
if (*endt != ')') {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "syntax error: missing ')' after conv keyword '%s'", ckw);
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
}
if (conv->in_type >= SMP_TYPES || conv->out_type >= SMP_TYPES) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "returns type of conv method '%s' is unknown", ckw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/* If impossible type conversion */
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
if (!sample_casts[prev_type][conv->in_type]) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "conv method '%s' cannot be applied", ckw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
goto out_error;
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
prev_type = conv->out_type;
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
conv_expr = calloc(1, sizeof(struct sample_conv_expr));
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
if (!conv_expr)
goto out_error;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
LIST_ADDQ(&(expr->conv_exprs), &(conv_expr->list));
conv_expr->conv = conv;
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
if (endt != endw) {
MEDIUM: pattern: report the precise argument parsing error when known. The argument parser knows what exact error it has faced, and the pattern parser is able to report errors, so let's make use of it. From now on, it becomes possible to detect such things : $ ./haproxy -db -f echo5.cfg [ALERT] 110/160344 (4791) : parsing [echo5.cfg:38] : 'stick': invalid arg 2 in fetch method 'payload' : Missing arguments (got 1/2), type 'unsigned integer' expected. [ALERT] 110/160344 (4791) : parsing [echo5.cfg:39] : 'stick': invalid args in fetch method 'payload' : payload length must be > 0. [ALERT] 110/160344 (4791) : parsing [echo5.cfg:40] : 'stick': invalid arg 3 in fetch method 'payload_lv' : Failed to parse 'x' as type 'signed integer'. [ALERT] 110/160344 (4791) : parsing [echo5.cfg:41] : 'stick': invalid arg 4 in fetch method 'payload_lv' : End of arguments expected at ',13'. [ALERT] 110/160344 (4791) : Error(s) found in configuration file : echo5.cfg [ALERT] 110/160344 (4791) : Fatal errors found in configuration.
2012-04-20 10:04:47 -04:00
int err_arg;
MEDIUM: pattern: add an argument validation callback to pattern descriptors This is used to validate that arguments are coherent. For instance, payload_lv expects that the last arg (if any) is not more negative than the sum of the first two. The error is reported if any.
2012-04-20 09:52:36 -04:00
MEDIUM: pattern: use the standard arg parser We don't need the pattern-specific args parsers anymore, make use of the common parser instead. We still need to improve this by adding a validation function to report abnormal argument values or combinations. We don't report precise parsing errors yet but this was not previously done either.
2012-04-20 08:45:49 -04:00
if (!conv->arg_mask) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "conv method '%s' does not support any args", ckw);
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
goto out_error;
}
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
al->kw = expr->fetch->kw;
al->conv = conv_expr->conv->kw;
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
if (make_arg_list(endw + 1, endt - endw - 1, conv->arg_mask, &conv_expr->arg_p, err_msg, NULL, &err_arg, al) < 0) {
memprintf(err_msg, "invalid arg %d in conv method '%s' : %s", err_arg+1, ckw, *err_msg);
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
goto out_error;
}
MEDIUM: pattern: add an argument validation callback to pattern descriptors This is used to validate that arguments are coherent. For instance, payload_lv expects that the last arg (if any) is not more negative than the sum of the first two. The error is reported if any.
2012-04-20 09:52:36 -04:00
MEDIUM: sample: pass an empty list instead of a null for fetch args ACL and sample fetches use args list and it is really not convenient to check for null args everywhere. Now for empty args we pass a constant list of end of lists. It will allow us to remove many useless checks.
2012-10-19 13:49:09 -04:00
if (!conv_expr->arg_p)
conv_expr->arg_p = empty_arg_list;
MINOR: configuration: File and line propagation This patch permits to communicate file and line of the configuration file at the configuration parser.
2014-02-11 08:00:19 -05:00
if (conv->val_args && !conv->val_args(conv_expr->arg_p, conv, file, line, err_msg)) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "invalid args in conv method '%s' : %s", ckw, *err_msg);
MEDIUM: pattern: add an argument validation callback to pattern descriptors This is used to validate that arguments are coherent. For instance, payload_lv expects that the last arg (if any) is not more negative than the sum of the first two. The error is reported if any.
2012-04-20 09:52:36 -04:00
goto out_error;
}
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
}
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
else if (ARGM(conv->arg_mask)) {
MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors Doing so ensures that we're consistent between all the functions in the whole chain. This is important so that we can extract the argument parsing from this function.
2013-12-12 17:16:54 -05:00
memprintf(err_msg, "missing args for conv method '%s'", ckw);
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
goto out_error;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
}
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
out:
free(fkw);
free(ckw);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return expr;
out_error:
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* TODO: prune_sample_expr(expr); */
MEDIUM: sample: handle comma-delimited converter list We now support having a comma-delimited converter list, which can start right after the fetch keyword. The immediate benefit is that it allows to use converters in log-format expressions, for example : set-header source-net %[src,ipmask(24)] The parser is also slightly improved and should be more resilient against configuration errors. Also, optional arguments in converters were mistakenly not allowed till now, so this was fixed.
2013-07-24 09:34:19 -04:00
expr = NULL;
goto out;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
/*
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Process a fetch + format conversion of defined by the sample expression <expr>
MEDIUM: acl/pattern: use the same direction scheme Patterns were using a bitmask to indicate if request or response was desired in fetch functions and keywords. ACLs were using a bitmask in fetch keywords and a single bit in fetch functions. ACLs were also using an ACL_PARTIAL bit in fetch functions indicating that a non-final fetch was performed, which was an abuse of the existing direction flag. The change now consists in using : - a capabilities field for fetch keywords => SMP_CAP_REQ/RES to indicate if a keyword supports requests, responses, both, etc... - an option field for fetch functions to indicate what the caller expects (request/response, final/non-final) The ACL_PARTIAL bit was reversed to get SMP_OPT_FINAL as it's more explicit to know we're working on a final buffer than on a non-final one. ACL_DIR_* were removed, as well as PATTERN_FETCH_*. L4 fetches were improved to support being called on responses too since they're still available. The <dir> field of all fetch functions was changed to <opt> which is now unsigned. The patch is large but mostly made of cosmetic changes to accomodate this, as almost no logic change happened.
2012-04-25 04:13:36 -04:00
* on request or response considering the <opt> parameter.
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* Returns a pointer on a typed sample structure containing the result or NULL if
* sample is not found or when format conversion failed.
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
* If <p> is not null, function returns results in structure pointed by <p>.
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
* If <p> is null, functions returns a pointer on a static sample structure.
MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description We need the pattern fetchers and converters to correctly set the output type so that they can be used by ACL fetchers. By using the sample type instead of the keyword type, we also open the possibility to create some multi-type pattern fetch methods later (eg: "src" being v4/v6). Right now the type in the keyword is used to validate the configuration.
2012-04-23 16:38:26 -04:00
*
* Note: the fetch functions are required to properly set the return type. The
* conversion functions must do so too. However the cast functions do not need
* to since they're made to cast mutiple types according to what is required.
BUG/MAJOR: tcp: fix a possible busy spinning loop in content track-sc* As a consequence of various recent changes on the sample conversion, a corner case has emerged where it is possible to wait forever for a sample in track-sc*. The issue is caused by the fact that functions relying on sample_process() don't all exactly work the same regarding the SMP_F_MAY_CHANGE flag and the output result. Here it was possible to wait forever for an output sample from stktable_fetch_key() without checking the SMP_OPT_FINAL flag. As a result, if the client connects and closes without sending the data and haproxy expects a sample which is capable of coming, it will ignore this impossible case and will continue to wait. This change adds control for SMP_OPT_FINAL before waiting for extra data. The various relevant functions have been better documented regarding their output values. This fix must be backported to 1.5 since it appeared there.
2014-07-30 02:56:35 -04:00
*
* The caller may indicate in <opt> if it considers the result final or not.
* The caller needs to check the SMP_F_MAY_CHANGE flag in p->flags to verify
* if the result is stable or not, according to the following table :
*
* return MAY_CHANGE FINAL Meaning for the sample
* NULL 0 * Not present and will never be (eg: header)
* NULL 1 0 Not present yet, could change (eg: POST param)
* NULL 1 1 Not present yet, will not change anymore
* smp 0 * Present and will not change (eg: header)
* smp 1 0 Present, may change (eg: request length)
* smp 1 1 Present, last known value (eg: request length)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
*/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample *sample_process(struct proxy *px, struct session *l4, void *l7,
unsigned int opt,
struct sample_expr *expr, struct sample *p)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
struct sample_conv_expr *conv_expr;
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MINOR: sample: fix sample_process handling of unstable data sample_process() used to return NULL on changing data, regardless of the SMP_OPT_FINAL flag. Let's change this so that it is now possible to include such data in logs or HTTP headers. Also, one unconvenient thing was that it used to always set the sample flags to zero, making it incompatible with ACLs which may need to call it multiple times. Only do this for locally-allocated samples.
2013-07-25 06:02:38 -04:00
if (p == NULL) {
MINOR: pattern: replace struct pattern with struct sample This change is pretty minor. Struct pattern is only used for pattern_process() now so changing it to use the common type is quite obvious. It's worth noting that the last argument of pattern_process() is never used so the function is self-sufficient. Note that pattern_process() does not initialize the pattern at all before calling fetch->process(), and that minimal initialization will be required when we later change the argument for the sample.
2012-04-23 15:35:11 -04:00
p = &temp_smp;
BUG/MAJOR: sample: correctly reinitialize sample fetch context before calling sample_process() We used to only clear flags when reusing the static sample before calling sample_process(), but that's not enough because there's a context in samples that can be used by some fetch functions such as auth, headers and cookies, and not reinitializing it risks that a pointer of a different type is used in the wrong context. An example configuration which triggers the case consists in mixing hdr() and http_auth_group() which both make use of contexts : http-request add-header foo2 %[hdr(host)],%[http_auth_group(foo)] The solution is simple, initialize all the sample and not just the flags. This fix must be backported into 1.5 since it was introduced in 1.5-dev19.
2014-06-25 10:56:41 -04:00
memset(p, 0, sizeof(*p));
MINOR: sample: fix sample_process handling of unstable data sample_process() used to return NULL on changing data, regardless of the SMP_OPT_FINAL flag. Let's change this so that it is now possible to include such data in logs or HTTP headers. Also, one unconvenient thing was that it used to always set the sample flags to zero, making it incompatible with ACLs which may need to call it multiple times. Only do this for locally-allocated samples.
2013-07-25 06:02:38 -04:00
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
MEDIUM: sample: systematically pass the keyword pointer to the keyword We're having a lot of duplicate code just because of minor variants between fetch functions that could be dealt with if the functions had the pointer to the original keyword, so let's pass it as the last argument. An earlier version used to pass a pointer to the sample_fetch element, but this is not the best solution for two reasons : - fetch functions will solely rely on the keyword string - some other smp_fetch_* users do not have the pointer to the original keyword and were forced to pass NULL. So finally we're passing a pointer to the keyword as a const char *, which perfectly fits the original purpose.
2013-07-22 10:29:32 -04:00
if (!expr->fetch->process(px, l4, l7, opt, expr->arg_p, p, expr->fetch->kw))
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return NULL;
list_for_each_entry(conv_expr, &expr->conv_exprs, list) {
MEDIUM: pattern: ensure that sample types always cast into other types. We want to ensure that a dynamically returned type will always have a cast before calling the cast function. This is done in pattern_process() and in stktable_fetch_key().
2012-04-25 11:21:49 -04:00
/* we want to ensure that p->type can be casted into
* conv_expr->conv->in_type. We have 3 possibilities :
* - NULL => not castable.
* - c_none => nothing to do (let's optimize it)
* - other => apply cast and prepare to fail
*/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
if (!sample_casts[p->type][conv_expr->conv->in_type])
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return NULL;
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
if (sample_casts[p->type][conv_expr->conv->in_type] != c_none &&
!sample_casts[p->type][conv_expr->conv->in_type](p))
MEDIUM: pattern: ensure that sample types always cast into other types. We want to ensure that a dynamically returned type will always have a cast before calling the cast function. This is done in pattern_process() and in stktable_fetch_key().
2012-04-25 11:21:49 -04:00
return NULL;
/* OK cast succeeded */
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
if (!conv_expr->conv->process(conv_expr->arg_p, p))
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
return NULL;
}
return p;
}
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
/*
* Resolve all remaining arguments in proxy <p>. Returns the number of
* errors or 0 if everything is fine.
*/
int smp_resolve_args(struct proxy *p)
{
struct arg_list *cur, *bak;
const char *ctx, *where;
const char *conv_ctx, *conv_pre, *conv_pos;
struct userlist *ul;
struct arg *arg;
int cfgerr = 0;
list_for_each_entry_safe(cur, bak, &p->conf.args.list, list) {
struct proxy *px;
struct server *srv;
char *pname, *sname;
arg = cur->arg;
/* prepare output messages */
conv_pre = conv_pos = conv_ctx = "";
if (cur->conv) {
conv_ctx = cur->conv;
conv_pre = "conversion keyword '";
conv_pos = "' for ";
}
where = "in";
ctx = "sample fetch keyword";
switch (cur->ctx) {
case ARGC_STK:where = "in stick rule in"; break;
case ARGC_TRK: where = "in tracking rule in"; break;
case ARGC_LOG: where = "in log-format string in"; break;
BUG/MINOR: arg: fix error reporting for add-header/set-header sample fetch arguments The 'add-header %[samples]' parsing errors associated to http-request and http-response are displayed with the wrong keyword. Configuration entry: http-request set-header mon-header %[res.hdr(user-agent)] Original error message: [WARNING] 323/150920 (16559) : parsing [haproxy.conf:36] : 'log-format' : sample fetch <res.hdr ... After commit error message: [WARNING] 323/150929 (16580) : parsing [haproxy.conf:36] : 'http-request' : sample fetch <res.hdr ...
2013-11-20 09:09:52 -05:00
case ARGC_HRQ: where = "in http-request header format string in"; break;
case ARGC_HRS: where = "in http-response header format string in"; break;
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
case ARGC_UIF: where = "in unique-id-format string in"; break;
MEDIUM: http: The redirect strings follows the log format rules. We handle "http-request redirect" with a log-format string now, but we leave "redirect" unaffected. Note that the control of the special "/" case is move from the runtime execution to the configuration parsing. If the format rule list is empty, the build_logline() function does nothing.
2013-11-29 06:15:45 -05:00
case ARGC_RDR: where = "in redirect format string in"; break;
MINOR: tcp: prepare support for the "capture" action A few minor entries will be needed to capture sample fetches in requests or responses. This patch just prepares the code for this.
2014-06-13 10:17:14 -04:00
case ARGC_CAP: where = "in capture rule in"; break;
MAJOR: sample: maintain a per-proxy list of the fetch args to resolve While ACL args were resolved after all the config was parsed, it was not the case with sample fetch args because they're almost everywhere now. The issue is that ACLs now solely rely on sample fetches, so their args resolving doesn't work anymore. And many fetches involving a server, a proxy or a userlist don't work at all. The real issue is that at the bottom layers we have no information about proxies, line numbers, even ACLs in order to report understandable errors, and that at the top layers we have no visibility over the locations where fetches are referenced (think log node). After failing multiple unsatisfying solutions attempts, we now have a new concept of args list. The principle is that every proxy has a list head which contains a number of indications such as the config keyword, the context where it's used, the file and line number, etc... and a list of arguments. This list head is of the same type as the elements, so it serves as a template for adding new elements. This way, it is filled from top to bottom by the callers with the information they have (eg: line numbers, ACL name, ...) and the lower layers just have to duplicate it and add an element when they face an argument they cannot resolve yet. Then at the end of the configuration parsing, a loop passes over each proxy's list and resolves all the args in sequence. And this way there is all necessary information to report verbose errors. The first immediate benefit is that for the first time we got very precise location of issues (arg number in a keyword in its context, ...). Second, in order to do this we had to parse log-format and unique-id-format a bit earlier, so that was a great opportunity for doing so when the directives are encountered (unless it's a default section). This way, the recorded line numbers for these args are the ones of the place where the log format is declared, not the end of the file. Userlists report slightly more information now. They're the only remaining ones in the ACL resolving function.
2013-04-02 10:34:32 -04:00
case ARGC_ACL: ctx = "ACL keyword"; break;
}
/* set a few default settings */
px = p;
pname = p->id;
switch (arg->type) {
case ARGT_SRV:
if (!arg->data.str.len) {
Alert("parsing [%s:%d] : missing server name in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
continue;
}
/* we support two formats : "bck/srv" and "srv" */
sname = strrchr(arg->data.str.str, '/');
if (sname) {
*sname++ = '\0';
pname = arg->data.str.str;
px = findproxy(pname, PR_CAP_BE);
if (!px) {
Alert("parsing [%s:%d] : unable to find proxy '%s' referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
}
else
sname = arg->data.str.str;
srv = findserver(px, sname);
if (!srv) {
Alert("parsing [%s:%d] : unable to find server '%s' in proxy '%s', referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, sname, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
free(arg->data.str.str);
arg->data.str.str = NULL;
arg->unresolved = 0;
arg->data.srv = srv;
break;
case ARGT_FE:
if (arg->data.str.len) {
pname = arg->data.str.str;
px = findproxy(pname, PR_CAP_FE);
}
if (!px) {
Alert("parsing [%s:%d] : unable to find frontend '%s' referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
if (!(px->cap & PR_CAP_FE)) {
Alert("parsing [%s:%d] : proxy '%s', referenced in arg %d of %s%s%s%s '%s' %s proxy '%s', has not frontend capability.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
free(arg->data.str.str);
arg->data.str.str = NULL;
arg->unresolved = 0;
arg->data.prx = px;
break;
case ARGT_BE:
if (arg->data.str.len) {
pname = arg->data.str.str;
px = findproxy(pname, PR_CAP_BE);
}
if (!px) {
Alert("parsing [%s:%d] : unable to find backend '%s' referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
if (!(px->cap & PR_CAP_BE)) {
Alert("parsing [%s:%d] : proxy '%s', referenced in arg %d of %s%s%s%s '%s' %s proxy '%s', has not backend capability.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
free(arg->data.str.str);
arg->data.str.str = NULL;
arg->unresolved = 0;
arg->data.prx = px;
break;
case ARGT_TAB:
if (arg->data.str.len) {
pname = arg->data.str.str;
px = find_stktable(pname);
}
if (!px) {
Alert("parsing [%s:%d] : unable to find table '%s' referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
if (!px->table.size) {
Alert("parsing [%s:%d] : no table in proxy '%s' referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, pname,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
free(arg->data.str.str);
arg->data.str.str = NULL;
arg->unresolved = 0;
arg->data.prx = px;
break;
case ARGT_USR:
if (!arg->data.str.len) {
Alert("parsing [%s:%d] : missing userlist name in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
if (p->uri_auth && p->uri_auth->userlist &&
!strcmp(p->uri_auth->userlist->name, arg->data.str.str))
ul = p->uri_auth->userlist;
else
ul = auth_find_userlist(arg->data.str.str);
if (!ul) {
Alert("parsing [%s:%d] : unable to find userlist '%s' referenced in arg %d of %s%s%s%s '%s' %s proxy '%s'.\n",
cur->file, cur->line, arg->data.str.str,
cur->arg_pos + 1, conv_pre, conv_ctx, conv_pos, ctx, cur->kw, where, p->id);
cfgerr++;
break;
}
free(arg->data.str.str);
arg->data.str.str = NULL;
arg->unresolved = 0;
arg->data.usr = ul;
break;
}
LIST_DEL(&cur->list);
free(cur);
} /* end of args processing */
return cfgerr;
}
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
/*
MINOR: sample: improve sample_fetch_string() to report partial contents Currently, all callers to sample_fetch_string() call it with SMP_OPT_FINAL. Now we improve it to support the case where this option is not set, and to make it return the original sample as-is. The purpose is to let the caller check the SMP_F_MAY_CHANGE flag in the result and know that it should wait to get complete contents. Currently this has no effect on existing code.
2014-06-13 10:04:35 -04:00
* Process a fetch + format conversion as defined by the sample expression
* <expr> on request or response considering the <opt> parameter. The output is
* always of type string. If a stable sample can be fetched, or an unstable one
* when <opt> contains SMP_OPT_FINAL, the sample is converted to a string and
* returned without the SMP_F_MAY_CHANGE flag. If an unstable sample is found
* and <opt> does not contain SMP_OPT_FINAL, then the sample is returned as-is
* with its SMP_F_MAY_CHANGE flag so that the caller can check it and decide to
* take actions (eg: wait longer). If a sample could not be found or could not
BUG/MAJOR: tcp: fix a possible busy spinning loop in content track-sc* As a consequence of various recent changes on the sample conversion, a corner case has emerged where it is possible to wait forever for a sample in track-sc*. The issue is caused by the fact that functions relying on sample_process() don't all exactly work the same regarding the SMP_F_MAY_CHANGE flag and the output result. Here it was possible to wait forever for an output sample from stktable_fetch_key() without checking the SMP_OPT_FINAL flag. As a result, if the client connects and closes without sending the data and haproxy expects a sample which is capable of coming, it will ignore this impossible case and will continue to wait. This change adds control for SMP_OPT_FINAL before waiting for extra data. The various relevant functions have been better documented regarding their output values. This fix must be backported to 1.5 since it appeared there.
2014-07-30 02:56:35 -04:00
* be converted, NULL is returned. The caller MUST NOT use the sample if the
* SMP_F_MAY_CHANGE flag is present, as it is used only as a hint that there is
* still hope to get it after waiting longer, and is not converted to string.
* The possible output combinations are the following :
*
* return MAY_CHANGE FINAL Meaning for the sample
* NULL * * Not present and will never be (eg: header)
* smp 0 * Final value converted (eg: header)
* smp 1 0 Not present yet, may appear later (eg: header)
* smp 1 1 never happens (either flag is cleared on output)
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
*/
struct sample *sample_fetch_string(struct proxy *px, struct session *l4, void *l7,
unsigned int opt, struct sample_expr *expr)
{
MINOR: sample: improve sample_fetch_string() to report partial contents Currently, all callers to sample_fetch_string() call it with SMP_OPT_FINAL. Now we improve it to support the case where this option is not set, and to make it return the original sample as-is. The purpose is to let the caller check the SMP_F_MAY_CHANGE flag in the result and know that it should wait to get complete contents. Currently this has no effect on existing code.
2014-06-13 10:04:35 -04:00
struct sample *smp = &temp_smp;
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
BUG/MAJOR: sample: correctly reinitialize sample fetch context before calling sample_process() We used to only clear flags when reusing the static sample before calling sample_process(), but that's not enough because there's a context in samples that can be used by some fetch functions such as auth, headers and cookies, and not reinitializing it risks that a pointer of a different type is used in the wrong context. An example configuration which triggers the case consists in mixing hdr() and http_auth_group() which both make use of contexts : http-request add-header foo2 %[hdr(host)],%[http_auth_group(foo)] The solution is simple, initialize all the sample and not just the flags. This fix must be backported into 1.5 since it was introduced in 1.5-dev19.
2014-06-25 10:56:41 -04:00
memset(smp, 0, sizeof(*smp));
MINOR: sample: improve sample_fetch_string() to report partial contents Currently, all callers to sample_fetch_string() call it with SMP_OPT_FINAL. Now we improve it to support the case where this option is not set, and to make it return the original sample as-is. The purpose is to let the caller check the SMP_F_MAY_CHANGE flag in the result and know that it should wait to get complete contents. Currently this has no effect on existing code.
2014-06-13 10:04:35 -04:00
if (!sample_process(px, l4, l7, opt, expr, smp)) {
if ((smp->flags & SMP_F_MAY_CHANGE) && !(opt & SMP_OPT_FINAL))
return smp;
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
return NULL;
MINOR: sample: improve sample_fetch_string() to report partial contents Currently, all callers to sample_fetch_string() call it with SMP_OPT_FINAL. Now we improve it to support the case where this option is not set, and to make it return the original sample as-is. The purpose is to let the caller check the SMP_F_MAY_CHANGE flag in the result and know that it should wait to get complete contents. Currently this has no effect on existing code.
2014-06-13 10:04:35 -04:00
}
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
if (!sample_casts[smp->type][SMP_T_STR])
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
return NULL;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
if (!sample_casts[smp->type][SMP_T_STR](smp))
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
return NULL;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->type = SMP_T_STR;
MINOR: sample: improve sample_fetch_string() to report partial contents Currently, all callers to sample_fetch_string() call it with SMP_OPT_FINAL. Now we improve it to support the case where this option is not set, and to make it return the original sample as-is. The purpose is to let the caller check the SMP_F_MAY_CHANGE flag in the result and know that it should wait to get complete contents. Currently this has no effect on existing code.
2014-06-13 10:04:35 -04:00
smp->flags &= ~SMP_F_MAY_CHANGE;
MINOR: samples: add a function to fetch and convert any sample to a string Any sample type can now easily be converted to a string that can be used anywhere. This will be used for logging and passing information in headers.
2012-12-20 18:02:32 -05:00
return smp;
}
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*****************************************************************/
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
/* Sample format convert functions */
MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description We need the pattern fetchers and converters to correctly set the output type so that they can be used by ACL fetchers. By using the sample type instead of the keyword type, we also open the possibility to create some multi-type pattern fetch methods later (eg: "src" being v4/v6). Right now the type in the keyword is used to validate the configuration.
2012-04-23 16:38:26 -04:00
/* These functions set the data type on return. */
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/*****************************************************************/
MINOR: ssl: adds sample converter base64 for binary type. The new converter encode binary type sample to base64 string. i.e. : ssl_c_serial,base64
2014-04-30 12:21:37 -04:00
static int sample_conv_bin2base64(const struct arg *arg_p, struct sample *smp)
{
struct chunk *trash = get_trash_chunk();
int b64_len;
trash->len = 0;
b64_len = a2base64(smp->data.str.str, smp->data.str.len, trash->str, trash->size);
if (b64_len < 0)
return 0;
trash->len = b64_len;
smp->data.str = *trash;
smp->type = SMP_T_STR;
smp->flags &= ~SMP_F_CONST;
return 1;
}
MINOR: sample: add hex converter This new filter converts BIN type to its hexadecimal representation in STR type. It is used to keep the compatibility with the original bin2str cast. It will be useful when bin2str changes to copy the string as-is without encoding anymore.
2014-03-12 10:01:52 -04:00
static int sample_conv_bin2hex(const struct arg *arg_p, struct sample *smp)
{
struct chunk *trash = get_trash_chunk();
unsigned char c;
int ptr = 0;
trash->len = 0;
while (ptr < smp->data.str.len && trash->len <= trash->size - 2) {
c = smp->data.str.str[ptr++];
trash->str[trash->len++] = hextab[(c >> 4) & 0xF];
trash->str[trash->len++] = hextab[c & 0xF];
}
smp->data.str = *trash;
smp->type = SMP_T_STR;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->flags &= ~SMP_F_CONST;
MINOR: sample: add hex converter This new filter converts BIN type to its hexadecimal representation in STR type. It is used to keep the compatibility with the original bin2str cast. It will be useful when bin2str changes to copy the string as-is without encoding anymore.
2014-03-12 10:01:52 -04:00
return 1;
}
MINOR: sample: add new converters to hash input From time to time it's useful to hash input data (scramble input, or reduce the space needed in a stick table). This patch provides 3 simple converters allowing use of the available hash functions to hash input data. The output is an unsigned integer which can be passed into a header, a log or used as an index for a stick table. One nice usage is to scramble source IP addresses before logging when there are requirements to hide them.
2014-07-15 14:15:37 -04:00
/* hashes the binary input into a 32-bit unsigned int */
static int sample_conv_djb2(const struct arg *arg_p, struct sample *smp)
{
smp->data.uint = hash_djb2(smp->data.str.str, smp->data.str.len);
if (arg_p && arg_p->data.uint)
smp->data.uint = full_hash(smp->data.uint);
smp->type = SMP_T_UINT;
return 1;
}
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
static int sample_conv_str2lower(const struct arg *arg_p, struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
int i;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
if (!smp_dup(smp))
return 0;
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
if (!smp->data.str.size)
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
return 0;
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
for (i = 0; i < smp->data.str.len; i++) {
if ((smp->data.str.str[i] >= 'A') && (smp->data.str.str[i] <= 'Z'))
smp->data.str.str[i] += 'a' - 'A';
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
return 1;
}
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
static int sample_conv_str2upper(const struct arg *arg_p, struct sample *smp)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
int i;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
if (!smp_dup(smp))
return 0;
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
if (!smp->data.str.size)
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 12:02:19 -04:00
return 0;
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
for (i = 0; i < smp->data.str.len; i++) {
if ((smp->data.str.str[i] >= 'a') && (smp->data.str.str[i] <= 'z'))
smp->data.str.str[i] += 'A' - 'a';
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
return 1;
}
MEDIUM: pattern: get rid of arg_i in all functions making use of arguments arg_i was almost unused, and since we migrated to use struct arg everywhere, the rare cases where arg_i was needed could be replaced by switching to arg->type = ARGT_STOP.
2012-04-20 08:03:29 -04:00
/* takes the netmask in arg_p */
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
static int sample_conv_ipmask(const struct arg *arg_p, struct sample *smp)
[MINOR] pattern: add the "ipmask()" converting function This converter can be applied on top of an IPv4-type pattern. It applies a netmask which is suited for IP address storage and matching. This can be used to make all hosts within a certain mask to share the same table entries and as such use the same server. The mask can be passed in dotted form (eg: 255.255.255.0) or in CIDR form (eg: 24).
2010-01-26 12:01:41 -05:00
{
MEDIUM: pattern: integrate pattern_data into sample and use sample everywhere Now there is no more reference to union pattern_data. All pattern fetch and conversion functions now make use of the common sample type. Note: none of them adjust the type right now so it's important to do it next otherwise we would risk sharing such functions with ACLs and seeing them fail.
2012-04-23 16:03:39 -04:00
smp->data.ipv4.s_addr &= arg_p->data.ipv4.s_addr;
MEDIUM: pattern: retrieve the sample type in the sample, not in the keyword description We need the pattern fetchers and converters to correctly set the output type so that they can be used by ACL fetchers. By using the sample type instead of the keyword type, we also open the possibility to create some multi-type pattern fetch methods later (eg: "src" being v4/v6). Right now the type in the keyword is used to validate the configuration.
2012-04-23 16:38:26 -04:00
smp->type = SMP_T_IPV4;
[MINOR] pattern: add the "ipmask()" converting function This converter can be applied on top of an IPv4-type pattern. It applies a netmask which is suited for IP address storage and matching. This can be used to make all hosts within a certain mask to share the same table entries and as such use the same server. The mask can be passed in dotted form (eg: 255.255.255.0) or in CIDR form (eg: 24).
2010-01-26 12:01:41 -05:00
return 1;
}
MINOR: samples: add two converters for the date format This patch adds two converters : ltime(<format>[,<offset>]) utime(<format>[,<offset>]) Both use strftime() to emit the output string from an input date. ltime() provides local time, while utime() provides the UTC time.
2014-07-10 10:37:47 -04:00
/* takes an UINT value on input supposed to represent the time since EPOCH,
* adds an optional offset found in args[1] and emits a string representing
* the local time in the format specified in args[1] using strftime().
*/
static int sample_conv_ltime(const struct arg *args, struct sample *smp)
{
struct chunk *temp;
time_t curr_date = smp->data.uint;
/* add offset */
if (args[1].type == ARGT_SINT || args[1].type == ARGT_UINT)
curr_date += args[1].data.sint;
temp = get_trash_chunk();
temp->len = strftime(temp->str, temp->size, args[0].data.str.str, localtime(&curr_date));
smp->data.str = *temp;
smp->type = SMP_T_STR;
return 1;
}
MINOR: sample: add new converters to hash input From time to time it's useful to hash input data (scramble input, or reduce the space needed in a stick table). This patch provides 3 simple converters allowing use of the available hash functions to hash input data. The output is an unsigned integer which can be passed into a header, a log or used as an index for a stick table. One nice usage is to scramble source IP addresses before logging when there are requirements to hide them.
2014-07-15 14:15:37 -04:00
/* hashes the binary input into a 32-bit unsigned int */
static int sample_conv_sdbm(const struct arg *arg_p, struct sample *smp)
{
smp->data.uint = hash_sdbm(smp->data.str.str, smp->data.str.len);
if (arg_p && arg_p->data.uint)
smp->data.uint = full_hash(smp->data.uint);
smp->type = SMP_T_UINT;
return 1;
}
MINOR: samples: add two converters for the date format This patch adds two converters : ltime(<format>[,<offset>]) utime(<format>[,<offset>]) Both use strftime() to emit the output string from an input date. ltime() provides local time, while utime() provides the UTC time.
2014-07-10 10:37:47 -04:00
/* takes an UINT value on input supposed to represent the time since EPOCH,
* adds an optional offset found in args[1] and emits a string representing
* the UTC date in the format specified in args[1] using strftime().
*/
static int sample_conv_utime(const struct arg *args, struct sample *smp)
{
struct chunk *temp;
time_t curr_date = smp->data.uint;
/* add offset */
if (args[1].type == ARGT_SINT || args[1].type == ARGT_UINT)
curr_date += args[1].data.sint;
temp = get_trash_chunk();
temp->len = strftime(temp->str, temp->size, args[0].data.str.str, gmtime(&curr_date));
smp->data.str = *temp;
smp->type = SMP_T_STR;
return 1;
}
MINOR: sample: add new converters to hash input From time to time it's useful to hash input data (scramble input, or reduce the space needed in a stick table). This patch provides 3 simple converters allowing use of the available hash functions to hash input data. The output is an unsigned integer which can be passed into a header, a log or used as an index for a stick table. One nice usage is to scramble source IP addresses before logging when there are requirements to hide them.
2014-07-15 14:15:37 -04:00
/* hashes the binary input into a 32-bit unsigned int */
static int sample_conv_wt6(const struct arg *arg_p, struct sample *smp)
{
smp->data.uint = hash_wt6(smp->data.str.str, smp->data.str.len);
if (arg_p && arg_p->data.uint)
smp->data.uint = full_hash(smp->data.uint);
smp->type = SMP_T_UINT;
return 1;
}
MINOR: sample: add "json" converter This converter escapes string to use it as json/ascii escaped string. It can read UTF-8 with differents behavior on errors and encode it in json/ascii. json([<input-code>]) Escapes the input string and produces an ASCII ouput string ready to use as a JSON string. The converter tries to decode the input string according to the <input-code> parameter. It can be "ascii", "utf8", "utf8s", "utf8"" or "utf8ps". The "ascii" decoder never fails. The "utf8" decoder detects 3 types of errors: - bad UTF-8 sequence (lone continuation byte, bad number of continuation bytes, ...) - invalid range (the decoded value is within a UTF-8 prohibited range), - code overlong (the value is encoded with more bytes than necessary). The UTF-8 JSON encoding can produce a "too long value" error when the UTF-8 character is greater than 0xffff because the JSON string escape specification only authorizes 4 hex digits for the value encoding. The UTF-8 decoder exists in 4 variants designated by a combination of two suffix letters : "p" for "permissive" and "s" for "silently ignore". The behaviors of the decoders are : - "ascii" : never fails ; - "utf8" : fails on any detected errors ; - "utf8s" : never fails, but removes characters corresponding to errors ; - "utf8p" : accepts and fixes the overlong errors, but fails on any other error ; - "utf8ps" : never fails, accepts and fixes the overlong errors, but removes characters corresponding to the other errors. This converter is particularly useful for building properly escaped JSON for logging to servers which consume JSON-formated traffic logs. Example: capture request header user-agent len 150 capture request header Host len 15 log-format {"ip":"%[src]","user-agent":"%[capture.req.hdr(1),json]"} Input request from client 127.0.0.1: GET / HTTP/1.0 User-Agent: Very "Ugly" UA 1/2 Output log: {"ip":"127.0.0.1","user-agent":"Very \"Ugly\" UA 1\/2"}
2014-08-12 04:20:47 -04:00
/* This function escape special json characters. The returned string can be
* safely set between two '"' and used as json string. The json string is
* defined like this:
*
* any Unicode character except '"' or '\' or control character
* \", \\, \/, \b, \f, \n, \r, \t, \u + four-hex-digits
*
* The enum input_type contain all the allowed mode for decoding the input
* string.
*/
enum input_type {
IT_ASCII = 0,
IT_UTF8,
IT_UTF8S,
IT_UTF8P,
IT_UTF8PS,
};
static int sample_conv_json_check(struct arg *arg, struct sample_conv *conv,
const char *file, int line, char **err)
{
if (!arg) {
memprintf(err, "Unexpected empty arg list");
return 0;
}
if (arg->type != ARGT_STR) {
memprintf(err, "Unexpected arg type");
return 0;
}
if (strcmp(arg->data.str.str, "") == 0) {
arg->type = ARGT_UINT;
arg->data.uint = IT_ASCII;
return 1;
}
else if (strcmp(arg->data.str.str, "ascii") == 0) {
arg->type = ARGT_UINT;
arg->data.uint = IT_ASCII;
return 1;
}
else if (strcmp(arg->data.str.str, "utf8") == 0) {
arg->type = ARGT_UINT;
arg->data.uint = IT_UTF8;
return 1;
}
else if (strcmp(arg->data.str.str, "utf8s") == 0) {
arg->type = ARGT_UINT;
arg->data.uint = IT_UTF8S;
return 1;
}
else if (strcmp(arg->data.str.str, "utf8p") == 0) {
arg->type = ARGT_UINT;
arg->data.uint = IT_UTF8P;
return 1;
}
else if (strcmp(arg->data.str.str, "utf8ps") == 0) {
arg->type = ARGT_UINT;
arg->data.uint = IT_UTF8PS;
return 1;
}
memprintf(err, "Unexpected input code type at file '%s', line %d. "
"Allowed value are 'ascii', 'utf8', 'utf8p' and 'utf8pp'", file, line);
return 0;
}
static int sample_conv_json(const struct arg *arg_p, struct sample *smp)
{
struct chunk *temp;
char _str[7]; /* \u + 4 hex digit + null char for sprintf. */
const char *str;
int len;
enum input_type input_type = IT_ASCII;
unsigned int c;
unsigned int ret;
char *p;
if (arg_p)
input_type = arg_p->data.uint;
temp = get_trash_chunk();
temp->len = 0;
p = smp->data.str.str;
while (p < smp->data.str.str + smp->data.str.len) {
if (input_type == IT_ASCII) {
/* Read input as ASCII. */
c = *(unsigned char *)p;
p++;
}
else {
/* Read input as UTF8. */
ret = utf8_next(p, smp->data.str.len - ( p - smp->data.str.str ), &c);
p += utf8_return_length(ret);
if (input_type == IT_UTF8 && utf8_return_code(ret) != UTF8_CODE_OK)
return 0;
if (input_type == IT_UTF8S && utf8_return_code(ret) != UTF8_CODE_OK)
continue;
if (input_type == IT_UTF8P && utf8_return_code(ret) & (UTF8_CODE_INVRANGE|UTF8_CODE_BADSEQ))
return 0;
if (input_type == IT_UTF8PS && utf8_return_code(ret) & (UTF8_CODE_INVRANGE|UTF8_CODE_BADSEQ))
continue;
/* Check too big values. */
if ((unsigned int)c > 0xffff) {
if (input_type == IT_UTF8 || input_type == IT_UTF8P)
return 0;
continue;
}
}
/* Convert character. */
if (c == '"') {
len = 2;
str = "\\\"";
}
else if (c == '\\') {
len = 2;
str = "\\\\";
}
else if (c == '/') {
len = 2;
str = "\\/";
}
else if (c == '\b') {
len = 2;
str = "\\b";
}
else if (c == '\f') {
len = 2;
str = "\\f";
}
else if (c == '\r') {
len = 2;
str = "\\r";
}
else if (c == '\n') {
len = 2;
str = "\\n";
}
else if (c == '\t') {
len = 2;
str = "\\t";
}
else if (c > 0xff || !isprint(c)) {
/* isprint generate a segfault if c is too big. The man says that
* c must have the value of an unsigned char or EOF.
*/
len = 6;
_str[0] = '\\';
_str[1] = 'u';
snprintf(&_str[2], 5, "%04x", (unsigned short)c);
str = _str;
}
else {
len = 1;
str = (char *)&c;
}
/* Check length */
if (temp->len + len > temp->size)
return 0;
/* Copy string. */
memcpy(temp->str + temp->len, str, len);
temp->len += len;
}
smp->flags &= ~SMP_F_CONST;
smp->data.str = *temp;
smp->type = SMP_T_STR;
return 1;
}
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
/************************************************************************/
/* All supported sample fetch functions must be declared here */
/************************************************************************/
/* force TRUE to be returned at the fetch level */
static int
smp_fetch_true(struct proxy *px, struct session *s, void *l7, unsigned int opt,
MEDIUM: sample: systematically pass the keyword pointer to the keyword We're having a lot of duplicate code just because of minor variants between fetch functions that could be dealt with if the functions had the pointer to the original keyword, so let's pass it as the last argument. An earlier version used to pass a pointer to the sample_fetch element, but this is not the best solution for two reasons : - fetch functions will solely rely on the keyword string - some other smp_fetch_* users do not have the pointer to the original keyword and were forced to pass NULL. So finally we're passing a pointer to the keyword as a const char *, which perfectly fits the original purpose.
2013-07-22 10:29:32 -04:00
const struct arg *args, struct sample *smp, const char *kw)
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
{
smp->type = SMP_T_BOOL;
smp->data.uint = 1;
return 1;
}
/* force FALSE to be returned at the fetch level */
static int
smp_fetch_false(struct proxy *px, struct session *s, void *l7, unsigned int opt,
MEDIUM: sample: systematically pass the keyword pointer to the keyword We're having a lot of duplicate code just because of minor variants between fetch functions that could be dealt with if the functions had the pointer to the original keyword, so let's pass it as the last argument. An earlier version used to pass a pointer to the sample_fetch element, but this is not the best solution for two reasons : - fetch functions will solely rely on the keyword string - some other smp_fetch_* users do not have the pointer to the original keyword and were forced to pass NULL. So finally we're passing a pointer to the keyword as a const char *, which perfectly fits the original purpose.
2013-07-22 10:29:32 -04:00
const struct arg *args, struct sample *smp, const char *kw)
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
{
smp->type = SMP_T_BOOL;
smp->data.uint = 0;
return 1;
}
/* retrieve environment variable $1 as a string */
static int
smp_fetch_env(struct proxy *px, struct session *s, void *l7, unsigned int opt,
MEDIUM: sample: systematically pass the keyword pointer to the keyword We're having a lot of duplicate code just because of minor variants between fetch functions that could be dealt with if the functions had the pointer to the original keyword, so let's pass it as the last argument. An earlier version used to pass a pointer to the sample_fetch element, but this is not the best solution for two reasons : - fetch functions will solely rely on the keyword string - some other smp_fetch_* users do not have the pointer to the original keyword and were forced to pass NULL. So finally we're passing a pointer to the keyword as a const char *, which perfectly fits the original purpose.
2013-07-22 10:29:32 -04:00
const struct arg *args, struct sample *smp, const char *kw)
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
{
char *env;
if (!args || args[0].type != ARGT_STR)
return 0;
env = getenv(args[0].data.str.str);
if (!env)
return 0;
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
smp->type = SMP_T_STR;
smp->flags = SMP_F_CONST;
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
smp->data.str.str = env;
smp->data.str.len = strlen(env);
return 1;
}
MINOR: sample: add a new "date" fetch to return the current date Returns the current date as the epoch (number of seconds since 01/01/1970). If an offset value is specified, then it is a number of seconds that is added to the current date before returning the value. This is particularly useful to compute relative dates, as both positive and negative offsets are allowed.
2013-07-25 08:28:25 -04:00
/* retrieve the current local date in epoch time, and applies an optional offset
* of args[0] seconds.
*/
static int
smp_fetch_date(struct proxy *px, struct session *s, void *l7, unsigned int opt,
MEDIUM: sample: systematically pass the keyword pointer to the keyword We're having a lot of duplicate code just because of minor variants between fetch functions that could be dealt with if the functions had the pointer to the original keyword, so let's pass it as the last argument. An earlier version used to pass a pointer to the sample_fetch element, but this is not the best solution for two reasons : - fetch functions will solely rely on the keyword string - some other smp_fetch_* users do not have the pointer to the original keyword and were forced to pass NULL. So finally we're passing a pointer to the keyword as a const char *, which perfectly fits the original purpose.
2013-07-22 10:29:32 -04:00
const struct arg *args, struct sample *smp, const char *kw)
MINOR: sample: add a new "date" fetch to return the current date Returns the current date as the epoch (number of seconds since 01/01/1970). If an offset value is specified, then it is a number of seconds that is added to the current date before returning the value. This is particularly useful to compute relative dates, as both positive and negative offsets are allowed.
2013-07-25 08:28:25 -04:00
{
smp->data.uint = date.tv_sec;
/* add offset */
if (args && (args[0].type == ARGT_SINT || args[0].type == ARGT_UINT))
smp->data.uint += args[0].data.sint;
smp->type = SMP_T_UINT;
smp->flags |= SMP_F_VOL_TEST | SMP_F_MAY_CHANGE;
return 1;
}
MINOR: sample: add a rand() sample fetch to return a sample. Sometimes it can be useful to generate a random value, at least for debugging purposes, but also to take routing decisions or to pass such a value to a backend server.
2014-02-14 05:59:04 -05:00
/* generate a random 32-bit integer for whatever purpose, with an optional
* range specified in argument.
*/
static int
smp_fetch_rand(struct proxy *px, struct session *s, void *l7, unsigned int opt,
const struct arg *args, struct sample *smp, const char *kw)
{
smp->data.uint = random();
/* reduce if needed. Don't do a modulo, use all bits! */
if (args && args[0].type == ARGT_UINT)
smp->data.uint = ((uint64_t)smp->data.uint * args[0].data.uint) >> 32;
smp->type = SMP_T_UINT;
smp->flags |= SMP_F_VOL_TEST | SMP_F_MAY_CHANGE;
return 1;
}
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
/* Note: must not be declared <const> as its list will be overwritten.
* Note: fetches that may return multiple types must be declared as the lowest
* common denominator, the type that can be casted into all other ones. For
* instance IPv4/IPv6 must be declared IPv4.
*/
static struct sample_fetch_kw_list smp_kws = {ILH, {
{ "always_false", smp_fetch_false, 0, NULL, SMP_T_BOOL, SMP_USE_INTRN },
{ "always_true", smp_fetch_true, 0, NULL, SMP_T_BOOL, SMP_USE_INTRN },
MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags The operations applied on types SMP_T_CSTR and SMP_T_STR are the same, but the check code and the declarations are double, because it must declare action for SMP_T_C* and SMP_T_*. The declared actions and checks are the same. this complexify the code. Only the "conv" functions can change from "C*" to "*" Now, if a function needs to modify input string, it can call the new function smp_dup(). This one duplicate data in a trash buffer.
2013-12-16 18:20:33 -05:00
{ "env", smp_fetch_env, ARG1(1,STR), NULL, SMP_T_STR, SMP_USE_INTRN },
MINOR: sample: add a new "date" fetch to return the current date Returns the current date as the epoch (number of seconds since 01/01/1970). If an offset value is specified, then it is a number of seconds that is added to the current date before returning the value. This is particularly useful to compute relative dates, as both positive and negative offsets are allowed.
2013-07-25 08:28:25 -04:00
{ "date", smp_fetch_date, ARG1(0,SINT), NULL, SMP_T_UINT, SMP_USE_INTRN },
MINOR: sample: add a rand() sample fetch to return a sample. Sometimes it can be useful to generate a random value, at least for debugging purposes, but also to take routing decisions or to pass such a value to a backend server.
2014-02-14 05:59:04 -05:00
{ "rand", smp_fetch_rand, ARG1(0,UINT), NULL, SMP_T_UINT, SMP_USE_INTRN },
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
{ /* END */ },
}};
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
/* Note: must not be declared <const> as its list will be overwritten */
BUG/MEDIUM: prevent gcc from moving empty keywords lists into BSS Benoit Dolez reported a failure to start haproxy 1.5-dev19. The process would immediately report an internal error with missing fetches from some crap instead of ACL names. The cause is that some versions of gcc seem to trim static structs containing a variable array when moving them to BSS, and only keep the fixed size, which is just a list head for all ACL and sample fetch keywords. This was confirmed at least with gcc 3.4.6. And we can't move these structs to const because they contain a list element which is needed to link all of them together during the parsing. The bug indeed appeared with 1.5-dev19 because it's the first one to have some empty ACL keyword lists. One solution is to impose -fno-zero-initialized-in-bss to everyone but this is not really nice. Another solution consists in ensuring the struct is never empty so that it does not move there. The easy solution consists in having a non-null list head since it's not yet initialized. A new "ILH" list head type was thus created for this purpose : create an Initialized List Head so that gcc cannot move the struct to BSS. This fixes the issue for this version of gcc and does not create any burden for the declarations.
2013-06-21 17:16:39 -04:00
static struct sample_conv_kw_list sample_conv_kws = {ILH, {
MINOR: ssl: adds sample converter base64 for binary type. The new converter encode binary type sample to base64 string. i.e. : ssl_c_serial,base64
2014-04-30 12:21:37 -04:00
{ "base64", sample_conv_bin2base64,0, NULL, SMP_T_BIN, SMP_T_STR },
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
{ "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
MINOR: sample: add hex converter This new filter converts BIN type to its hexadecimal representation in STR type. It is used to keep the compatibility with the original bin2str cast. It will be useful when bin2str changes to copy the string as-is without encoding anymore.
2014-03-12 10:01:52 -04:00
{ "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
{ "ipmask", sample_conv_ipmask, ARG1(1,MSK4), NULL, SMP_T_IPV4, SMP_T_IPV4 },
MINOR: samples: add two converters for the date format This patch adds two converters : ltime(<format>[,<offset>]) utime(<format>[,<offset>]) Both use strftime() to emit the output string from an input date. ltime() provides local time, while utime() provides the UTC time.
2014-07-10 10:37:47 -04:00
{ "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_UINT, SMP_T_STR },
{ "utime", sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_UINT, SMP_T_STR },
MINOR: sample: add new converters to hash input From time to time it's useful to hash input data (scramble input, or reduce the space needed in a stick table). This patch provides 3 simple converters allowing use of the available hash functions to hash input data. The output is an unsigned integer which can be passed into a header, a log or used as an index for a stick table. One nice usage is to scramble source IP addresses before logging when there are requirements to hide them.
2014-07-15 14:15:37 -04:00
{ "djb2", sample_conv_djb2, ARG1(0,UINT), NULL, SMP_T_BIN, SMP_T_UINT },
{ "sdbm", sample_conv_sdbm, ARG1(0,UINT), NULL, SMP_T_BIN, SMP_T_UINT },
{ "wt6", sample_conv_wt6, ARG1(0,UINT), NULL, SMP_T_BIN, SMP_T_UINT },
MINOR: sample: add "json" converter This converter escapes string to use it as json/ascii escaped string. It can read UTF-8 with differents behavior on errors and encode it in json/ascii. json([<input-code>]) Escapes the input string and produces an ASCII ouput string ready to use as a JSON string. The converter tries to decode the input string according to the <input-code> parameter. It can be "ascii", "utf8", "utf8s", "utf8"" or "utf8ps". The "ascii" decoder never fails. The "utf8" decoder detects 3 types of errors: - bad UTF-8 sequence (lone continuation byte, bad number of continuation bytes, ...) - invalid range (the decoded value is within a UTF-8 prohibited range), - code overlong (the value is encoded with more bytes than necessary). The UTF-8 JSON encoding can produce a "too long value" error when the UTF-8 character is greater than 0xffff because the JSON string escape specification only authorizes 4 hex digits for the value encoding. The UTF-8 decoder exists in 4 variants designated by a combination of two suffix letters : "p" for "permissive" and "s" for "silently ignore". The behaviors of the decoders are : - "ascii" : never fails ; - "utf8" : fails on any detected errors ; - "utf8s" : never fails, but removes characters corresponding to errors ; - "utf8p" : accepts and fixes the overlong errors, but fails on any other error ; - "utf8ps" : never fails, accepts and fixes the overlong errors, but removes characters corresponding to the other errors. This converter is particularly useful for building properly escaped JSON for logging to servers which consume JSON-formated traffic logs. Example: capture request header user-agent len 150 capture request header Host len 15 log-format {"ip":"%[src]","user-agent":"%[capture.req.hdr(1),json]"} Input request from client 127.0.0.1: GET / HTTP/1.0 User-Agent: Very "Ugly" UA 1/2 Output log: {"ip":"127.0.0.1","user-agent":"Very \"Ugly\" UA 1\/2"}
2014-08-12 04:20:47 -04:00
{ "json", sample_conv_json, ARG1(1,STR), sample_conv_json_check, SMP_T_STR, SMP_T_STR },
MEDIUM: pattern: use the standard arg parser We don't need the pattern-specific args parsers anymore, make use of the common parser instead. We still need to improve this by adding a validation function to report abnormal argument values or combinations. We don't report precise parsing errors yet but this was not previously done either.
2012-04-20 08:45:49 -04:00
{ NULL, NULL, 0, 0, 0 },
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}};
__attribute__((constructor))
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
static void __sample_init(void)
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
{
CLEANUP: acl: move the 3 remaining sample fetches to samples.c There is no more reason for having "always_true", "always_false" and "env" in acl.c while they're the most basic sample fetch keywords, so let's move them to sample.c where it's easier to find them.
2013-07-25 06:17:57 -04:00
/* register sample fetch and format conversion keywords */
sample_register_fetches(&smp_kws);
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 15:37:17 -04:00
sample_register_convs(&sample_conv_kws);
[MEDIUM] Add pattern fetch management types and functions
2010-01-04 10:16:05 -05:00
}
Reference in a new issue Copy permalink