upstream/haproxy
1
0
Fork 0
mirror of https://github.com/haproxy/haproxy.git synced 2026-06-18 13:59:56 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 7
haproxy/src
History
Willy Tarreau 3cfaa8d1e0 BUG/MEDIUM: task: bound the number of tasks picked from the wait queue at once 
There is a theorical problem in the wait queue, which is that with many
threads, one could spend a lot of time looping on the newly expired tasks,
causing a lot of contention on the global wq_lock and on the global
rq_lock. This initially sounds bening, but if another thread does just
a task_schedule() or task_queue(), it might end up waiting for a long
time on this lock, and this wait time will count on its execution budget,
degrading the end user's experience and possibly risking to trigger the
watchdog if that lasts too long.

The simplest (and backportable) solution here consists in bounding the
number of expired tasks that may be picked from the global wait queue at
once by a thread, given that all other ones will do it as well anyway.

We don't need to pick more than global.tune.runqueue_depth tasks at once
as we won't process more, so this counter is updated for both the local
and the global queues: threads with more local expired tasks will pick
less global tasks and conversely, keeping the load balanced between all
threads. This will guarantee a much lower latency if/when wakeup storms
happen (e.g. hundreds of thousands of synchronized health checks).

Note that some crashes have been witnessed with 1/4 of the threads in
wake_expired_tasks() and, while the issue might or might not be related,
not having reasonable bounds here definitely justifies why we can spend
so much time there.

This patch should be backported, probably as far as 2.0 (maybe with
some adaptations).
2020-10-16 15:18:48 +02:00
..
51d.c CLEANUP: Do not use a fixed type for 'sizeof' in 'calloc' 2020-09-12 20:31:25 +02:00
acl.c MINOR: arg: Use chunk_destroy() to release string arguments 2020-08-07 14:27:54 +02:00
action.c MEDIUM: tcp-rules: Warn if a track-sc* content rule doesn't depend on content 2020-10-02 15:50:26 +02:00
activity.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
applet.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
arg.c CLEANUP: Do not use a fixed type for 'sizeof' in 'calloc' 2020-09-12 20:31:25 +02:00
auth.c BUG/MINOR: auth: report valid crypto(3) support depending on build options 2020-09-08 14:34:04 +02:00
backend.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
base64.c REORG: include: move base64.h, errors.h and hash.h from common to to haproxy/ 2020-06-11 10:18:56 +02:00
cache.c CLEANUP: cache: Fix leak of cconf->c.name during config check 2020-10-07 14:07:29 +02:00
calltrace.c BUILD: trace: include tools.h 2020-09-25 17:54:48 +02:00
cfgparse-global.c MEDIUM: config: remove the deprecated and dangerous global "debug" directive 2020-10-09 19:18:45 +02:00
cfgparse-listen.c MEDIUM: proxy: remove obsolete "monitor-net" 2020-10-15 21:47:04 +02:00
cfgparse-ssl.c BUG/MINOR: ssl: ssl-skip-self-issued-ca requires >= 1.0.2 2020-08-10 17:31:10 +02:00
cfgparse-tcp.c MINOR: receiver: move the FOREIGN and V6ONLY options from listener to settings 2020-09-16 22:08:07 +02:00
cfgparse-unix.c MINOR: listener: create a new struct "settings" in bind_conf 2020-09-16 20:13:13 +02:00
cfgparse.c MEDIUM: proxy: remove obsolete "monitor-net" 2020-10-15 21:47:04 +02:00
channel.c MINOR: channel: new getword and getchar functions on channel. 2020-10-07 17:17:27 +02:00
check.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
chunk.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
cli.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
compression.c BUILD: compression: make gcc 10 happy with free_zlib() 2020-06-14 08:00:19 +02:00
connection.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
da.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
debug.c BUG/MINOR: debug: Don't dump the lua stack if it is not initialized 2020-07-27 09:37:18 +02:00
dgram.c REORG: dgram: rename proto_udp to dgram 2020-06-11 10:18:59 +02:00
dict.c REORG: include: move THREAD_LOCAL and __decl_thread() to compiler.h 2020-06-11 10:18:59 +02:00
dns.c MINOR: dns/stats: integrate dns counters in stats 2020-10-05 12:02:14 +02:00
dynbuf.c REORG: buffer: rename buffer.c to dynbuf.c 2020-06-29 09:26:59 +02:00
eb32sctree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
eb32tree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
eb64tree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
ebimtree.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
ebistree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
ebmbtree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
ebpttree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
ebsttree.c REORG: ebtree: move the include files from ebtree to include/import/ 2020-06-11 09:31:11 +02:00
ebtree.c BUG/MEDIUM: ebtree: use a byte-per-byte memcmp() to compare memory blocks 2020-06-16 11:30:33 +02:00
ev_epoll.c MINOR: debug: add a new DEBUG_FD build option 2020-06-23 10:04:54 +02:00
ev_evports.c MINOR: debug: add a new DEBUG_FD build option 2020-06-23 10:04:54 +02:00
ev_kqueue.c MINOR: debug: add a new DEBUG_FD build option 2020-06-23 10:04:54 +02:00
ev_poll.c MINOR: debug: add a new DEBUG_FD build option 2020-06-23 10:04:54 +02:00
ev_select.c MINOR: debug: add a new DEBUG_FD build option 2020-06-23 10:04:54 +02:00
extcheck.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
fcgi-app.c BUG/MEDIUM: fcgi-app: fix memory leak in fcgi_flt_http_headers 2020-07-15 20:23:29 +02:00
fcgi.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
fd.c MINOR: fd: report an error message when failing initial allocations 2020-10-13 18:15:33 +02:00
filters.c REORG: include: move cfgparse.h to haproxy/cfgparse.h 2020-06-11 10:18:58 +02:00
flt_http_comp.c CLEANUP: assorted typo fixes in the code and comments 2020-07-06 14:34:32 +02:00
flt_spoe.c MINOR: buffer: use MT_LIST_ADDQ() for buffer_wait lists additions 2020-07-10 08:52:13 +02:00
flt_trace.c CLEANUP: assorted typo fixes in the code and comments 2020-07-31 11:18:07 +02:00
freq_ctr.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
frontend.c REORG: listener: move the receiving FD to struct receiver 2020-09-16 22:08:03 +02:00
h1.c BUILD: tree-wide: cast arguments to tolower/toupper to unsigned char 2020-07-05 21:50:02 +02:00
h1_htx.c MEDIUM: htx: Add a flag on a HTX message when no more data are expected 2020-07-22 16:43:32 +02:00
h2.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
haproxy.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
hash.c REORG: include: move base64.h, errors.h and hash.h from common to to haproxy/ 2020-06-11 10:18:56 +02:00
hlua.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
hlua_fcn.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
hpack-dec.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
hpack-enc.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
hpack-huff.c REORG: include: move hpack*.h to haproxy/ and split hpack-tbl 2020-06-11 10:18:57 +02:00
hpack-tbl.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
http.c BUG/MINOR: http: Fix content-length of the default 500 error 2020-10-09 10:02:09 +02:00
http_acl.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
http_act.c MINOR: http-rules: Add set-pathq and replace-pathq actions 2020-09-04 11:41:46 +02:00
http_ana.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
http_conv.c CLEANUP: assorted typo fixes in the code and comments 2020-07-06 14:34:32 +02:00
http_fetch.c BUG/MINOR: http-fetch: Don't set the sample type during the htx prefetch 2020-09-18 11:06:24 +02:00
http_htx.c BUG/MINOR: http-htx: Expect no body for 204/304 internal HTTP responses 2020-10-09 10:02:09 +02:00
http_rules.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
htx.c CLEANUP: assorted typo fixes in the code and comments 2020-07-06 14:34:32 +02:00
lb_chash.c CLEANUP: Do not use a fixed type for 'sizeof' in 'calloc' 2020-09-12 20:31:25 +02:00
lb_fas.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
lb_fwlc.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
lb_fwrr.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
lb_map.c CLEANUP: Do not use a fixed type for 'sizeof' in 'calloc' 2020-09-12 20:31:25 +02:00
listener.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
log.c MINOR: log: set the UDP receiver's I/O handler in the receiver 2020-10-15 21:47:56 +02:00
lru.c MINOR: lru: new function to delete <nb> least recently used keys 2016-01-11 07:31:35 +01:00
mailers.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
map.c MINOR: arg: Use chunk_destroy() to release string arguments 2020-08-07 14:27:54 +02:00
mux_fcgi.c MEDIUM: fcgi: remove conn from session on detach 2020-10-15 15:19:34 +02:00
mux_h1.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
mux_h2.c MEDIUM: h2: remove conn from session on detach 2020-10-15 15:19:34 +02:00
mux_pt.c MINOR: connection: Set the conncetion target during its initialisation 2020-07-15 14:08:14 +02:00
mworker-prog.c REORG: include: move the error reporting functions to from log.h to errors.h 2020-06-11 10:18:59 +02:00
mworker.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
namespace.c REORG: include: move the error reporting functions to from log.h to errors.h 2020-06-11 10:18:59 +02:00
pattern.c BUG/MEDIUM: pattern: Renew the pattern expression revision when it is pruned 2020-09-11 09:54:34 +02:00
payload.c MINOR: arg: Use chunk_destroy() to release string arguments 2020-08-07 14:27:54 +02:00
peers.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
pipe.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
pool.c MINOR: pools: move the LRU cache heads to thread_info 2020-06-29 10:36:37 +02:00
proto_sockpair.c CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind() 2020-10-15 21:47:56 +02:00
proto_tcp.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
proto_udp.c CLEANUP: protocol: intitialize all of the sockaddr when disconnecting 2020-10-14 10:54:15 +02:00
proto_uxst.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
protocol.c CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind() 2020-10-15 21:47:56 +02:00
proxy.c BUG/MEDIUM: proxy: properly stop backends 2020-10-16 15:16:17 +02:00
queue.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
raw_sock.c MINOR: raw_sock: Report the number of bytes emitted using the splicing 2020-07-15 14:08:14 +02:00
regex.c OPTIM: regex: PCRE2 use JIT match when JIT optimisation occured. 2020-08-14 07:53:40 +02:00
ring.c CLEANUP: fix all duplicated semicolons 2020-08-10 08:49:38 +02:00
sample.c MINOR: sample: Add iif(<true>,<false>) converter 2020-09-11 16:59:27 +02:00
server.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
session.c MINOR: session: simplify error path in session_accept_fd() 2020-10-15 21:47:56 +02:00
sha1.c BUILD: use inttypes.h instead of stdint.h 2019-04-01 07:44:56 +02:00
shctx.c REORG: include: split global.h into haproxy/global{,-t}.h 2020-06-11 10:18:58 +02:00
signal.c REORG: include: move the error reporting functions to from log.h to errors.h 2020-06-11 10:18:59 +02:00
sink.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
sock.c MINOR: protocol: register the receiver's I/O handler and not the protocol's 2020-10-15 21:47:56 +02:00
sock_inet.c CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind() 2020-10-15 21:47:56 +02:00
sock_unix.c CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind() 2020-10-15 21:47:56 +02:00
ssl_ckch.c CLEANUP: ssl/cli: remove test on 'multi' variable in CLI functions 2020-09-16 16:28:26 +02:00
ssl_crtlist.c CLEANUP: ssl: "bundle" is not an OpenSSL wording 2020-10-02 18:11:47 +02:00
ssl_sample.c BUG/MINOR: ssl: double free w/ smp_fetch_ssl_x_chain_der() 2020-08-11 11:18:46 +02:00
ssl_sock.c CLEANUP: ssl: Release cached SSL sessions on deinit 2020-10-07 14:07:29 +02:00
ssl_utils.c CLEANUP: ssl: ssl_sock_crt2der semicolon and spaces 2020-08-07 15:38:40 +02:00
stats.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
stick_table.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
stream.c MEDIUM: proxy: replace proxy->state with proxy->disabled 2020-10-09 11:27:30 +02:00
stream_interface.c MINOR: stream-int: Be sure to have a mux to do sends and receives 2020-07-30 09:39:20 +02:00
task.c BUG/MEDIUM: task: bound the number of tasks picked from the wait queue at once 2020-10-16 15:18:48 +02:00
tcp_act.c REORG: tcp: move TCP actions from proto_tcp.c to tcp_act.c 2020-08-28 18:51:36 +02:00
tcp_rules.c MEDIUM: tcp-rules: Use a dedicated expiration date for tcp ruleset 2020-07-30 09:31:09 +02:00
tcp_sample.c MINOR: listener: prefer to retrieve the socket's settings via the receiver 2020-09-16 22:08:07 +02:00
tcpcheck.c MINOR: connection: make sockaddr_alloc() take the address to be copied 2020-10-15 21:47:56 +02:00
thread.c BUILD: threads: better workaround for late loading of libgcc_s 2020-09-09 19:10:46 +02:00
time.c CLEANUP: assorted typo fixes in the code and comments 2020-07-06 14:34:32 +02:00
tools.c BUILD: tools: fix minor build issue on isspace() 2020-10-01 18:05:48 +02:00
trace.c MEDIUM: log/sink: re-work and merge of build message API. 2020-07-15 17:50:12 +02:00
uri_auth.c CLEANUP: Do not use a fixed type for 'sizeof' in 'calloc' 2020-09-12 20:31:25 +02:00
vars.c CLEANUP: Add static void vars_deinit() 2020-07-07 16:52:35 +02:00
version.c BUILD: Fix build by including haproxy/global.h 2020-06-16 23:36:04 +02:00
wdt.c REORG: include: move the error reporting functions to from log.h to errors.h 2020-06-11 10:18:59 +02:00
wurfl.c CLEANUP: include: tree-wide alphabetical sort of include files 2020-06-11 10:18:59 +02:00
xprt_handshake.c REORG: include: move stream_interface.h to haproxy/stream_interface{,-t}.h 2020-06-11 10:18:58 +02:00
xxhash.c BUILD: use inttypes.h instead of stdint.h 2019-04-01 07:44:56 +02:00