upstream/redis
1
0
Fork 0
mirror of https://github.com/redis/redis.git synced 2026-06-14 20:12:48 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 7
redis/src
History
Ozan Tezcan 6790d848c5
Reuse temporary client objects for blocked clients by module (#9940) 
Added a pool for temporary client objects to reuse in module operations.
By reusing temporary clients, we are avoiding expensive createClient()/freeClient()
calls and improving performance of RM_BlockClient() and  RM_GetThreadSafeContext() calls. 

This commit contains two optimizations: 

1 - RM_BlockClient() and RM_GetThreadSafeContext() calls create temporary clients and they are freed in
RM_UnblockClient() and RM_FreeThreadSafeContext() calls respectively. Creating/destroying client object
takes quite time. To avoid that, added a pool of temporary clients. Pool expands when more clients are needed.
Also, added a cron function to shrink the pool and free unused clients after some time. Pool starts with zero
clients in it. It does not have max size and can grow unbounded as we need it. We will keep minimum of 8
temporary clients in the pool once created. Keeping small amount of clients to avoid client allocation costs
if temporary clients are required after some idle period.

2 - After unblocking a client (RM_UnblockClient()), one byte is written to pipe to wake up Redis main thread.
If there are many clients that will be unblocked, each operation requires one write() call which is quite expensive.
Changed code to avoid subsequent calls if possible. 

There are a few more places that need temporary client objects (e.g RM_Call()). These are now using the same
temporary client pool to make things more centralized.
2022-01-11 19:00:56 +02:00
..
commands Move doc metadata from COMMAND to COMMAND DOCS (#10056) 2022-01-11 17:16:16 +02:00
modules fix memory leak in example module hellocluster (#9813) 2021-11-20 00:48:08 +02:00
.gitignore Ignore gcov/lcov artifacts 2012-04-13 17:52:33 -07:00
acl.c Sharded pubsub implementation (#8621) 2022-01-02 16:54:47 -08:00
adlist.c Free value if dup succeed but listAddNodeTail failed. (#8901) 2021-05-03 13:39:07 +03:00
adlist.h Speedup: unblock clients on keys in O(1). 2020-04-08 12:55:57 +02:00
ae.c Improve doc comment about AE_DONT_WAIT (#9165) 2021-06-29 14:37:02 +03:00
ae.h Use H/W Monotonic clock and updates to AE (#7644) 2020-08-28 11:54:10 +03:00
ae_epoll.c Fail fast when systemic error occurs in poll (#8749) 2021-04-26 15:52:06 +03:00
ae_evport.c Typo: Change port_get to port_getn in aeApiPoll panic message. (#8920) 2021-05-07 15:34:35 +03:00
ae_kqueue.c Fix the timing of read and write events under kqueue (#9416) 2021-09-02 11:07:51 +03:00
ae_select.c Fail fast when systemic error occurs in poll (#8749) 2021-04-26 15:52:06 +03:00
anet.c Fix some nonsense came from LGTM (#9962) 2021-12-19 17:52:23 +02:00
anet.h Implement anetPipe() to combine creating pipe and setting flags (#9511) 2021-10-06 16:08:13 +03:00
aof.c Support whitespace characters in appendfilename, and ban them in appenddirname (#10049) 2022-01-10 09:09:39 +02:00
asciilogo.h Changes http to https in texts (#8495) 2021-03-10 19:11:16 +02:00
atomicvar.h Several (mostly Solaris-related) cleanups (#8171) 2020-12-13 17:09:54 +02:00
bio.c Delete unused 'time' fields from struct bio_job (#9622) 2021-10-10 08:17:54 +03:00
bio.h Delete some unimplemented prototype. (#8882) 2021-04-29 08:25:10 +03:00
bitops.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
blocked.c Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00
call_reply.c Fix callReplyParseCollection memleak when use AutoMemory (#9446) 2021-09-09 11:03:05 +03:00
call_reply.h Unified Lua and modules reply parsing and added RESP3 support to RM_Call (#9202) 2021-08-04 16:28:07 +03:00
childinfo.c Implement anetPipe() to combine creating pipe and setting flags (#9511) 2021-10-06 16:08:13 +03:00
cli_common.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
cli_common.h Added URI support to redis-benchmark (cli and benchmark share the same uri-parsing methods) (#9314) 2021-09-14 19:45:06 +03:00
cluster.c Implement clusterbus message extensions and cluster hostname support (#9530) 2022-01-02 19:48:29 -08:00
cluster.h Implement clusterbus message extensions and cluster hostname support (#9530) 2022-01-02 19:48:29 -08:00
commands.c Move doc metadata from COMMAND to COMMAND DOCS (#10056) 2022-01-11 17:16:16 +02:00
config.c Support whitespace characters in appendfilename, and ban them in appenddirname (#10049) 2022-01-10 09:09:39 +02:00
config.h Check somaxconn system settings on macOS, FreeBSD and OpenBSD. (#9972) 2021-12-28 09:20:10 +02:00
connection.c Fix EINTR test failures. (#9751) 2021-11-08 16:09:33 +02:00
connection.h Fix EINTR test failures. (#9751) 2021-11-08 16:09:33 +02:00
connhelpers.h Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
crc16.c RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
crc16_slottable.h Added basic support for clusters to redis-benchmark. 2019-03-01 17:53:14 +01:00
crc64.c Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
crc64.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
crcspeed.c Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
crcspeed.h Added crcspeed library 2020-04-24 17:11:21 -07:00
db.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
debug.c Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) 2022-01-03 19:14:13 +02:00
debugmacro.h Fix some redundancy use of semicolon in do-while macros (#8221) 2020-12-21 22:57:45 -08:00
defrag.c Remove EVAL script verbatim replication, propagation, and deterministic execution logic (#9812) 2021-12-21 08:32:42 +02:00
dict.c Fix some nonsense came from LGTM (#9962) 2021-12-19 17:52:23 +02:00
dict.h Fix possible int overflow when hashing an sds. (#9916) 2021-12-13 21:16:25 +02:00
endianconv.c Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
endianconv.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
eval.c Remove EVAL script verbatim replication, propagation, and deterministic execution logic (#9812) 2021-12-21 08:32:42 +02:00
evict.c use startEvictionTimeProc() in config set maxmemory (#10019) 2022-01-04 13:08:10 +02:00
expire.c Sort out mess around propagation and MULTI/EXEC (#9890) 2021-12-23 00:03:48 +02:00
fmacros.h Fix failed tests on Linux Alpine and add a CI job. (#8532) 2021-02-23 12:57:45 +02:00
function_lua.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
functions.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
functions.h Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
geo.c Fix some nonsense came from LGTM (#9962) 2021-12-19 17:52:23 +02:00
geo.h RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
geohash.c FIX: truncate max/min longitude,latitude related geo_point (ex: {180, 85.05112878} ) 2020-04-09 17:48:29 +08:00
geohash.h Delete some unimplemented prototype. (#8882) 2021-04-29 08:25:10 +03:00
geohash_helper.c Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
geohash_helper.h Delete some unimplemented prototype. (#8882) 2021-04-29 08:25:10 +03:00
help.h Remove useless - from help.h on commands with no arguments (#8939) 2021-05-18 17:13:10 +03:00
hyperloglog.c Sort out the mess around writable replicas and lookupKeyRead/Write (#9572) 2021-11-28 11:26:28 +02:00
intset.c Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
intset.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
latency.c Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00
latency.h Disable THP if enabled (#7381) 2020-10-27 15:04:18 +02:00
lazyfree.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
listpack.c Santize dump payload: fix invalid listpack entry start with EOF (#9889) 2021-12-04 16:43:08 +02:00
listpack.h Replace ziplist with listpack in quicklist (#9740) 2021-11-24 13:34:13 +02:00
listpack_malloc.h Optimize listpack for stream usage to avoid repeated reallocs (#6281) 2021-02-16 16:17:38 +02:00
localtime.c update leap year comment when div by 400 2018-09-18 17:04:00 +08:00
lolwut.c Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
lolwut.h add include guard for lolwut.h 2020-05-05 23:35:08 -04:00
lolwut5.c Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
lolwut6.c Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
lzf.h Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
lzf_c.c Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
lzf_d.c Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
lzfP.h Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
Makefile Build commands.c in Makefile (#10039) 2022-01-06 17:59:37 +02:00
memtest.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
mkreleasehdr.sh Update mkreleasehdr.sh 2019-11-19 17:23:47 +08:00
module.c Reuse temporary client objects for blocked clients by module (#9940) 2022-01-11 19:00:56 +02:00
monotonic.c Use H/W Monotonic clock and updates to AE (#7644) 2020-08-28 11:54:10 +03:00
monotonic.h Use H/W Monotonic clock and updates to AE (#7644) 2020-08-28 11:54:10 +03:00
mt19937-64.c Fix random element selection for large hash tables. (#8133) 2020-12-23 15:52:07 +02:00
mt19937-64.h Fix random element selection for large hash tables. (#8133) 2020-12-23 15:52:07 +02:00
multi.c Sort out mess around propagation and MULTI/EXEC (#9890) 2021-12-23 00:03:48 +02:00
networking.c Reuse temporary client objects for blocked clients by module (#9940) 2022-01-11 19:00:56 +02:00
notify.c Fixed some typos, add a spell check ci and others minor fix (#8890) 2021-06-10 15:39:33 +03:00
object.c Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) 2022-01-03 19:14:13 +02:00
pqsort.c No more trailing spaces in Redis source code. 2014-06-26 18:48:40 +02:00
pqsort.h BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
pubsub.c Sharded pubsub implementation (#8621) 2022-01-02 16:54:47 -08:00
quicklist.c Fix abnormal compression due to out-of-control recompress (#9849) 2021-11-29 07:57:01 +02:00
quicklist.h Fix recent daily CI test failures (#9966) 2021-12-20 12:31:13 +02:00
rand.c Use 'void' for zero-argument functions 2014-08-08 10:05:32 +02:00
rand.h BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
rax.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
rax.h Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-09-10 13:43:38 +03:00
rax_malloc.h Cluster: hash slots tracking using a radix tree. 2017-03-27 16:37:22 +02:00
rdb.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
rdb.h Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
redis-benchmark.c Cleanup typos, incorrect comments, and fixed small memory leak in redis-cli (#9153) 2021-10-02 22:19:33 -07:00
redis-check-aof.c Add checks for overflow in redis-check-aof and loadAppendOnlyFile (#9669) 2021-11-02 17:03:07 +02:00
redis-check-rdb.c Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) 2022-01-03 19:14:13 +02:00
redis-cli.c Stringify JSON key of --json option result (#10046) 2022-01-04 17:24:29 +02:00
redis-trib.rb Redis-trib deprecated: it no longer works and it 2018-07-13 10:51:58 +02:00
redisassert.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
redisassert.h Sanitize dump payload: performance optimizations and tuning 2020-12-06 14:54:34 +02:00
redismodule.h Module events: Fail RM_SubscribeToServerEvent if event is too new (#9987) 2022-01-11 14:33:39 +02:00
release.c Speedup INFO server section. 2019-10-02 11:30:20 +02:00
replication.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
resp_parser.c Unified Lua and modules reply parsing and added RESP3 support to RM_Call (#9202) 2021-08-04 16:28:07 +03:00
resp_parser.h Unified Lua and modules reply parsing and added RESP3 support to RM_Call (#9202) 2021-08-04 16:28:07 +03:00
rio.c Fix when the master connection is disconnected, replication retry read indefinitely (#10032) 2021-12-31 14:08:04 +02:00
rio.h diskless replication rdb transfer uses pipe, and writes to sockets form the parent process. 2019-10-07 21:06:30 +03:00
script.c fix redis crached by using eval with access to volatile keys (#10080) 2022-01-11 07:56:31 +02:00
script.h Sort out mess around propagation and MULTI/EXEC (#9890) 2021-12-23 00:03:48 +02:00
script_lua.c Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
script_lua.h Redis Function Libraries (#10004) 2022-01-06 13:39:38 +02:00
sds.c There is mismach between function sdssplitlen() comments and implementation (#4909) 2021-12-22 10:00:21 +02:00
sds.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
sdsalloc.h Sanitize dump payload: fail RESTORE if memory allocation fails 2020-12-06 14:54:34 +02:00
sentinel.c Fix SENTINEL subcommands's arity (#9909) 2021-12-08 08:59:02 +02:00
server.c Reuse temporary client objects for blocked clients by module (#9940) 2022-01-11 19:00:56 +02:00
server.h Reuse temporary client objects for blocked clients by module (#9940) 2022-01-11 19:00:56 +02:00
setcpuaffinity.c cpu affinity: DragonFlyBSD support (#7956) 2020-10-25 14:14:05 +02:00
setproctitle.c Fix failed tests on Linux Alpine and add a CI job. (#8532) 2021-02-23 12:57:45 +02:00
sha1.c Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
sha1.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
sha256.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
sha256.h fix explanation of sha256 (#9220) 2021-07-10 10:04:54 -05:00
siphash.c Add sanitizer support and clean up sanitizer findings (#9601) 2021-11-11 13:51:33 +02:00
slowlog.c slowlog get command supports passing in -1 to get all logs. (#9018) 2021-06-14 16:46:45 +03:00
slowlog.h Auto-generate the command table from JSON files (#9656) 2021-12-15 21:23:15 +02:00
solarisfixes.h Check for __sun macro in solarisfixes.h, not in includers. 2015-01-09 11:23:22 +01:00
sort.c Redis Functions - Introduce script unit. 2021-12-01 23:54:23 +02:00
sparkline.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-09-10 13:43:38 +03:00
sparkline.h LATENCY GRAPH implemented. 2014-07-02 16:31:22 +02:00
stream.h Adds auto-seq-only-generation via XADD ... <ms>-* (#9217) 2021-11-30 19:56:39 +02:00
syncio.c syncWithMaster(): non blocking state machine. 2015-08-06 18:12:20 +02:00
t_hash.c Fix false positive leak reported by GCC ASAN (#9816) 2021-11-21 18:47:10 +02:00
t_list.c LPOP/RPOP with count against non existing list return null array (#10095) 2022-01-11 14:26:13 +02:00
t_set.c Sort out the mess around writable replicas and lookupKeyRead/Write (#9572) 2021-11-28 11:26:28 +02:00
t_stream.c lpGetInteger returns int64_t, avoid overflow (#10068) 2022-01-07 15:31:05 +02:00
t_string.c Prevent LCS from allocating temp memory over proto-max-bulk-len (#9817) 2021-11-21 14:30:20 +02:00
t_zset.c Sort out the mess around writable replicas and lookupKeyRead/Write (#9572) 2021-11-28 11:26:28 +02:00
testhelp.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
timeout.c Add missing calls to raxStop (#7532) 2020-07-21 08:13:05 +03:00
tls.c Multiparam config set (#9748) 2021-12-01 10:15:11 +02:00
tracking.c Sharded pubsub implementation (#8621) 2022-01-02 16:54:47 -08:00
util.c Support whitespace characters in appendfilename, and ban them in appenddirname (#10049) 2022-01-10 09:09:39 +02:00
util.h Support whitespace characters in appendfilename, and ban them in appenddirname (#10049) 2022-01-10 09:09:39 +02:00
valgrind.sup Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed 2020-12-06 14:54:34 +02:00
version.h Add Module API for version and compatibility checks (#7865) 2020-10-11 17:21:58 +03:00
ziplist.c fix invalid read on corrupt ziplist (#9831) 2021-11-23 14:56:52 +02:00
ziplist.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
zipmap.c Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
zipmap.h Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784) 2021-11-16 08:55:10 +02:00
zmalloc.c Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00
zmalloc.h Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00