mirror of
https://github.com/postgres/postgres.git
synced 2026-04-15 22:10:45 -04:00
8c4040edf4
Should a hash join exceed memory limit, the hashtable is split up into multiple batches. The number of batches is doubled each time a given batch is determined not to fit in memory. Each batch file is allocated with a block-sized buffer for buffering tuples and parallel hash join has additional sharedtuplestore accessor buffers. In some pathological cases requiring a lot of batches, often with skewed data, bad stats, or very large datasets, users can run out-of-memory solely from the memory overhead of all the batch files' buffers. Batch files were allocated in the ExecutorState memory context, making it very hard to identify when this batch explosion was the source of an OOM. This commit allocates the batch files in a dedicated memory context, making it easier to identify the cause of an OOM and work to avoid it. Based on initial draft by Tomas Vondra, with significant reworks and improvements by Jehan-Guillaume de Rorthais. Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com> Author: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Discussion: https://postgr.es/m/20190421114618.z3mpgmimc3rmubi4@development Discussion: https://postgr.es/m/20230504193006.1b5b9622%40karst#273020ff4061fc7a2fbb1ba96b281f17 |
||
|---|---|---|
| .. | ||
| backend | ||
| bin | ||
| common | ||
| fe_utils | ||
| include | ||
| interfaces | ||
| makefiles | ||
| pl | ||
| port | ||
| template | ||
| test | ||
| timezone | ||
| tools | ||
| tutorial | ||
| .gitignore | ||
| DEVELOPERS | ||
| Makefile | ||
| Makefile.global.in | ||
| Makefile.shlib | ||
| meson.build | ||
| nls-global.mk | ||