postgresql/src/include
Thomas Munro ba02d24bac Fix corruption when relation truncation fails.
RelationTruncate() does three things, while holding an
AccessExclusiveLock and preventing checkpoints:

1. Logs the truncation.
2. Drops buffers, even if they're dirty.
3. Truncates some number of files.

Step 2 could previously be canceled if it had to wait for I/O, and step
3 could and still can fail in file APIs.  All orderings of these
operations have data corruption hazards if interrupted, so we can't give
up until the whole operation is done.  When dirty pages were discarded
but the corresponding blocks were left on disk due to ERROR, old page
versions could come back from disk, reviving deleted data (see
pgsql-bugs #18146 and several like it).  When primary and standby were
allowed to disagree on relation size, standbys could panic (see
pgsql-bugs #18426) or revive data unknown to visibility management on
the primary (theorized).

Changes:

 * WAL is now unconditionally flushed first
 * smgrtruncate() is now called in a critical section, preventing
   interrupts and causing PANIC on file API failure
 * smgrtruncate() has a new parameter for existing fork sizes,
   because it can't call smgrnblocks() itself inside a critical section

The changes apply to RelationTruncate(), smgr_redo() and
pg_truncate_visibility_map().  That last is also brought up to date with
other evolutions of the truncation protocol.

The VACUUM FileTruncate() failure mode had been discussed in older
reports than the ones referenced below, with independent analysis from
many people, but earlier theories on how to fix it were too complicated
to back-patch.  The more recently invented cancellation bug was
diagnosed by Alexander Lakhin.  Other corruption scenarios were spotted
by me while iterating on this patch and earlier commit 75818b3a.

Back-patch to all supported releases.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reported-by: rootcause000@gmail.com
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18146-04e908c662113ad5%40postgresql.org
Discussion: https://postgr.es/m/18426-2d18da6586f152d6%40postgresql.org
2024-12-20 23:57:26 +13:00
..
access Ensure that pg_amop/amproc entries depend on their lefttype/righttype. 2024-12-07 15:56:28 -05:00
archive Redesign archive modules 2023-02-17 14:26:42 +09:00
backup Fix some typos and some incorrectly duplicated words 2023-04-18 14:03:49 +12:00
bootstrap Update copyright for 2023 2023-01-02 15:00:37 -05:00
catalog Cope with inplace update making catcache stale during TOAST fetch. 2024-06-27 19:21:10 -07:00
commands doc: Add better description for rewrite functions in event triggers 2024-10-29 15:35:18 +09:00
common Fix corner-case 64-bit integer subtraction bug on some platforms. 2023-11-09 09:53:05 +00:00
datatype Code review for commit 05a7be935. 2023-01-26 12:51:00 -05:00
executor Simplify executor's determination of whether to use parallelism. 2024-12-09 14:38:19 -05:00
fe_utils Pre-beta mechanical code beautification. 2023-05-19 17:24:48 -04:00
foreign Update copyright for 2023 2023-01-02 15:00:37 -05:00
jit Monkey-patch LLVM code to fix ARM relocation bug. 2024-11-06 23:09:03 +13:00
lib simplehash: Free collisions array in SH_STAT 2024-04-07 19:09:02 -07:00
libpq Remove a few unused global variables and declarations. 2023-06-12 16:25:37 +03:00
mb Move is_valid_ascii() to ascii.h. 2024-01-29 12:09:03 -06:00
nodes Fix C23 compiler warning 2024-11-26 13:37:06 +01:00
optimizer Account for optimized MinMax aggregates during SS_finalize_plan. 2024-05-18 14:31:35 -04:00
parser Avoid unnecessary plancache revalidation of utility statements. 2023-08-24 12:02:40 -04:00
partitioning Revert "Move PartitionPruneInfo out of plan nodes into PlannedStmt" 2023-05-04 12:09:59 +02:00
pch Update copyright for 2023 2023-01-02 15:00:37 -05:00
port Fix gai_strerror() thread-safety on Windows. 2024-02-12 11:14:42 +13:00
portability instr_time: Represent time as an int64 on all platforms 2023-01-20 21:16:47 -08:00
postmaster Fix wal_writer_flush_after initializer value. 2023-05-15 11:19:54 +12:00
regex Avoid assertion due to disconnected NFA sub-graphs in regex parsing. 2024-11-15 18:23:38 -05:00
replication Fix possibility of logical decoding partial transaction changes. 2024-07-11 22:48:18 +09:00
rewrite Make Vars be outer-join-aware. 2023-01-30 13:16:20 -05:00
snowball Update copyright for 2023 2023-01-02 15:00:37 -05:00
statistics Update copyright for 2023 2023-01-02 15:00:37 -05:00
storage Fix corruption when relation truncation fails. 2024-12-20 23:57:26 +13:00
tcop Restrict accesses to non-system views and foreign tables during pg_dump. 2024-08-05 06:05:28 -07:00
tsearch Update tsearch regex memory management. 2023-04-08 22:09:17 +12:00
utils Simplify executor's determination of whether to use parallelism. 2024-12-09 14:38:19 -05:00
.gitignore Refactor dlopen() support 2018-09-06 11:33:04 +02:00
c.h Assume that <stdbool.h> conforms to the C standard. 2024-11-25 20:54:05 +13:00
fmgr.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
funcapi.h Pre-beta mechanical code beautification. 2023-05-19 17:24:48 -04:00
getopt_long.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
Makefile Include headers of archive/ in installation 2023-03-10 20:08:10 +09:00
meson.build meson: Make some Meson style more consistent with surrounding code 2023-06-29 13:06:02 +02:00
miscadmin.h Fix improper interactions between session_authorization and role. 2024-11-11 10:29:54 -05:00
pg_config.h.in Assume that <stdbool.h> conforms to the C standard. 2024-11-25 20:54:05 +13:00
pg_config_ext.h.in Autoconfiscate selection of 64-bit int type for 64-bit large object API. 2012-10-07 21:52:43 -04:00
pg_config_ext.h.meson meson: Add initial version of meson based build system 2022-09-21 22:37:17 -07:00
pg_config_manual.h Fix typos in comments 2023-05-02 12:23:08 +09:00
pg_getopt.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
pg_trace.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
pgstat.h Add writeback to pg_stat_io 2023-05-17 11:18:35 -07:00
pgtar.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
pgtime.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
port.h Add SHELL_ERROR and SHELL_EXIT_CODE magic variables to psql. 2023-03-21 13:03:56 -04:00
postgres.h New header varatt.h split off from postgres.h 2023-01-10 05:54:36 +01:00
postgres_ext.h Move RelFileNumber declarations to common/relpath.h. 2022-09-27 12:01:57 -04:00
postgres_fe.h Update copyright for 2023 2023-01-02 15:00:37 -05:00
varatt.h New header varatt.h split off from postgres.h 2023-01-10 05:54:36 +01:00
windowapi.h Update copyright for 2023 2023-01-02 15:00:37 -05:00