"prev_tuple" was overwritten with a new tuple coming from
CopyIndexTuple() on each loop, leaking memory for every tuple processed
on entry tree pages. The function uses a dedicated memory context, but
this could leave unused large areas of memory while processing a large
GIN index, the larger the worse.
Oversight in 14ffaece0f.
Author: Kirill Reshke <reshkekirill@gmail.com>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Discussion: https://postgr.es/m/CALdSSPjTS6TYe5=5NfMUBYZyQu5cn=ABL6K5_OZjzGWqnwXeBw@mail.gmail.com
Backpatch-through: 18
psql decided whether to use the pager in expanded output without
accounting for possible wrapping of column values. This could
allow it to not use the pager in cases where it should do so.
To fix, move the IsPagerNeeded decision in print_aligned_vertical()
down until after the wrapped data width is known. Then, if we're in
wrapped mode, prepare a width_wrap array specifying that width (which,
in vertical mode, is the same for all columns).
This is fixing an omission in 27da1a796, so back-patch to v19
where that came in.
Author: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Erik Wienhold <ewie@ewie.name>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/A44110E7-6A03-4C67-95AD-527192A6C768@gmail.com
Backpatch-through: 19
AlterPropGraph() cleans up pg_propgraph_property entries that are
orphaned by dropping an element or by dropping properties associated
with an element. But it did not clean up pg_propgraph_property
entries that are orphaned by dropping labels associated with an
element. Fix this missing case.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Author: zengman <zengman@halodbtech.com>
Discussion: https://www.postgresql.org/message-id/flat/tencent_76F6ACA2364EAA1E5DBD7A47%40qq.com
ON SELECT rules must be named "_RETURN", while other kinds of rules
must not be; this ancient restriction is depended on by various client
code. We successfully enforced this convention in most places, but
ALTER RULE allowed renaming a non-SELECT rule to "_RETURN". Notably,
that would break dump/restore, since the eventual CREATE RULE command
would reject the name.
While at it, remove DefineQueryRewrite's hack to substitute "_RETURN"
for the convention that was used before 7.3. We dropped other
server-side code that supported restoring pre-7.3 dumps some time ago
(notably in e58a59975 and nearby commits), but this bit was missed.
Bug: #19543
Reported-by: Adam Pickering <adamkpickering@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19543-461228e77f3b32fc@postgresql.org
Backpatch-through: 14
getObjectDescription() currently constructs property graph-related
object descriptions incrementally with appendStringInfo(). This
effectively fixes the word order in English, which makes the messages
difficult to translate naturally into languages such as Japanese.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/20260528.121622.1662808269492494574.horikyota.ntt%40gmail.com
We were skipping a bunch of things that are mostly unnecessary for
REPACK. However, one thing that seems would be better to pass closer to
truth, is the updatedCols bitmapset in the range table entry for the
repacked table. Cons up an RTE and install it into the EState.
This only has an effect on btree indexes, because certain operations are
optimized in the case of unchanged columns; and even then, correctnesss
is not being compromised.
The values we pass after this commit are not fully trustworthy either,
because we simply say "all columns were updated" for all insert/updates,
regardless of whether their values were actually modified or not.
However, this way we err to the side of caution rather than to the
opposite direction as we were originally doing. This could be refined
in the future, but there's a trade-off: determining whether the column
was in fact updated could be expensive.
Author: Antonin Houska <ah@cybertec.at>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/18222.1782126731@localhost
gbt_var_consistent() handled the <> (BtreeGistNotEqual) strategy without
distinguishing leaf from internal pages, unlike every other strategy.
In particular, it tried to apply the datatype-specific f_eq method,
which is completely wrong since internal keys might not have the same
representation as leaf keys. This led to OOB reads and potentially
crashes, and most likely to wrong query results as well.
On leaf pages we can apply the inverse of what the Equal strategy does.
On internal pages, use a correct implementation of what the previous
code intended: we can descend if the query value equals both bounds,
*so long as the bounds aren't truncated*. With truncated bounds we
don't quite know the range of what's below, so we must always descend.
Adjust the code in gbt_num_consistent() to look similar, too. This
fixes a performance buglet in that there's no need to do two comparisons
on a leaf entry, but the main point is just to keep code consistency.
Reported-by: 王跃林 <violin0613@tju.edu.cn>
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/AH*AvQCYKhQGVvPWi1GiU4oY.8.1781609375063.Hmail.3020001251@tju.edu.cn
Backpatch-through: 14
If we're dealing with leaf entries, the function to call is bitcmp
not byteacmp. Using byteacmp didn't lead to any obvious failure,
but it did result in sorting the entries in a way not matching the
datatype's actual sort order. Hence the constructed index would be
less efficient than one would expect, and in particular worse than
what you got before this code was added in v18 (by commit e4309f73f).
We might want to recommend that users reindex btree_gist indexes
on bit/varbit columns.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/AH*AvQCYKhQGVvPWi1GiU4oY.8.1781609375063.Hmail.3020001251@tju.edu.cn
Backpatch-through: 18
Commit ce146621 ensures that ERROR is raised if a session tries to read
pages of another session's temp table. But there is a corner case where
the other session's temp table is empty -- in this case the INSERT
command bypasses our checks and executes without any errors.
Such behavior is inconsistent and erroneous: it leaves an invalid buffer
in the temp buffers pool. Since the buffer was created for another
session's temp table, we get an error "no such file or directory" when
trying to flush it.
This commit fixes it by adding a RELATION_IS_OTHER_TEMP check in the
relation-extension path.
Backpatch to 16, because it is the first release after 31966b151e, which
introduced a separate local relation extension function
ExtendBufferedRelLocal(), which lacks of RELATION_IS_OTHER_TEMP() check.
As this fix introduces more checks to 013_temp_obj_multisession.pl, backpatch
the whole test script to 16.
Discussion: https://postgr.es/m/CAJDiXgiX2XZBHDNo%2BzBbvku%2BtchrUurvPRaN1_40mEQ1_sG90g%40mail.gmail.com
Author: Daniil Davydov <3danissimo@gmail.com>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Reviewed-by: Imran Zaheer <imran.zhir@gmail.com>
Reviewed-by: ZizhuanLiu X-MAN <44973863@qq.com>
Backpatch-through: 16
When dropping a property by name from a label, the code checked only
whether the property existed in the graph's property catalog. It did
not verify that the property was actually associated with the given
label, resulting in passing InvalidOid to performDeletion(). Fix it
by explicilty checking the label property association.
While at it also rearrange the code so as to avoid multiple ereport
calls for the same error in the same block.
Author: Chao Li <lic@highgo.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/1DA5D52A-4AFA-426E-83F7-42ED974D682B%40gmail.com
BackendKeyData length was increased from 4 bytes to a variable-length
length (up to 256 bytes) in a460251f0a. However, pqTrace still traces
it as a 4 bytes key, leading to a "mismatched message length" warning
message. The same issue impacts the tracing of CancelRequest.
This patch fixes the issue by using pqTraceOutputNchar instead of
pqTraceOutputInt32 in both cases.
Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://www.postgresql.org/message-id/CAO6_Xqo6gTv9=76H=k2qDRFU+KHuBiY2S=bQynEr6J8gS7L6xA@mail.gmail.com
Backpatch-through: 18
In order to replay concurrent changes, REPACK CONCURRENTLY needs the
pg_attrdef tuples for the transient table to be there, in case a tuple
is modified concurrently with REPACK and requires to store the value
from the generated column (which, with the current arrangements, means
all tuples concurrently updated or inserted). Fix by creating a copy of
them from the original table. Add a test that tickles the bug.
Author: Antonin Houska <ah@cybertec.at>
Reported-by: Ewan Young <kdbase.hack@gmail.com>
Diagnosed-by: Ewan Young <kdbase.hack@gmail.com>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Backpatch-through: 19
Discussion: https://postgr.es/m/CAON2xHMrELwx9vKg6niSf8fMBA=-MGXmG=MPQU6+vMVhGjF8kQ@mail.gmail.com
Per SQL/PGQ standard, every graph element must have at least one
label. When dropping a label from a graph element, ensure that there
exists at least one other label on the element. If the label being
dropped is the only label on the element, raise an error.
We hold a ShareRowExclusiveLock when modifying a property graph.
Hence the label will not be dropped even when multiple labels are
being dropped concurrently.
Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Author: Satyanarayana Narlapuram <satyanarlapuram@gmail.com>
Reported-by: Satyanarayana Narlapuram <satyanarlapuram@gmail.com>
Discussion: https://www.postgresql.org/message-id/CAHg+QDeP=mTHTV48R23zKMy1SBmCKZ_L7-z5zKnYyw+K0x-gCg@mail.gmail.com
Commit fb421231da extended \df to include procedures, but its tab
completion continued not to show procedures.
Update \df tab completion to include procedures as well.
Backpatch to all supported versions.
Author: Erik Wienhold <ewie@ewie.name>
Reviewed-by: Surya Poondla <suryapoondla4@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/10fbfdfe-80f6-4ef9-b8b3-f7be0eb53a50@ewie.name
Backpatch-through: 14
Previously, MultiXactId wraparound hints suggested dropping stale
replication slots. While that advice is appropriate for transaction ID
wraparound, where replication slots can hold back XID horizons,
it was misleading for MultiXactId wraparound. Following it could lead
users to drop replication slots unnecessarily without helping resolve
the MultiXactId wraparound condition.
MultiXact cleanup is not directly delayed by replication slots.
Instead, it depends on whether old MultiXactIds can still be seen
as live by running transactions.
This commit removes the replication slot advice from MultiXactId
wraparound hints, and documents that stale replication slots are
normally not relevant to resolving MultiXactId wraparound problems.
Backpatch to all supported branches.
BUG #18876
Reported-by: Haruka Takatsuka <harukat@sraoss.co.jp>
Author: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/18876-0d0b53bad5a1f4c1@postgresql.org
Backpatch-through: 14
A foreign scan can target a single relation while still reaching the
fs_relids branch of pgpa_build_scan() -- for example, when postgres_fdw
pushes an aggregate down over one foreign table. In that case, no
advice should be emitted.
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
Co-authored-by: Robert Haas <rhaas@postgresql.org>
Discussion: http://postgr.es/m/CAKYtNAofuAJBz6++SeikpCb=Y=MO1QgEuZNJ+KZOP2johF1r4Q@mail.gmail.com
Previously, test_custom_var_stats and test_custom_fixed_stats silently
skipped pgstat_register_kind() when not loaded via
shared_preload_libraries, behavior inherited from injection_points.
This left the SQL functions callable without the kind registered,
leading to various issues on the backend side.
This code is not designed to work without the pgstats kinds registered.
pgstat_register_kind() gets now called when these libraries are loaded,
with or without shared_preload_libraries, letting the registration fail
if loading the modules at a later step than startup. test_custom_rmgrs
does the same thing.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Discussion: https://postgr.es/m/akS/ldidWeqG1FWk@bdtpg
Backpatch-through: 19
The jsonpath .decimal(precision[, scale]) method built its numeric
typmod by calling numerictypmodin() through DirectFunctionCall1(), which
can throw a hard error for an incorrect set of precision and/or scale
vaulues. This breaks the silent mode supported by this function, that
should not fail.
Most of the jsonpath code uses the soft error reporting to bypass
errors, which is what this fix does by avoiding a direct use of
numerictypmodin(). Its code is refactored to use a new routine called
make_numeric_typmod_safe(), able to take an error context in input.
numerictypmodin() sets no context, mapping to its previous behavior.
The jsonpath code sets or not a context depending on the use of the
silent mode. This result leads to some nice simplifications:
numerictypmodin() feeds on an array, we can now pass directly values for
the scale and precision.
Oversight in 66ea94e8e6.
Author: Ewan Young <kdbase.hack@gmail.com>
Discussion: https://postgr.es/m/CAON2xHMaigKABiyPBBq3Sjd3gp7uWMJXnnMHt=s85V1ij3KP1w@mail.gmail.com
Backpatch-through: 17
The float4 and float8 btree_gist opclasses compared keys with raw C
operators (==, <, >). IEEE 754 makes every comparison involving NaN
false, so GiST disagreed with the regular float comparison operators
and with the btree opclass, which uses float[4|8]_cmp_internal()
(so that all NaNs are equal and NaN sorts after every non-NaN value).
In addition, the penalty and distance functions were not careful
about NaNs, and the penalty functions could also misbehave for IEEE
infinities. Wrong answers from the penalty functions would probably
do no more than make the index non-optimal, but the distance mistakes
were visible from SQL.
To fix, make the comparison functions rely on the same NaN-aware
comparison functions the core code uses, and rewrite the penalty
and distance functions to follow the rules that NaNs are equal
but maximally far away from non-NaNs. The penalty_num() code was
formerly shared between integral and float cases, but I chose to make
two copies so that the integral cases are not saddled with the extra
logic for NaNs and infinities/overflows. I also rewrote it as static
inline functions instead of an unreadable and uncommented macro.
The float penalty functions were previously unreached by the
regression tests, so add new test cases to exercise them.
There's no on-disk format change, but users who have NaN entries
in a btree_gist index would be well advised to reindex it.
Bug: #19501
Bug: #19524
Reported-by: Man Zeng <zengman@halodbtech.com>
Reported-by: Yuelin Wang <3020001251@tju.edu.cn>
Author: Bill Kim <billkimjh@gmail.com>
Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org
Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org
Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com
Backpatch-through: 14
The descriptions of the component scores state that values greater
than or equal to the corresponding weight parameter mean autovacuum
will process the table. However, since the code that determines
whether to vacuum or analyze a table actually checks whether the
threshold is exceeded, it's more accurate to say "greater than"
there.
Author: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Discussion: https://postgr.es/m/E3ABDC6B-80CA-4C37-BA0B-A519D49F4C66%40gmail.com
Backpatch-through: 19
Previously, if a base backup failed after it had started streaming
files, pg_stat_progress_basebackup could continue to show a stale
progress entry even though the backup was no longer running. This could
be observed when the client kept the replication connection open after
the error. It is normally not observable when using pg_basebackup,
because the client disconnects after the error.
The problem was that progress reporting was cleared only after
successful completion.
This commit moves the progress reporting cleanup into the progress
sink's cleanup callback so that it is cleared after both successful
and failed backups.
Backpatch to v15. v14 has the same issue, but the fix does not apply
cleanly because it lacks the base backup sink infrastructure. Since
the bug does not affect the backup itself and is normally not
observable when using pg_basebackup, skip the v14 backpatch.
Author: Chao Li <lic@highgo.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/EA1A6CD2-EFA6-462B-9A02-03003555AB4A@gmail.com
Backpatch-through: 15
Commit bc60ee860 added a connection warning after successful MD5
authentication, but only for the md5 authentication method. A role with
an MD5-encrypted password can also authenticate via the password method,
which left that path without the same deprecation warning.
Emit the MD5 deprecation connection warning after successful
password authentication as well, when the stored password is
MD5-encrypted.
Backpatch to v19, where the MD5 connection warning was introduced.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/CAHGQGwGkWfn5rtHzvdRbVk+PCefQU3gun3hc7QnaMXHFa5Bu3w@mail.gmail.com
Backpatch-through: 19
Fixup for commit c05dee1911. It fits better with the style and APIs
to print separate primary and a detail messages instead of one
multiline message.
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CAHut+PsvQJQnQO0KT0S2oegenkvJ8FUuY-QS5syyqmT24R2xFQ@mail.gmail.com
off_t is most likely a 64-bit integer, so casting it to a 32-bit type
for output could lose data. There are more issues like this in the
tree, but this is an instance where this could actually happen in
practice, since base backups are routinely larger than 4 GB. So this
is separated out as a bug fix.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/20ce62fa-47fc-457b-b504-12f3c1651726%40eisentraut.org
Replace pg_attribute_aligned with C11 alignas, for consistency with
current conventions.
(These new uses were added by commit fbc57f2bc2, which was developed
concurrently with the switch from pg_attribute_aligned to C11 standard
alignas, and it ended up being committed with the "old" style.)
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/CANWCAZaKhE+RD5KKouUFoxx1EbUNrNhcduM1VQ=DkSDadNEFng@mail.gmail.com
MultiXactOffsetStorageSize() and GetMultiXactInfo() are called to gather
the information reported by the function, but were wasteful for the case
where a role does not have the privileges of pg_read_all_stats, where we
return a set of NULLs. These calls are moved to the code path where
their results are used.
Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQAonQh7be=wOR-CJFW=bgMBz5wW_bv4t0OFxbgn-794JCQ@mail.gmail.com
Backpatch-through: 19
Commit 4035cd5d4 added LZ4 compression for full-page writes in WAL, and
retained "on" as a backward-compatible way to specify the builtin PGLZ
method. Document this meaning of "on" and update postgresql.conf.sample
to make the equivalence clear.
Author: Christoph Berg <myon@debian.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/akJDHRtXwGLTppsQ@msg.df7cb.de
Backpatch-through: 15
Commit a1643d40b removed RADIUS authentication, but apparently
overlooked initdb's list of accepted authentication methods. As a
result, initdb still accepted radius for --auth, --auth-host, and
--auth-local, allowing it to create a pg_hba.conf that the server could
not load.
Remove radius from initdb's local and host authentication method lists.
Backpatch-through: 19
Author: Chao Li <lic@highgo.com>
Discussion: https://postgr.es/m/983F946B-A7CE-4C93-B5F0-665616F72254%40gmail.com
We previously allowed this, but it leads to odd behaviors, basically
because putting a SRF there is inconsistent with the principle that a
window function doesn't change the number of rows in the query result.
There doesn't seem to be a strong reason to try to make such cases
behave consistently. Users should put their SRFs in lateral FROM
clauses instead.
This issue has been sitting on the back burner for multiple years
now, partially because it didn't seem wise to back-patch such a
change. Let's squeeze it into v19 before it's too late.
Bug: #17502
Bug: #19535
Reported-by: Daniel Farkaš <daniel.farkas@datoris.com>
Reported-by: Qifan Liu <imchifan@163.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/17502-281a7aaacfaa872a@postgresql.org
Discussion: https://postgr.es/m/19535-376081d7cc07c86d@postgresql.org
Backpatch-through: 19
The existing description says the ranges of merged range-partitions
"must be adjacent" only under the heading "If the DEFAULT partition is
not in the list of merged partitions". That could be misread as
a restriction tied to the presence of a default partition. In fact,
merging non-adjacent ranges is rejected regardless of whether
the partitioned table has a default partition; spell that out explicitly.
Also, this commit removes a small redundancy in the documentation sentence
stating that "merged range-partitions" are "to be merged".
Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/aj6BPoziSb-F8aJz%40pryzbyj2023
Reported-by: Pavel Borisov <pashkin.elfe@gmail.com>
Backpatch-through: 19
Fixup for commit 8a27d418f8, which converted many functions to use
"const Datum *" instead of "Datum *", including some SPI functions.
For SPI_cursor_open(), the code was updated but not the documentation.
For SPI_cursor_open_with_args(), the documentation was updated but not
the code. (Possibly, these two were confused with each other.) Also,
SPI_execp() and SPI_modifytuple() were not updated, even though they
are closely related to the functions touched by the previous commit
and now look inconsistent. Fix all these.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/86b5162f-c472-40fa-997b-0450dece1dec%40eisentraut.org
Other statistics views (pg_stat_io, pg_stat_database, etc.) use float8
for all measured-time columns, the new pg_stat_lock standing out as an
outlier by using bigint.
This commit aligns pg_stat_lock with the other stats views for
consistency. Like pg_stat_io, the time is stored in microseconds, and
is displayed in milliseconds with a conversion done when the view is
queried.
While on it, replace a use of "long" by PgStat_Counter, the former could
overflow for large wait times where sizeof(long) is 4 bytes (aka WIN32).
Bump catalog version.
Author: Tatsuya Kawata <kawatatatsuya0913@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAHza6qerEiQehrbW5xaXyxvR0qJe3KBX1R4kocDz1+7Ygu8x-g@mail.gmail.com
Backpatch-through: 19
LockBufferForCleanup() acquires the exclusive content lock, checks the
buffer's shared pin count, and, if other pins remain, registers itself
as the BM_PIN_COUNT_WAITER before waiting for an unpin notification.
Since commits 5310fac6e0 and c75ebc657f, however, a shared buffer
pin can be released while BM_LOCKED is set, introducing the following
race:
- LockBufferForCleanup() observes a refcount greater than one.
- Before it sets BM_PIN_COUNT_WAITER, another backend releases the
last conflicting pin.
- Since BM_PIN_COUNT_WAITER is not yet set, no wakeup is sent.
- LockBufferForCleanup() then sets BM_PIN_COUNT_WAITER and goes to
sleep, even though only its own pin remains.
As a result, LockBufferForCleanup() can sleep indefinitely because
the wakeup corresponding to the last conflicting unpin has already been
missed.
Fix this by setting BM_PIN_COUNT_WAITER while holding the buffer
header lock, then rechecking the refcount before releasing the content
lock. If only our pin remains, clear the waiter state and proceed
without sleeping. Otherwise, wait as before.
This issue was reported by buildfarm member skink, where it manifested
as intermittent timeouts in 048_vacuum_horizon_floor.pl.
Backpatch to v19, where commits 5310fac6e0 and c75ebc657f
introduced the race.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/7685519a-0bf9-4e17-93ca-7e3aa10fa29c@gmail.com
Backpatch-through: 19
Previously, if an unlogged sequence was created on the primary and
replicated to a standby, reading the sequence after promoting the
standby (for example, with nextval()) could trigger the following
assertion failure:
TRAP: failed Assert("((const PageHeaderData *) page)->pd_special >= SizeOfPageHeaderData")
In non-assert builds, the same operation could instead fail with an
error such as:
ERROR: bad magic number in sequence
The problem was that seq_redo() updated the init fork page in shared
buffers but did not flush it to disk. During promotion,
ResetUnloggedRelations() recreates the main fork of unlogged
relations by copying the init fork from disk, bypassing shared
buffers. As a result, the main fork could be recreated from a stale
init fork instead of the WAL-replayed page.
Fix this by introducing a helper to flush init fork buffers
immediately, and make seq_redo() use it. As a result, the main fork
of an unlogged sequence is recreated from the up-to-date init fork on
disk, allowing the unlogged sequence to be read successfully after
standby promotion.
Backpatch to v15, where unlogged sequences were introduced.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwH1Ssze3XM6wjoTjSLVOR041c6xP+vsdLP951=w8oG8bA@mail.gmail.com
Backpatch-through: 15
statatt_build_stavalues() and array_in_safe() have been relying on
InitFunctionCallInfoData() with a locally-filled state to call a data
type input function. InputFunctionCallSafe() can be used to achieve the
same job, simplifying some code.
This fixes an over-allocation of FunctionCallInfoBaseData done in
statatt_build_stavalues(), where there was space for 8 elements but only
3 were needed. The over-allocation exists since REL_18_STABLE, and was
harmless in practice.
While on it, fix some comments for both routines, where elemtypid was
mentioned.
Backpatch down to v19. This code has been reworked during the last
development cycle while working on the restore of extended statistics,
so this keeps the code consistent across all branches.
Author: Jian He <jian.universality@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CACJufxEGah9PaiTQ=cG14GMMBsUQ3ohGct9tdSwbMQPQ0-nbbQ@mail.gmail.com
Backpatch-through: 19
Per the checklist in RELEASE_CHANGES for the creation of a new stable
branch, this commit does the following things:
- Update URLs of top-level README and Makefile to point to the new
stable version.
This is required before the creation of a new branch. pgindent is
clean, as well as is reformat-dat-files.
perltidy version is v20230309, as documented in pgindent's README.
It is not clear how the implicit condition of FOR PORTION OF should
interact with the use of a cursor. Normally, we forbid combining
WHERE CURRENT OF with other WHERE conditions. The SQL standard only
includes FOR PORTION OF with <update statement: searched> and <delete
statement: searched>, not <update statement: positioned> or <delete
statement: positioned>, so it is easy for us to exclude the
functionality, at least for now.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BrenyUEKPexUYsH4qeU8_o1jqKsUkEWca1keS6n21shgG1g%2BA%40mail.gmail.com
Treat copy_file_range() return value of zero as an error: it indicates
that no bytes could be copied (perhaps the source file is shorter than
expected), and the existing retry loop would otherwise spin forever
since nwritten would never reach BLCKSZ.
The other uses of copy_file_range() in the tree don't have this
problem.
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Yingying Chen <cyy9255@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/3208cf7a-c7f3-41eb-92f6-33cbeff4df40%40eisentraut.org
While on it, let's add links pointing to the set of SQL commands
generated by these functions. The descriptions of the functions are
exactly the same, just moved around.
Author: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Ian Lawrence Barwick <barwick@gmail.com>
Discussion: https://postgr.es/m/CAHut+Pun9Z8qZFJTa9fLgdhM=Cip9d-cnx2YXDW6eFrSwbQj1g@mail.gmail.com
PL/Python and its hstore and jsonb transforms build SQL values from
Python containers by calling Python C API functions that can return
NULL, and in several places the result was used without first checking
it.
On the sequence side, PySequence_GetItem() is used when converting a
returned sequence into a SQL array or composite value, when reading
the argument list passed to plpy.execute() or plpy.cursor(), and when
reading the list of type names given to plpy.prepare(). On the
mapping side, the hstore and jsonb transforms call PyMapping_Size()
and PyMapping_Items() and then index the result with PyList_GetItem()
and PyTuple_GetItem().
All of these return NULL (or -1), with a Python exception set, for a
broken object: for example one whose __getitem__() or items() raises,
or which reports a length that disagrees with what it actually yields.
The unchecked result was then dereferenced, crashing the backend.
Fix this by checking the result of each call and reporting a regular
error if it failed, so that the underlying Python exception is
surfaced instead of taking down the session.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49BKM9wP6m8bCXEpHwQKp7usvOGV6Jf=J7FYr_BCpxLqg@mail.gmail.com
Backpatch-through: 14
Commit b7b27eb41a, which added foreign-key fast-path batching to
ri_triggers.c, registered ri_FastPathXactCallback() via
RegisterXactCallback() to clear the fast-path batching state at end of
transaction. RegisterXactCallback() is documented as intended for
dynamically loaded modules; built-in code is supposed to hardwire its
end-of-xact hooks into xact.c, mainly so callback ordering can be
controlled where it matters (see the header comment on
RegisterXactCallback()).
Convert the callback into a plain AtEOXact_RI() function and call it
directly from CommitTransaction(), PrepareTransaction() and
AbortTransaction(), alongside the other AtEOXact_* cleanup steps, and
drop the RegisterXactCallback() registration.
Like the other AtEOXact_* routines, AtEOXact_RI() takes an isCommit
argument and treats the two paths differently. On commit or prepare
the fast-path cache must already have been flushed and torn down by
the after-trigger batch callback, so a surviving cache indicates a
trigger batch was never flushed -- which would have silently skipped
FK checks -- and draws an Assert plus a WARNING. On abort a surviving
cache is expected (a flush may have errored out partway) and is simply
reset.
There is no ordering dependency here: AtEOXact_RI() only resets
backend-local static state (the cache pointer, the
callback-registered flag, and the in-flush guard). It touches no
relations, locks, buffers or catalogs, so its position relative to
ResourceOwnerRelease() and the surrounding AtEOXact_* calls does not
matter. On a normal commit the fast-path cache has already been
flushed and torn down by ri_FastPathEndBatch() (an
AfterTriggerBatchCallback fired from AfterTriggerFireDeferred(), well
before any end-of-xact callback), so the reset is a no-op; its real
job is the abort path, where teardown may not have run and the static
pointers would otherwise dangle into the next transaction. The cache
memory itself lives in TopTransactionContext and is freed by the
end-of-transaction memory-context reset on both paths.
The companion RegisterSubXactCallback() use from b7b27eb41a was
already removed by commit 4113873a, which confined fast-path batching
to the top transaction level, so only the RegisterXactCallback() use
remained.
Reported-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/ajypPeEWceXRGAEW@bdtpg
