Given a WHERE clause like "int[] @@ query_int" or "query_int ~~ int[]"
where the query_int side is a table column having statistics,
_int_matchsel() exited without remembering to free the statistics
tuple. This would typically lead to warnings about cache refcount
leakage, like
WARNING: resource was not closed: cache pg_statistic (73), tuple 42/12 has count 1
It's been wrong since this code was added, in commit c6fbe6d6f.
Bug: #19492
Reported-by: Man Zeng <zengman@halodbtech.com>
Author: Man Zeng <zengman@halodbtech.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19492-ddcd0e22399ef85a@postgresql.org
Backpatch-through: 14
Previously, dblink accepted the use_scram_passthrough option on
foreign-data wrappers via ALTER FOREIGN DATA WRAPPER dblink_fdw
OPTIONS, even though the setting had no effect there.
use_scram_passthrough should be only meaningful for foreign servers
and user mappings, so this commit updates dblink to accept the option
only in those contexts.
Backpatch to v18, where use_scram_passthrough was introduced.
Author: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwEJ8rZjmbOvCicyr4vbuLio082bNTde0WNoSWaWr9wVcg@mail.gmail.com
Backpatch-through: 18
Commit 97f6fc10ff changed postgres_fdw so that user-mapping settings
override foreign server settings for use_scram_passthrough. This commit
applies the same behavior to dblink.
Backpatch to v18, where use_scram_passthrough was introduced.
Author: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwEJ8rZjmbOvCicyr4vbuLio082bNTde0WNoSWaWr9wVcg@mail.gmail.com
Backpatch-through: 18
Previously, when use_scram_passthrough was specified on both a foreign server
and a user mapping, the server-level setting took precedence over the
user-mapping setting. This was inconsistent with the usual semantics of
postgres_fdw options, where foreign server options provide shared defaults
and user mapping options override them on a per-user basis.
This commit updates postgres_fdw so that the user-mapping setting takes
precedence when use_scram_passthrough is specified in both places. This
matches the behavior of other connection options such as sslcert and sslkey.
Backpatch to v18, where use_scram_passthrough was introduced. In v18,
this only affects limited configurations that specify conflicting values
at both the foreign server and user-mapping levels. In such cases, users
would naturally expect the user-mapping setting to override the server-level
setting, so changing the behavior should be minimally disruptive.
Also keeping v18 as the only branch with different semantics for
use_scram_passthrough would be unnecessarily confusing, so backpatch
this fix to v18.
Author: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwEJ8rZjmbOvCicyr4vbuLio082bNTde0WNoSWaWr9wVcg@mail.gmail.com
Backpatch-through: 18
This reverts commit 0d3dba38c7, which was determined to have
fundamental flaws. This restricts REPACK (CONCURRENTLY) so that only
one process can run it concurrently on different tables and even on
different databases; we'll lift that restriction in another way during
the next development cycle.
Reported-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAA4eK1Jg21ODQ7fS2fvN5W_S5kDRhAP5inj3XMRQaa=s-GbYhw@mail.gmail.com
Commit 112faf1378 added custom notice receivers for replication,
postgres_fdw, and dblink so that remote NOTICE, WARNING, and similar
messages are reported via ereport(). However, those notice receivers were
installed only after libpqsrv_connect() and libpqsrv_connect_params()
returned, by which point libpq connection startup had already completed.
As a result, messages emitted during connection establishment could be
missed.
This commit fixes the issue by splitting libpqsrv_connect() and
libpqsrv_connect_params() into separate start and complete phases:
libpqsrv_connect_start(), libpqsrv_connect_params_start(), and
libpqsrv_connect_complete(). This allows callers to perform
per-connection setup, such as installing a notice receiver, after the
connection has been started but before startup completes.
Note that callers of libpqsrv_connect_start() and
libpqsrv_connect_params_start() must still call
libpqsrv_connect_complete(), even if the start function returns NULL, so
that any external FDs reserved during startup are released properly.
Author: Chao Li <lic@highgo.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Rafia Sabih <rafia.pghackers@gmail.com>
Discussion: https://postgr.es/m/A2B8B7DE-C119-492F-A9FA-14CF86849777@gmail.com
Commit 28972b6fc ("Add support for importing statistics from remote
servers.") stored the names of local/remote columns for a foreign table
into the buffers of NAMEDATALEN bytes in this structure, without
accounting for the possibility that the remote column name in particular
could be longer than NAMEDATALEN - 1. If it was longer than that, this
would leave it unterminated/truncated in the buffer, invoking undefined
behavior when match_attrmap() processes it, which assumes that it's
fully-contained/terminated in the buffer.
To fix, replace the buffers with char pointers, pstrdup the local/remote
column names, and store the results into the pointers. This commit also
adds a function to clean up the nested data structure.
Per Coverity and Tom Lane.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Author: Corey Huinker <corey.huinker@gmail.com>
Reviewed-by: Etsuro Fujita <etsuro.fujita@gmail.com>
Discussion: https://postgr.es/m/342868.1776017700%40sss.pgh.pa.us
These tests have been removed by 906ea101d0, due to some of them being
unstable in the buildfarm with low max_stack_depth values. They are now
reworked so as they should be more portable.
The tests to cover the findoprnd() overflows use a balanced tree to
avoid using too much stack, per a suggestion and an investigation by Tom
Lane.
Note: This is initially applied only on HEAD; a backpatch will follow
should the buildfarm be fine with the situation.
Discussion: https://postgr.es/m/agZc6XecyE7E7fep@paquier.xyz
Backpatch-through: 14
When an UPDATE statement triggers check_foreign_key() with the
action set to "cascade", it generates more UPDATE statements to
modify the key values in referencing relations. If a new key value
is NULL, SPI_getvalue() returns a NULL pointer, which is
subsequently passed to quote_literal_cstr(), causing a segfault.
To fix, skip quoting when a new key value is NULL and insert an
unquoted NULL keyword instead.
Oversight in commit 260e97733b. While the refint documentation
recommends marking primary key columns NOT NULL, the aforementioned
scenario accidentally worked on platforms where snprintf()
substitutes "(null)" for NULL pointers. Note that for
character-type columns, the old code quoted "(null)" as a string
literal, so this didn't always produce correct results. But it
still seems better to fix this than to reject cases that previously
worked.
Reported-by: Nikita Kalinin <n.kalinin@postgrespro.ru>
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Pierre Forstmann <pierre.forstmann@gmail.com>
Discussion: https://postgr.es/m/19476-bd04ea6241345303%40postgresql.org
Backpatch-through: 14
build_remattrmap() deparses a list of remote column names for a query
that retrieves attribute stats for them from the remote server.
Previously, it did so by using the array-literal syntax with each column
name individually quoted by quote_identifier(), causing the query to
fail on the remote server with a syntax error or no results when that
column name included a single quote or backslash, as quote_identifier()
doesn't escape those characters, making the query invalid or incorrect.
Fix by switching from the array-literal syntax to the ARRAY constructor
syntax with each column name individually quoted by
deparseStringLiteral().
Oversight in commit 28972b6fc.
Reported-by: Satya Narlapuram <satyanarlapuram@gmail.com>
Reported-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Alex Guo <guo.alex.hengchen@gmail.com>
Reviewed-by: Zhenwei Shang <a934172442@gmail.com>
Reviewed-by: Etsuro Fujita <etsuro.fujita@gmail.com>
Discussion: https://postgr.es/m/CAHg%2BQDc9%3DWtYi%3DJW6QUL6ASOJc6PcGPTuxoMkhnkQ7oi7j5atg%40mail.gmail.com
Discussion: https://postgr.es/m/CAJTYsWWGhVDFjr%2BsmdYdU-Q_TT9YMzXA4QcLCr7rizDOyrEEow%40mail.gmail.com
Update typedefs.list from the buildfarm, and run pgindent.
The changes from the new typedefs list are pretty minimal,
since we'd been pretty good (not perfect) about updating
typedefs.list by hand. But the pgindent behavior changes
installed by a3e6beba6, b518ba4af, and 60f9467c3 add up
to make this a relatively sizable diff.
As mentioned in 8268e41aca, pgss_ProcessUtility() may free the
PlannedStmt after an internal ROLLBACK. This commit sets the
PlannedStmt "pstmt" to NULL once it is no longer safe to rely on it,
making bugs similar to the one fixed by the previous commit easier to
detect.
Suggested-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/0A9A8DAC-BC3C-4C7A-9504-2C6050405544@anarazel.de
pgss_ProcessUtility() included a reference to a portion of a PlannedStmt
after the point where this data's structure could have been freed,
causing an incorrect memory access. There was a comment documenting
this requirement, missed in 3357471cf9.
This commit includes a test able to make valgrind complain with a
PlannedStmt freed by an internal ROLLBACK query. Similarly to what is
mentioned in 495e73c207, this can be triggered by using the extended
query protocol, something that can be now tested thanks to the recent
meta-command additions in psql. This commit mentions potential other
cases, but as far as I can see the extended protocol case with an
internal ROLLBACK is the only problematic pattern reachable in practice.
Issue introduced by 3357471cf9, gone unnoticed due to a lack of test
coverage. The fix is authored by Chao, my contribution being the new
test.
Author: Chao Li <li.evan.chao@gmail.com>
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/2F91906A-F2B5-4A6B-9695-D136957D4545@gmail.com
These checks are failing in the buildfarm, reporting stack overflows
rather than the expected errors, though seemingly only on ppc64 and
s390x platforms. Perhaps there is something off about our tests
for stack depth on those architectures? But there's no time to
debug that right now, and surely these tests aren't too essential.
Revert for now and plan to revisit after the release dust settles.
Backpatch-through: 14
Security: CVE-2026-6473
Maliciously crafted key value updates could achieve SQL injection
within check_foreign_key(). To fix, ensure new key values are
properly quoted and escaped in the internally generated SQL
statements. While at it, avoid potential buffer overruns by
replacing the stack buffers for internally generated SQL statements
with StringInfo.
Reported-by: Nikolay Samokhvalov <nik@postgres.ai>
Author: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Security: CVE-2026-6637
Backpatch-through: 14
contrib/intarray's query_int type uses an int16 field to hold the
offset from a binary operator node to its left operand. However, it
allows the number of nodes to be as much as will fit in MaxAllocSize,
so there is a risk of overflowing int16 depending on the precise shape
of the tree. Simple right-associative cases like "a | b | c | ..."
work fine, so we should not solve this by restricting the overall
number of nodes. Instead add a direct test of whether each individual
offset is too large.
contrib/ltree's ltxtquery type uses essentially the same logic and
has the same 16-bit restriction.
(The core backend's tsquery.c has a variant of this logic too, but
in that case the target field is 32 bits, so it is okay so long
as varlena datums are restricted to 1GB.)
In v16 and up, these types support soft error reporting, so we have
to complicate the recursive findoprnd function's API a bit to allow
the complaint to be reported softly. v14/v15 don't need that.
Undocumented and overcomplicated code like this makes my head hurt,
so add some comments and simplify while at it.
Reported-by: Xint Code
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Backpatch-through: 14
Security: CVE-2026-6473
The lquery parser in contrib/ltree/ had two overflow problems:
- A single lquery level with many OR-separated variants (e.g.,
'label1|label2|...'), could cause an overflow of totallen, this being
stored as a uint16, meaning a maximum value of UINT16_MAX or 65k. Each
variant contributes MAXALIGN(LVAR_HDRSIZE + len) bytes. With enough
long variants, the value would wraparound. This would corrupt the data
written by LQL_NEXT(), leading to a stack corruption, most likely
translating into a crash, but it would allow incorrect memory access.
- numvar, labelled as a uint16, counts the number of OR-variants in a
single level, and it is incremented without bounds checking. With more
than PG_UINT16_MAX (65k) variants in a single level, and a minimum of
131kB of input data, it would wrap to 0. When a (wildcard) '*' is
used, this would change the query results silently.
For both issues, a set of overflows checks are added to guard against
these problematic patterns.
The first issue has been reported by the three people listed below,
affecting v16 and newer versions due to b1665bf01e. Its coding was
still unsafe in v14 and v15. The second issue affects all the stable
branches; I have bumped into while reviewing the code of the module.
Reported-by: Vergissmeinnicht <vergissmeinnichtzh@gmail.com>
Reported-by: A1ex <alex000young@gmail.com>
Reported-by: Jihe Wang <wangjihe.mail@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Security: CVE-2026-6473
Backpatch-through: 14
When importing remote stats for a foreign table backed by a pre-v17
remote server, the query built/executed in this function has three NULL
placeholders for the range stats supported in v17 at the end of the
SELECT list. Previously, it included a trailing comma after the last
NULL, like "SELECT ..., NULL, NULL, NULL, FROM pg_catalog.pg_stats ...",
causing a syntax error on the remote server. Fix by removing the comma.
Oversight in commit 28972b6fc.
Author: Satya Narlapuram <satyanarlapuram@gmail.com>
Discussion: https://postgr.es/m/CAHg%2BQDdEE7wp1S60Fn9Kmna8KfdMo5Tu6dROLpMn_-EOUBKmWQ%40mail.gmail.com
As connections that failed abort cleanup can't safely be further used,
if a remote query tries to get such a connection, we reject it.
Previously, this rejection involved dropping the connection if it was
open, without accounting for the possibility of open cursors using it,
causing a server crash when such an open cursor tried to use an
already-dropped connection, as a cursor-handling function
(create_cursor, fetch_more_data, or close_cursor) was called on a freed
PGconn. To fix, delay dropping failed connections until abort cleanup
of the main transaction, to ensure open cursors using such a connection
can safely refer to the PGconn for it.
Oversight in commit 8bf58c0d9.
Reported-by: Zhibai Song <songzhibai1234@gmail.com>
Diagnosed-by: Zhibai Song <songzhibai1234@gmail.com>
Author: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Discussion: https://postgr.es/m/CAPmGK176y6JP017-Cn%2BhS9CEJx_6iVhRoYbAqzuLU4d8-XPPNg%40mail.gmail.com
Backpatch-through: 14
Commit 28d534e2ae introduced reform_tuple() with a fast path that
returns the source tuple verbatim when no dropped columns require fixing
up. I (Álvaro) failed to realize that this broke handling of columns
with a 'missingval' defined: after a VACUUM FULL, CLUSTER, or REPACK
operation, the catalogued missingval is thrown away, so the tuples are
no longer correct.
Fix by forcing the rewrite when the tuple is shorter than the tuple
descriptor.
Author: Satya Narlapuram <satyanarlapuram@gmail.com>
Discussion: https://postgr.es/m/CAHg+QDeoccU5CudrJpmSKZfKZ1gRMNY=5BxSC=JpHgkonzgcOw@mail.gmail.com
This reverts portions of commit 6dcfac9696, which is wrong in trying
to use a *GetDatum() that matches with the C types of the values read.
*GetDatum() should match with the output argument types of the SQL
functions.
The portions of 6dcfac9696 that are right regarding this rule are:
- gistget.c, where the GiST support functions use DatumGetUInt16() to
retrieve the strategy number.
- The BRIN code for strategynum, used in syscache lookups.
The adjustments done in this commit are for pageinspect, pg_buffercache
and pg_lock_status().
While double-checking the whole state of the tree regarding non-matching
pairs of DatumGet*() and *GetDatum(), I have found much more code paths
that are incorrect, unrelated to 6dcfac9696. These may be adjusted in
the future, in a different patch (perhaps not for v19, as we are already
past feature freeze).
Reported-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/97f9375a-be61-4272-a44d-408337fe8fa6@eisentraut.org
Discussion: https://postgr.es/m/CAJ7c6TMcGu8qmRe1gZfJ-gOzVnZq-t=fwn-UuyStx1w6ZyydMw@mail.gmail.com
These tests sometimes run with wal_level=minimal, which does not allow
to run REPACK (CONCURRENTLY). Move them to test_decoding, which is
ensured to run with high enough wal_level.
Discussion: https://postgr.es/m/260901.1776696126@sss.pgh.pa.us
Make sure that function declarations use names that exactly match the
corresponding names from function definitions in a few places. Most of
these inconsistencies were introduced during Postgres 19 development.
This commit was written with help from clang-tidy, by mechanically
applying the same rules as similar clean-up commits (the earliest such
commit was commit 035ce1fe).
The btree_gist enum test expects a bitmap heap scan. Since b46e1e54d0
enabled setting the VM during on-access pruning and 378a21618 set
pd_prune_xid on INSERT, scans of enumtmp may set pages all-visible.
If autovacuum or autoanalyze then updates pg_class.relallvisible, the
planner could choose an index-only scan instead.
Make the enumtmp a temp table to exclude it from autovacuum/autoanalyze.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/46733d68-aec0-4d09-8120-4c66b87047a4%40gmail.com
This batch is similar to 462fe0ff62 and addresses a variety of code
style issues, including grammar mistakes, typos, inconsistent variable
names in function declarations, and incorrect function names in comments
and documentation. These fixes have accumulated on the community
mailing lists since the commit mentioned above.
Notably, Alexander Lakhin previously submitted a patch identifying many
of the trivial typos and grammar issues that had been reported on
pgsql-hackers. His patch covered a somewhat large portion of the issues
addressed here, though not all of them.
The documentation changes only affect HEAD.
This one occurs when an outer join appears beneath the made-unique
side of a semijoin. The issue is that join RTEs are not featured
out of sj_unique_rels entries. Fix, and add a test case.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Analyzed-by: Tender Wang <tndrwang@gmail.com>
Discussion: http://postgr.es/m/c0c63979-43c2-4424-8fe8-56949934c9d8@gmail.com
overexplain_range_table() emitted the "Unprunable RTIs" and "Result
RTIs" properties before closing the "Range Table" group. In the JSON
and YAML formats the Range Table group is rendered as an array of RTE
objects, so emitting key/value pairs inside it produced structurally
invalid output. The XML format had a related oddity, with these
elements nested inside <Range-Table> rather than appearing as its
siblings.
These fields are properties of the PlannedStmt as a whole, not of any
individual RTE, so close the Range Table group before emitting them.
They now appear as siblings of "Range Table" in the parent Query
object, which is what was intended.
Also add a test exercising FORMAT JSON with RANGE_TABLE so that any
future regression in the output structure is caught.
Reported-by: Satyanarayana Narlapuram <satyanarlapuram@gmail.com>
Author: Satyanarayana Narlapuram <satyanarlapuram@gmail.com>
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAHg+QDdDrdqMr98a_OBYDYmK3RaT7XwCEShZfvDYKZpZTfOEjQ@mail.gmail.com
Backpatch-through: 18
It turns out that our main regression test suite queries tables upon
which concurrent DDL is occurring, which can, rarely, cause
test_plan_advice failures. We're not quite ready to fix that problem
just yet, because we want to gather some more information about how
often it actually happens first. But, our plan is going to require
test_plan_advice to access a few bits of pg_plan_advice that have
been considered internal up until now, so this commit rejiggers
things to expose those bits.
First, test_plan_advice is going to need to be able to interpret
the PGPA_TE_* constants which have been declared in pgpa_trove.h.
The "TE" stands for "trove entry" but that's kind of a silly name;
change the naming to "FB" (for "feedback") and move the declarations
to pg_plan_advice.h, which is a header file that's already installed.
This has the side benefit of making these constants available to any
other extensions that may want to examine plan advice feedback.
Second, test_plan_advice is going to call pgpa_planner_feedback_warning,
so make that function non-static and mark it PGDLLEXPORT.
Discussion: http://postgr.es/m/CA+TgmobOOmmXSJz3e+cjTY-bA1+W0dqVDqzxUBEvGtW62whYGg@mail.gmail.com
If a subquery is proven empty, and if that subquery contained a
semijoin, and if making one side or the other of that semijoin
unique and performing an inner join was a possible strategy, then
the previous code would fail with ERROR: no rtoffset for plan %s
when attempting to generate advice. Fix that.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: http://postgr.es/m/CA+TgmobOOmmXSJz3e+cjTY-bA1+W0dqVDqzxUBEvGtW62whYGg@mail.gmail.com
When a tablesample routine says that it is not repeatable across
scans, set_tablesample_rel_pathlist will (usually) materialize it,
confusing pg_plan_advice's plan walker machinery. To fix, update that
machinery to view such Material paths as essentially an extension of
the underlying scan.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: http://postgr.es/m/CA+TgmobOOmmXSJz3e+cjTY-bA1+W0dqVDqzxUBEvGtW62whYGg@mail.gmail.com
Add a new FDW callback routine that allows importing remote statistics
for a foreign table directly to the local server, instead of collecting
statistics locally. The new callback routine is called at the beginning
of the ANALYZE operation on the table, and if the FDW failed to import
the statistics, the existing callback routine is called on the table to
collect statistics locally.
Also implement this for postgres_fdw. It is enabled by "restore_stats"
option both at the server and table level. Currently, it is the user's
responsibility to ensure remote statistics to import are up-to-date, so
the default is false.
Author: Corey Huinker <corey.huinker@gmail.com>
Co-authored-by: Etsuro Fujita <etsuro.fujita@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Etsuro Fujita <etsuro.fujita@gmail.com>
Discussion: https://postgr.es/m/CADkLM%3DchrYAx%3DX2KUcDRST4RLaRLivYDohZrkW4LLBa0iBhb5w%40mail.gmail.com
Until now extensions that wanted to measure overall query execution could
create QueryDesc->totaltime, which the core executor would then start and
stop. That's a bit odd and composes badly, e.g. extensions always had to use
INSTRUMENT_ALL, because otherwise another extension might not get what they
need.
Instead this introduces a new field, QueryDesc->query_instr_options, that
extensions can use to indicate whether they need query level instrumentation
populated, and with which instrumentation options. Extensions should take care
to only add options they need, instead of replacing the options of others.
The prior name of the field, totaltime, sounded like it would only measure
time, but these days the instrumentation infrastructure can track more
resources. The secondary benefit is that this will make it obvious to
extensions that they may not create the Instrumentation struct themselves
anymore (often extensions build only against a postgres build without
assertions).
Adjust pg_stat_statements and auto_explain to match, and lower the
requested instrumentation level for auto_explain to INSTRUMENT_TIMER,
since the summary instrumentation it needs is only runtime.
The reason to push this now, rather in the PG 20 cycle, is that 5a79e78501
already required extensions using query level instrumentations to adjust their
code, and it seemed undesirable to require them to do so again for 20.
Author: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAP53Pkyqsht+exJQYRsjhSWYKu+vFGHhPub7m6PmFD6Or0=p1g@mail.gmail.com
If pg_stash_advice.persist = true, stashed advice will be written to
pg_stash_advice.tsv in the data directory, periodically and at
shutdown. On restart, stash modifications are locked out until this
file has been reloaded, but queries will not be, so there may be a
short window after startup during which previously-stashed advice is
not automatically applied.
Author: Robert Haas <rhaas@postgresql.org>
Co-authored-by: Lukas Fittl <lukas@fittl.com>
Discussion: https://postgr.es/m/CA+Tgmob87qsWa-VugofU6epuV0H5XjWZGMbQas4Q-ADKmvSyBg@mail.gmail.com
Use templated qsort() so that the comparison function can be
inlined. To speed up qunique(), use a specialized comparison function
that only checks for equality.
Author: David Geier <geidav.pg@gmail.com>
Reviewed-by: Kirill Reshke <reshkekirill@gmail.com>
Discussion: https://www.postgresql.org/message-id/2a76b5ef-4b12-4023-93a1-eed6e64968f3@gmail.com
By default, the logical decoding assumes access to shared catalogs, so
the snapshot builder needs to consider cluster-wide XIDs during startup.
That in turn means that, if any transaction is already running (and has
XID assigned), the snapshot builder needs to wait for its completion, as
it does not know if that transaction performed catalog changes earlier.
A possible problem with this concept is that if REPACK (CONCURRENTLY) is
running in some database, backends running the same command in other
databases get stuck until the first one has committed. Thus only a
single backend in the cluster can run REPACK (CONCURRENTLY) at any time.
Likewise, REPACK (CONCURRENTLY) can block walsenders starting on behalf
of subscriptions throughout the cluster.
This patch adds a new option to logical replication output plugin, to
declare that it does not use shared catalogs (i.e. catalogs that can be
changed by transactions running in other databases in the cluster). In
that case, no snapshot the backend will use during the decoding needs to
contain information about transactions running in other databases. Thus
the snapshot builder only needs to wait for completion of transactions
in the current database.
Currently we only use this option in the REPACK background worker. It
could possibly be used in the plugin for logical replication too,
however that would need thorough analysis of that plugin.
Bump WAL version number, due to a new field in xl_running_xacts.
Author: Antonin Houska <ah@cybertec.at>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/90475.1775218118@localhost
This commit changes the post_parse_analyze_hook_type() hook to take a
const JumbleState, to tell external modules that they are not allowed to
touch the JumbleState that has been compiled by the core code. This
fixes a pretty old problem with pg_stat_statements, that had always the
idea of modifying the lengths of the constants stored in the
JumbleState. The previous state could confuse extensions that need to
look at a JumbleState depending on the loading order, if
pg_stat_statements is part of the stack loaded.
Another piece included in this commit is the move of the routine
fill_in_constant_lengths() to queryjumblefuncs.c, to give an option to
extensions to compile the lengths of the constants, if necessary. I was
surprised by the number of external code that carries a copy of this
routine (see the thread for details). Previously, this routine modified
JumbleState. It now copies the set of LocationLens from JumbleState,
and fills the constant lengths for separate use.
pg_stat_statements is updated to use the new ComputeConstantLengths().
JumbleState is now marked with a const in the module, where relevant.
Author: Sami Imseih <samimseih@gmail.com>
Co-authored-by: Lukas Fittl <lukas@fittl.com>
Discussion: https://postgr.es/m/CAA5RZ0tZp5qU0ikZEEqJnxvdSNGh1DWv80sb-k4QAUmiMoOp_Q@mail.gmail.com
The associated value should look like something that could be
part of an EXPLAIN options list, but restricted to EXPLAIN options
added by extensions.
For example, if pg_overexplain is loaded, you could set
auto_explain.log_extension_options = 'DEBUG, RANGE_TABLE'.
You can also specify arguments to these options in the same manner
as normal e.g. 'DEBUG 1, RANGE_TABLE false'.
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Discussion: http://postgr.es/m/CA+Tgmob-0W8306mvrJX5Urtqt1AAasu8pi4yLrZ1XfwZU-Uj1w@mail.gmail.com
contrib/pg_stash_advice and src/test/modules/test_shmem
missed these, leading to complaints from git after an
in-tree check-world run.
Use our standard boilerplate list of ignorable subdirectories,
although the two modules presently create different subsets
of that.
It would be useful to be able to tell auto_explain to set a custom
EXPLAIN option, but it would be bad if it tried to do so and the
option name or value wasn't valid, because then every query would fail
with a complaint about the EXPLAIN option. So add a guc_check_handler
that auto_explain will be able to use to only try to set option
name/value/type combinations that have been determined to be legal,
and to emit useful messages about ones that aren't.
Reviewed-by: Matheus Alcantara <matheusssilv97@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Discussion: http://postgr.es/m/CA+Tgmob-0W8306mvrJX5Urtqt1AAasu8pi4yLrZ1XfwZU-Uj1w@mail.gmail.com
This module allows plan advice strings to be provided automatically
from an in-memory advice stash. Advice stashes are stored in dynamic
shared memory and must be recreated and repopulated after a server
restart. If pg_stash_advice.stash_name is set to the name of an advice
stash, and if query identifiers are enabled, the query identifier
for each query will be looked up in the advice stash and the
associated advice string, if any, will be used each time that query
is planned.
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com>
Reviewed-by: David G. Johnston <david.g.johnston@gmail.com>
Reviewed-by: Jakub Wartak <jakub.wartak@enterprisedb.com>
Discussion: http://postgr.es/m/CA+TgmoaeNuHXQ60P3ZZqJLrSjP3L1KYokW9kPfGbWDyt+1t=Ng@mail.gmail.com
As part of this, embed the LWLock it needs in the shared memory struct
itself, so that we don't need to use RequestNamedLWLockTranche()
anymore. LWLockNewTrancheId() + LWLockInitialize() is more convenient
to use in extensions.
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com
