Tomas Vondra d2d8a229bc Implement Incremental Sort ... Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com		2020-04-06 21:35:10 +02:00
..
.gitignore	Add a SECURITY LABEL command.	2010-09-27 20:55:27 -04:00
advisory_lock.sql	Add transaction-level advisory locks.	2011-02-18 14:05:12 +09:00
aggregates.sql	Reduce test time for disk-based Hash Aggregation.	2020-03-23 19:03:49 -07:00
alter_generic.sql	Implement operator class parameters	2020-03-30 19:17:23 +03:00
alter_operator.sql	Avoid unnecessary use of pg_strcasecmp for already-downcased identifiers.	2018-01-26 18:25:14 -05:00
alter_table.sql	Preserve clustered index after rewrites with ALTER TABLE	2020-04-06 11:03:49 +09:00
amutils.sql	Add support for nearest-neighbor (KNN) searches to SP-GiST	2018-09-19 01:54:10 +03:00
arrays.sql	Improve expression evaluation test coverage.	2017-03-11 15:41:34 -08:00
async.sql	Add new function pg_notification_queue_usage.	2015-07-17 09:12:03 -04:00
bit.sql	Fix bitshiftright()'s zero-padding some more.	2019-10-04 10:34:40 -04:00
bitmapops.sql	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
boolean.sql	Expand AND/OR regression tests around NULL handling.	2018-03-13 16:12:31 -07:00
box.sql	Add support for <-> (box, point) operator to SP-GiST box_ops	2019-07-14 15:09:23 +03:00
brin.sql	Add BRIN test case	2020-01-22 18:37:39 -03:00
btree_index.sql	Implement operator class parameters	2020-03-30 19:17:23 +03:00
case.sql	Fix interaction of CASE and ArrayCoerceExpr.	2018-10-30 15:26:11 -04:00
char.sql	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
circle.sql	Back off output precision in circle.sql regression test.	2019-08-25 12:14:50 -04:00
cluster.sql	Fix CLUSTER on expression indexes.	2019-10-15 10:40:13 -07:00
collate.icu.utf8.sql	Fix random regression failure in test case "collate.icu.utf8"	2019-08-14 13:37:48 +09:00
collate.linux.utf8.sql	Allow to_date/to_timestamp to recognize non-English month/day names.	2020-03-03 11:06:47 -05:00
collate.sql	Un-hide most cascaded-drop details in regression test results.	2019-03-24 19:15:37 -04:00
combocid.sql	Fix locking a tuple updated by an aborted (sub)transaction	2016-09-09 15:54:29 -03:00
comments.sql	Implement nested block comments in the backend and in psql.	2000-07-14 15:43:57 +00:00
conversion.sql	Establish conventions about global object names used in regression tests.	2016-07-17 18:42:43 -04:00
copy2.sql	Allow COPY FROM to filter data using WHERE conditions	2019-01-20 00:22:14 +01:00
copydml.sql	COPY (INSERT/UPDATE/DELETE .. RETURNING ..)	2015-11-27 19:11:22 +03:00
copyselect.sql	Have \copy go through SendQuery	2012-01-25 18:22:00 -03:00
create_aggregate.sql	Introduce "anycompatible" family of polymorphic types.	2020-03-19 11:43:11 -04:00
create_am.sql	Stop using spelling "nonexistant".	2019-06-08 10:12:26 -07:00
create_cast.sql	Make CREATE TYPE print warnings if a datatype's I/O functions are volatile.	2014-11-05 11:44:06 -05:00
create_function_3.sql	Un-hide most cascaded-drop details in regression test results.	2019-03-24 19:15:37 -04:00
create_index.sql	Preserve pg_index.indisclustered across REINDEX CONCURRENTLY	2020-03-03 10:12:28 +09:00
create_index_spgist.sql	Provide stable test for NULL-values in KNN SP-GiST	2019-09-20 15:33:45 +03:00
create_misc.sql	ANALYZE a_star and its children to avoid plan instability in tests.	2019-09-27 11:28:24 -04:00
create_operator.sql	Fix lexing of standard multi-character operators in edge cases.	2018-08-23 21:42:40 +01:00
create_procedure.sql	Add collation assignment to CALL statement	2019-02-07 08:25:47 +01:00
create_table.sql	Skip WAL for new relfilenodes, under wal_level=minimal.	2020-04-04 12:25:34 -07:00
create_table_like.sql	Ensure that CREATE TABLE LIKE copies any NO INHERIT constraint property.	2020-03-10 14:54:00 -04:00
create_type.sql	Allow ALTER TYPE to change some properties of a base type.	2020-03-06 12:19:29 -05:00
create_view.sql	Improve test coverage of ruleutils.c.	2019-12-06 17:40:30 -05:00
date.sql	Remove explicit error handling for obsolete date/time values	2019-06-30 10:27:35 +02:00
dbsize.sql	Add pg_size_bytes() to parse human-readable size strings.	2016-02-20 09:57:27 +00:00
delete.sql	Regression tests for TOAST.	2011-04-25 09:48:10 -04:00
dependency.sql	Un-hide most cascaded-drop details in regression test results.	2019-03-24 19:15:37 -04:00
domain.sql	Drop no-op CoerceToDomain nodes from expressions at planning time.	2018-12-13 13:24:43 -05:00
drop_if_exists.sql	Introduce the 'force' option for the Drop Database command.	2019-11-13 08:25:33 +05:30
drop_operator.sql	Fix DROP OPERATOR to reset oprcom/oprnegate links to the dropped operator.	2016-03-25 12:33:16 -04:00
enum.sql	Relax transactional restrictions on ALTER TYPE ... ADD VALUE (redux).	2018-10-09 12:51:01 +13:00
equivclass.sql	Suppress unnecessary RelabelType nodes in more cases.	2020-02-26 18:14:12 -05:00
errors.sql	Re-stabilize infinite_recurse() test case.	2020-04-06 12:00:37 -04:00
event_trigger.sql	Fix confusion about event trigger vs. plain function in plpgsql.	2020-02-19 14:45:17 -05:00
explain.sql	Improve stability of explain regression test.	2020-04-04 01:26:39 +09:00
expressions.sql	Add some test cases to improve test coverage of parse_expr.c.	2019-07-05 23:56:34 -04:00
fast_default.sql	Fix heap_getattr() handling of fast defaults.	2019-02-06 01:09:32 -08:00
float4.sql	Improve coverage of utils/float.h	2019-08-28 12:28:16 +09:00
float8.sql	Un-hide most cascaded-drop details in regression test results.	2019-03-24 19:15:37 -04:00
foreign_data.sql	Fix partitioned index creation with foreign partitions	2019-06-26 18:38:51 -04:00
foreign_key.sql	Simplify FK-to-partitioned regression test query	2020-02-20 14:14:20 -03:00
functional_deps.sql	Recognize functional dependency on primary keys. This allows a table's	2010-08-07 02:44:09 +00:00
generated.sql	ALTER TABLE ... ALTER COLUMN ... DROP EXPRESSION	2020-01-14 13:36:03 +01:00
geometry.sql	Add missing commutators for distance operators	2019-07-14 14:55:01 +03:00
gin.sql	Fix bugs in gin_fuzzy_search_limit processing.	2020-04-03 13:15:45 -04:00
gist.sql	Add support for <-> (box, point) operator to GiST box_ops	2019-07-14 15:09:15 +03:00
groupingsets.sql	Reduce test time for disk-based Hash Aggregation.	2020-03-23 19:03:49 -07:00
guc.sql	Give up on testing guc.c's behavior for "infinity" inputs.	2019-03-11 17:53:09 -04:00
hash_func.sql	Introduce 64-bit hash functions with a 64-bit seed.	2017-08-31 22:21:21 -04:00
hash_index.sql	Add more tests for reloptions	2017-10-19 14:22:05 +02:00
hash_part.sql	Fix failure with textual partition hash keys.	2019-04-15 16:47:09 -04:00
horology.sql	Fix an oversight in commit 4c70098ff.	2020-01-23 16:15:32 -05:00
hs_primary_extremes.sql	Remove all references to "xlog" from SQL-callable functions in pg_proc.	2017-02-09 15:10:09 -05:00
hs_primary_setup.sql	Remove all references to "xlog" from SQL-callable functions in pg_proc.	2017-02-09 15:10:09 -05:00
hs_standby_allowed.sql	Allow UNLISTEN in hot-standby mode.	2019-01-25 21:14:49 -05:00
hs_standby_check.sql	Allow read only connections during recovery, known as Hot Standby.	2009-12-19 01:32:45 +00:00
hs_standby_disallowed.sql	Allow UNLISTEN in hot-standby mode.	2019-01-25 21:14:49 -05:00
hs_standby_functions.sql	Remove all references to "xlog" from SQL-callable functions in pg_proc.	2017-02-09 15:10:09 -05:00
identity.sql	Fix INSERT OVERRIDING USER VALUE behavior	2020-03-31 08:50:39 +02:00
incremental_sort.sql	Implement Incremental Sort	2020-04-06 21:35:10 +02:00
index_including.sql	Fix nbtree high key "continuescan" row compare bug.	2019-03-31 17:24:04 -07:00
index_including_gist.sql	Support for INCLUDE attributes in GiST indexes	2019-03-10 11:37:17 +03:00
indexing.sql	Fix assertion failure with ALTER TABLE ATTACH PARTITION and indexes	2020-03-03 13:55:41 +09:00
indirect_toast.sql	Clean up duplicate table and function names in regression tests.	2018-03-15 17:09:02 -04:00
inet.sql	Add sort support routine for the inet data type.	2019-08-01 09:34:14 -07:00
inherit.sql	Add regression tests for constraint errors in partitioned tables.	2020-03-23 15:06:11 -07:00
init_privs.sql	Fix typos in comments.	2017-02-06 11:33:58 +02:00
insert.sql	Add tests for error message generation in partition tuple routing	2019-05-06 21:44:24 +09:00
insert_conflict.sql	Don't mark partitioned indexes invalid unnecessarily	2018-12-05 13:31:51 -03:00
int2.sql	Add documentation and regression tests concerning rounding of numerics.	2015-07-03 17:04:39 -04:00
int4.sql	Add functions gcd() and lcm() for integer and numeric types.	2020-01-25 14:00:59 +00:00
int8.sql	Add functions gcd() and lcm() for integer and numeric types.	2020-01-25 14:00:59 +00:00
interval.sql	Fix integer-overflow edge case detection in interval_mul and pgbench.	2019-11-07 11:22:58 -05:00
join.sql	Prevent overly-aggressive collapsing of joins to RTE_RESULT relations.	2019-12-14 13:49:15 -05:00
join_hash.sql	Fix representation of hash keys in Hash/HashJoin nodes.	2019-08-02 00:02:46 -07:00
json.sql	Remove unnecessary test dependency on the contents of pg_pltemplate.	2019-08-21 10:43:23 -04:00
json_encoding.sql	Allow Unicode escapes in any server encoding, not only UTF-8.	2020-03-06 14:17:43 -05:00
jsonb.sql	Add a non-strict version of jsonb_set	2020-01-17 11:52:39 +10:30
jsonb_jsonpath.sql	Allow to_date/to_timestamp to recognize non-English month/day names.	2020-03-03 11:06:47 -05:00
jsonpath.sql	Implement jsonpath .datetime() method	2019-09-25 22:51:51 +03:00
jsonpath_encoding.sql	Allow Unicode escapes in any server encoding, not only UTF-8.	2020-03-06 14:17:43 -05:00
limit.sql	Add regression tests exercising more code paths in nodeLimit.c.	2017-08-11 17:28:01 -04:00
line.sql	Improve test coverage of geometric types	2018-09-26 10:45:21 +02:00
lock.sql	Make inherited LOCK TABLE perform access permission checks on parent table only.	2020-02-18 13:13:15 +09:00
lseg.sql	Improve test coverage of geometric types	2018-09-26 10:45:21 +02:00
macaddr.sql	Allow input format xxxx-xxxx-xxxx for macaddr type	2014-10-21 16:16:39 -04:00
macaddr8.sql	Add support for EUI-64 MAC addresses as macaddr8	2017-03-15 11:16:25 -04:00
matview.sql	Remove useless and rather expensive stanza in matview regression test.	2017-05-03 19:37:01 -04:00
misc_functions.sql	Avoid holding a directory FD open across assorted SRF calls.	2020-03-16 21:05:52 -04:00
misc_sanity.sql	Remove WITH OIDS support, change oid catalog column visibility.	2018-11-20 16:00:17 -08:00
money.sql	Fix loss of fractional digits for large values in cash_numeric().	2019-07-26 11:59:00 -04:00
name.sql	Introduce parse_ident()	2016-03-18 18:16:14 +03:00
namespace.sql	Clean up duplicate role and schema names in regression tests.	2018-03-15 14:00:31 -04:00
numeric.sql	Improve the performance and accuracy of numeric sqrt() and ln().	2020-03-28 14:37:53 +00:00
numeric_big.sql	Fix corner-case loss of precision in numeric ln().	2020-03-01 14:49:25 +00:00
numerology.sql	Remove extra newlines at end and beginning of files, add missing newlines	2010-08-19 05:57:36 +00:00
object_address.sql	Warn if wal_level is too low when creating a publication.	2019-07-13 10:35:34 +12:00
oid.sql	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
oidjoins.sql	Rework the pg_statistic_ext catalog	2019-06-16 01:20:31 +02:00
opr_sanity.sql	Implement operator class parameters	2020-03-30 19:17:23 +03:00
partition_aggregate.sql	Implement Incremental Sort	2020-04-06 21:35:10 +02:00
partition_info.sql	Fix crash with pg_partition_root	2019-03-22 17:27:38 +09:00
partition_join.sql	Clean up some misplaced comments in partition_join.sql regression test.	2019-12-16 17:00:15 +09:00
partition_prune.sql	Fix "force_parallel_mode = regress" to work with ANALYZE + VERBOSE.	2019-12-16 20:14:35 -05:00
password.sql	Fix use of term "verifier"	2019-10-12 21:41:59 +02:00
path.sql	Improve test coverage of geometric types	2018-09-26 10:45:21 +02:00
pg_lsn.sql	Add min() and max() aggregates for pg_lsn	2019-07-05 12:21:11 +09:00
plancache.sql	Add plan_cache_mode setting	2018-07-16 13:35:41 +02:00
plpgsql.sql	Introduce "anycompatible" family of polymorphic types.	2020-03-19 11:43:11 -04:00
point.sql	Change floating-point output format for improved performance.	2019-02-13 15:20:33 +00:00
polygon.sql	Fix double-word typos	2019-06-13 10:03:56 -04:00
polymorphism.sql	Remove bogus Assert, add some regression test cases showing why.	2020-04-04 18:03:30 -04:00
portals.sql	Fix failure in WHERE CURRENT OF after rewinding the referenced cursor.	2018-09-23 16:05:45 -04:00
portals_p2.sql	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
prepare.sql	Add more tests for CREATE TABLE AS with WITH NO DATA	2019-02-07 09:21:57 +09:00
prepared_xacts.sql	Remove unnecessary dependency on statement_timeout in prepared_xacts test.	2017-03-13 16:46:32 -04:00
privileges.sql	Make inherited LOCK TABLE perform access permission checks on parent table only.	2020-02-18 13:13:15 +09:00
psql.sql	Show opclass and opfamily related information in psql	2020-03-08 13:33:16 +03:00
psql_crosstab.sql	Fix incorrect error reporting for duplicate data in \crosstabview.	2016-12-25 16:04:45 -05:00
publication.sql	Support adding partitioned tables to publication	2020-03-10 09:09:32 +01:00
random.sql	Remove extra newlines at end and beginning of files, add missing newlines	2010-08-19 05:57:36 +00:00
rangefuncs.sql	Introduce "anycompatible" family of polymorphic types.	2020-03-19 11:43:11 -04:00
rangetypes.sql	Introduce "anycompatible" family of polymorphic types.	2020-03-19 11:43:11 -04:00
regex.linux.utf8.sql	Make locale-dependent regex character classes work for large char codes.	2016-09-05 17:06:29 -04:00
regex.sql	Fix misoptimization of "{1,1}" quantifiers in regular expressions.	2019-05-12 18:53:38 -04:00
regproc.sql	Implement type regcollation	2020-03-18 21:21:00 +01:00
reindex_catalog.sql	Remove reindex_catalog test from test schedules.	2019-05-10 12:44:31 -07:00
reloptions.sql	Add vacuum_truncate reloption.	2019-04-08 16:43:57 +09:00
replica_identity.sql	Preserve replica identity index across ALTER TABLE rewrite	2020-03-13 11:57:06 +01:00
returning.sql	Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.	2015-05-08 05:43:10 +02:00
roleattributes.sql	Remove WITH OIDS support, change oid catalog column visibility.	2018-11-20 16:00:17 -08:00
rowsecurity.sql	Fix usage of whole-row variables in WCO and RLS policy expressions.	2019-09-12 18:29:45 -04:00
rowtypes.sql	Allow extracting fields from a ROW() expression in more cases.	2019-10-28 15:08:24 -04:00
rules.sql	Improve test coverage of ruleutils.c.	2019-12-06 17:40:30 -05:00
sanity_check.sql	Don't create relfilenode for relations without storage	2019-01-04 14:51:17 -03:00
security_label.sql	Establish conventions about global object names used in regression tests.	2016-07-17 18:42:43 -04:00
select.sql	Make some subquery-using test cases a bit more robust.	2018-10-14 14:02:59 -04:00
select_distinct.sql	Disk-based Hash Aggregation.	2020-03-18 15:42:02 -07:00
select_distinct_on.sql	Fix assertion failure when a SELECT DISTINCT ON expression is volatile.	2009-09-12 00:04:59 +00:00
select_having.sql	Make the behavior of HAVING without GROUP BY conform to the SQL spec.	2005-03-10 23:21:26 +00:00
select_implicit.sql	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
select_into.sql	Add more tests for CREATE TABLE AS with WITH NO DATA	2019-02-07 09:21:57 +09:00
select_parallel.sql	Don't shut down Gather[Merge] early under Limit.	2019-11-26 08:30:24 +05:30
select_views.sql	Avoid locale-dependent output in select_views regression test.	2017-05-28 14:52:18 -04:00
sequence.sql	Make command order in test more sensible	2019-10-22 10:35:54 +02:00
spgist.sql	Repair double-free in SP-GIST rescan (bug #15378)	2018-09-11 18:14:19 +01:00
stats.sql	Partially revert "Insert temporary debugging output in regression tests."	2019-08-11 18:55:32 -04:00
stats_ext.sql	Still another try at stabilizing stats_ext test results.	2020-03-31 16:09:25 -04:00
strings.sql	Allow Unicode escapes in any server encoding, not only UTF-8.	2020-03-06 14:17:43 -05:00
subscription.sql	Fix regression tests to use only global names beginning with "regress_".	2019-06-29 11:09:03 -04:00
subselect.sql	Correctly re-use hash tables in buildSubPlanHash().	2020-02-29 13:48:09 -05:00
sysviews.sql	Invent pg_hba_file_rules view to show the content of pg_hba.conf.	2017-01-30 18:00:26 -05:00
tablesample.sql	Make tablesample work with partitioned tables.	2017-02-24 12:23:28 +05:30
temp.sql	Fix misbehavior with expression indexes on ON COMMIT DELETE ROWS tables.	2019-12-01 13:09:26 -05:00
text.sql	Move checking an explicit VARIADIC "any" argument into the parser.	2013-07-18 11:52:12 -04:00
tidscan.sql	Fix bug in Tid scan.	2020-02-07 22:06:31 +09:00
time.sql	Please find attached two patches for documentation and regression tests	2006-07-06 01:46:38 +00:00
timestamp.sql	Avoid low-probability regression test failures in timestamp[tz] tests.	2019-12-22 18:00:22 -05:00
timestamptz.sql	Avoid low-probability regression test failures in timestamp[tz] tests.	2019-12-22 18:00:22 -05:00
timetz.sql	Fix some incorrect parsing of time with time zone strings	2019-08-07 18:16:31 +09:00
transactions.sql	Improve new AND CHAIN tests	2019-09-09 10:30:22 +02:00
triggers.sql	Enable BEFORE row-level triggers for partitioned tables	2020-03-18 18:58:05 -03:00
truncate.sql	Fix TRUNCATE .. CASCADE on partitions	2020-02-07 17:09:36 -03:00
tsdicts.sql	Preserve integer and float values accurately in (de)serialize_deflist.	2020-03-10 12:30:02 -04:00
tsearch.sql	Implement operator class parameters	2020-03-30 19:17:23 +03:00
tsrf.sql	Fix handling of targetlist SRFs when scan/join relation is known empty.	2019-03-07 14:22:13 -05:00
tstypes.sql	More float test and portability fixes.	2019-02-13 19:35:50 +00:00
tuplesort.sql	Try to stabilize results of new tuplesort regression test.	2019-12-14 15:01:56 -05:00
txid.sql	Add a txid_status function.	2017-03-24 12:00:53 -04:00
type_sanity.sql	Introduce "anycompatible" family of polymorphic types.	2020-03-19 11:43:11 -04:00
typed_table.sql	Suppress less info in regression tests using DROP CASCADE.	2017-08-01 16:49:23 -04:00
unicode.sql	Add SQL functions for Unicode normalization	2020-04-02 08:56:27 +02:00
union.sql	Repair issues with faulty generation of merge-append plans.	2019-05-09 16:53:05 -04:00
updatable_views.sql	Make rewriter prevent auto-updates on views with conditional INSTEAD rules.	2020-01-14 09:52:21 +00:00
update.sql	Fix system column accesses in ON CONFLICT ... RETURNING.	2019-07-24 18:45:58 -07:00
uuid.sql	Add gen_random_uuid function	2019-07-14 14:30:27 +02:00
vacuum.sql	Allow vacuum command to process indexes in parallel.	2020-01-20 07:57:49 +05:30
varchar.sql	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
window.sql	Selectively include window frames in expression walks/mutates.	2019-10-03 10:54:52 +01:00
with.sql	Revert MERGE patch	2018-04-12 11:22:56 +01:00
write_parallel.sql	Allow DML commands that create tables to use parallel query.	2017-10-05 11:40:48 -04:00
xml.sql	Avoid failure when selecting a namespace node in XMLTABLE.	2019-10-25 15:22:45 -04:00
xmlmap.sql	Fix cursor_to_xml in tableforest false mode	2017-05-03 21:41:10 -04:00