postgresql/src
Peter Eisentraut 5e1963fb76 Collations with nondeterministic comparison
This adds a flag "deterministic" to collations.  If that is false,
such a collation disables various optimizations that assume that
strings are equal only if they are byte-wise equal.  That then allows
use cases such as case-insensitive or accent-insensitive comparisons
or handling of strings with different Unicode normal forms.

This functionality is only supported with the ICU provider.  At least
glibc doesn't appear to have any locales that work in a
nondeterministic way, so it's not worth supporting this for the libc
provider.

The term "deterministic comparison" in this context is from Unicode
Technical Standard #10
(https://unicode.org/reports/tr10/#Deterministic_Comparison).

This patch makes changes in three areas:

- CREATE COLLATION DDL changes and system catalog changes to support
  this new flag.

- Many executor nodes and auxiliary code are extended to track
  collations.  Previously, this code would just throw away collation
  information, because the eventually-called user-defined functions
  didn't use it since they only cared about equality, which didn't
  need collation information.

- String data type functions that do equality comparisons and hashing
  are changed to take the (non-)deterministic flag into account.  For
  comparison, this just means skipping various shortcuts and tie
  breakers that use byte-wise comparison.  For hashing, we first need
  to convert the input string to a canonical "sort key" using the ICU
  analogue of strxfrm().

Reviewed-by: Daniel Verite <daniel@manitou-mail.org>
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:12:43 +01:00
..
backend Collations with nondeterministic comparison 2019-03-22 12:12:43 +01:00
bin Collations with nondeterministic comparison 2019-03-22 12:12:43 +01:00
common Refactor more code logic to update the control file 2019-03-18 12:59:35 +09:00
fe_utils More unconstify use 2019-02-13 11:50:16 +01:00
include Collations with nondeterministic comparison 2019-03-22 12:12:43 +01:00
interfaces Restructure libpq's handling of send failures. 2019-03-19 16:20:28 -04:00
makefiles Move port-specific parts of with_temp_install to port makefile. 2019-02-04 18:54:56 +00:00
pl PL/Tcl: Improve trigger tests organization 2019-03-15 12:42:07 +01:00
port Fix error handling of readdir() port implementation on first file lookup 2019-03-04 09:49:06 +09:00
template Yet further rethinking of build changes for macOS Mojave. 2018-11-02 18:54:00 -04:00
test Collations with nondeterministic comparison 2019-03-22 12:12:43 +01:00
timezone More unconstify use 2019-02-13 11:50:16 +01:00
tools Get rid of jsonpath_gram.h and jsonpath_scanner.h 2019-03-20 11:13:34 +03:00
tutorial SQL comment: remove extra word in heading comment 2019-01-25 18:57:21 -05:00
.gitignore
DEVELOPERS
Makefile Fix partial-build problems introduced by having more generated headers. 2018-04-09 16:42:10 -04:00
Makefile.global.in Revert attempts to use POPCNT etc instructions 2019-02-15 16:32:30 -03:00
Makefile.shlib Ensure static libraries have correct mod time even if ranlib messes it up. 2018-11-29 15:53:44 -05:00
nls-global.mk Replace @postgresql.org with @lists.postgresql.org for mailinglists 2019-01-19 19:06:35 +01:00