in schemas other than the system namespace; however, there's no search
path yet, and not all operations work yet on tables outside the system
namespace.
objects to be privilege-checked. Some change in their APIs would be
necessary no matter what in the schema environment, and simply getting
rid of the name-based interface entirely seems like the best way.
the parsetree representation. As yet we don't *do* anything with schema
names, just drop 'em on the floor; but you can enter schema-compatible
command syntax, and there's even a primitive CREATE SCHEMA command.
No doc updates yet, except to note that you can now extract a field
from a function-returning-row's result with (foo(...)).fieldname.
PostgreSQL. This hash function replaces the one used by hash indexes and
the catalog cache. Hash joins use a different, relatively poor-quality
hash function, but I'll fix that later.
As suggested by Tom Lane, this patch also changes the size of the fixed
hash table used by the catalog cache to be a power-of-2 (instead of a
prime: I chose 256 instead of 257). This allows the catcache to lookup
hash buckets using a simple bitmask. This should improve the performance
of the catalog cache slightly, since the previous method (modulo a
prime) was slow.
In my tests, this improves the performance of hash indexes by between 4%
and 8%; the performance when using btree indexes or seqscans is
basically unchanged.
Neil Conway <neilconway@rogers.com>
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
now just below FATAL in server_min_messages. Added more text to
highlight ordering difference between it and client_min_messages.
---------------------------------------------------------------------------
REALLYFATAL => PANIC
STOP => PANIC
New INFO level the prints to client by default
New LOG level the prints to server log by default
Cause VACUUM information to print only to the client
NOTICE => INFO where purely information messages are sent
DEBUG => LOG for purely server status messages
DEBUG removed, kept as backward compatible
DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1 added
DebugLvl removed in favor of new DEBUG[1-5] symbols
New server_min_messages GUC parameter with values:
DEBUG[5-1], INFO, NOTICE, ERROR, LOG, FATAL, PANIC
New client_min_messages GUC parameter with values:
DEBUG[5-1], LOG, INFO, NOTICE, ERROR, FATAL, PANIC
Server startup now logged with LOG instead of DEBUG
Remove debug_level GUC parameter
elog() numbers now start at 10
Add test to print error message if older elog() values are passed to elog()
Bootstrap mode now has a -d that requires an argument, like postmaster
both input streams to the end. If one variable's range is much less
than the other, an indexscan-based merge can win by not scanning all
of the other table. Per example from Reinhard Max.
are now both invoked once per received SQL command (raw parsetree) from
pg_exec_query_string. BeginCommand is actually just an empty routine
at the moment --- all its former operations have been pushed into tuple
receiver setup routines in printtup.c. This makes for a clean distinction
between BeginCommand/EndCommand (once per command) and the tuple receiver
setup/teardown routines (once per ExecutorRun call), whereas the old code
was quite ad hoc. Along the way, clean up the calling conventions for
ExecutorRun a little bit.
report for each received SQL command, regardless of rewriting activity.
Also ensure that this report comes from the 'original' command, not the
last command generated by rewrite; this fixes 7.2 breakage for INSERT
commands that have actions added by rules. Fernando Nasser and Tom Lane.
Improve 'pg_internal.init' relcache entry preload mechanism so that it is
safe to use for all system catalogs, and arrange to preload a realistic
set of system-catalog entries instead of only the three nailed-in-cache
indexes that were formerly loaded this way. Fix mechanism for deleting
out-of-date pg_internal.init files: this must be synchronized with transaction
commit, not just done at random times within transactions. Drive it off
relcache invalidation mechanism so that no special-case tests are needed.
Cache additional information in relcache entries for indexes (their pg_index
tuples and index-operator OIDs) to eliminate repeated lookups. Also cache
index opclass info at the per-opclass level to avoid repeated lookups during
relcache load.
Generalize 'systable scan' utilities originally developed by Hiroshi,
move them into genam.c, use in a number of places where there was formerly
ugly code for choosing either heap or index scan. In particular this allows
simplification of the logic that prevents infinite recursion between syscache
and relcache during startup: we can easily switch to heapscans in relcache.c
when and where needed to avoid recursion, so IndexScanOK becomes simpler and
does not need any expensive initialization.
Eliminate useless opening of a heapscan data structure while doing an indexscan
(this saves an mdnblocks call and thus at least one kernel call).
originally created with, so that the set of visible tuples does not
change as a result of other activity. This essentially makes PG cursors
INSENSITIVE per the SQL92 definition. See bug report of 13-Feb-02.
inner indexscan (ie, one with runtime keys). ExecIndexReScan must
compute or recompute runtime keys even if we are rescanning in the
EPQ case. TidScan seems to have comparable problems. Per bug
noted by Barry Lind 11-Feb-02.
readability. Bizarre '(long *) TRUE' return convention is gone,
in favor of just raising an error internally in dynahash.c when
we detect hashtable corruption. HashTableWalk is gone, in favor
of using hash_seq_search directly, since it had no hope of working
with non-LONGALIGNable datatypes. Simplify some other code that was
made undesirably grotty by promixity to HashTableWalk.
Frank Miles 7-Sep-01. This is really just sticking a finger in the dike.
Frank's case works now, but we still couldn't support a recursive function
returning a set. Really need to restructure querytrees and execution
state so that the querytree is *read only*. We've run into this over and
over and over again ... it has to happen sometime soon.
default, but OIDS are removed from many system catalogs that don't need them.
Some interesting side effects: TOAST pointers are 20 bytes not 32 now;
pg_description has a three-column key instead of one.
Bugs fixed in passing: BINARY cursors work again; pg_class.relhaspkey
has some usefulness; pg_dump dumps comments on indexes, rules, and
triggers in a valid order.
initdb forced.
Note: I didn't force an initdb, figuring that one today was enough.
However, there is a new function in pg_proc.h, and pg_dump won't be
able to dump partial indexes until you add that function.
per previous discussion on pghackers. Most of the duplicate code in
different AMs' ambuild routines has been moved out to a common routine
in index.c; this means that all index types now do the right things about
inserting recently-dead tuples, etc. (I also removed support for EXTEND
INDEX in the ambuild routines, since that's about to go away anyway, and
it cluttered the code a lot.) The retail indextuple deletion routines have
been replaced by a "bulk delete" routine in which the indexscan is inside
the access method. I haven't pushed this change as far as it should go yet,
but it should allow considerable simplification of the internal bookkeeping
for deletions. Also, add flag columns to pg_am to eliminate various
hardcoded tests on AM OIDs, and remove unused pg_am columns.
Fix rtree and gist index types to not attempt to store NULLs; before this,
gist usually crashed, while rtree managed not to crash but computed wacko
bounding boxes for NULL entries (which might have had something to do with
the performance problems we've heard about occasionally).
Add AtEOXact routines to hash, rtree, and gist, all of which have static
state that needs to be reset after an error. We discovered this need long
ago for btree, but missed the other guys.
Oh, one more thing: concurrent VACUUM is now the default.
tests to return the correct results per SQL9x when given NULL inputs.
Reimplement these tests as well as IS [NOT] NULL to have their own
expression node types, instead of depending on special functions.
From Joe Conway, with a little help from Tom Lane.
report on old-style functions invoked by RI triggers. We had a number of
other places that were being sloppy about which memory context FmgrInfo
subsidiary data will be allocated in. Turns out none of them actually
cause a problem in 7.1, but this is for arcane reasons such as the fact
that old-style triggers aren't supported anyway. To avoid getting burnt
later, I've restructured the trigger support so that we don't keep trigger
FmgrInfo structs in relcache memory. Some other related cleanups too:
it's not really necessary to call fmgr_info at all while setting up
the index support info in relcache entries, because those ScanKeyEntry
structs are never used to invoke the functions. This should speed up
relcache initialization a tiny bit.
the same tuple slot that the raw tuple came from, because that slot has
the wrong tuple descriptor. Store it into its own slot with the correct
descriptor, instead. This repairs problems with SPI functions seeing
inappropriate tuple descriptors --- for example, plpgsql code failing to
cope with SELECT FOR UPDATE.
- New functions to create a portal using a prepared/saved
SPI plan or lookup an existing portal by name.
- Functions to fetch/move from/in portals. Results are placed
in the usual SPI_processed and SPI_tuptable, so the entire
set of utility functions can be used to gain attribute access.
- Prepared/saved SPI plans now use their own memory context
and SPI_freeplan(plan) can remove them.
- Tuple result sets (SPI_tuptable) now uses it's own memory
context and can be free'd by SPI_freetuptable(tuptab).
Enhancement of PL/pgSQL
- Uses generic named portals internally in FOR ... SELECT
loops to avoid running out of memory on huge result sets.
- Support for CURSOR and REFCURSOR syntax using the new SPI
functionality. Cursors used internally only need no explicit
transaction block. Refcursor variables can be used inside
of explicit transaction block to pass cursors between main
application and functions.
Jan
trees (mostly my fault). Repair. Also fix long-standing bug in ExecReplace:
after recomputing a concurrently updated tuple, we must recheck constraints.
Make EvalPlanQual leak memory with somewhat less enthusiasm than before,
although plugging leaks fully will require more changes than I care to risk
in a dot-release.
to their children, leading to misbehavior if they had any children that paid
attention to chgParam (most plan node types don't). Append's bug has been
there a long time, but nobody had noticed because it used to be difficult
to create a query where an Append would be used below the top level of a
plan; so there were never any parameters getting passed down. SubqueryScan
is new in 7.1 ... and I'd modeled its behavior on Append :-(
a separate statement (though it can still be invoked as part of VACUUM, too).
pg_statistic redesigned to be more flexible about what statistics are
stored. ANALYZE now collects a list of several of the most common values,
not just one, plus a histogram (not just the min and max values). Random
sampling is used to make the process reasonably fast even on very large
tables. The number of values and histogram bins collected is now
user-settable via an ALTER TABLE command.
There is more still to do; the new stats are not being used everywhere
they could be in the planner. But the remaining changes for this project
should be localized, and the behavior is already better than before.
A not-very-related change is that sorting now makes use of btree comparison
routines if it can find one, rather than invoking '<' twice.
function arguments in join queries: copy the tuples into
TransactionCommandContext so they don't get recycled too soon. This is
horrid, but not any worse than 7.0 or before, which also leaked such
tuples until end of query. A proper fix will require allowing tuple
datums to be physically stored inside larger tuple datums, which opens
up a bunch of issues that can't realistically be solved for 7.1.1.
1. If there is exactly one pg_operator entry of the right name and oprkind,
oper() and related routines would return that entry whether its input type
had anything to do with the request or not. This is just premature
optimization: we shouldn't return the single candidate until after we verify
that it really is a valid candidate, ie, is at least coercion-compatible
with the given types.
2. oper() and related routines only promise a coercion-compatible result.
Unfortunately, there were quite a few callers that assumed the returned
operator is binary-compatible with the given datatype; they would proceed
to call it without making any datatype coercions. These callers include
sorting, grouping, aggregation, and VACUUM ANALYZE. In general I think
it is appropriate for these callers to require an exact or binary-compatible
match, so I've added a new routine compatible_oper() that only succeeds if
it can find an operator that doesn't require any run-time conversions.
Callers now call oper() or compatible_oper() depending on whether they are
prepared to deal with type conversion or not.
The upshot of these bugs is revealed by the following silliness in PL/Tcl's
selftest: it creates an operator @< on int4, and then tries to use it to
sort a char(N) column. The system would let it do that :-( (and evidently
has done so since 6.3 :-( :-(). The result in this case was just a silly
sort order, but the reverse combination would've provoked coredump from
trying to dereference integers. With this fix you get more reasonable
behavior:
pltcl_test=# select * from T_pkey1 order by key1, key2 using @<;
ERROR: Unable to identify an operator '@<' for types 'bpchar' and 'bpchar'
You will have to retype this query using an explicit cast
allocated by plan nodes are not leaked at end of query. This doesn't
really matter for normal queries, but it sure does for queries invoked
repetitively inside SQL functions. Clean up some other grotty code
associated with tupdescs, and fix a few other memory leaks exposed by
tests with simple SQL functions.
are treated more like 'cancel' interrupts: the signal handler sets a
flag that is examined at well-defined spots, rather than trying to cope
with an interrupt that might happen anywhere. See pghackers discussion
of 1/12/01.
1. Distinguish cases where a Datum representing a tuple datatype is an OID
from cases where it is a pointer to TupleTableSlot, and make sure we use
the right typlen in each case.
2. Make fetchatt() and related code support 8-byte by-value datatypes on
machines where Datum is 8 bytes. Centralize knowledge of the available
by-value datatype sizes in two macros in tupmacs.h, so that this will be
easier if we ever have to do it again.
an error as we used to. In an OUTER JOIN scenario, retrieving a null
CTID from one of the input relations is entirely expected. We still
want to lock the input rows from the other relations, so just ignore
the null and keep going.
maintained for each cache entry. A cache entry will not be freed until
the matching ReleaseSysCache call has been executed. This eliminates
worries about cache entries getting dropped while still in use. See
my posting to pg-hackers of even date for more info.
joins, and clean things up a good deal at the same time. Append plan node
no longer hacks on rangetable at runtime --- instead, all child tables are
given their own RT entries during planning. Concept of multiple target
tables pushed up into execMain, replacing bug-prone implementation within
nodeAppend. Planner now supports generating Append plans for inheritance
sets either at the top of the plan (the old way) or at the bottom. Expanding
at the bottom is appropriate for tables used as sources, since they may
appear inside an outer join; but we must still expand at the top when the
target of an UPDATE or DELETE is an inheritance set, because we actually need
a different targetlist and junkfilter for each target table in that case.
Fortunately a target table can't be inside an outer join... Bizarre mutual
recursion between union_planner and prepunion.c is gone --- in fact,
union_planner doesn't really have much to do with union queries anymore,
so I renamed it grouping_planner.
(WAL logging for this is not done yet, however.) Clean up a number of really
crufty things that are no longer needed now that DROP behaves nicely. Make
temp table mapper do the right things when drop or rename affecting a temp
table is rolled back. Also, remove "relation modified while in use" error
check, in favor of locking tables at first reference and holding that lock
throughout the statement.
ExecutorRun. This allows LIMIT to work in a view. Also, LIMIT in a
cursor declaration will behave in a reasonable fashion, whereas before
it was overridden by the FETCH count.
SQL92 semantics, including support for ALL option. All three can be used
in subqueries and views. DISTINCT and ORDER BY work now in views, too.
This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT
where the SELECT yields different datatypes than the INSERT needs. I did
that by making UNION subqueries and SELECT in INSERT be treated like
subselects-in-FROM, thereby allowing an extra level of targetlist where the
datatype conversions can be inserted safely.
INITDB NEEDED!
(Don't forget that an alias is required.) Views reimplemented as expanding
to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually
work now (he says optimistically). No UNION support in subselects/views
yet, but I have some ideas about that. Rule-related permissions checking
moved out of rewriter and into executor.
INITDB REQUIRED!
for views. Views are now have a "relkind" of
RELKIND_VIEW instead of RELKIND_RELATION.
Also, views no longer have actual heap storage
files.
The following changes were made
1. CREATE VIEW sets the new relkind
2. The executor complains if a DELETE or
INSERT references a view.
3. DROP RULE complains if an attempt is made
to delete a view SELECT rule.
4. CREATE RULE "_RETmytable" AS ON SELECT TO mytable DO INSTEAD ...
1. checks to make sure mytable is empty.
2. sets the relkind to RELKIND_VIEW.
3. deletes the heap storage files.
5. LOCK myview is not allowed. :)
6. the regression test type_sanity was changed to
account for the new relkind value.
7. CREATE INDEX ON myview ... is not allowed.
8. VACUUM myview is not allowed.
VACUUM automatically skips views when do the entire
database.
9. TRUNCATE myview is not allowed.
THINGS LEFT TO THINK ABOUT
o pg_views
o pg_dump
o pgsql (\d \dv)
o Do we really want to be able to inherit from views?
o Is 'DROP TABLE myview' OK?
--
Mark Hollomon
user is now defined in terms of the user id, the user name is only computed
upon request (for display purposes). This is kind of the opposite of the
previous state, which would maintain the user name and compute the user id
for permission checks.
Besides perhaps saving a few cycles (integer vs string), this now creates a
single point of attack for changing the user id during a connection, for
purposes of "setuid" functions, etc.
for example, an SQL function can be used in a functional index. (I make
no promises about speed, but it'll work ;-).) Clean up and simplify
handling of functions returning sets.
right circumstances a hash join executed as a DECLARE CURSOR/FETCH
query would crash the backend. Problem as seen in current sources was
that the hash tables were stored in a context that was a child of
TransactionCommandContext, which got zapped at completion of the FETCH
command --- but cursor cleanup executed at COMMIT expected the tables
to still be valid. I haven't chased down the details as seen in 7.0.*
but I'm sure it's the same general problem.
pg_proc.c (where it's actually used). Fix it to correctly handle tlists
that contain resjunk target items, and improve error messages. This
addresses bug reported by Krupnikov 6-July-00.
right thing with variable-free clauses that contain noncachable functions,
such as 'WHERE random() < 0.5' --- these are evaluated once per
potential output tuple. Expressions that contain only Params are
now candidates to be indexscan quals --- for example, 'var = ($1 + 1)'
can now be indexed. Cope with RelabelType nodes atop potential indexscan
variables --- this oversight prevents 7.0.* from recognizing some
potentially indexscanable situations.
from Param nodes, per discussion a few days ago on pghackers. Add new
expression node type FieldSelect that implements the functionality where
it's actually needed. Clean up some other unused fields in Func nodes
as well.
NOTE: initdb forced due to change in stored expression trees for rules.
thing when there are multiple result relations. Formerly, during
something like 'UPDATE foo*', foo's constraints and *only* foo's
constraints would be applied to all foo's children. Wrong-o ...
These two routines will now ALWAYS elog() on failure, whether you ask for
a lock or not. If you really want to get a NULL return on failure, call
the new routines heap_open_nofail()/heap_openr_nofail(). By my count there
are only about three places that actually want that behavior. There were
rather more than three places that were missing the check they needed to
make under the old convention :-(.
varlena elements work now. Allow assignment to previously-nonexistent
subscript position to extend array, but only for 1-D arrays and only
if adjacent to existing positions (could do more if we had a way to
represent nulls in arrays, but I don't want to tackle that now).
Arrange for assignment of NULL to an array element in UPDATE to be a
no-op, rather than setting the entire array to NULL as it used to.
(Throwing an error would be a reasonable alternative, but it's never
done that...) Update regress test accordingly.
Remove a bunch of crufty code for large-object-based arrays, which is
superseded by TOAST and likely hasn't worked in a long time anyway.
Clean up array code a little, and in particular eliminate its habit
of scribbling on the input array (ie, modifying the input tuple :-().
There's now only one transition value and transition function.
NULL handling in aggregates is a lot cleaner. Also, use Numeric
accumulators instead of integer accumulators for sum/avg on integer
datatypes --- this avoids overflow at the cost of being a little slower.
Implement VARIANCE() and STDDEV() aggregates in the standard backend.
Also, enable new LIKE selectivity estimators by default. Unrelated
change, but as long as I had to force initdb anyway...
pass-by-ref data types --- eg, an index on lower(textfield) --- no longer
leak memory during index creation or update. Clean up a lot of redundant
code ... did you know that copy, vacuum, truncate, reindex, extend index,
and bootstrap each basically duplicated the main executor's logic for
extracting information about an index and preparing index entries?
Functional indexes should be a little faster now too, due to removal
of repeated function lookups.
CREATE INDEX 'opt_type' clause is deimplemented by these changes,
but I haven't removed it from the parser yet (need to merge with
Thomas' latest change set first).
memory contexts. Currently, only leaks in expressions executed as
quals or projections are handled. Clean up some old dead cruft in
executor while at it --- unused fields in state nodes, that sort of thing.
Special handling of TOAST relations during VACUUM. TOAST relations
are vacuumed while the lock on the master table is still active.
The ANALYZE flag doesn't propagate to their vacuuming because the
toaster access routines allways use index access ignoring stats, so
why compute them at all.
Protection of TOAST relations against normal INSERT/UPDATE/DELETE
while offering SELECT for debugging purposes.
Jan
Don't go through pg_exec_query_dest(), but directly to the execution
routines. Also, extend parameter lists so that there's no need to
change the global setting of allowSystemTableMods, a hack that was
certain to cause trouble in the event of any error.
for details). It doesn't really do that much yet, since there are no
short-term memory contexts in the executor, but the infrastructure is
in place and long-term contexts are handled reasonably. A few long-
standing bugs have been fixed, such as 'VACUUM; anything' in a single
query string crashing. Also, out-of-memory is now considered a
recoverable ERROR, not FATAL.
Eliminate a large amount of crufty, now-dead code in and around
memory management.
Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and
backend startup.
are opened in a consistent order by different backends (I ordered them
by index OID because that's easy, but any other consistent order would
do as well). This avoids potential deadlock for index types that we
acquire exclusive locks on ... ie, rtree.
materialized tupleset is small enough) instead of a temporary relation.
This was something I was thinking of doing anyway for performance, and Jan
says he needs it for TOAST because he doesn't want to cope with toasting
noname relations. With this change, the 'noname table' support in heap.c
is dead code, and I have accordingly removed it. Also clean up 'noname'
plan handling in planner --- nonames are either sort or materialize plans,
and it seems less confusing to handle them separately under those names.
discussion of 5/19/00). pg_index is now searched for indexes of a
relation using an indexscan. Moreover, this is done once and cached
in the relcache entry for the relation, in the form of a list of OIDs
for the indexes. This list is used by the parser and executor to drive
lookups in the pg_index syscache when they want to know the properties
of the indexes. Net result: index information will be fully cached
for repetitive operations such as inserts.
That means you can now set your options in either or all of $PGDATA/configuration,
some postmaster option (--enable-fsync=off), or set a SET command. The list of
options is in backend/utils/misc/guc.c, documentation will be written post haste.
pg_options is gone, so is that pq_geqo config file. Also removed were backend -K,
-Q, and -T options (no longer applicable, although -d0 does the same as -Q).
Added to configure an --enable-syslog option.
changed all callers from TPRINTF to elog(DEBUG)
key call sites are changed, but most called functions are still oldstyle.
An exception is that the PL managers are updated (so, for example, NULL
handling now behaves as expected in plperl and plpgsql functions).
NOTE initdb is forced due to added column in pg_proc.
think that both sides of indexqual look like index keys. An example is
create table inside (f1 float8 primary key);
create table outside (g1 float8, g2 float8);
select * from inside,outside where f1 = atan2(g1+1, g2);
ERROR: ExecInitIndexScan: both left and right ops are rel-vars
(note that failure is potentially platform-dependent). Solution is a
cleanup I had had in mind to make anyway: functional index keys should
be represented as Var nodes in the fixed indexqual, just like regular
index keys.
table for an average of NTUP_PER_BUCKET tuples/bucket, but cost_hashjoin
was assuming a target load of one tuple/bucket. This was causing a
noticeable underestimate of hashjoin costs.
Ensure that outer tuple link needed for inner indexscan qual evaluation
gets set in the EvalPlanQual case. This stops coredump, but we still
have resource leaks due to failure to clean up EvalPlanQual properly...
would crash, due to premature invocation of SetQuerySnapshot(). Clean
up problems with handling of multiple queries by splitting
pg_parse_and_plan into two routines. The old code would not, for
example, do the right thing with END; SELECT... submitted in one query
string when it had been in transaction abort state, because it'd decide
to skip planning the SELECT before it had executed the END. New
arrangement is simpler and doesn't force caller to plan if only
parse+rewrite is needed.
nodes. The former version failed to check permissions of relations that
were referenced in second and later clauses of UNIONs, and it did not
check permissions of tables referenced via inheritance.
represent the result of a binary-compatible type coercion. At runtime
it just evaluates its argument --- but during type resolution, exprType
will pick up the output type of the RelabelType node instead of the type
of the argument. This solves some longstanding problems with dropped
type coercions, an example being 'select now()::abstime::int4' which
used to produce date-formatted output, not an integer, because the
coercion to int4 was dropped on the floor.
SELECT a FROM t1 tx (a);
Allow join syntax, including queries like
SELECT * FROM t1 NATURAL JOIN t2;
Update RTE structure to hold column aliases in an Attr structure.
SELECT DISTINCT ON (expr [, expr ...]) targetlist ...
and there is a check to make sure that the user didn't specify an ORDER BY
that's incompatible with the DISTINCT operation.
Reimplement nodeUnique and nodeGroup to use the proper datatype-specific
equality function for each column being compared --- they used to do
bitwise comparisons or convert the data to text strings and strcmp().
(To add insult to injury, they'd look up the conversion functions once
for each tuple...) Parse/plan representation of DISTINCT is now a list
of SortClause nodes.
initdb forced by querytree change...
from a constraint condition does not violate the constraint (cf. discussion
on pghackers 12/9/99). Implemented by adding a parameter to ExecQual,
specifying whether to return TRUE or FALSE when the qual result is
really NULL in three-valued boolean logic. Currently, ExecRelCheck is
the only caller that asks for TRUE, but if we find any other places that
have the wrong response to NULL, it'll be easy to fix them.
Make all system indexes unique.
Make all cache loads use system indexes.
Rename *rel to *relid in inheritance tables.
Rename cache names to be clearer.
subselects can only appear on the righthand side of a binary operator.
That's still true for quantified predicates like x = ANY (SELECT ...),
but a subselect that delivers a single result can now appear anywhere
in an expression. This is implemented by changing EXPR_SUBLINK sublinks
to represent just the (SELECT ...) expression, without any 'left hand
side' or combining operator --- so they're now more like EXISTS_SUBLINK.
To handle the case of '(x, y, z) = (SELECT ...)', I added a new sublink
type MULTIEXPR_SUBLINK, which acts just like EXPR_SUBLINK used to.
But the grammar will only generate one for a multiple-left-hand-side
row expression.
nulls with non-nulls using proper three-valued boolean logic. Also clean
up ExecQual to make it clearer that ExecQual *does* follow the SQL spec
for boolean nulls. See '[BUGS] (null) != (null)' thread around 10/26/99
for more detail.
Now indexes of pg_class and pg_type are unique indexes
and guarantee the uniqueness of correponding attributes.
heap_create() was changed to take another boolean parameter
which allows to postpone the creation of disk file.
The name of rd_nonameunlinked was changed to rd_unlinked.
It is used generally(not only for noname relations) now.
Requires initdb.
in the TupleDesc that the caller already has (for call from ExecMain) or
can make just as easily as ExecInitJunkFilter() can (for call from
ExecAppend). Also, don't bother to build a junk filter for an INSERT
operation that doesn't actually need one, which is the normal case.
before calling execProject, when the outerPlan has returned zero tuples.
I took this out under the mistaken impression that the input tuple
couldn't be referenced by execProject if we weren't in GROUP BY mode.
But it can, if we're in an UPDATE or DELETE...
a generalized module 'tuplesort.c' that can sort either HeapTuples or
IndexTuples, and is not tied to execution of a Sort node. Clean up
memory leakages in sorting, and replace nbtsort.c's private implementation
of mergesorting with calls to tuplesort.c.
BufFile so that it handles multi-segment temporary files transparently.
This allows sorts and hashes to work with data exceeding 2Gig (or whatever
the local limit on file size is). Change psort.c to use relative seeks
instead of absolute seeks for backwards scanning, so that it won't fail
when the data volume exceeds 2Gig.
mentioned in FROM but not elsewhere in the query: such tables should be
joined over anyway. Aside from being more standards-compliant, this allows
removal of some very ugly hacks for COUNT(*) processing. Also, allow
HAVING clause without aggregate functions, since SQL does. Clean up
CREATE RULE statement-list syntax the same way Bruce just fixed the
main stmtmulti production.
CAUTION: addition of a field to RangeTblEntry nodes breaks stored rules;
you will have to initdb if you have any rules.
Implements the CREATE CONSTRAINT TRIGGER and SET CONSTRAINTS commands.
TODO:
Generic builtin trigger procedures
Automatic execution of appropriate CREATE CONSTRAINT... at CREATE TABLE
Support of new trigger type in pg_dump
Swapping of huge # of events to disk
Jan
an empty targetlist *and* fails to return any tuples, as will happen
for example with 'SELECT COUNT(1) FROM table WHERE ...' if the where-
clause selects no tuples. It's so nice to make a fix by diking out code,
instead of adding more...
with no input rows, per pghackers discussions around 7/22/99. Clean up
a bunch of ugly coding while at it; remove redundant re-lookup of
aggregate info at start of each new GROUP. Arrange to pfree intermediate
values when they are pass-by-ref types, so that aggregates on pass-by-ref
types no longer eat memory. This takes care of a couple of TODO items...
Frankpitt, plus some improvements from yours truly. The simplifier depends
on the proiscachable field of pg_proc to tell it whether a function is
safe to pre-evaluate --- things like nextval() are not, for example.
Update pg_proc.h to contain reasonable cacheability information; as of
6.5.* hardly any functions were marked cacheable. I may have erred too
far in the other direction; see recent mail to pghackers for more info.
This update does not force an initdb, exactly, but you won't see much
benefit from the simplifier until you do one.
* Buffer refcount cleanup (per my "progress report" to pghackers, 9/22).
* Add links to backend PROC structs to sinval's array of per-backend info,
and use these links for routines that need to check the state of all
backends (rather than the slow, complicated search of the ShmemIndex
hashtable that was used before). Add databaseOID to PROC structs.
* Use this to implement an interlock that prevents DESTROY DATABASE of
a database containing running backends. (It's a little tricky to prevent
a concurrently-starting backend from getting in there, since the new
backend is not able to lock anything at the time it tries to look up
its database in pg_database. My solution is to recheck that the DB is
OK at the end of InitPostgres. It may not be a 100% solution, but it's
a lot better than no interlock at all...)
* In ALTER TABLE RENAME, flush buffers for the relation before doing the
rename of the physical files, to ensure we don't get failures later from
mdblindwrt().
* Update TRUNCATE patch so that it actually compiles against current
sources :-(.
You should do "make clean all" after pulling these changes.
additional argument specifying the kind of lock to acquire/release (or
'NoLock' to do no lock processing). Ensure that all relations are locked
with some appropriate lock level before being examined --- this ensures
that relevant shared-inval messages have been processed and should prevent
problems caused by concurrent VACUUM. Fix several bugs having to do with
mismatched increment/decrement of relation ref count and mismatched
heap_open/close (which amounts to the same thing). A bogus ref count on
a relation doesn't matter much *unless* a SI Inval message happens to
arrive at the wrong time, which is probably why we got away with this
sloppiness for so long. Repair missing grab of AccessExclusiveLock in
DROP TABLE, ALTER/RENAME TABLE, etc, as noted by Hiroshi.
Recommend 'make clean all' after pulling this update; I modified the
Relation struct layout slightly.
Will post further discussion to pghackers list shortly.
documented intepretation of the lefthand and oper fields. Fix a number of
obscure problems while at it --- for example, the old code failed if the parser
decided to insert a type-coercion function just below the operator of a
SubLink.
CAUTION: this will break stored rules that contain subplans. You may
need to initdb.
and fix_opids processing to a single recursive pass over the plan tree
executed at the very tail end of planning, rather than haphazardly here
and there at different places. Now that tlist Vars do not get modified
until the very end, it's possible to get rid of the klugy var_equal and
match_varid partial-matching routines, and just use plain equal()
throughout the optimizer. This is a step towards allowing merge and
hash joins to be done on expressions instead of only Vars ...
sort order down into planner, instead of handling it only at the very top
level of the planner. This fixes many things. An explicit sort is now
avoided if there is a cheaper alternative (typically an indexscan) not
only for ORDER BY, but also for the internal sort of GROUP BY. It works
even when there is no other reason (such as a WHERE condition) to consider
the indexscan. It works for indexes on functions. It works for indexes
on functions, backwards. It's just so cool...
CAUTION: I have changed the representation of SortClause nodes, therefore
THIS UPDATE BREAKS STORED RULES. You will need to initdb.
contains much code that looks like it will handle indexquals with the index
key on either side of the operator, in fact indexquals must have the index
key on the left because of limitations of the ScanKey machinery. Perhaps
someone will be motivated to fix that someday...
> >
> > was implemented by Jan Wieck.
> > His work is for ascending order cases.
> >
> > Here is a patch to prevent sorting also in descending
> > order cases.
> > Because I had already changed _bt_first() to position
> > backward correctly before v6.5,this patch would work.
> >
Hiroshi Inoue
Inoue@tpf.co.jp
of the SELECT part of the statement is just like a plain SELECT. All
INSERT-specific processing happens after the SELECT parsing is done.
This eliminates many problems, e.g. INSERT ... SELECT ... GROUP BY using
the wrong column labels. Ensure that DEFAULT clauses are coerced to
the target column type, whether or not stored clause produces the right
type. Substantial cleanup of parser's array support.
pointer to palloc'd but uninitialized memory. This is not cool; anyone looking
at the returned 'tuple' would at best coredump and at worst behave in a
bizarre and irreproducible way. Fix it to return a predictable value,
namely a correctly-set-up palloc'd tuple containing zero attributes.
I believe this fix is both safe and critical.
this one could be useful for people experiencing out-of-memory crashes while
executing queries which retrieve or use a very large number of tuples.
The problem happens when storage is allocated for functions results used in
a large query, for example:
select upper(name) from big_table;
select big_table.array[1] from big_table;
select count(upper(name)) from big_table;
This patch is a dirty hack that fixes the out-of-memory problem for the most
common cases, like the above ones. It is not the final solution for the
problem but it can work for some people, so I'm posting it.
The patch should be safe because all changes are under #ifdef. Furthermore
the feature can be enabled or disabled at runtime by the `free_tuple_memory'
options in the pg_options file. The option is disabled by default and must
be explicitly enabled at runtime to have any effect.
To enable the patch add the follwing line to Makefile.custom:
CUSTOM_COPT += -DFREE_TUPLE_MEMORY
To enable the option at runtime add the following line to pg_option:
free_tuple_memory=1
Massimo
after ExecEndNode. It must be done! Or we'll be out of free
tuple slots very soon, though slots are freed by ExecEndNode
and ready for reusing.
We didn't see this problem before because of
int nSlots = ExecCountSlotsNode(plan);
TupleTable tupleTable = ExecCreateTupleTable(nSlots + 10);
/* why add ten? - jolly */
code in InitPlan - i.e. extra 10 slots. Simple select uses
3 slots and so it was possible to re-use evaluation plan
3 additional times and didn't get
elog(NOTICE, "Plan requires more slots than are available");
elog(ERROR, "send mail to your local executor guru to fix this");
Changes are obvious and shouldn't be problems with them.
Though, I added Assert(epqstate->es_tupleTable->next == 0)
before EvalPlanQual():ExecInitNode and we'll notice if
something is still wrong. Is it better to change Assert
to elog(ERROR) ?
fixed-size hashtable. This should prevent 'hashtable out of memory' errors,
unless you really do run out of memory. Note: target size for hashtable
is now taken from -S postmaster switch, not -B, since it is local memory
in the backend rather than shared memory.
lists are now plain old garden-variety Lists, allocated with palloc,
rather than specialized expansible-array data allocated with malloc.
This substantially simplifies their handling and eliminates several
sources of memory leakage.
Several basic types of erroneous queries (syntax error, attempt to
insert a duplicate key into a unique index) now demonstrably leak
zero bytes per query.
about certain to fail anytime it decided the relation to be hashed was
too big to fit in memory --- the code for 'batching' a series of hashjoins
had multiple errors. I've fixed the easier problems. A remaining big
problem is that you can get 'hashtable out of memory' if the code's
guesstimate about how much overflow space it will need turns out wrong.
That will require much more extensive revisions to fix, so I'm committing
these fixes now before I start on that problem.
expression context (ie, not at the top level of a WHERE clause). Examples
like this one work now:
SELECT name, value FROM t1 as touter WHERE
(value/(SELECT AVG(value) FROM t1 WHERE name = touter.name)) > 0.75;
indexes.
1. Index Scan using plural indexids never scan backward
as to the order of indexids.
2. The cursor using Index scan is not usable after moving
past the end.
This patch solves above bugs.
Moreover the change of _bt_first() would be useful to extend
ORDER BY patch by Jan Wieck for all descending order cases.
Hiroshi Inoue
hashjoin's hashFunc() so that it does the right thing with pass-by-value
data types (the old code would always return 0 for int2 or char values,
which would work but would slow things down a lot). Extend opr_sanity
regress test to catch more kinds of errors.
change functionality, but makes the code more ANSI C'ish.
My AIX xlc compiler barfs on all of these. Can someone please
review and apply to current.
<<port.patch>>
Thanks
Andreas
Ok. I made patches replacing all of "#if FALSE" or "#if 0" to "#ifdef
NOT_USED" for current. I have tested these patches in that the
postgres binaries are identical.
qualification expression trees in the execution state. Prevents from
memory exhaustion on INSERT, UPDATE or COPY to tables that have CHECK
constraints. Speedup against the variant using freeObject() is more than
factor 2.
Jan
so that fetching an attribute value needs only one SearchSysCacheTuple call
instead of two redundant searches. This speeds up a large SELECT by about
ten percent, and probably will help GROUP BY and SELECT DISTINCT too.
INTERSECT and EXCEPT is available for postgresql-v6.4!
The patch against v6.4 is included at the end of the current text
(in uuencoded form!)
I also included the text of my Master's Thesis. (a postscript
version). I hope that you find something of it useful and would be
happy if parts of it find their way into the PostgreSQL documentation
project (If so, tell me, then I send the sources of the document!)
The contents of the document are:
-) The first chapter might be of less interest as it gives only an
overview on SQL.
-) The second chapter gives a description on much of PostgreSQL's
features (like user defined types etc. and how to use these features)
-) The third chapter starts with an overview of PostgreSQL's internal
structure with focus on the stages a query has to pass (i.e. parser,
planner/optimizer, executor). Then a detailed description of the
implementation of the Having clause and the Intersect/Except logic is
given.
Originally I worked on v6.3.2 but never found time enough to prepare
and post a patch. Now I applied the changes to v6.4 to get Intersect
and Except working with the new version. Chapter 3 of my documentation
deals with the changes against v6.3.2, so keep that in mind when
comparing the parts of the code printed there with the patched sources
of v6.4.
Here are some remarks on the patch. There are some things that have
still to be done but at the moment I don't have time to do them
myself. (I'm doing my military service at the moment) Sorry for that
:-(
-) I used a rewrite technique for the implementation of the Except/Intersect
logic which rewrites the query to a semantically equivalent query before
it is handed to the rewrite system (for views, rules etc.), planner,
executor etc.
-) In v6.3.2 the types of the attributes of two select statements
connected by the UNION keyword had to match 100%. In v6.4 the types
only need to be familiar (i.e. int and float can be mixed). Since this
feature did not exist when I worked on Intersect/Except it
does not work correctly for Except/Intersect queries WHEN USED IN
COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the
resulting table. This is because until now the types of the attributes of
the first select statement have been used for the resulting table.
When Intersects and/or Excepts are used in combination with Unions it
might happen, that the first select statement of the original query
appears at another position in the query which will be executed. The reason
for this is the technique used for the implementation of
Except/Intersect which does a query rewrite!)
NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT
queries!!!
-) I had to add the field intersect_clause to some data structures
but did not find time to implement printfuncs for the new field.
This does NOT break the debug modes but when an Except/Intersect
is used the query debug output will be the already rewritten query.
-) Massive changes to the grammar rules for SELECT and INSERT statements
have been necessary (see comments in gram.y and documentation for
deatails) in order to be able to use mixed queries like
(SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...;
-) When using UNION/EXCEPT/INTERSECT you will get:
NOTICE: equal: "Don't know if nodes of type xxx are equal".
I did not have time to add comparsion support for all the needed nodes,
but the default behaviour of the function equal met my requirements.
I did not dare to supress this message!
That's the reason why the regression test for union will fail: These
messages are also included in the union.out file!
-) Somebody of you changed the union_planner() function for v6.4
(I copied the targetlist to new_tlist and that was removed and
replaced by a cleanup of the original targetlist). These chnages
violated some having queries executed against views so I changed
it back again. I did not have time to examine the differences between the
two versions but now it works :-)
If you want to find out, try the file queries/view_having.sql on
both versions and compare the results . Two queries won't produce a
correct result with your version.
regards
Stefan
where you state a format and arguments. the old behavior required
each appendStringInfo to have to have a sprintf() before it if any
formatting was required.
Also shortened several instances where there were multiple appendStringInfo()
calls in a row, doing nothing more then adding one more word to the String,
instead of doing them all in one call.
patch is applied:
Rewrite rules on relation level work fine now.
Event qualifications on insert/update/delete rules work
fine now.
I added the new keyword OLD to reference the CURRENT
tuple. CURRENT will be removed in 6.5.
Update rules can reference NEW and OLD in the rule
qualification and the actions.
Insert/update/delete rules on views can be established to
let them behave like real tables.
For insert/update/delete rules multiple actions are
supported now. The actions can also be surrounded by
parantheses to make psql happy. Multiple actions are
required if update to a view requires updates to multiple
tables.
Regular users are permitted to create/drop rules on
tables they have RULE permissions for
(DefineQueryRewrite() is now able to get around the
access restrictions on pg_rewrite). This enables view
creation for regular users too. This required an extra
boolean parameter to pg_parse_and_plan() that tells to
set skipAcl on all rangetable entries of the resulting
queries. There is a new function
pg_exec_query_acl_override() that could be used by
backend utilities to use this facility.
All rule actions (not only views) inherit the permissions
of the event relations owner. Sample: User A creates
tables T1 and T2, creates rules that log
INSERT/UPDATE/DELETE on T1 in T2 (like in the regression
tests for rules I created) and grants ALL but RULE on T1
to user B. User B can now fully access T1 and the
logging happens in T2. But user B cannot access T2 at
all, only the rule actions can. And due to missing RULE
permissions on T1, user B cannot disable logging.
Rules on the attribute level are disabled (they don't
work properly and since regular users are now permitted
to create rules I decided to disable them).
Rules on select must have exactly one action that is a
select (so select rules must be a view definition).
UPDATE NEW/OLD rules are disabled (still broken, but
triggers can do it).
There are two new system views (pg_rule and pg_view) that
show the definition of the rules or views so the db admin
can see what the users do. They use two new functions
pg_get_ruledef() and pg_get_viewdef() that are builtins.
The functions pg_get_ruledef() and pg_get_viewdef() could
be used to implement rule and view support in pg_dump.
PostgreSQL is now the only database system I know, that
has rewrite rules on the query level. All others (where I
found a rule statement at all) use stored database
procedures or the like (triggers as we call them) for
active rules (as some call them).
Future of the rule system:
The now disabled parts of the rule system (attribute
level, multiple actions on select and update new stuff)
require a complete new rewrite handler from scratch. The
old one is too badly wired up.
After 6.4 I'll start to work on a new rewrite handler,
that fully supports the attribute level rules, multiple
actions on select and update new. This will be available
for 6.5 so we get full rewrite rule capabilities.
Jan
no longer returns buffer pointer, can be gotten from scan;
descriptor; bootstrap can create multi-key indexes;
pg_procname index now is multi-key index; oidint2, oidint4, oidname
are gone (must be removed from regression tests); use System Cache
rather than sequential scan in many places; heap_modifytuple no
longer takes buffer parameter; remove unused buffer parameter in
a few other functions; oid8 is not index-able; remove some use of
single-character variable names; cleanup Buffer variables usage
and scan descriptor looping; cleaned up allocation and freeing of
tuples; 18k lines of diff;
now. Here some tested features, (examples included in the patch):
1.1) Subselects in the having clause 1.2) Double nested subselects
1.3) Subselects used in the where clause and in the having clause
simultaneously 1.4) Union Selects using having 1.5) Indexes
on the base relations are used correctly 1.6) Unallowed Queries
are prevented (e.g. qualifications in the
having clause that belong to the where clause) 1.7) Insert
into as select
2) Queries using the having clause on view relations also work
but there are some restrictions:
2.1) Create View as Select ... Having ...; using base tables in
the select 2.1.1) The Query rewrite system:
2.1.2) Why are only simple queries allowed against a view from 2.1)
? 2.2) Select ... from testview1, testview2, ... having...; 3) Bug
in ExecMergeJoin ??
Regards Stefan
1. Removes the unnecessary "#define AbcRegProcedure 123"'s from
pg_proc.h.
2. Changes those #defines to use the names already defined in
fmgr.h.
3. Forces the make of fmgr.h in backend/Makefile instead of having
it
made as a dependency in access/common/Makefile *hack*hack*hack*
4. Rearranged the #includes to a less helter-skelter arrangement,
also
changing <file.h> to "file.h" to signify a non-system header.
5. Removed "pg_proc.h" from files where its only purpose was for
the
#defines removed in item #1.
6. Added "fmgr.h" to each file changed for completeness sake.
Turns out that #6 was not necessary for some files because fmgr.h
was being included in a roundabout way SIX levels deep by the first
include.
"access/genam.h"
->"access/relscan.h"
->"utils/rel.h"
->"access/strat.h"
->"access/skey.h"
->"fmgr.h"
So adding fmgr.h really didn't add anything to the compile, hopefully
just made it clearer to the programmer.
S Darren.
