upstream/postgresql
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2026-07-15 12:51:05 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Revert non-text output formats for pg_dumpall

Browse source 
This reverts the non-text (custom/directory/tar) output format support
for pg_dumpall added by 763aaa06f0 and its feature-specific follow-ups,
in line with Noah Misch's post-commit review which recommends reverting
and finishing the work through the commitfest.

Scope is deliberately minimal: only the feature itself is removed.
Independent improvements that merely touched the same files, or that were
committed alongside the feature but do not depend on its design, are
preserved.

Reverted (the feature):

  763aaa06f0  Add non-text output formats to pg_dumpall
  d6d9b96b40  Clean up nodes that are no longer of use in 007_pgdumpall.pl
  01c729e0c7  Fix casting away const-ness in pg_restore.c
  c7572cd48d  Improve writing map.dat preamble
  3c19983cc0  pg_restore: add --no-globals option to skip globals
  abff4492d0  Fix options listing of pg_restore --no-globals
  bb53b8d359  Fix small memory leak in get_dbname_oid_list_from_mfile()
  a793677e57  pg_restore: Remove dead code in restore_all_databases()
  a198c26ded  pg_dumpall: simplify coding of dropDBs()
  ec80215c03  pg_restore: Remove unnecessary strlen() calls in options parsing

Preserved (independent of the feature):

  b2898baaf7  the check_mut_excl_opts() helper in src/fe_utils/option_utils.c
               and its use in pg_dump
  7c8280eeb5  pg_dump's conflicting-option refactor (and tests 002/005)
  be0d0b457c  pg_dumpall's rejection of --clean together with --data-only
               (re-expressed directly, since pg_dumpall.c is otherwise
               returned to its pre-feature state)
  74b4438a70  the dangling-grantor-OID GRANT fix (back-patched through 16)
  273d26b75e, d4cb9c3776  independent pg_restore.sgml clarifications

Because the feature restructured pg_dumpall.c and pg_restore.c (pg_restore's
main() was split into restore_one_database() plus a dispatcher) and
interleaved its option checks with the conflicting-option refactor in the
same regions, the cosmetic check_mut_excl_opts() reflow of those two files'
option blocks is inseparable from the feature and comes out with it; the
behavior is unchanged.  The reusable helper and pg_dump's use of it are
unaffected.

Discussion: https://postgr.es/m/20260607000218.96.noahmisch@microsoft.com
This commit is contained in:
Andrew Dunstan 2026-06-15 15:58:04 -04:00
parent ff8bec8c46
commit 7ca548f23a
16 changed files with 273 additions and 2512 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

135
doc/src/sgml/ref/pg_dumpall.sgml
View file

@ -16,10 +16,7 @@ PostgreSQL documentation
<refnamediv>
<refname>pg_dumpall</refname>
<refpurpose>
export a <productname>PostgreSQL</productname> database cluster as an SQL script or to other formats
</refpurpose>
<refpurpose>extract a <productname>PostgreSQL</productname> database cluster into a script file</refpurpose>
</refnamediv>
<refsynopsisdiv>
@ -36,33 +33,14 @@ PostgreSQL documentation
<para>
<application>pg_dumpall</application> is a utility for writing out
(<quote>dumping</quote>) all <productname>PostgreSQL</productname> databases
of a cluster into an SQL script file or an archive. It does this by
of a cluster into one script file. The script file contains
<acronym>SQL</acronym> commands that can be used as input to <xref
linkend="app-psql"/> to restore the databases. It does this by
calling <xref linkend="app-pgdump"/> for each database in the cluster.
The output contains <acronym>SQL</acronym> commands that can be used
as input to <xref linkend="app-psql"/> or <xref linkend="app-pgrestore"/>
to restore the databases.
<application>pg_dumpall</application> also dumps global objects
that are common to all databases, namely database roles, tablespaces,
and privilege grants for configuration parameters.
(<application>pg_dump</application> does not save these objects.)
The only parts of a database cluster's state that
are <emphasis>not</emphasis> included in the default output
of <application>pg_dumpall</application> are the configuration files
and any database parameter setting changes made with
<xref linkend="sql-altersystem"/>.
</para>
<para>
If the output format is a
plain text SQL script, it will be written to the standard output. Use the
<option>-f</option>/<option>--file</option> option or shell operators to
redirect it into a file.
</para>
<para>
If another output format is selected, the archive will be placed in a
directory named using the <option>-f</option>/<option>--file</option>
option, which is required in this case.
</para>
<para>
@ -73,6 +51,12 @@ PostgreSQL documentation
allowed to add roles and create databases.
</para>
<para>
The SQL script will be written to the standard output. Use the
<option>-f</option>/<option>--file</option> option or shell operators to
redirect it into a file.
</para>
<para>
<application>pg_dumpall</application> needs to connect several
times to the <productname>PostgreSQL</productname> server (once per
@ -147,93 +131,16 @@ PostgreSQL documentation
<para>
Send output to the specified file. If this is omitted, the
standard output is used.
This option can only be omitted when <option>--format</option> is plain.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-F <replaceable class="parameter">format</replaceable></option></term>
<term><option>--format=<replaceable class="parameter">format</replaceable></option></term>
<listitem>
<para>
Specify the format of dump files. In plain format, all the dump data is
sent in a single text stream. This is the default.
In all other modes, <application>pg_dumpall</application> first creates two files,
<filename>toc.glo</filename> and <filename>map.dat</filename>, in the directory
specified by <option>--file</option>.
The first file contains global data (roles and tablespaces) in custom format. The second
contains a mapping between database OIDs and names. These files are used by
<application>pg_restore</application>. Data for individual databases is placed in
the <filename>databases</filename> subdirectory, named using the database's OID.
<variablelist>
<varlistentry>
<term><literal>d</literal></term>
<term><literal>directory</literal></term>
<listitem>
<para>
Output directory-format archives for each database,
suitable for input into pg_restore. The directory
will have database <type>oid</type> as its name.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>p</literal></term>
<term><literal>plain</literal></term>
<listitem>
<para>
Output a plain-text SQL script file (the default).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>c</literal></term>
<term><literal>custom</literal></term>
<listitem>
<para>
Output a custom-format archive for each database,
suitable for input into pg_restore. The archive
will be named <filename>dboid.dmp</filename> where <type>dboid</type> is the
<type>oid</type> of the database.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>t</literal></term>
<term><literal>tar</literal></term>
<listitem>
<para>
Output a tar-format archive for each database,
suitable for input into pg_restore. The archive
will be named <filename>dboid.tar</filename> where <type>dboid</type> is the
<type>oid</type> of the database.
</para>
</listitem>
</varlistentry>
</variablelist>
See <xref linkend="app-pgdump"/> for details on how the
various non-plain-text archive formats work.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-g</option></term>
<term><option>--globals-only</option></term>
<listitem>
<para>
Dump only global objects (roles and tablespaces), no databases.
Note: <option>--globals-only</option> cannot be used with
<option>--clean</option> with non-text dump format.
</para>
</listitem>
</varlistentry>
@ -1029,21 +936,13 @@ exclude database <replaceable class="parameter">PATTERN</replaceable>
<refsect1 id="app-pg-dumpall-ex">
<title>Examples</title>
<para>
To dump all databases in plain text format (the default):
To dump all databases:
<screen>
<prompt>$</prompt> <userinput>pg_dumpall &gt; db.out</userinput>
</screen>
</para>
<para>
To dump all databases using other formats:
<screen>
<prompt>$</prompt> <userinput>pg_dumpall --format=directory -f db.out</userinput>
<prompt>$</prompt> <userinput>pg_dumpall --format=custom -f db.out</userinput>
<prompt>$</prompt> <userinput>pg_dumpall --format=tar -f db.out</userinput>
</screen>
</para>
<para>
To restore database(s) from this file, you can use:
<screen>
@ -1057,16 +956,6 @@ exclude database <replaceable class="parameter">PATTERN</replaceable>
the script will attempt to drop other databases immediately, and that
will fail for the database you are connected to.
</para>
<para>
If the dump was taken in a non-plain-text format, use
<application>pg_restore</application> to restore the databases:
<screen>
<prompt>$</prompt> <userinput>pg_restore db.out -d postgres -C</userinput>
</screen>
This will restore all databases. To restore only some databases, use
the <option>--exclude-database</option> option to skip those not wanted.
</para>
</refsect1>
<refsect1>

129
doc/src/sgml/ref/pg_restore.sgml
View file

@ -18,9 +18,8 @@ PostgreSQL documentation
<refname>pg_restore</refname>
<refpurpose>
restore <productname>PostgreSQL</productname> databases from archives
created by <application>pg_dump</application> or
<application>pg_dumpall</application>
restore a <productname>PostgreSQL</productname> database from an
archive file created by <application>pg_dump</application>
</refpurpose>
</refnamediv>
@ -39,14 +38,13 @@ PostgreSQL documentation
<para>
<application>pg_restore</application> is a utility for restoring a
<productname>PostgreSQL</productname> database or cluster from an archive
created by <xref linkend="app-pgdump"/> or
<xref linkend="app-pg-dumpall"/> in one of the non-plain-text
<productname>PostgreSQL</productname> database from an archive
created by <xref linkend="app-pgdump"/> in one of the non-plain-text
formats. It will issue the commands necessary to reconstruct the
database or cluster to the state it was in at the time it was saved. The
archives also allow <application>pg_restore</application> to
database to the state it was in at the time it was saved. The
archive files also allow <application>pg_restore</application> to
be selective about what is restored, or even to reorder the items
prior to being restored. The archive formats are designed to be
prior to being restored. The archive files are designed to be
portable across architectures.
</para>
@ -54,34 +52,14 @@ PostgreSQL documentation
<application>pg_restore</application> can operate in two modes.
If a database name is specified, <application>pg_restore</application>
connects to that database and restores archive contents directly into
the database.
When restoring from a dump made by <application>pg_dumpall</application>,
each database will be created and then the restoration will be run in that
database.
Otherwise, when a database name is not specified, a script containing the SQL
commands necessary to rebuild the database or cluster is created and written
the database. Otherwise, a script containing the SQL
commands necessary to rebuild the database is created and written
to a file or standard output. This script output is equivalent to
the plain text output format of <application>pg_dump</application> or
<application>pg_dumpall</application>.
the plain text output format of <application>pg_dump</application>.
Some of the options controlling the output are therefore analogous to
<application>pg_dump</application> options.
</para>
<para>
A non-plain-text archive made using <application>pg_dumpall</application>
is a directory containing a <filename>toc.glo</filename> file with global
objects (roles and tablespaces), a <filename>map.dat</filename> file
listing the databases, and a subdirectory for each database containing
its archive. When restoring such an archive,
<application>pg_restore</application> first restores global objects from
<filename>toc.glo</filename>, then processes each database listed in
<filename>map.dat</filename>. Lines in <filename>map.dat</filename> can
be commented out with <literal>#</literal> to skip restoring specific
databases.
</para>
<para>
Obviously, <application>pg_restore</application> cannot restore information
that is not present in the archive file. For instance, if the
@ -152,12 +130,6 @@ PostgreSQL documentation
ignorable error messages will be reported,
unless <option>--if-exists</option> is also specified.
</para>
<para>
When restoring a <application>pg_dumpall</application> archive,
<option>--if-exists</option> is implied by <option>--clean</option>,
since global objects such as roles and tablespaces may not exist
in the target cluster.
</para>
</listitem>
</varlistentry>
@ -180,8 +152,6 @@ PostgreSQL documentation
commands that mention this database.
Access privileges for the database itself are also restored,
unless <option>--no-acl</option> is specified.
<option>--create</option> is required when restoring multiple databases
from a non-plain-text archive made using <application>pg_dumpall</application>.
</para>
<para>
@ -282,29 +252,6 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>-g</option></term>
<term><option>--globals-only</option></term>
<listitem>
<para>
Restore only global objects (roles and tablespaces), no databases.
</para>
<para>
This option is only relevant when restoring from a non-plain-text archive made using <application>pg_dumpall</application>.
Note: <option>--globals-only</option> cannot be used with
<option>--data-only</option>,
<option>--schema-only</option>,
<option>--statistics-only</option>,
<option>--statistics</option>,
<option>--no-globals</option>,
<option>--exit-on-error</option>,
<option>--single-transaction</option>,
<option>--clean</option>, or
<option>--transaction-size</option>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>-I <replaceable class="parameter">index</replaceable></option></term>
<term><option>--index=<replaceable class="parameter">index</replaceable></option></term>
@ -641,28 +588,6 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>--exclude-database=<replaceable class="parameter">pattern</replaceable></option></term>
<listitem>
<para>
Do not restore databases whose name matches
<replaceable class="parameter">pattern</replaceable>.
Multiple patterns can be excluded by writing multiple
<option>--exclude-database</option> switches. The
<replaceable class="parameter">pattern</replaceable> parameter is
interpreted as a pattern according to the same rules used by
<application>psql</application>'s <literal>\d</literal>
commands (see <xref linkend="app-psql-patterns"/>),
so multiple databases can also be excluded by writing wildcard
characters in the pattern. When using wildcards, be careful to
quote the pattern if needed to prevent shell wildcard expansion.
</para>
<para>
This option is only relevant when restoring from a non-plain-text archive made using <application>pg_dumpall</application>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--filter=<replaceable class="parameter">filename</replaceable></option></term>
<listitem>
@ -751,9 +676,7 @@ PostgreSQL documentation
in <option>--clean</option> mode. This suppresses <quote>does not
exist</quote> errors that might otherwise be reported. This
option is not valid unless <option>--clean</option> is also
specified. This option is implied when restoring a
<application>pg_dumpall</application> archive with
<option>--clean</option>.
specified.
</para>
</listitem>
</varlistentry>
@ -800,21 +723,6 @@ PostgreSQL documentation
</listitem>
</varlistentry>
<varlistentry>
<term><option>--no-globals</option></term>
<listitem>
<para>
Do not restore global objects (roles and tablespaces). When
<option>-C</option>/<option>--create</option> is not specified,
databases that do not already exist on the target server are skipped.
</para>
<para>
This option is only relevant when restoring from a non-plain-text
archive made using <application>pg_dumpall</application>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><option>--no-policies</option></term>
<listitem>
@ -1240,21 +1148,6 @@ CREATE DATABASE foo WITH TEMPLATE template0;
</para>
</listitem>
<listitem>
<para>
The following options cannot be used when restoring from a non-plain-text
archive made using <application>pg_dumpall</application>:
<option>-a/--data-only</option>,
<option>-l/--list</option>,
<option>-L/--use-list</option>,
<option>--no-schema</option>,
<option>--statistics-only</option>, and
<option>--strict-names</option>.
Also, if the <option>--section</option> option is used, it must
include <option>pre-data</option>.
</para>
</listitem>
</itemizedlist>
</para>

19
doc/src/sgml/release-19.sgml
View file

@ -2688,25 +2688,6 @@ Report nanoseconds instead of microseconds. In addition to histogram output, ou
linkend="app-pgrestore"><application>pg_restore</application></link></title>
<itemizedlist>
<!--
Author: Andrew Dunstan <andrew@dunslane.net>
2026-02-26 [763aaa06f] Add non-text output formats to pg_dumpall
Author: Andrew Dunstan <andrew@dunslane.net>
2026-03-04 [3c19983cc] pg_restore: add - -no-globals option to skip globals
-->
<listitem>
<para>
Allow <link linkend="app-pg-dumpall"><application>pg_dumpall</application></link> to produce output in non-text formats (Mahendra Singh Thalor, Andrew Dunstan)
<ulink url="&commit_baseurl;763aaa06f">&sect;</ulink>
<ulink url="&commit_baseurl;3c19983cc">&sect;</ulink>
</para>
<para>
The new output formats are <literal>custom</literal>, <literal>directory</literal>, or <literal>tar</literal>.
</para>
</listitem>
<!--
Author: Michael Paquier <michael@paquier.xyz>
2026-01-27 [c32fb29e9] Include extended statistics data in pg_dump

1
src/bin/pg_dump/meson.build
View file

@ -103,7 +103,6 @@ tests += {
't/004_pg_dump_parallel.pl',
't/005_pg_dump_filterfile.pl',
't/006_pg_dump_compress.pl',
't/007_pg_dumpall.pl',
't/010_dump_connstr.pl',
],
},

14
src/bin/pg_dump/parallel.c
View file

@ -333,20 +333,6 @@ on_exit_close_archive(Archive *AHX)
on_exit_nicely(archive_close_connection, &shutdown_info);
}
/*
* Update the archive handle in the on_exit callback registered by
* on_exit_close_archive(). When pg_restore processes a pg_dumpall archive
* containing multiple databases, each database is restored from a separate
* archive. After closing one archive and opening the next, we update the
* shutdown_info to reference the new archive handle so the cleanup callback
* will close the correct archive on exit.
*/
void
replace_on_exit_close_archive(Archive *AHX)
{
shutdown_info.AHX = AHX;
}
/*
* on_exit_nicely handler for shutting down database connections and
* worker processes cleanly.

2
src/bin/pg_dump/pg_backup.h
View file

@ -313,7 +313,7 @@ extern void SetArchiveOptions(Archive *AH, DumpOptions *dopt, RestoreOptions *ro
extern void ProcessArchiveRestoreOptions(Archive *AHX);
extern void RestoreArchive(Archive *AHX, bool append_data);
extern void RestoreArchive(Archive *AHX);
/* Open an existing archive */
extern Archive *OpenArchive(const char *FileSpec, const ArchiveFormat fmt);

67
src/bin/pg_dump/pg_backup_archiver.c
View file

@ -87,7 +87,7 @@ static int RestoringToDB(ArchiveHandle *AH);
static void dump_lo_buf(ArchiveHandle *AH);
static void dumpTimestamp(ArchiveHandle *AH, const char *msg, time_t tim);
static void SetOutput(ArchiveHandle *AH, const char *filename,
const pg_compress_specification compression_spec, bool append_data);
const pg_compress_specification compression_spec);
static CompressFileHandle *SaveOutput(ArchiveHandle *AH);
static void RestoreOutput(ArchiveHandle *AH, CompressFileHandle *savedOutput);
@ -340,14 +340,9 @@ ProcessArchiveRestoreOptions(Archive *AHX)
StrictNamesCheck(ropt);
}
/*
* RestoreArchive
*
* If append_data is set, then append data into file as we are restoring dump
* of multiple databases which was taken by pg_dumpall.
*/
/* Public */
void
RestoreArchive(Archive *AHX, bool append_data)
RestoreArchive(Archive *AHX)
{
ArchiveHandle *AH = (ArchiveHandle *) AHX;
RestoreOptions *ropt = AH->public.ropt;
@ -464,7 +459,7 @@ RestoreArchive(Archive *AHX, bool append_data)
*/
sav = SaveOutput(AH);
if (ropt->filename || ropt->compression_spec.algorithm != PG_COMPRESSION_NONE)
SetOutput(AH, ropt->filename, ropt->compression_spec, append_data);
SetOutput(AH, ropt->filename, ropt->compression_spec);
ahprintf(AH, "--\n-- PostgreSQL database dump\n--\n\n");
@ -767,19 +762,6 @@ RestoreArchive(Archive *AHX, bool append_data)
if ((te->reqs & (REQ_SCHEMA | REQ_DATA | REQ_STATS)) == 0)
continue; /* ignore if not to be dumped at all */
/* Skip if no-tablespace is given. */
if (ropt->noTablespace && te && te->desc &&
(strcmp(te->desc, "TABLESPACE") == 0))
continue;
/*
* Skip DROP DATABASE/ROLES/TABLESPACE if we didn't specify
* --clean
*/
if (!ropt->dropSchema && te && te->desc &&
strcmp(te->desc, "DROP_GLOBAL") == 0)
continue;
switch (_tocEntryRestorePass(te))
{
case RESTORE_PASS_MAIN:
@ -1335,7 +1317,7 @@ PrintTOCSummary(Archive *AHX)
sav = SaveOutput(AH);
if (ropt->filename)
SetOutput(AH, ropt->filename, out_compression_spec, false);
SetOutput(AH, ropt->filename, out_compression_spec);
if (strftime(stamp_str, sizeof(stamp_str), PGDUMP_STRFTIME_FMT,
localtime(&AH->createDate)) == 0)
@ -1710,15 +1692,11 @@ archprintf(Archive *AH, const char *fmt, ...)
/*******************************
* Stuff below here should be 'private' to the archiver routines
*
* If append_data is set, then append data into file as we are restoring dump
* of multiple databases which was taken by pg_dumpall.
*******************************/
static void
SetOutput(ArchiveHandle *AH, const char *filename,
const pg_compress_specification compression_spec,
bool append_data)
const pg_compress_specification compression_spec)
{
CompressFileHandle *CFH;
const char *mode;
@ -1738,7 +1716,7 @@ SetOutput(ArchiveHandle *AH, const char *filename,
else
fn = fileno(stdout);
if (append_data || AH->mode == archModeAppend)
if (AH->mode == archModeAppend)
mode = PG_BINARY_A;
else
mode = PG_BINARY_W;
@ -2414,7 +2392,7 @@ _allocAH(const char *FileSpec, const ArchiveFormat fmt,
/* initialize for backwards compatible string processing */
AH->public.encoding = 0; /* PG_SQL_ASCII */
AH->public.std_strings = true;
AH->public.std_strings = false;
/* sql error handling */
AH->public.exit_on_error = true;
@ -3050,16 +3028,6 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
return 0;
}
/*
* Global object TOC entries (e.g., ROLEs or TABLESPACEs) must not be
* ignored.
*/
if (strcmp(te->desc, "ROLE") == 0 ||
strcmp(te->desc, "ROLE PROPERTIES") == 0 ||
strcmp(te->desc, "TABLESPACE") == 0 ||
strcmp(te->desc, "DROP_GLOBAL") == 0)
return REQ_SCHEMA;
/*
* Process exclusions that affect certain classes of TOC entries.
*/
@ -3095,14 +3063,6 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
if (ropt->no_subscriptions &&
strncmp(te->tag, "SUBSCRIPTION", strlen("SUBSCRIPTION")) == 0)
return 0;
/*
* Comments on global objects (ROLEs or TABLESPACEs) should not be
* skipped, since global objects themselves are never skipped.
*/
if (strncmp(te->tag, "ROLE", strlen("ROLE")) == 0 ||
strncmp(te->tag, "TABLESPACE", strlen("TABLESPACE")) == 0)
return REQ_SCHEMA;
}
/*
@ -3132,14 +3092,6 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
if (ropt->no_subscriptions &&
strncmp(te->tag, "SUBSCRIPTION", strlen("SUBSCRIPTION")) == 0)
return 0;
/*
* Security labels on global objects (ROLEs or TABLESPACEs) should not
* be skipped, since global objects themselves are never skipped.
*/
if (strncmp(te->tag, "ROLE", strlen("ROLE")) == 0 ||
strncmp(te->tag, "TABLESPACE", strlen("TABLESPACE")) == 0)
return REQ_SCHEMA;
}
/* If it's a subscription, maybe ignore it */
@ -3952,9 +3904,6 @@ _getObjectDescription(PQExpBuffer buf, const TocEntry *te)
else if (strcmp(type, "CAST") == 0 ||
strcmp(type, "CHECK CONSTRAINT") == 0 ||
strcmp(type, "CONSTRAINT") == 0 ||
strcmp(type, "DROP_GLOBAL") == 0 ||
strcmp(type, "ROLE PROPERTIES") == 0 ||
strcmp(type, "ROLE") == 0 ||
strcmp(type, "DATABASE PROPERTIES") == 0 ||
strcmp(type, "DEFAULT") == 0 ||
strcmp(type, "FK CONSTRAINT") == 0 ||

1
src/bin/pg_dump/pg_backup_archiver.h
View file

@ -394,7 +394,6 @@ struct _tocEntry
extern int parallel_restore(ArchiveHandle *AH, TocEntry *te);
extern void on_exit_close_archive(Archive *AHX);
extern void replace_on_exit_close_archive(Archive *AHX);
extern void warn_or_exit_horribly(ArchiveHandle *AH, const char *fmt, ...) pg_attribute_printf(2, 3);

2
src/bin/pg_dump/pg_backup_tar.c
View file

@ -826,7 +826,7 @@ _CloseArchive(ArchiveHandle *AH)
savVerbose = AH->public.verbose;
AH->public.verbose = 0;
RestoreArchive((Archive *) AH, false);
RestoreArchive((Archive *) AH);
SetArchiveOptions((Archive *) AH, savDopt, savRopt);

2
src/bin/pg_dump/pg_dump.c
View file

@ -1279,7 +1279,7 @@ main(int argc, char **argv)
* right now.
*/
if (plainText)
RestoreArchive(fout, false);
RestoreArchive(fout);
CloseArchive(fout);

859
src/bin/pg_dump/pg_dumpall.c
View file

File diff suppressed because it is too large Load diff

776
src/bin/pg_dump/pg_restore.c
View file

@ -2,7 +2,7 @@
*
* pg_restore.c
* pg_restore is an utility extracting postgres database definitions
* from a backup archive created by pg_dump/pg_dumpall using the archiver
* from a backup archive created by pg_dump using the archiver
* interface.
*
* pg_restore will read the backup archive and
@ -41,16 +41,12 @@
#include "postgres_fe.h"
#include <ctype.h>
#include <sys/stat.h>
#ifdef HAVE_TERMIOS_H
#include <termios.h>
#endif
#include "common/string.h"
#include "connectdb.h"
#include "dumputils.h"
#include "fe_utils/option_utils.h"
#include "fe_utils/string_utils.h"
#include "filter.h"
#include "getopt_long.h"
#include "parallel.h"
@ -58,41 +54,18 @@
static void usage(const char *progname);
static void read_restore_filters(const char *filename, RestoreOptions *opts);
static bool file_exists_in_directory(const char *dir, const char *filename);
static int restore_one_database(const char *inputFileSpec, RestoreOptions *opts,
int numWorkers, bool append_data);
static int restore_global_objects(const char *inputFileSpec, RestoreOptions *opts);
static int restore_all_databases(const char *inputFileSpec,
SimpleStringList db_exclude_patterns, RestoreOptions *opts, int numWorkers);
static int get_dbnames_list_to_restore(PGconn *conn,
SimplePtrList *dbname_oid_list,
SimpleStringList db_exclude_patterns);
static int get_dbname_oid_list_from_mfile(const char *dumpdirpath,
SimplePtrList *dbname_oid_list);
/*
* Stores a database OID and the corresponding name.
*/
typedef struct DbOidName
{
Oid oid;
char str[FLEXIBLE_ARRAY_MEMBER]; /* null-terminated string here */
} DbOidName;
int
main(int argc, char **argv)
{
RestoreOptions *opts;
int c;
int exit_code;
int numWorkers = 1;
Archive *AH;
char *inputFileSpec;
bool data_only = false;
bool schema_only = false;
int n_errors = 0;
bool globals_only = false;
SimpleStringList db_exclude_patterns = {NULL, NULL};
static int disable_triggers = 0;
static int enable_row_security = 0;
static int if_exists = 0;
@ -107,7 +80,6 @@ main(int argc, char **argv)
static int no_schema = 0;
static int no_security_labels = 0;
static int no_statistics = 0;
static int no_globals = 0;
static int no_subscriptions = 0;
static int strict_names = 0;
static int statistics_only = 0;
@ -117,7 +89,6 @@ main(int argc, char **argv)
{"clean", 0, NULL, 'c'},
{"create", 0, NULL, 'C'},
{"data-only", 0, NULL, 'a'},
{"globals-only", 0, NULL, 'g'},
{"dbname", 1, NULL, 'd'},
{"exit-on-error", 0, NULL, 'e'},
{"exclude-schema", 1, NULL, 'N'},
@ -165,14 +136,12 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-schema", no_argument, &no_schema, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-globals", no_argument, &no_globals, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
{"no-statistics", no_argument, &no_statistics, 1},
{"statistics", no_argument, &with_statistics, 1},
{"statistics-only", no_argument, &statistics_only, 1},
{"filter", required_argument, NULL, 4},
{"restrict-key", required_argument, NULL, 6},
{"exclude-database", required_argument, NULL, 7},
{NULL, 0, NULL, 0}
};
@ -201,7 +170,7 @@ main(int argc, char **argv)
}
}
while ((c = getopt_long(argc, argv, "acCd:ef:F:gh:I:j:lL:n:N:Op:P:RsS:t:T:U:vwWx1",
while ((c = getopt_long(argc, argv, "acCd:ef:F:h:I:j:lL:n:N:Op:P:RsS:t:T:U:vwWx1",
cmdopts, NULL)) != -1)
{
switch (c)
@ -225,15 +194,14 @@ main(int argc, char **argv)
opts->filename = pg_strdup(optarg);
break;
case 'F':
opts->formatName = pg_strdup(optarg);
break;
case 'g':
/* restore only global sql commands. */
globals_only = true;
if (strlen(optarg) != 0)
opts->formatName = pg_strdup(optarg);
break;
case 'h':
opts->cparams.pghost = pg_strdup(optarg);
if (strlen(optarg) != 0)
opts->cparams.pghost = pg_strdup(optarg);
break;
case 'j': /* number of restore jobs */
if (!option_parse_int(optarg, "-j/--jobs", 1,
PG_MAX_JOBS,
@ -262,7 +230,8 @@ main(int argc, char **argv)
break;
case 'p':
opts->cparams.pgport = pg_strdup(optarg);
if (strlen(optarg) != 0)
opts->cparams.pgport = pg_strdup(optarg);
break;
case 'R':
/* no-op, still accepted for backwards compatibility */
@ -352,10 +321,6 @@ main(int argc, char **argv)
opts->restrict_key = pg_strdup(optarg);
break;
case 7: /* database patterns to skip */
simple_string_list_append(&db_exclude_patterns, optarg);
break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@ -382,20 +347,23 @@ main(int argc, char **argv)
if (!opts->cparams.dbname && !opts->filename && !opts->tocSummary)
pg_fatal("one of -d/--dbname and -f/--file must be specified");
/* --exclude-database and --globals-only are incompatible */
check_mut_excl_opts(db_exclude_patterns.head, "--exclude-database",
globals_only, "-g/--globals-only");
/* Should get at most one of -d and -f, else user is confused */
check_mut_excl_opts(opts->cparams.dbname, "-d/--dbname",
opts->filename, "-f/--file");
/* --dbname and --restrict-key are incompatible */
check_mut_excl_opts(opts->cparams.dbname, "-d/--dbname",
opts->restrict_key, "--restrict-key");
if (opts->cparams.dbname)
{
if (opts->filename)
{
pg_log_error("options %s and %s cannot be used together",
"-d/--dbname", "-f/--file");
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
exit_nicely(1);
}
if (opts->restrict_key)
pg_fatal("options %s and %s cannot be used together",
"-d/--dbname", "--restrict-key");
opts->useDB = 1;
}
else
{
/*
@ -409,54 +377,56 @@ main(int argc, char **argv)
pg_fatal("invalid restrict key");
}
/* *-only options are incompatible with each other */
check_mut_excl_opts(data_only, "-a/--data-only",
globals_only, "-g/--globals-only",
schema_only, "-s/--schema-only",
statistics_only, "--statistics-only");
/* reject conflicting "-only" options */
if (data_only && schema_only)
pg_fatal("options %s and %s cannot be used together",
"-s/--schema-only", "-a/--data-only");
if (schema_only && statistics_only)
pg_fatal("options %s and %s cannot be used together",
"-s/--schema-only", "--statistics-only");
if (data_only && statistics_only)
pg_fatal("options %s and %s cannot be used together",
"-a/--data-only", "--statistics-only");
/* --no-* and *-only for same thing are incompatible */
check_mut_excl_opts(data_only, "-a/--data-only",
no_data, "--no-data");
check_mut_excl_opts(globals_only, "-g/--globals-only",
no_globals, "--no-globals");
check_mut_excl_opts(schema_only, "-s/--schema-only",
no_schema, "--no-schema");
check_mut_excl_opts(statistics_only, "--statistics-only",
no_statistics, "--no-statistics");
/* reject conflicting "-only" and "no-" options */
if (data_only && no_data)
pg_fatal("options %s and %s cannot be used together",
"-a/--data-only", "--no-data");
if (schema_only && no_schema)
pg_fatal("options %s and %s cannot be used together",
"-s/--schema-only", "--no-schema");
if (statistics_only && no_statistics)
pg_fatal("options %s and %s cannot be used together",
"--statistics-only", "--no-statistics");
/* --statistics and --no-statistics are incompatible */
check_mut_excl_opts(with_statistics, "--statistics",
no_statistics, "--no-statistics");
/* reject conflicting "no-" options */
if (with_statistics && no_statistics)
pg_fatal("options %s and %s cannot be used together",
"--statistics", "--no-statistics");
/* --statistics is incompatible with *-only (except --statistics-only) */
check_mut_excl_opts(with_statistics, "--statistics",
data_only, "-a/--data-only",
globals_only, "-g/--globals-only",
schema_only, "-s/--schema-only");
/* reject conflicting "only-" options */
if (data_only && with_statistics)
pg_fatal("options %s and %s cannot be used together",
"-a/--data-only", "--statistics");
if (schema_only && with_statistics)
pg_fatal("options %s and %s cannot be used together",
"-s/--schema-only", "--statistics");
/* --clean and --data-only are incompatible */
check_mut_excl_opts(opts->dropSchema, "-c/--clean",
data_only, "-a/--data-only");
if (data_only && opts->dropSchema)
pg_fatal("options %s and %s cannot be used together",
"-c/--clean", "-a/--data-only");
/*
* --globals-only, --single-transaction, and --transaction-size are
* incompatible.
*/
check_mut_excl_opts(globals_only, "-g/--globals-only",
opts->single_txn, "-1/--single-transaction",
opts->txn_size, "--transaction-size");
/* --exit-on-error and --globals-only are incompatible */
check_mut_excl_opts(opts->exit_on_error, "--exit-on-error",
globals_only, "-g/--globals-only");
if (opts->single_txn && opts->txn_size > 0)
pg_fatal("options %s and %s cannot be used together",
"-1/--single-transaction", "--transaction-size");
/*
* -C is not compatible with -1, because we can't create a database inside
* a transaction block.
*/
check_mut_excl_opts(opts->createDB, "-C/--create",
opts->single_txn, "-1/--single-transaction");
if (opts->createDB && opts->single_txn)
pg_fatal("options %s and %s cannot be used together",
"-C/--create", "-1/--single-transaction");
/* Can't do single-txn mode with multiple connections */
if (opts->single_txn && numWorkers > 1)
@ -515,187 +485,6 @@ main(int argc, char **argv)
opts->formatName);
}
/*
* If toc.glo file is present, then restore all the databases from
* map.dat, but skip restoring those matching --exclude-database patterns.
*/
if (inputFileSpec != NULL &&
(file_exists_in_directory(inputFileSpec, "toc.glo")))
{
char global_path[MAXPGPATH];
RestoreOptions *tmpopts = pg_malloc0_object(RestoreOptions);
opts->format = archUnknown;
memcpy(tmpopts, opts, sizeof(RestoreOptions));
/*
* Can only use --list or --use-list options with a single database
* dump.
*/
if (opts->tocSummary)
pg_fatal("option %s cannot be used when restoring an archive created by pg_dumpall",
"-l/--list");
if (opts->tocFile)
pg_fatal("option %s cannot be used when restoring an archive created by pg_dumpall",
"-L/--use-list");
if (opts->strict_names)
pg_fatal("option %s cannot be used when restoring an archive created by pg_dumpall",
"--strict-names");
if (globals_only && opts->dropSchema)
pg_fatal("options %s and %s cannot be used together when restoring an archive created by pg_dumpall",
"--clean", "-g/--globals-only");
/*
* For pg_dumpall archives, --clean implies --if-exists since global
* objects may not exist in the target cluster.
*/
if (opts->dropSchema && !opts->if_exists)
{
opts->if_exists = 1;
pg_log_info("--if-exists is implied by --clean for pg_dumpall archives");
}
if (no_schema)
pg_fatal("option %s cannot be used when restoring an archive created by pg_dumpall",
"--no-schema");
if (data_only)
pg_fatal("option %s cannot be used when restoring an archive created by pg_dumpall",
"-a/--data-only");
if (statistics_only)
pg_fatal("option %s cannot be used when restoring an archive created by pg_dumpall",
"--statistics-only");
if (!(opts->dumpSections & DUMP_PRE_DATA))
pg_fatal("option %s cannot exclude %s when restoring a pg_dumpall archive",
"--section", "--pre-data");
/*
* To restore from a pg_dumpall archive, -C (create database) option
* must be specified unless we are only restoring globals or we are
* skipping globals.
*/
if (!no_globals && !globals_only && opts->createDB != 1)
{
pg_log_error("option %s must be specified when restoring an archive created by pg_dumpall",
"-C/--create");
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
pg_log_error_hint("Individual databases can be restored using their specific archives.");
exit_nicely(1);
}
/*
* Restore global objects, even if --exclude-database results in zero
* databases to process. If 'globals-only' is set, exit immediately.
*/
snprintf(global_path, MAXPGPATH, "%s/toc.glo", inputFileSpec);
if (!no_globals)
n_errors = restore_global_objects(global_path, tmpopts);
else
pg_log_info("skipping restore of global objects because %s was specified",
"--no-globals");
if (globals_only)
pg_log_info("database restoring skipped because option %s was specified",
"-g/--globals-only");
else
{
/* Now restore all the databases from map.dat */
n_errors = n_errors + restore_all_databases(inputFileSpec, db_exclude_patterns,
opts, numWorkers);
}
/* Free db pattern list. */
simple_string_list_destroy(&db_exclude_patterns);
}
else
{
if (db_exclude_patterns.head != NULL)
{
simple_string_list_destroy(&db_exclude_patterns);
pg_fatal("option %s can be used only when restoring an archive created by pg_dumpall",
"--exclude-database");
}
if (globals_only)
pg_fatal("option %s can be used only when restoring an archive created by pg_dumpall",
"-g/--globals-only");
/* Process if toc.glo file does not exist. */
n_errors = restore_one_database(inputFileSpec, opts, numWorkers, false);
}
/* Done, print a summary of ignored errors during restore. */
if (n_errors)
{
pg_log_warning("errors ignored on restore: %d", n_errors);
return 1;
}
return 0;
}
/*
* restore_global_objects
*
* This restore all global objects.
*/
static int
restore_global_objects(const char *inputFileSpec, RestoreOptions *opts)
{
Archive *AH;
int nerror = 0;
/* Set format as custom so that toc.glo file can be read. */
opts->format = archCustom;
opts->txn_size = 0;
AH = OpenArchive(inputFileSpec, opts->format);
SetArchiveOptions(AH, NULL, opts);
on_exit_close_archive(AH);
/* Let the archiver know how noisy to be */
AH->verbose = opts->verbose;
/* Don't output TOC entry comments when restoring globals */
((ArchiveHandle *) AH)->noTocComments = 1;
AH->exit_on_error = false;
/* Parallel execution is not supported for global object restoration. */
AH->numWorkers = 1;
ProcessArchiveRestoreOptions(AH);
RestoreArchive(AH, false);
nerror = AH->n_errors;
/* AH may be freed in CloseArchive? */
CloseArchive(AH);
return nerror;
}
/*
* restore_one_database
*
* This will restore one database using toc.dat file.
*
* returns the number of errors while doing restore.
*/
static int
restore_one_database(const char *inputFileSpec, RestoreOptions *opts,
int numWorkers, bool append_data)
{
Archive *AH;
int n_errors;
AH = OpenArchive(inputFileSpec, opts->format);
SetArchiveOptions(AH, NULL, opts);
@ -703,15 +492,9 @@ restore_one_database(const char *inputFileSpec, RestoreOptions *opts,
/*
* We don't have a connection yet but that doesn't matter. The connection
* is initialized to NULL and if we terminate through exit_nicely() while
* it's still NULL, the cleanup function will just be a no-op. If we are
* restoring multiple databases, then only update AH handle for cleanup as
* the previous entry was already in the array and we had closed previous
* connection, so we can use the same array slot.
* it's still NULL, the cleanup function will just be a no-op.
*/
if (!append_data)
on_exit_close_archive(AH);
else
replace_on_exit_close_archive(AH);
on_exit_close_archive(AH);
/* Let the archiver know how noisy to be */
AH->verbose = opts->verbose;
@ -731,21 +514,25 @@ restore_one_database(const char *inputFileSpec, RestoreOptions *opts,
else
{
ProcessArchiveRestoreOptions(AH);
RestoreArchive(AH, append_data);
RestoreArchive(AH);
}
n_errors = AH->n_errors;
/* done, print a summary of ignored errors */
if (AH->n_errors)
pg_log_warning("errors ignored on restore: %d", AH->n_errors);
/* AH may be freed in CloseArchive? */
exit_code = AH->n_errors ? 1 : 0;
CloseArchive(AH);
return n_errors;
return exit_code;
}
static void
usage(const char *progname)
{
printf(_("%s restores PostgreSQL databases from archives created by pg_dump or pg_dumpall.\n\n"), progname);
printf(_("%s restores a PostgreSQL database from an archive created by pg_dump.\n\n"), progname);
printf(_("Usage:\n"));
printf(_(" %s [OPTION]... [FILE]\n"), progname);
@ -763,7 +550,6 @@ usage(const char *progname)
printf(_(" -c, --clean clean (drop) database objects before recreating\n"));
printf(_(" -C, --create create the target database\n"));
printf(_(" -e, --exit-on-error exit on error, default is to continue\n"));
printf(_(" -g, --globals-only restore only global objects, no databases\n"));
printf(_(" -I, --index=NAME restore named index\n"));
printf(_(" -j, --jobs=NUM use this many parallel jobs to restore\n"));
printf(_(" -L, --use-list=FILENAME use table of contents from this file for\n"
@ -780,7 +566,6 @@ usage(const char *progname)
printf(_(" -1, --single-transaction restore as a single transaction\n"));
printf(_(" --disable-triggers disable triggers during data-only restore\n"));
printf(_(" --enable-row-security enable row security\n"));
printf(_(" --exclude-database=PATTERN do not restore the specified database(s)\n"));
printf(_(" --filter=FILENAME restore or skip objects based on expressions\n"
" in FILENAME\n"));
printf(_(" --if-exists use IF EXISTS when dropping objects\n"));
@ -788,7 +573,6 @@ usage(const char *progname)
printf(_(" --no-data do not restore data\n"));
printf(_(" --no-data-for-failed-tables do not restore data of tables that could not be\n"
" created\n"));
printf(_(" --no-globals do not restore global objects (roles and tablespaces)\n"));
printf(_(" --no-policies do not restore row security policies\n"));
printf(_(" --no-publications do not restore publications\n"));
printf(_(" --no-schema do not restore schema\n"));
@ -817,8 +601,8 @@ usage(const char *progname)
printf(_(" --role=ROLENAME do SET ROLE before restore\n"));
printf(_("\n"
"The options -I, -n, -N, -P, -t, -T, --section, and --exclude-database can be\n"
"combined and specified multiple times to select multiple objects.\n"));
"The options -I, -n, -N, -P, -t, -T, and --section can be combined and specified\n"
"multiple times to select multiple objects.\n"));
printf(_("\nIf no input file name is supplied, then standard input is used.\n\n"));
printf(_("Report bugs to <%s>.\n"), PACKAGE_BUGREPORT);
printf(_("%s home page: <%s>\n"), PACKAGE_NAME, PACKAGE_URL);
@ -923,409 +707,3 @@ read_restore_filters(const char *filename, RestoreOptions *opts)
filter_free(&fstate);
}
/*
* file_exists_in_directory
*
* Returns true if the file exists in the given directory.
*/
static bool
file_exists_in_directory(const char *dir, const char *filename)
{
struct stat st;
char buf[MAXPGPATH];
if (snprintf(buf, MAXPGPATH, "%s/%s", dir, filename) >= MAXPGPATH)
pg_fatal("directory name too long: \"%s\"", dir);
return (stat(buf, &st) == 0 && S_ISREG(st.st_mode));
}
/*
* get_dbnames_list_to_restore
*
* This will mark for skipping any entries from dbname_oid_list that pattern match an
* entry in the db_exclude_patterns list.
*
* Returns the number of database to be restored.
*
*/
static int
get_dbnames_list_to_restore(PGconn *conn,
SimplePtrList *dbname_oid_list,
SimpleStringList db_exclude_patterns)
{
int count_db = 0;
PQExpBuffer query;
PQExpBuffer db_lit;
PGresult *res;
query = createPQExpBuffer();
db_lit = createPQExpBuffer();
/*
* Process one by one all dbnames and if specified to skip restoring, then
* remove dbname from list.
*/
for (SimplePtrListCell *db_cell = dbname_oid_list->head;
db_cell; db_cell = db_cell->next)
{
DbOidName *dbidname = (DbOidName *) db_cell->ptr;
bool skip_db_restore = false;
resetPQExpBuffer(query);
resetPQExpBuffer(db_lit);
appendStringLiteralConn(db_lit, dbidname->str, conn);
for (SimpleStringListCell *pat_cell = db_exclude_patterns.head; pat_cell; pat_cell = pat_cell->next)
{
/*
* If there is an exact match then we don't need to try a pattern
* match
*/
if (pg_strcasecmp(dbidname->str, pat_cell->val) == 0)
skip_db_restore = true;
/* Otherwise, try a pattern match if there is a connection */
else
{
int dotcnt;
appendPQExpBufferStr(query, "SELECT 1 ");
processSQLNamePattern(conn, query, pat_cell->val, false,
false, NULL, db_lit->data,
NULL, NULL, NULL, &dotcnt);
if (dotcnt > 0)
{
pg_log_error("improper qualified name (too many dotted names): %s",
dbidname->str);
PQfinish(conn);
exit_nicely(1);
}
res = executeQuery(conn, query->data);
if (PQntuples(res))
{
skip_db_restore = true;
pg_log_info("database name \"%s\" matches --exclude-database pattern \"%s\"", dbidname->str, pat_cell->val);
}
PQclear(res);
resetPQExpBuffer(query);
}
if (skip_db_restore)
break;
}
/*
* Mark db to be skipped or increment the counter of dbs to be
* restored
*/
if (skip_db_restore)
{
pg_log_info("excluding database \"%s\"", dbidname->str);
dbidname->oid = InvalidOid;
}
else
count_db++;
}
destroyPQExpBuffer(query);
destroyPQExpBuffer(db_lit);
return count_db;
}
/*
* get_dbname_oid_list_from_mfile
*
* Open map.dat file and read line by line and then prepare a list of database
* names and corresponding db_oid.
*
* Returns, total number of database names in map.dat file.
*/
static int
get_dbname_oid_list_from_mfile(const char *dumpdirpath,
SimplePtrList *dbname_oid_list)
{
StringInfoData linebuf;
FILE *pfile;
char map_file_path[MAXPGPATH];
int count = 0;
/*
* If there is no map.dat file in the dump, then return from here as there
* is no database to restore.
*/
if (!file_exists_in_directory(dumpdirpath, "map.dat"))
{
pg_log_info("database restoring is skipped because file \"%s\" does not exist in directory \"%s\"", "map.dat", dumpdirpath);
return 0;
}
snprintf(map_file_path, MAXPGPATH, "%s/map.dat", dumpdirpath);
/* Open map.dat file. */
pfile = fopen(map_file_path, PG_BINARY_R);
if (pfile == NULL)
pg_fatal("could not open file \"%s\": %m", map_file_path);
initStringInfo(&linebuf);
/* Append all the dbname/db_oid combinations to the list. */
while (pg_get_line_buf(pfile, &linebuf))
{
Oid db_oid = InvalidOid;
char *dbname;
DbOidName *dbidname;
int namelen;
char *p = linebuf.data;
/* look for the dboid. */
while (isdigit((unsigned char) *p))
p++;
/* ignore lines that don't begin with a digit */
if (p == linebuf.data)
continue;
if (*p == ' ')
{
sscanf(linebuf.data, "%u", &db_oid);
p++;
}
/* dbname is the rest of the line */
dbname = p;
namelen = strlen(dbname);
/* Strip trailing newline */
if (namelen > 0 && dbname[namelen - 1] == '\n')
dbname[--namelen] = '\0';
/* Report error and exit if the file has any corrupted data. */
if (!OidIsValid(db_oid) || namelen < 1)
pg_fatal("invalid entry in file \"%s\" on line %d", map_file_path,
count + 1);
dbidname = pg_malloc(offsetof(DbOidName, str) + namelen + 1);
dbidname->oid = db_oid;
strlcpy(dbidname->str, dbname, namelen + 1);
pg_log_info("found database \"%s\" (OID: %u) in file \"%s\"",
dbidname->str, db_oid, map_file_path);
simple_ptr_list_append(dbname_oid_list, dbidname);
count++;
}
/* Close map.dat file. */
fclose(pfile);
pfree(linebuf.data);
return count;
}
/*
* restore_all_databases
*
* This will restore databases those dumps are present in
* directory based on map.dat file mapping.
*
* This will skip restoring for databases that are specified with
* exclude-database option.
*
* returns, number of errors while doing restore.
*/
static int
restore_all_databases(const char *inputFileSpec,
SimpleStringList db_exclude_patterns, RestoreOptions *opts,
int numWorkers)
{
SimplePtrList dbname_oid_list = {NULL, NULL};
int num_db_restore = 0;
int num_total_db;
int n_errors_total = 0;
char *connected_db = NULL;
PGconn *conn = NULL;
RestoreOptions *original_opts = pg_malloc0_object(RestoreOptions);
RestoreOptions *tmpopts = pg_malloc0_object(RestoreOptions);
memcpy(original_opts, opts, sizeof(RestoreOptions));
/* Save db name to reuse it for all the database. */
if (opts->cparams.dbname)
connected_db = opts->cparams.dbname;
num_total_db = get_dbname_oid_list_from_mfile(inputFileSpec, &dbname_oid_list);
pg_log_info(ngettext("found %d database name in \"%s\"",
"found %d database names in \"%s\"",
num_total_db),
num_total_db, "map.dat");
/*
* If exclude-patterns is given, connect to the database to process them.
*/
if (db_exclude_patterns.head != NULL)
{
if (opts->cparams.dbname)
{
conn = ConnectDatabase(opts->cparams.dbname, NULL, opts->cparams.pghost,
opts->cparams.pgport, opts->cparams.username, TRI_DEFAULT,
false, progname, NULL, NULL, NULL, NULL);
if (!conn)
pg_fatal("could not connect to database \"%s\"", opts->cparams.dbname);
}
if (!conn)
{
pg_log_info("trying to connect to database \"%s\"", "postgres");
conn = ConnectDatabase("postgres", NULL, opts->cparams.pghost,
opts->cparams.pgport, opts->cparams.username, TRI_DEFAULT,
false, progname, NULL, NULL, NULL, NULL);
/* Try with template1. */
if (!conn)
{
pg_log_info("trying to connect to database \"%s\"", "template1");
conn = ConnectDatabase("template1", NULL, opts->cparams.pghost,
opts->cparams.pgport, opts->cparams.username, TRI_DEFAULT,
false, progname, NULL, NULL, NULL, NULL);
if (!conn)
{
pg_log_error("could not connect to databases \"postgres\" or \"template1\"\n"
"Please specify an alternative database.");
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
exit_nicely(1);
}
}
}
/* Filter the db list according to the exclude patterns. */
num_db_restore = get_dbnames_list_to_restore(conn, &dbname_oid_list,
db_exclude_patterns);
PQfinish(conn);
}
else
num_db_restore = num_total_db;
/* Exit if no db needs to be restored. */
if (num_db_restore == 0)
{
pg_log_info(ngettext("no database needs restoring out of %d database",
"no database needs restoring out of %d databases", num_total_db),
num_total_db);
pg_free(original_opts);
pg_free(tmpopts);
return 0;
}
pg_log_info("need to restore %d databases out of %d databases", num_db_restore, num_total_db);
/*
* We have a list of databases to restore after processing the
* exclude-database switch(es). Now we can restore them one by one.
*/
for (SimplePtrListCell *db_cell = dbname_oid_list.head;
db_cell; db_cell = db_cell->next)
{
DbOidName *dbidname = (DbOidName *) db_cell->ptr;
char subdirpath[MAXPGPATH];
char subdirdbpath[MAXPGPATH];
char dbfilename[MAXPGPATH];
int n_errors;
/* ignore dbs marked for skipping */
if (dbidname->oid == InvalidOid)
continue;
/*
* Since pg_backup_archiver.c may modify RestoreOptions during the
* previous restore, we must provide a fresh copy of the original
* "opts" for each call to restore_one_database.
*/
memcpy(tmpopts, original_opts, sizeof(RestoreOptions));
snprintf(subdirdbpath, MAXPGPATH, "%s/databases", inputFileSpec);
/*
* Look for the database dump file/dir. If there is an {oid}.tar or
* {oid}.dmp file, use it. Otherwise try to use a directory called
* {oid}
*/
snprintf(dbfilename, MAXPGPATH, "%u.tar", dbidname->oid);
if (file_exists_in_directory(subdirdbpath, dbfilename))
snprintf(subdirpath, MAXPGPATH, "%s/databases/%u.tar", inputFileSpec, dbidname->oid);
else
{
snprintf(dbfilename, MAXPGPATH, "%u.dmp", dbidname->oid);
if (file_exists_in_directory(subdirdbpath, dbfilename))
snprintf(subdirpath, MAXPGPATH, "%s/databases/%u.dmp", inputFileSpec, dbidname->oid);
else
snprintf(subdirpath, MAXPGPATH, "%s/databases/%u", inputFileSpec, dbidname->oid);
}
pg_log_info("restoring database \"%s\"", dbidname->str);
/* If database is already created, then don't set createDB flag. */
if (tmpopts->cparams.dbname)
{
PGconn *test_conn;
test_conn = ConnectDatabase(dbidname->str, NULL, tmpopts->cparams.pghost,
tmpopts->cparams.pgport, tmpopts->cparams.username, TRI_DEFAULT,
false, progname, NULL, NULL, NULL, NULL);
if (test_conn)
{
PQfinish(test_conn);
/* Use already created database for connection. */
tmpopts->createDB = 0;
tmpopts->cparams.dbname = dbidname->str;
}
else
{
if (!tmpopts->createDB)
{
pg_log_info("skipping restore of database \"%s\": database does not exist and %s was not specified",
dbidname->str, "-C/--create");
continue;
}
/* We'll have to create it */
tmpopts->createDB = 1;
tmpopts->cparams.dbname = connected_db;
}
}
/* Restore the single database. */
n_errors = restore_one_database(subdirpath, tmpopts, numWorkers, true);
n_errors_total += n_errors;
/* Print a summary of ignored errors during single database restore. */
if (n_errors)
pg_log_warning("errors ignored on database \"%s\" restore: %d", dbidname->str, n_errors);
}
/* Log number of processed databases. */
pg_log_info("number of restored databases is %d", num_db_restore);
/* Free dbname and dboid list. */
simple_ptr_list_destroy(&dbname_oid_list);
pg_free(original_opts);
pg_free(tmpopts);
return n_errors_total;
}

112
src/bin/pg_dump/t/001_basic.pl
View file

@ -87,8 +87,8 @@ command_fails_like(
command_fails_like(
[ 'pg_restore', '-s', '-a', '-f -' ],
qr/\Qpg_restore: error: options -a\/--data-only and -s\/--schema-only cannot be used together\E/,
'pg_restore: options -a/--data-only and -s/--schema-only cannot be used together'
qr/\Qpg_restore: error: options -s\/--schema-only and -a\/--data-only cannot be used together\E/,
'pg_restore: options -s/--schema-only and -a/--data-only cannot be used together'
);
command_fails_like(
@ -205,12 +205,7 @@ command_fails_like(
command_fails_like(
[ 'pg_restore', '-f -', '-F', 'garbage' ],
qr/\Qpg_restore: error: unrecognized archive format "garbage";\E/,
'pg_restore: unrecognized archive format');
command_fails_like(
[ 'pg_restore', '-f -', '-F', '' ],
qr/\Qpg_restore: error: unrecognized archive format "";\E/,
'pg_restore: empty archive format');
'pg_dump: unrecognized archive format');
command_fails_like(
[ 'pg_dump', '--on-conflict-do-nothing' ],
@ -251,105 +246,8 @@ command_fails_like(
# also fails for -r and -t, but it seems pointless to add more tests for those.
command_fails_like(
[ 'pg_dumpall', '--exclude-database=foo', '--globals-only' ],
qr/\Qpg_dumpall: error: options --exclude-database and -g\/--globals-only cannot be used together\E/,
'pg_dumpall: options --exclude-database and -g/--globals-only cannot be used together'
qr/\Qpg_dumpall: error: option --exclude-database cannot be used together with -g\/--globals-only\E/,
'pg_dumpall: option --exclude-database cannot be used together with -g/--globals-only'
);
command_fails_like(
[ 'pg_dumpall', '-a', '--no-data' ],
qr/\Qpg_dumpall: error: options -a\/--data-only and --no-data cannot be used together\E/,
'pg_dumpall: options -a\/--data-only and --no-data cannot be used together'
);
command_fails_like(
[ 'pg_dumpall', '-s', '--no-schema' ],
qr/\Qpg_dumpall: error: options -s\/--schema-only and --no-schema cannot be used together\E/,
'pg_dumpall: options -s\/--schema-only and --no-schema cannot be used together'
);
command_fails_like(
[ 'pg_dumpall', '--statistics-only', '--no-statistics' ],
qr/\Qpg_dumpall: error: options --statistics-only and --no-statistics cannot be used together\E/,
'pg_dumpall: options --statistics-only and --no-statistics cannot be used together'
);
command_fails_like(
[ 'pg_dumpall', '--statistics', '--no-statistics' ],
qr/\Qpg_dumpall: error: options --statistics and --no-statistics cannot be used together\E/,
'pg_dumpall: options --statistics-only and --no-statistics cannot be used together'
);
command_fails_like(
[ 'pg_dumpall', '--statistics', '--tablespaces-only' ],
qr/\Qpg_dumpall: error: options --statistics and -t\/--tablespaces-only cannot be used together\E/,
'pg_dumpall: options --statistics and -t\/--tablespaces-only cannot be used together'
);
command_fails_like(
[ 'pg_dumpall', '--format', 'x' ],
qr/\Qpg_dumpall: error: unrecognized output format "x";\E/,
'pg_dumpall: unrecognized output format');
command_fails_like(
[ 'pg_dumpall', '--format', 'd', '--restrict-key=uu', '-f dumpfile' ],
qr/\Qpg_dumpall: error: option --restrict-key can only be used with --format=plain\E/,
'pg_dumpall: --restrict-key can only be used with plain dump format');
command_fails_like(
[
'pg_dumpall', '--format', 'd', '--globals-only',
'--clean', '-f', 'dumpfile'
],
qr/\Qpg_dumpall: error: options --clean and -g\/--globals-only cannot be used together in non-text dump\E/,
'pg_dumpall: --clean and -g/--globals-only cannot be used together in non-text dump'
);
command_fails_like(
[ 'pg_dumpall', '--format', 'd' ],
qr/\Qpg_dumpall: error: option -F\/--format=d|c|t requires option -f\/--file\E/,
'pg_dumpall: non-plain format requires --file option');
command_fails_like(
[ 'pg_restore', '--exclude-database=foo', '--globals-only', '-d', 'xxx' ],
qr/\Qpg_restore: error: options --exclude-database and -g\/--globals-only cannot be used together\E/,
'pg_restore: options --exclude-database and -g/--globals-only cannot be used together'
);
command_fails_like(
[ 'pg_restore', '--data-only', '--globals-only', '-d', 'xxx' ],
qr/\Qpg_restore: error: options -a\/--data-only and -g\/--globals-only cannot be used together\E/,
'pg_restore: error: options -a/--data-only and -g/--globals-only cannot be used together'
);
command_fails_like(
[ 'pg_restore', '--schema-only', '--globals-only', '-d', 'xxx' ],
qr/\Qpg_restore: error: options -g\/--globals-only and -s\/--schema-only cannot be used together\E/,
'pg_restore: error: options -g/--globals-only and -s/--schema-only cannot be used together'
);
command_fails_like(
[ 'pg_restore', '--statistics-only', '--globals-only', '-d', 'xxx' ],
qr/\Qpg_restore: error: options -g\/--globals-only and --statistics-only cannot be used together\E/,
'pg_restore: error: options -g/--globals-only and --statistics-only cannot be used together'
);
command_fails_like(
[ 'pg_restore', '--exclude-database=foo', '-d', 'xxx', 'dumpdir' ],
qr/\Qpg_restore: error: option --exclude-database can be used only when restoring an archive created by pg_dumpall\E/,
'When option --exclude-database is used in pg_restore with dump of pg_dump'
);
command_fails_like(
[ 'pg_restore', '--globals-only', '-d', 'xxx', 'dumpdir' ],
qr/\Qpg_restore: error: option -g\/--globals-only can be used only when restoring an archive created by pg_dumpall\E/,
'When option --globals-only is used in pg_restore with the dump of pg_dump'
);
command_fails_like(
[
'pg_restore', '--globals-only', '--no-globals', '-d', 'xxx',
'dumpdir'
],
qr/\Qpg_restore: error: options -g\/--globals-only and --no-globals cannot be used together\E/,
'options --no-globals and --globals-only cannot be used together');
done_testing();

4
src/bin/pg_dump/t/005_pg_dump_filterfile.pl
View file

@ -570,8 +570,8 @@ command_fails_like(
'--filter' => "$tempdir/inputfile.txt",
'--globals-only'
],
qr/\Qpg_dumpall: error: options --exclude-database and -g\/--globals-only cannot be used together\E/,
'pg_dumpall: options --exclude-database and -g/--globals-only cannot be used together'
qr/\Qpg_dumpall: error: option --exclude-database cannot be used together with -g\/--globals-only\E/,
'pg_dumpall: option --exclude-database cannot be used together with -g/--globals-only'
);
# Test invalid filter command

661
src/bin/pg_dump/t/007_pg_dumpall.pl
View file

@ -1,661 +0,0 @@
# Copyright (c) 2021-2026, PostgreSQL Global Development Group
use strict;
use warnings FATAL => 'all';
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
my $run_db = 'postgres';
my $sep = $windows_os ? "\\" : "/";
# Tablespace locations used by "restore_tablespace" test case.
my $tablespace1 = "${tempdir}${sep}tbl1";
my $tablespace2 = "${tempdir}${sep}tbl2";
mkdir($tablespace1) || die "mkdir $tablespace1 $!";
mkdir($tablespace2) || die "mkdir $tablespace2 $!";
# escape tablespace locations on Windows.
my $tablespace2_orig = $tablespace2;
$tablespace1 = $windows_os ? ($tablespace1 =~ s/\\/\\\\/gr) : $tablespace1;
$tablespace2 = $windows_os ? ($tablespace2 =~ s/\\/\\\\/gr) : $tablespace2;
# Where pg_dumpall will be executed.
my $node = PostgreSQL::Test::Cluster->new('node');
$node->init;
$node->start;
###############################################################
# Definition of the pg_dumpall test cases to run.
#
# Each of these test cases are named and those names are used for fail
# reporting and also to save the dump and restore information needed for the
# test to assert.
#
# The "setup_sql" is a psql valid script that contains SQL commands to execute
# before of actually execute the tests. The setups are all executed before of
# any test execution.
#
# The "dump_cmd" and "restore_cmd" are the commands that will be executed. The
# "restore_cmd" must have the --file flag to save the restore output so that we
# can assert on it.
#
# The "like" and "unlike" is a regexp that is used to match the pg_restore
# output. It must have at least one of then filled per test cases but it also
# can have both. See "excluding_databases" test case for example.
my %pgdumpall_runs = (
restore_roles => {
setup_sql => '
CREATE ROLE dumpall WITH ENCRYPTED PASSWORD \'admin\' SUPERUSER;
CREATE ROLE dumpall2 WITH REPLICATION CONNECTION LIMIT 10;',
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/restore_roles",
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'directory',
'--file' => "$tempdir/restore_roles.sql",
"$tempdir/restore_roles",
],
like => qr/
\s*\QCREATE ROLE dumpall2;\E
\s*\QALTER ROLE dumpall2 WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB NOLOGIN REPLICATION NOBYPASSRLS CONNECTION LIMIT 10;\E
/xm
},
restore_tablespace => {
setup_sql => "
CREATE ROLE tap;
CREATE TABLESPACE tbl1 OWNER tap LOCATION '$tablespace1';
CREATE TABLESPACE tbl2 OWNER tap LOCATION '$tablespace2' WITH (seq_page_cost=1.0);",
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/restore_tablespace",
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'directory',
'--file' => "$tempdir/restore_tablespace.sql",
"$tempdir/restore_tablespace",
],
# Match "E" as optional since it is added on LOCATION when running on
# Windows.
like => qr/^
\n\QCREATE TABLESPACE tbl2 OWNER tap LOCATION \E(?:E)?\Q'$tablespace2_orig';\E
\n\QALTER TABLESPACE tbl2 SET (seq_page_cost=1.0);\E
/xm,
},
restore_grants => {
setup_sql => "
CREATE DATABASE tapgrantsdb;
CREATE SCHEMA private;
CREATE SEQUENCE serial START 101;
CREATE FUNCTION fn() RETURNS void AS \$\$
BEGIN
END;
\$\$ LANGUAGE plpgsql;
CREATE ROLE super;
CREATE ROLE grant1;
CREATE ROLE grant2;
CREATE ROLE grant3;
CREATE ROLE grant4;
CREATE ROLE grant5;
CREATE ROLE grant6;
CREATE ROLE grant7;
CREATE ROLE grant8;
CREATE TABLE t (id int);
INSERT INTO t VALUES (1), (2), (3), (4);
GRANT SELECT ON TABLE t TO grant1;
GRANT INSERT ON TABLE t TO grant2;
GRANT ALL PRIVILEGES ON TABLE t to grant3;
GRANT CONNECT, CREATE ON DATABASE tapgrantsdb TO grant4;
GRANT USAGE, CREATE ON SCHEMA private TO grant5;
GRANT USAGE, SELECT, UPDATE ON SEQUENCE serial TO grant6;
GRANT super TO grant7;
GRANT EXECUTE ON FUNCTION fn() TO grant8;
",
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/restore_grants",
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'directory',
'--file' => "$tempdir/restore_grants.sql",
"$tempdir/restore_grants",
],
like => qr/^
\n\QGRANT ALL ON SCHEMA private TO grant5;\E
(.*\n)*
\n\QGRANT ALL ON FUNCTION public.fn() TO grant8;\E
(.*\n)*
\n\QGRANT ALL ON SEQUENCE public.serial TO grant6;\E
(.*\n)*
\n\QGRANT SELECT ON TABLE public.t TO grant1;\E
\n\QGRANT INSERT ON TABLE public.t TO grant2;\E
\n\QGRANT ALL ON TABLE public.t TO grant3;\E
(.*\n)*
\n\QGRANT CREATE,CONNECT ON DATABASE tapgrantsdb TO grant4;\E
/xm,
},
excluding_databases => {
setup_sql => 'CREATE DATABASE db1;
\c db1
CREATE TABLE t1 (id int);
INSERT INTO t1 VALUES (1), (2), (3), (4);
CREATE TABLE t2 (id int);
INSERT INTO t2 VALUES (1), (2), (3), (4);
CREATE DATABASE db2;
\c db2
CREATE TABLE t3 (id int);
INSERT INTO t3 VALUES (1), (2), (3), (4);
CREATE TABLE t4 (id int);
INSERT INTO t4 VALUES (1), (2), (3), (4);
CREATE DATABASE dbex3;
\c dbex3
CREATE TABLE t5 (id int);
INSERT INTO t5 VALUES (1), (2), (3), (4);
CREATE TABLE t6 (id int);
INSERT INTO t6 VALUES (1), (2), (3), (4);
CREATE DATABASE dbex4;
\c dbex4
CREATE TABLE t7 (id int);
INSERT INTO t7 VALUES (1), (2), (3), (4);
CREATE TABLE t8 (id int);
INSERT INTO t8 VALUES (1), (2), (3), (4);
CREATE DATABASE db5;
\c db5
CREATE TABLE t9 (id int);
INSERT INTO t9 VALUES (1), (2), (3), (4);
CREATE TABLE t10 (id int);
INSERT INTO t10 VALUES (1), (2), (3), (4);
',
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/excluding_databases",
'--exclude-database' => 'dbex*',
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'directory',
'--file' => "$tempdir/excluding_databases.sql",
'--exclude-database' => 'db5',
"$tempdir/excluding_databases",
],
like => qr/^
\n\QCREATE DATABASE db1\E
(.*\n)*
\n\QCREATE TABLE public.t1 (\E
(.*\n)*
\n\QCREATE TABLE public.t2 (\E
(.*\n)*
\n\QCREATE DATABASE db2\E
(.*\n)*
\n\QCREATE TABLE public.t3 (\E
(.*\n)*
\n\QCREATE TABLE public.t4 (/xm,
unlike => qr/^
\n\QCREATE DATABASE db3\E
(.*\n)*
\n\QCREATE TABLE public.t5 (\E
(.*\n)*
\n\QCREATE TABLE public.t6 (\E
(.*\n)*
\n\QCREATE DATABASE db4\E
(.*\n)*
\n\QCREATE TABLE public.t7 (\E
(.*\n)*
\n\QCREATE TABLE public.t8 (\E
\n\QCREATE DATABASE db5\E
(.*\n)*
\n\QCREATE TABLE public.t9 (\E
(.*\n)*
\n\QCREATE TABLE public.t10 (\E
/xm,
},
format_directory => {
setup_sql => "CREATE TABLE format_directory(a int, b boolean, c text);
INSERT INTO format_directory VALUES (1, true, 'name1'), (2, false, 'name2');",
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/format_directory",
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'directory',
'--file' => "$tempdir/format_directory.sql",
"$tempdir/format_directory",
],
like => qr/^\n\QCOPY public.format_directory (a, b, c) FROM stdin;/xm
},
format_tar => {
setup_sql => "CREATE TABLE format_tar(a int, b boolean, c text);
INSERT INTO format_tar VALUES (1, false, 'name3'), (2, true, 'name4');",
dump_cmd => [
'pg_dumpall',
'--format' => 'tar',
'--file' => "$tempdir/format_tar",
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'tar',
'--file' => "$tempdir/format_tar.sql",
"$tempdir/format_tar",
],
like => qr/^\n\QCOPY public.format_tar (a, b, c) FROM stdin;/xm
},
format_custom => {
setup_sql => "CREATE TABLE format_custom(a int, b boolean, c text);
INSERT INTO format_custom VALUES (1, false, 'name5'), (2, true, 'name6');",
dump_cmd => [
'pg_dumpall',
'--format' => 'custom',
'--file' => "$tempdir/format_custom",
],
restore_cmd => [
'pg_restore', '-C',
'--format' => 'custom',
'--file' => "$tempdir/format_custom.sql",
"$tempdir/format_custom",
],
like => qr/^ \n\QCOPY public.format_custom (a, b, c) FROM stdin;/xm
},
dump_globals_only => {
setup_sql => "CREATE TABLE format_dir(a int, b boolean, c text);
INSERT INTO format_dir VALUES (1, false, 'name5'), (2, true, 'name6');",
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--globals-only',
'--file' => "$tempdir/dump_globals_only",
],
restore_cmd => [
'pg_restore', '-C', '--globals-only',
'--format' => 'directory',
'--file' => "$tempdir/dump_globals_only.sql",
"$tempdir/dump_globals_only",
],
like => qr/
^\s*\QCREATE ROLE dumpall;\E\s*\n
/xm
},
restore_no_globals => {
setup_sql => "CREATE TABLE no_globals_test(a int, b text);
INSERT INTO no_globals_test VALUES (1, 'hello'), (2, 'world');",
dump_cmd => [
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/restore_no_globals",
],
restore_cmd => [
'pg_restore', '-C', '--no-globals',
'--format' => 'directory',
'--file' => "$tempdir/restore_no_globals.sql",
"$tempdir/restore_no_globals",
],
like => qr/^\n\QCOPY public.no_globals_test (a, b) FROM stdin;\E/xm,
unlike => qr/^\QCREATE ROLE dumpall;\E/xm,
},);
# First execute the setup_sql
foreach my $run (sort keys %pgdumpall_runs)
{
if ($pgdumpall_runs{$run}->{setup_sql})
{
$node->safe_psql($run_db, $pgdumpall_runs{$run}->{setup_sql});
}
}
# Execute the tests
foreach my $run (sort keys %pgdumpall_runs)
{
# Create a new target cluster to pg_restore each test case run so that we
# don't need to take care of the cleanup from the target cluster after each
# run.
my $target_node = PostgreSQL::Test::Cluster->new("target_$run");
$target_node->init;
$target_node->start;
# Dumpall from node cluster.
$node->command_ok(\@{ $pgdumpall_runs{$run}->{dump_cmd} },
"$run: pg_dumpall runs");
# Restore the dump on "target_node" cluster.
my @restore_cmd = (
@{ $pgdumpall_runs{$run}->{restore_cmd} },
'--host', $target_node->host, '--port', $target_node->port);
my ($stdout, $stderr) = run_command(\@restore_cmd);
# pg_restore --file output file.
my $output_file = slurp_file("$tempdir/${run}.sql");
if ( !($pgdumpall_runs{$run}->{like})
&& !($pgdumpall_runs{$run}->{unlike}))
{
die "missing \"like\" or \"unlike\" in test \"$run\"";
}
if ($pgdumpall_runs{$run}->{like})
{
like($output_file, $pgdumpall_runs{$run}->{like}, "should dump $run");
}
if ($pgdumpall_runs{$run}->{unlike})
{
unlike(
$output_file,
$pgdumpall_runs{$run}->{unlike},
"should not dump $run");
}
$target_node->stop;
$target_node->clean_node;
}
# Some negative test case with dump of pg_dumpall and restore using pg_restore
# report an error when -C is not used in pg_restore with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom",
'--format' => 'custom',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option -C\/--create must be specified when restoring an archive created by pg_dumpall\E/,
'When -C is not used in pg_restore with dump of pg_dumpall');
# report an error when \l/--list option is used with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--list',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option -l\/--list cannot be used when restoring an archive created by pg_dumpall\E/,
'When --list is used in pg_restore with dump of pg_dumpall');
# report an error when -L/--use-list option is used with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--use-list' => 'use',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option -L\/--use-list cannot be used when restoring an archive created by pg_dumpall\E/,
'When -L/--use-list is used in pg_restore with dump of pg_dumpall');
# report an error when --strict-names option is used with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--strict-names',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option --strict-names cannot be used when restoring an archive created by pg_dumpall\E/,
'When --strict-names is used in pg_restore with dump of pg_dumpall');
# report an error when --clean and -g/--globals-only are used in pg_restore with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--clean',
'--globals-only',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options --clean and -g\/--globals-only cannot be used together when restoring an archive created by pg_dumpall\E/,
'When --clean and -g/--globals-only are used in pg_restore with dump of pg_dumpall'
);
# report an error when non-exist database is given with -d option
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'-d' => 'dbpq',
],
qr/\QFATAL: database "dbpq" does not exist\E/,
'When non-existent database is given with -d option in pg_restore with dump of pg_dumpall'
);
# report an error when --no-schema is used with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--no-schema',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option --no-schema cannot be used when restoring an archive created by pg_dumpall\E/,
'When --no-schema is used in pg_restore with dump of pg_dumpall');
# report an error when --data-only is used with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--data-only',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option -a\/--data-only cannot be used when restoring an archive created by pg_dumpall\E/,
'When --data-only is used in pg_restore with dump of pg_dumpall');
# report an error when --statistics-only is used with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--statistics-only',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option --statistics-only cannot be used when restoring an archive created by pg_dumpall\E/,
'When --statistics-only is used in pg_restore with dump of pg_dumpall');
# report an error when --section excludes pre-data with dump of pg_dumpall
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--section' => 'post-data',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: option --section cannot exclude --pre-data when restoring a pg_dumpall archive\E/,
'When --section=post-data is used in pg_restore with dump of pg_dumpall');
# report an error when --globals-only and --data-only are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--data-only',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options -a\/--data-only and -g\/--globals-only cannot be used together\E/,
'When --globals-only and --data-only are used together');
# report an error when --globals-only and --schema-only are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--schema-only',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options -g\/--globals-only and -s\/--schema-only cannot be used together\E/,
'When --globals-only and --schema-only are used together');
# report an error when --globals-only and --statistics-only are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--statistics-only',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options -g\/--globals-only and --statistics-only cannot be used together\E/,
'When --globals-only and --statistics-only are used together');
# report an error when --globals-only and --statistics are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--statistics',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options --statistics and -g\/--globals-only cannot be used together\E/,
'When --globals-only and --statistics are used together');
# report an error when --globals-only and --exit-on-error are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--exit-on-error',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options --exit-on-error and -g\/--globals-only cannot be used together\E/,
'When --globals-only and --exit-on-error are used together');
# report an error when --globals-only and --single-transaction are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--single-transaction',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options -g\/--globals-only and -1\/--single-transaction cannot be used together\E/,
'When --globals-only and --single-transaction are used together');
# report an error when --globals-only and --transaction-size are used together
$node->command_fails_like(
[
'pg_restore',
"$tempdir/format_custom", '-C',
'--format' => 'custom',
'--globals-only',
'--transaction-size' => '100',
'--file' => "$tempdir/error_test.sql",
],
qr/\Qpg_restore: error: options -g\/--globals-only and --transaction-size cannot be used together\E/,
'When --globals-only and --transaction-size are used together');
# verify map.dat preamble exists
my $map_dat_content = slurp_file("$tempdir/format_directory/map.dat");
like(
$map_dat_content,
qr/^# map\.dat\n.*# This file maps oids to database names/ms,
'map.dat contains expected preamble');
# verify commenting out a line in map.dat skips that database
$node->safe_psql(
$run_db, 'CREATE DATABASE comment_test_db;
\c comment_test_db
CREATE TABLE comment_test_table (id int);');
$node->command_ok(
[
'pg_dumpall',
'--format' => 'directory',
'--file' => "$tempdir/comment_test",
],
'pg_dumpall for comment test');
# Modify map.dat to comment out the comment_test_db entry
my $map_content = slurp_file("$tempdir/comment_test/map.dat");
$map_content =~ s/^(\d+ comment_test_db)$/# $1/m;
open(my $fh, '>', "$tempdir/comment_test/map.dat")
or die "Cannot open map.dat: $!";
print $fh $map_content;
close($fh);
# Create a target node and restore - commented db should be skipped
my $target_comment = PostgreSQL::Test::Cluster->new("target_comment");
$target_comment->init;
$target_comment->start;
$node->command_ok(
[
'pg_restore', '-C',
'--format' => 'directory',
'--file' => "$tempdir/comment_test_restore.sql",
'--host', $target_comment->host,
'--port', $target_comment->port,
"$tempdir/comment_test",
],
'pg_restore with commented out database in map.dat');
my $restore_output = slurp_file("$tempdir/comment_test_restore.sql");
unlike(
$restore_output,
qr/CREATE DATABASE comment_test_db/,
'commented out database in map.dat is not restored');
# Test that --clean implies --if-exists for pg_dumpall archives
$node->command_ok(
[
'pg_restore', '-C',
'--format' => 'custom',
'--clean',
'--file' => "$tempdir/clean_test.sql",
"$tempdir/format_custom",
],
'pg_restore with --clean on pg_dumpall archive');
my $clean_output = slurp_file("$tempdir/clean_test.sql");
like(
$clean_output,
qr/DROP ROLE IF EXISTS/,
'--clean implies --if-exists: DROP ROLE IF EXISTS in output');
$node->stop('fast');
done_testing();

1
src/tools/pgindent/typedefs.list
View file

@ -648,7 +648,6 @@ DatumTupleFields
DbInfo
DbInfoArr
DbLocaleInfo
DbOidName
DdlOptType
DdlOption
DeClonePtrType