mirror of
https://github.com/postgres/postgres.git
synced 2026-04-28 01:26:56 -04:00
eaef111396
used within WAL files. Historically this was the same as the data file BLCKSZ, but there's no necessary connection, and it's possible that performance gains might ensue from reducing XLOG_BLCKSZ. In any case distinguishing two symbols should improve code clarity. This commit does not actually change the page size, only provide the infrastructure to make it possible to do so. initdb forced because of addition of a field to pg_control. Mark Wong, with some help from Simon Riggs and Tom Lane.
1638 lines
62 KiB
Text
1638 lines
62 KiB
Text
<!-- $PostgreSQL: pgsql/doc/src/sgml/runtime.sgml,v 1.366 2006/04/03 23:35:02 tgl Exp $ -->
|
|
|
|
<chapter Id="runtime">
|
|
<title>Operating System Environment</title>
|
|
|
|
<para>
|
|
This chapter discusses how to set up and run the database server
|
|
and its interactions with the operating system.
|
|
</para>
|
|
|
|
<sect1 id="postgres-user">
|
|
<title>The <productname>PostgreSQL</productname> User Account</title>
|
|
|
|
<indexterm>
|
|
<primary>postgres user</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
As with any other server daemon that is accessible to the outside world,
|
|
it is advisable to run <productname>PostgreSQL</productname> under a
|
|
separate user account. This user account should only own the data
|
|
that is managed by the server, and should not be shared with other
|
|
daemons. (For example, using the user <literal>nobody</literal> is a bad
|
|
idea.) It is not advisable to install executables owned by this
|
|
user because compromised systems could then modify their own
|
|
binaries.
|
|
</para>
|
|
|
|
<para>
|
|
To add a Unix user account to your system, look for a command
|
|
<command>useradd</command> or <command>adduser</command>. The user
|
|
name <systemitem>postgres</systemitem> is often used, and is assumed
|
|
throughout this book, but you can use another name if you like.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="creating-cluster">
|
|
<title>Creating a Database Cluster</title>
|
|
|
|
<indexterm>
|
|
<primary>database cluster</primary>
|
|
</indexterm>
|
|
|
|
<indexterm>
|
|
<primary>data area</primary>
|
|
<see>database cluster</see>
|
|
</indexterm>
|
|
|
|
<para>
|
|
Before you can do anything, you must initialize a database storage
|
|
area on disk. We call this a <firstterm>database cluster</firstterm>.
|
|
(<acronym>SQL</acronym> uses the term catalog cluster.) A
|
|
database cluster is a collection of databases that is managed by a
|
|
single instance of a running database server. After initialization, a
|
|
database cluster will contain a database named <literal>postgres</literal>,
|
|
which is meant as a default database for use by utilities, users and third
|
|
party applications. The database server itself does not require the
|
|
<literal>postgres</literal> database to exist, but many external utility
|
|
programs assume it exists. Another database created within each cluster
|
|
during initialization is called
|
|
<literal>template1</literal>. As the name suggests, this will be used
|
|
as a template for subsequently created databases; it should not be
|
|
used for actual work. (See <xref linkend="managing-databases"> for
|
|
information about creating new databases within a cluster.)
|
|
</para>
|
|
|
|
<para>
|
|
In file system terms, a database cluster will be a single directory
|
|
under which all data will be stored. We call this the <firstterm>data
|
|
directory</firstterm> or <firstterm>data area</firstterm>. It is
|
|
completely up to you where you choose to store your data. There is no
|
|
default, although locations such as
|
|
<filename>/usr/local/pgsql/data</filename> or
|
|
<filename>/var/lib/pgsql/data</filename> are popular. To initialize a
|
|
database cluster, use the command <xref
|
|
linkend="app-initdb">,<indexterm><primary>initdb</></> which is
|
|
installed with <productname>PostgreSQL</productname>. The desired
|
|
file system location of your database cluster is indicated by the
|
|
<option>-D</option> option, for example
|
|
<screen>
|
|
<prompt>$</> <userinput>initdb -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
Note that you must execute this command while logged into the
|
|
<productname>PostgreSQL</productname> user account, which is
|
|
described in the previous section.
|
|
</para>
|
|
|
|
<tip>
|
|
<para>
|
|
As an alternative to the <option>-D</option> option, you can set
|
|
the environment variable <envar>PGDATA</envar>.
|
|
<indexterm><primary><envar>PGDATA</envar></primary></indexterm>
|
|
</para>
|
|
</tip>
|
|
|
|
<para>
|
|
<command>initdb</command> will attempt to create the directory you
|
|
specify if it does not already exist. It is likely that it will not
|
|
have the permission to do so (if you followed our advice and created
|
|
an unprivileged account). In that case you should create the
|
|
directory yourself (as root) and change the owner to be the
|
|
<productname>PostgreSQL</productname> user. Here is how this might
|
|
be done:
|
|
<screen>
|
|
root# <userinput>mkdir /usr/local/pgsql/data</userinput>
|
|
root# <userinput>chown postgres /usr/local/pgsql/data</userinput>
|
|
root# <userinput>su postgres</userinput>
|
|
postgres$ <userinput>initdb -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> will refuse to run if the data directory
|
|
looks like it has already been initialized.</para>
|
|
|
|
<para>
|
|
Because the data directory contains all the data stored in the
|
|
database, it is essential that it be secured from unauthorized
|
|
access. <command>initdb</command> therefore revokes access
|
|
permissions from everyone but the
|
|
<productname>PostgreSQL</productname> user.
|
|
</para>
|
|
|
|
<para>
|
|
However, while the directory contents are secure, the default
|
|
client authentication setup allows any local user to connect to the
|
|
database and even become the database superuser. If you do not
|
|
trust other local users, we recommend you use one of
|
|
<command>initdb</command>'s <option>-W</option>, <option>--pwprompt</option>
|
|
or <option>--pwfile</option> options to assign a password to the
|
|
database superuser.<indexterm><primary>password</><secondary>of the
|
|
superuser</></indexterm> Also, specify <option>-A md5</> or
|
|
<option>-A password</> so that the default <literal>trust</> authentication
|
|
mode is not used; or modify the generated <filename>pg_hba.conf</filename>
|
|
file after running <command>initdb</command>,
|
|
<emphasis>before</> you start the server for the first time. (Other
|
|
reasonable approaches include using <literal>ident</literal> authentication
|
|
or file system permissions to restrict connections. See <xref
|
|
linkend="client-authentication"> for more information.)
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> also initializes the default
|
|
locale<indexterm><primary>locale</></> for the database cluster.
|
|
Normally, it will just take the locale settings in the environment
|
|
and apply them to the initialized database. It is possible to
|
|
specify a different locale for the database; more information about
|
|
that can be found in <xref linkend="locale">. The sort order used
|
|
within a particular database cluster is set by
|
|
<command>initdb</command> and cannot be changed later, short of
|
|
dumping all data, rerunning <command>initdb</command>, and reloading
|
|
the data. There is also a performance impact for using locales
|
|
other than <literal>C</> or <literal>POSIX</>. Therefore, it is
|
|
important to make this choice correctly the first time.
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> also sets the default character set encoding
|
|
for the database cluster. Normally this should be chosen to match the
|
|
locale setting. For details see <xref linkend="multibyte">.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="postmaster-start">
|
|
<title>Starting the Database Server</title>
|
|
|
|
<para>
|
|
Before anyone can access the database, you must start the database
|
|
server. The database server program is called
|
|
<command>postmaster</command>.<indexterm><primary>postmaster</></>
|
|
The <command>postmaster</command> must know where to
|
|
find the data it is supposed to use. This is done with the
|
|
<option>-D</option> option. Thus, the simplest way to start the
|
|
server is:
|
|
<screen>
|
|
$ <userinput>postmaster -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
which will leave the server running in the foreground. This must be
|
|
done while logged into the <productname>PostgreSQL</productname> user
|
|
account. Without <option>-D</option>, the server will try to use
|
|
the data directory named by the environment variable <envar>PGDATA</envar>.
|
|
If that variable is not provided either, it will fail.
|
|
</para>
|
|
|
|
<para>
|
|
Normally it is better to start the <command>postmaster</command> in the
|
|
background. For this, use the usual shell syntax:
|
|
<screen>
|
|
$ <userinput>postmaster -D /usr/local/pgsql/data >logfile 2>&1 &</userinput>
|
|
</screen>
|
|
It is important to store the server's <systemitem>stdout</> and
|
|
<systemitem>stderr</> output somewhere, as shown above. It will help
|
|
for auditing purposes and to diagnose problems. (See <xref
|
|
linkend="logfile-maintenance"> for a more thorough discussion of log
|
|
file handling.)
|
|
</para>
|
|
|
|
<para>
|
|
The <command>postmaster</command> also takes a number of other
|
|
command line options. For more information, see the
|
|
<xref linkend="app-postmaster"> reference page
|
|
and <xref linkend="runtime-config"> below.
|
|
</para>
|
|
|
|
<para>
|
|
This shell syntax can get tedious quickly. Therefore the wrapper
|
|
program
|
|
<xref linkend="app-pg-ctl"><indexterm><primary>pg_ctl</primary></indexterm>
|
|
is provided to simplify some tasks. For example:
|
|
<programlisting>
|
|
pg_ctl start -l logfile
|
|
</programlisting>
|
|
will start the server in the background and put the output into the
|
|
named log file. The <option>-D</option> option has the same meaning
|
|
here as in the <command>postmaster</command>. <command>pg_ctl</command>
|
|
is also capable of stopping the server.
|
|
</para>
|
|
|
|
<para>
|
|
Normally, you will want to start the database server when the
|
|
computer boots.<indexterm><primary>booting</><secondary>starting
|
|
the server during</></> Autostart scripts are operating-system-specific.
|
|
There are a few distributed with
|
|
<productname>PostgreSQL</productname> in the
|
|
<filename>contrib/start-scripts</> directory. Installing one will require
|
|
root privileges.
|
|
</para>
|
|
|
|
<para>
|
|
Different systems have different conventions for starting up daemons
|
|
at boot time. Many systems have a file
|
|
<filename>/etc/rc.local</filename> or
|
|
<filename>/etc/rc.d/rc.local</filename>. Others use
|
|
<filename>rc.d</> directories. Whatever you do, the server must be
|
|
run by the <productname>PostgreSQL</productname> user account
|
|
<emphasis>and not by root</emphasis> or any other user. Therefore you
|
|
probably should form your commands using <literal>su -c '...'
|
|
postgres</literal>. For example:
|
|
<programlisting>
|
|
su -c 'pg_ctl start -D /usr/local/pgsql/data -l serverlog' postgres
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
Here are a few more operating-system-specific suggestions. (In each
|
|
case be sure to use the proper installation directory and user
|
|
name where we show generic values.)
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
For <productname>FreeBSD</productname>, look at the file
|
|
<filename>contrib/start-scripts/freebsd</filename> in the
|
|
<productname>PostgreSQL</productname> source distribution.
|
|
<indexterm><primary>FreeBSD</><secondary>start script</secondary></>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>OpenBSD</productname>, add the following lines
|
|
to the file <filename>/etc/rc.local</filename>:
|
|
<indexterm><primary>OpenBSD</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
if [ -x /usr/local/pgsql/bin/pg_ctl -a -x /usr/local/pgsql/bin/postmaster ]; then
|
|
su - -c '/usr/local/pgsql/bin/pg_ctl start -l /var/postgresql/log -s' postgres
|
|
echo -n ' postgresql'
|
|
fi
|
|
</programlisting>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>Linux</productname> systems either add
|
|
<indexterm><primary>Linux</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data
|
|
</programlisting>
|
|
to <filename>/etc/rc.d/rc.local</filename> or look at the file
|
|
<filename>contrib/start-scripts/linux</filename> in the
|
|
<productname>PostgreSQL</productname> source distribution.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>NetBSD</productname>, either use the
|
|
<productname>FreeBSD</productname> or
|
|
<productname>Linux</productname> start scripts, depending on
|
|
preference. <indexterm><primary>NetBSD</><secondary>start script</secondary></>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>Solaris</productname>, create a file called
|
|
<filename>/etc/init.d/postgresql</filename> that contains
|
|
the following line:
|
|
<indexterm><primary>Solaris</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data"
|
|
</programlisting>
|
|
Then, create a symbolic link to it in <filename>/etc/rc3.d</> as
|
|
<filename>S99postgresql</>.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</para>
|
|
|
|
<para>
|
|
While the <command>postmaster</command> is running, its
|
|
<acronym>PID</acronym> is stored in the file
|
|
<filename>postmaster.pid</filename> in the data directory. This is
|
|
used to prevent multiple <command>postmaster</command> processes
|
|
running in the same data directory and can also be used for
|
|
shutting down the <command>postmaster</command> process.
|
|
</para>
|
|
|
|
<sect2 id="postmaster-start-failures">
|
|
<title>Server Start-up Failures</title>
|
|
|
|
<para>
|
|
There are several common reasons the server might fail to
|
|
start. Check the server's log file, or start it by hand (without
|
|
redirecting standard output or standard error) and see what error
|
|
messages appear. Below we explain some of the most common error
|
|
messages in more detail.
|
|
</para>
|
|
|
|
<para>
|
|
<screen>
|
|
LOG: could not bind IPv4 socket: Address already in use
|
|
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
|
|
FATAL: could not create TCP/IP listen socket
|
|
</screen>
|
|
This usually means just what it suggests: you tried to start
|
|
another <command>postmaster</command> on the same port where one is already running.
|
|
However, if the kernel error message is not <computeroutput>Address
|
|
already in use</computeroutput> or some variant of that, there may
|
|
be a different problem. For example, trying to start a <command>postmaster</command>
|
|
on a reserved port number may draw something like:
|
|
<screen>
|
|
$ <userinput>postmaster -p 666</userinput>
|
|
LOG: could not bind IPv4 socket: Permission denied
|
|
HINT: Is another postmaster already running on port 666? If not, wait a few seconds and retry.
|
|
FATAL: could not create TCP/IP listen socket
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
A message like
|
|
<screen>
|
|
FATAL: could not create shared memory segment: Invalid argument
|
|
DETAIL: Failed system call was shmget(key=5440001, size=4011376640, 03600).
|
|
</screen>
|
|
probably means your kernel's limit on the size of shared memory is
|
|
smaller than the work area <productname>PostgreSQL</productname>
|
|
is trying to create (4011376640 bytes in this example). Or it could
|
|
mean that you do not have System-V-style shared memory support
|
|
configured into your kernel at all. As a temporary workaround, you
|
|
can try starting the server with a smaller-than-normal number of
|
|
buffers (<xref linkend="guc-shared-buffers">). You will eventually want
|
|
to reconfigure your kernel to increase the allowed shared memory
|
|
size. You may also see this message when trying to start multiple
|
|
servers on the same machine, if their total space requested
|
|
exceeds the kernel limit.
|
|
</para>
|
|
|
|
<para>
|
|
An error like
|
|
<screen>
|
|
FATAL: could not create semaphores: No space left on device
|
|
DETAIL: Failed system call was semget(5440126, 17, 03600).
|
|
</screen>
|
|
does <emphasis>not</emphasis> mean you've run out of disk
|
|
space. It means your kernel's limit on the number of <systemitem
|
|
class="osname">System V</> semaphores is smaller than the number
|
|
<productname>PostgreSQL</productname> wants to create. As above,
|
|
you may be able to work around the problem by starting the
|
|
server with a reduced number of allowed connections
|
|
(<xref linkend="guc-max-connections">), but you'll eventually want to
|
|
increase the kernel limit.
|
|
</para>
|
|
|
|
<para>
|
|
If you get an <quote>illegal system call</> error, it is likely that
|
|
shared memory or semaphores are not supported in your kernel at
|
|
all. In that case your only option is to reconfigure the kernel to
|
|
enable these features.
|
|
</para>
|
|
|
|
<para>
|
|
Details about configuring <systemitem class="osname">System V</>
|
|
<acronym>IPC</> facilities are given in <xref linkend="sysvipc">.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="client-connection-problems">
|
|
<title>Client Connection Problems</title>
|
|
|
|
<para>
|
|
Although the error conditions possible on the client side are quite
|
|
varied and application-dependent, a few of them might be directly
|
|
related to how the server was started up. Conditions other than
|
|
those shown below should be documented with the respective client
|
|
application.
|
|
</para>
|
|
|
|
<para>
|
|
<screen>
|
|
psql: could not connect to server: Connection refused
|
|
Is the server running on host "server.joe.com" and accepting
|
|
TCP/IP connections on port 5432?
|
|
</screen>
|
|
This is the generic <quote>I couldn't find a server to talk
|
|
to</quote> failure. It looks like the above when TCP/IP
|
|
communication is attempted. A common mistake is to forget to
|
|
configure the server to allow TCP/IP connections.
|
|
</para>
|
|
|
|
<para>
|
|
Alternatively, you'll get this when attempting Unix-domain socket
|
|
communication to a local server:
|
|
<screen>
|
|
psql: could not connect to server: No such file or directory
|
|
Is the server running locally and accepting
|
|
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
The last line is useful in verifying that the client is trying to
|
|
connect to the right place. If there is in fact no server
|
|
running there, the kernel error message will typically be either
|
|
<computeroutput>Connection refused</computeroutput> or
|
|
<computeroutput>No such file or directory</computeroutput>, as
|
|
illustrated. (It is important to realize that
|
|
<computeroutput>Connection refused</computeroutput> in this context
|
|
does <emphasis>not</emphasis> mean that the server got your
|
|
connection request and rejected it. That case will produce a
|
|
different message, as shown in <xref
|
|
linkend="client-authentication-problems">.) Other error messages
|
|
such as <computeroutput>Connection timed out</computeroutput> may
|
|
indicate more fundamental problems, like lack of network
|
|
connectivity.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="kernel-resources">
|
|
<title>Managing Kernel Resources</title>
|
|
|
|
<para>
|
|
A large <productname>PostgreSQL</> installation can quickly exhaust
|
|
various operating system resource limits. (On some systems, the
|
|
factory defaults are so low that you don't even need a really
|
|
<quote>large</> installation.) If you have encountered this kind of
|
|
problem, keep reading.
|
|
</para>
|
|
|
|
<sect2 id="sysvipc">
|
|
<title>Shared Memory and Semaphores</title>
|
|
|
|
<indexterm zone="sysvipc">
|
|
<primary>shared memory</primary>
|
|
</indexterm>
|
|
|
|
<indexterm zone="sysvipc">
|
|
<primary>semaphores</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
Shared memory and semaphores are collectively referred to as
|
|
<quote><systemitem class="osname">System V</>
|
|
<acronym>IPC</></quote> (together with message queues, which are not
|
|
relevant for <productname>PostgreSQL</>). Almost all modern
|
|
operating systems provide these features, but not all of them have
|
|
them turned on or sufficiently sized by default, especially systems
|
|
with BSD heritage. (For the <systemitem class="osname">Windows</>
|
|
port, <productname>PostgreSQL</> provides its own replacement
|
|
implementation of these facilities.)
|
|
</para>
|
|
|
|
<para>
|
|
The complete lack of these facilities is usually manifested by an
|
|
<errorname>Illegal system call</> error upon server start. In
|
|
that case there's nothing left to do but to reconfigure your
|
|
kernel. <productname>PostgreSQL</> won't work without them.
|
|
</para>
|
|
|
|
<para>
|
|
When <productname>PostgreSQL</> exceeds one of the various hard
|
|
<acronym>IPC</> limits, the server will refuse to start and
|
|
should leave an instructive error message describing the problem
|
|
encountered and what to do about it. (See also <xref
|
|
linkend="postmaster-start-failures">.) The relevant kernel
|
|
parameters are named consistently across different systems; <xref
|
|
linkend="sysvipc-parameters"> gives an overview. The methods to set
|
|
them, however, vary. Suggestions for some platforms are given below.
|
|
Be warned that it is often necessary to reboot your machine, and
|
|
possibly even recompile the kernel, to change these settings.
|
|
</para>
|
|
|
|
|
|
<table id="sysvipc-parameters">
|
|
<title><systemitem class="osname">System V</> <acronym>IPC</> parameters</>
|
|
|
|
<tgroup cols="3">
|
|
<thead>
|
|
<row>
|
|
<entry>Name</>
|
|
<entry>Description</>
|
|
<entry>Reasonable values</>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<row>
|
|
<entry><varname>SHMMAX</></>
|
|
<entry>Maximum size of shared memory segment (bytes)</>
|
|
<entry>at least several megabytes (see text)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMMIN</></>
|
|
<entry>Minimum size of shared memory segment (bytes)</>
|
|
<entry>1</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMALL</></>
|
|
<entry>Total amount of shared memory available (bytes or pages)</>
|
|
<entry>if bytes, same as <varname>SHMMAX</varname>; if pages, <literal>ceil(SHMMAX/PAGE_SIZE)</literal></>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMSEG</></>
|
|
<entry>Maximum number of shared memory segments per process</>
|
|
<entry>only 1 segment is needed, but the default is much higher</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMMNI</></>
|
|
<entry>Maximum number of shared memory segments system-wide</>
|
|
<entry>like <varname>SHMSEG</> plus room for other applications</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMNI</></>
|
|
<entry>Maximum number of semaphore identifiers (i.e., sets)</>
|
|
<entry>at least <literal>ceil(max_connections / 16)</literal></>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMNS</></>
|
|
<entry>Maximum number of semaphores system-wide</>
|
|
<entry><literal>ceil(max_connections / 16) * 17</literal> plus room for other applications</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMSL</></>
|
|
<entry>Maximum number of semaphores per set</>
|
|
<entry>at least 17</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMAP</></>
|
|
<entry>Number of entries in semaphore map</>
|
|
<entry>see text</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMVMX</></>
|
|
<entry>Maximum value of semaphore</>
|
|
<entry>at least 1000 (The default is often 32767, don't change unless forced to)</>
|
|
</row>
|
|
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
|
|
|
|
<para>
|
|
<indexterm><primary>SHMMAX</primary></indexterm> The most important
|
|
shared memory parameter is <varname>SHMMAX</>, the maximum size, in
|
|
bytes, of a shared memory segment. If you get an error message from
|
|
<function>shmget</> like <errorname>Invalid argument</>, it is
|
|
likely that this limit has been exceeded. The size of the required
|
|
shared memory segment varies depending on several
|
|
<productname>PostgreSQL</> configuration parameters, as shown in
|
|
<xref linkend="shared-memory-parameters">.
|
|
You can, as a temporary solution, lower some of those settings to
|
|
avoid the failure. As a rough approximation, you can estimate the
|
|
required segment size as 500 kB plus the variable amounts shown in
|
|
the table. (Any error message you might get will include the exact
|
|
size of the failed allocation request.) While it is possible to get
|
|
<productname>PostgreSQL</> to run with <varname>SHMMAX</> as small as
|
|
1 MB, you need at least 4 MB for acceptable performance, and desirable
|
|
settings are in the tens of megabytes.
|
|
</para>
|
|
|
|
<para>
|
|
Some systems also have a limit on the total amount of shared memory in
|
|
the system (<varname>SHMALL</>). Make sure this is large enough
|
|
for <productname>PostgreSQL</> plus any other applications that
|
|
are using shared memory segments. (Caution: <varname>SHMALL</>
|
|
is measured in pages rather than bytes on many systems.)
|
|
</para>
|
|
|
|
<para>
|
|
Less likely to cause problems is the minimum size for shared
|
|
memory segments (<varname>SHMMIN</>), which should be at most
|
|
approximately 500 kB for <productname>PostgreSQL</> (it is
|
|
usually just 1). The maximum number of segments system-wide
|
|
(<varname>SHMMNI</>) or per-process (<varname>SHMSEG</>) are unlikely
|
|
to cause a problem unless your system has them set to zero.
|
|
</para>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> uses one semaphore per allowed connection
|
|
(<xref linkend="guc-max-connections">), in sets of 16. Each such set will
|
|
also contain a 17th semaphore which contains a <quote>magic
|
|
number</quote>, to detect collision with semaphore sets used by
|
|
other applications. The maximum number of semaphores in the system
|
|
is set by <varname>SEMMNS</>, which consequently must be at least
|
|
as high as <varname>max_connections</> plus one extra for each 16
|
|
allowed connections (see the formula in <xref
|
|
linkend="sysvipc-parameters">). The parameter <varname>SEMMNI</>
|
|
determines the limit on the number of semaphore sets that can
|
|
exist on the system at one time. Hence this parameter must be at
|
|
least <literal>ceil(max_connections / 16)</>. Lowering the number
|
|
of allowed connections is a temporary workaround for failures,
|
|
which are usually confusingly worded <errorname>No space
|
|
left on device</>, from the function <function>semget</>.
|
|
</para>
|
|
|
|
<para>
|
|
In some cases it might also be necessary to increase
|
|
<varname>SEMMAP</> to be at least on the order of
|
|
<varname>SEMMNS</>. This parameter defines the size of the semaphore
|
|
resource map, in which each contiguous block of available semaphores
|
|
needs an entry. When a semaphore set is freed it is either added to
|
|
an existing entry that is adjacent to the freed block or it is
|
|
registered under a new map entry. If the map is full, the freed
|
|
semaphores get lost (until reboot). Fragmentation of the semaphore
|
|
space could over time lead to fewer available semaphores than there
|
|
should be.
|
|
</para>
|
|
|
|
<para>
|
|
The <varname>SEMMSL</> parameter, which determines how many
|
|
semaphores can be in a set, must be at least 17 for
|
|
<productname>PostgreSQL</>.
|
|
</para>
|
|
|
|
<para>
|
|
Various other settings related to <quote>semaphore undo</>, such as
|
|
<varname>SEMMNU</> and <varname>SEMUME</>, are not of concern
|
|
for <productname>PostgreSQL</>.
|
|
</para>
|
|
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">BSD/OS</></term>
|
|
<indexterm><primary>BSD/OS</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<formalpara>
|
|
<title>Shared Memory</>
|
|
<para>
|
|
By default, only 4 MB of shared memory is supported. Keep in
|
|
mind that shared memory is not pageable; it is locked in RAM.
|
|
To increase the amount of shared memory supported by your
|
|
system, add something like the following to your kernel configuration
|
|
file:
|
|
<programlisting>
|
|
options "SHMALL=8192"
|
|
options "SHMMAX=\(SHMALL*PAGE_SIZE\)"
|
|
</programlisting>
|
|
<varname>SHMALL</> is measured in 4KB pages, so a value of
|
|
1024 represents 4 MB of shared memory. Therefore the above increases
|
|
the maximum shared memory area to 32 MB.
|
|
For those running 4.3 or later, you will probably also need to increase
|
|
<varname>KERNEL_VIRTUAL_MB</> above the default <literal>248</>.
|
|
Once all changes have been made, recompile the kernel, and reboot.
|
|
</para>
|
|
</formalpara>
|
|
|
|
<para>
|
|
For those running 4.0 and earlier releases, use <command>bpatch</>
|
|
to find the <varname>sysptsize</> value in the current
|
|
kernel. This is computed dynamically at boot time.
|
|
<screen>
|
|
$ <userinput>bpatch -r sysptsize</>
|
|
<computeroutput>0x9 = 9</>
|
|
</screen>
|
|
Next, add <varname>SYSPTSIZE</> as a hard-coded value in the
|
|
kernel configuration file. Increase the value you found using
|
|
<command>bpatch</>. Add 1 for every additional 4 MB of
|
|
shared memory you desire.
|
|
<programlisting>
|
|
options "SYSPTSIZE=16"
|
|
</programlisting>
|
|
<varname>sysptsize</> cannot be changed by <command>sysctl</command>.
|
|
</para>
|
|
|
|
<formalpara>
|
|
<title>Semaphores</>
|
|
<para>
|
|
You will probably want to increase the number of semaphores
|
|
as well; the default system total of 60 will only allow about
|
|
50 <productname>PostgreSQL</productname> connections. Set the
|
|
values you want in your kernel configuration file, e.g.:
|
|
<programlisting>
|
|
options "SEMMNI=40"
|
|
options "SEMMNS=240"
|
|
</programlisting>
|
|
</para>
|
|
</formalpara>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">FreeBSD</></term>
|
|
<indexterm><primary>FreeBSD</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings are only suitable for small installations
|
|
(for example, default <varname>SHMMAX</varname> is 32
|
|
MB). Changes can be made via the <command>sysctl</command> or
|
|
<command>loader</command> interfaces. The following
|
|
parameters can be set using <command>sysctl</command>:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmall=32768</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmmax=134217728</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.semmap=256</userinput>
|
|
</screen>
|
|
To have these settings persist over reboots, modify
|
|
<filename>/etc/sysctl.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
The remaining semaphore settings are read-only as far as
|
|
<command>sysctl</command> is concerned, but can be changed
|
|
before boot using the <command>loader</command> prompt:
|
|
<screen>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmni=256</userinput>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmns=512</userinput>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmnu=256</userinput>
|
|
</screen>
|
|
Similarly these can be saved between reboots in
|
|
<filename>/boot/loader.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
You might also want to configure your kernel to lock shared
|
|
memory into RAM and prevent it from being paged out to swap.
|
|
This can be accomplished using the <command>sysctl</command>
|
|
setting <literal>kern.ipc.shm_use_phys</literal>.
|
|
</para>
|
|
|
|
<para>
|
|
<systemitem class="osname">FreeBSD</> versions before 4.0 work like
|
|
<systemitem class="osname">NetBSD</> and <systemitem class="osname">
|
|
OpenBSD</> (see below).
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">NetBSD</></term>
|
|
<term><systemitem class="osname">OpenBSD</></term>
|
|
<indexterm><primary>NetBSD</><secondary>IPC configuration</></>
|
|
<indexterm><primary>OpenBSD</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The options <varname>SYSVSHM</> and <varname>SYSVSEM</> need
|
|
to be enabled when the kernel is compiled. (They are by
|
|
default.) The maximum size of shared memory is determined by
|
|
the option <varname>SHMMAXPGS</> (in pages). The following
|
|
shows an example of how to set the various parameters
|
|
(<systemitem class="osname">OpenBSD</> uses <literal>option</> instead):
|
|
<programlisting>
|
|
options SYSVSHM
|
|
options SHMMAXPGS=4096
|
|
options SHMSEG=256
|
|
|
|
options SYSVSEM
|
|
options SEMMNI=256
|
|
options SEMMNS=512
|
|
options SEMMNU=256
|
|
options SEMMAP=256
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
You might also want to configure your kernel to lock shared
|
|
memory into RAM and prevent it from being paged out to swap.
|
|
This can be accomplished using the <command>sysctl</command>
|
|
setting <literal>kern.ipc.shm_use_phys</literal>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">HP-UX</></term>
|
|
<indexterm><primary>HP-UX</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings tend to suffice for normal installations.
|
|
On <productname>HP-UX</> 10, the factory default for
|
|
<varname>SEMMNS</> is 128, which might be too low for larger
|
|
database sites.
|
|
</para>
|
|
<para>
|
|
<acronym>IPC</> parameters can be set in the <application>System
|
|
Administration Manager</> (<acronym>SAM</>) under
|
|
<menuchoice><guimenu>Kernel
|
|
Configuration</><guimenuitem>Configurable Parameters</></>. Hit
|
|
<guibutton>Create A New Kernel</> when you're done.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Linux</></term>
|
|
<indexterm><primary>Linux</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings are only suitable for small installations
|
|
(the default max segment size is 32 MB). However the remaining
|
|
defaults are quite generously sized, and usually do not require
|
|
changes. The max segment size can be changed via the
|
|
<command>sysctl</command> interface. For example, to allow 128 MB,
|
|
and explicitly set the maximum total shared memory size to 2097152
|
|
pages (the default):
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>sysctl -w kernel.shmmax=134217728</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kernel.shmall=2097152</userinput>
|
|
</screen>
|
|
In addition these settings can be saved between reboots in
|
|
<filename>/etc/sysctl.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
Older distributions may not have the <command>sysctl</command> program,
|
|
but equivalent changes can be made by manipulating the
|
|
<filename>/proc</filename> file system:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>echo 134217728 >/proc/sys/kernel/shmmax</userinput>
|
|
<prompt>$</prompt> <userinput>echo 2097152 >/proc/sys/kernel/shmall</userinput>
|
|
</screen>
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">MacOS X</></term>
|
|
<indexterm><primary>MacOS X</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
In OS X 10.2 and earlier, edit the file
|
|
<filename>/System/Library/StartupItems/SystemTuning/SystemTuning</>
|
|
and change the values in the following commands:
|
|
<programlisting>
|
|
sysctl -w kern.sysv.shmmax
|
|
sysctl -w kern.sysv.shmmin
|
|
sysctl -w kern.sysv.shmmni
|
|
sysctl -w kern.sysv.shmseg
|
|
sysctl -w kern.sysv.shmall
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
In OS X 10.3 and later, these commands have been moved to
|
|
<filename>/etc/rc</> and must be edited there. Note that
|
|
<filename>/etc/rc</> is usually overwritten by OS X updates (such as
|
|
10.3.6 to 10.3.7) so you should expect to have to redo your editing
|
|
after each update.
|
|
</para>
|
|
|
|
<para>
|
|
In OS X 10.3.9 and later, instead of editing <filename>/etc/rc</>
|
|
you may create a file named <filename>/etc/sysctl.conf</>,
|
|
containing variable assignments such as
|
|
<programlisting>
|
|
kern.sysv.shmmax=4194304
|
|
kern.sysv.shmmin=1
|
|
kern.sysv.shmmni=32
|
|
kern.sysv.shmseg=8
|
|
kern.sysv.shmall=1024
|
|
</programlisting>
|
|
This method is better than editing <filename>/etc/rc</> because
|
|
your changes will be preserved across system updates. Note that
|
|
<emphasis>all five</> shared-memory parameters must be set in
|
|
<filename>/etc/sysctl.conf</>, else the values will be ignored.
|
|
</para>
|
|
|
|
<para>
|
|
Beware that recent releases of OS X ignore attempts to set
|
|
<varname>SHMMAX</> to a value that isn't an exact multiple of 4096.
|
|
</para>
|
|
|
|
<para>
|
|
<varname>SHMALL</> is measured in 4KB pages on this platform.
|
|
</para>
|
|
|
|
<para>
|
|
In all OS X versions, you'll need to reboot to make changes in the
|
|
shared memory parameters take effect.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">SCO OpenServer</></term>
|
|
<indexterm><primary>SCO OpenServer</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
In the default configuration, only 512 kB of shared memory per
|
|
segment is allowed. To increase the setting, first change to the
|
|
directory <filename>/etc/conf/cf.d</>. To display the current value of
|
|
<varname>SHMMAX</>, run
|
|
<programlisting>
|
|
./configure -y SHMMAX
|
|
</programlisting>
|
|
To set a new value for <varname>SHMMAX</>, run
|
|
<programlisting>
|
|
./configure SHMMAX=<replaceable>value</>
|
|
</programlisting>
|
|
where <replaceable>value</> is the new value you want to use
|
|
(in bytes). After setting <varname>SHMMAX</>, rebuild the kernel:
|
|
<programlisting>
|
|
./link_unix
|
|
</programlisting>
|
|
and reboot.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">AIX</></term>
|
|
<indexterm><primary>AIX</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
At least as of version 5.1, it should not be necessary to do
|
|
any special configuration for such parameters as
|
|
<varname>SHMMAX</varname>, as it appears this is configured to
|
|
allow all memory to be used as shared memory. That is the
|
|
sort of configuration commonly used for other databases such
|
|
as <application>DB/2</application>.</para>
|
|
|
|
<para> It may, however, be necessary to modify the global
|
|
<command>ulimit</command> information in
|
|
<filename>/etc/security/limits</filename>, as the default hard
|
|
limits for file sizes (<varname>fsize</varname>) and numbers of
|
|
files (<varname>nofiles</varname>) may be too low.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Solaris</></term>
|
|
<indexterm><primary>Solaris</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
At least in version 2.6, the default maximum size of a shared
|
|
memory segments is too low for <productname>PostgreSQL</>. The
|
|
relevant settings can be changed in <filename>/etc/system</>,
|
|
for example:
|
|
<programlisting>
|
|
set shmsys:shminfo_shmmax=0x2000000
|
|
set shmsys:shminfo_shmmin=1
|
|
set shmsys:shminfo_shmmni=256
|
|
set shmsys:shminfo_shmseg=256
|
|
|
|
set semsys:seminfo_semmap=256
|
|
set semsys:seminfo_semmni=512
|
|
set semsys:seminfo_semmns=512
|
|
set semsys:seminfo_semmsl=32
|
|
</programlisting>
|
|
You need to reboot for the changes to take effect.
|
|
</para>
|
|
|
|
<para>
|
|
See also <ulink
|
|
url="http://sunsite.uakom.sk/sunworldonline/swol-09-1997/swol-09-insidesolaris.html"></>
|
|
for information on shared memory under
|
|
<productname>Solaris</>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">UnixWare</></term>
|
|
<indexterm><primary>UnixWare</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
On <productname>UnixWare</> 7, the maximum size for shared
|
|
memory segments is only 512 kB in the default configuration.
|
|
To display the current value of <varname>SHMMAX</>, run
|
|
<programlisting>
|
|
/etc/conf/bin/idtune -g SHMMAX
|
|
</programlisting>
|
|
which displays the current, default, minimum, and maximum
|
|
values. To set a new value for <varname>SHMMAX</>,
|
|
run
|
|
<programlisting>
|
|
/etc/conf/bin/idtune SHMMAX <replaceable>value</>
|
|
</programlisting>
|
|
where <replaceable>value</> is the new value you want to use
|
|
(in bytes). After setting <varname>SHMMAX</>, rebuild the
|
|
kernel:
|
|
<programlisting>
|
|
/etc/conf/bin/idbuild -B
|
|
</programlisting>
|
|
and reboot.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
|
|
<table id="shared-memory-parameters">
|
|
<title>Configuration parameters affecting
|
|
<productname>PostgreSQL</productname>'s shared memory usage</>
|
|
|
|
<tgroup cols="2">
|
|
<thead>
|
|
<row>
|
|
<entry>Name</>
|
|
<entry>Approximate multiplier (bytes per increment)</>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<row>
|
|
<entry><xref linkend="guc-max-connections"></>
|
|
<entry>400 + 270 * <xref linkend="guc-max-locks-per-transaction"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-max-prepared-transactions"></>
|
|
<entry>600 + 270 * <xref linkend="guc-max-locks-per-transaction"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-shared-buffers"></>
|
|
<entry>8300 (assuming 8K <symbol>BLCKSZ</>)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-wal-buffers"></>
|
|
<entry>8200 (assuming 8K <symbol>XLOG_BLCKSZ</>)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-max-fsm-relations"></>
|
|
<entry>70</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><xref linkend="guc-max-fsm-pages"></>
|
|
<entry>6</>
|
|
</row>
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
</sect2>
|
|
|
|
|
|
<sect2>
|
|
<title>Resource Limits</title>
|
|
|
|
<para>
|
|
Unix-like operating systems enforce various kinds of resource limits
|
|
that might interfere with the operation of your
|
|
<productname>PostgreSQL</productname> server. Of particular
|
|
importance are limits on the number of processes per user, the
|
|
number of open files per process, and the amount of memory available
|
|
to each process. Each of these have a <quote>hard</quote> and a
|
|
<quote>soft</quote> limit. The soft limit is what actually counts
|
|
but it can be changed by the user up to the hard limit. The hard
|
|
limit can only be changed by the root user. The system call
|
|
<function>setrlimit</function> is responsible for setting these
|
|
parameters. The shell's built-in command <command>ulimit</command>
|
|
(Bourne shells) or <command>limit</command> (<application>csh</>) is
|
|
used to control the resource limits from the command line. On
|
|
BSD-derived systems the file <filename>/etc/login.conf</filename>
|
|
controls the various resource limits set during login. See the
|
|
operating system documentation for details. The relevant
|
|
parameters are <varname>maxproc</varname>,
|
|
<varname>openfiles</varname>, and <varname>datasize</varname>. For
|
|
example:
|
|
<programlisting>
|
|
default:\
|
|
...
|
|
:datasize-cur=256M:\
|
|
:maxproc-cur=256:\
|
|
:openfiles-cur=256:\
|
|
...
|
|
</programlisting>
|
|
(<literal>-cur</literal> is the soft limit. Append
|
|
<literal>-max</literal> to set the hard limit.)
|
|
</para>
|
|
|
|
<para>
|
|
Kernels can also have system-wide limits on some resources.
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
On <productname>Linux</productname>
|
|
<filename>/proc/sys/fs/file-max</filename> determines the
|
|
maximum number of open files that the kernel will support. It can
|
|
be changed by writing a different number into the file or by
|
|
adding an assignment in <filename>/etc/sysctl.conf</filename>.
|
|
The maximum limit of files per process is fixed at the time the
|
|
kernel is compiled; see
|
|
<filename>/usr/src/linux/Documentation/proc.txt</filename> for
|
|
more information.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
The <productname>PostgreSQL</productname> server uses one process
|
|
per connection so you should provide for at least as many processes
|
|
as allowed connections, in addition to what you need for the rest
|
|
of your system. This is usually not a problem but if you run
|
|
several servers on one machine things might get tight.
|
|
</para>
|
|
|
|
<para>
|
|
The factory default limit on open files is often set to
|
|
<quote>socially friendly</quote> values that allow many users to
|
|
coexist on a machine without using an inappropriate fraction of
|
|
the system resources. If you run many servers on a machine this
|
|
is perhaps what you want, but on dedicated servers you may want to
|
|
raise this limit.
|
|
</para>
|
|
|
|
<para>
|
|
On the other side of the coin, some systems allow individual
|
|
processes to open large numbers of files; if more than a few
|
|
processes do so then the system-wide limit can easily be exceeded.
|
|
If you find this happening, and you do not want to alter the
|
|
system-wide limit, you can set <productname>PostgreSQL</>'s <xref
|
|
linkend="guc-max-files-per-process"> configuration parameter to
|
|
limit the consumption of open files.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Linux Memory Overcommit</title>
|
|
|
|
<para>
|
|
In Linux 2.4 and later, the default virtual memory behavior is not
|
|
optimal for <productname>PostgreSQL</productname>. Because of the
|
|
way that the kernel implements memory overcommit, the kernel may
|
|
terminate the <productname>PostgreSQL</productname> server (the
|
|
<filename>postmaster</filename> process) if the memory demands of
|
|
another process cause the system to run out of virtual memory.
|
|
</para>
|
|
|
|
<para>
|
|
If this happens, you will see a kernel message that looks like
|
|
this (consult your system documentation and configuration on where
|
|
to look for such a message):
|
|
<programlisting>
|
|
Out of Memory: Killed process 12345 (postmaster).
|
|
</programlisting>
|
|
This indicates that the <filename>postmaster</filename> process
|
|
has been terminated due to memory pressure.
|
|
Although existing database connections will continue to function
|
|
normally, no new connections will be accepted. To recover,
|
|
<productname>PostgreSQL</productname> will need to be restarted.
|
|
</para>
|
|
|
|
<para>
|
|
One way to avoid this problem is to run
|
|
<productname>PostgreSQL</productname>
|
|
on a machine where you can be sure that other processes will not
|
|
run the machine out of memory.
|
|
</para>
|
|
|
|
<para>
|
|
On Linux 2.6 and later, a better solution is to modify the kernel's
|
|
behavior so that it will not <quote>overcommit</> memory. This is
|
|
done by selecting strict overcommit mode via <command>sysctl</command>:
|
|
<programlisting>
|
|
sysctl -w vm.overcommit_memory=2
|
|
</programlisting>
|
|
or placing an equivalent entry in <filename>/etc/sysctl.conf</>.
|
|
You may also wish to modify the related setting
|
|
<literal>vm.overcommit_ratio</>. For details see the kernel documentation
|
|
file <filename>Documentation/vm/overcommit-accounting</>.
|
|
</para>
|
|
|
|
<para>
|
|
Some vendors' Linux 2.4 kernels are reported to have early versions
|
|
of the 2.6 overcommit <command>sysctl</command> parameter. However, setting
|
|
<literal>vm.overcommit_memory</> to 2
|
|
on a kernel that does not have the relevant code will make
|
|
things worse not better. It is recommended that you inspect
|
|
the actual kernel source code (see the function
|
|
<function>vm_enough_memory</> in the file <filename>mm/mmap.c</>)
|
|
to verify what is supported in your copy before you try this in a 2.4
|
|
installation. The presence of the <filename>overcommit-accounting</>
|
|
documentation file should <emphasis>not</> be taken as evidence that the
|
|
feature is there. If in any doubt, consult a kernel expert or your
|
|
kernel vendor.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
|
|
<sect1 id="postmaster-shutdown">
|
|
<title>Shutting Down the Server</title>
|
|
|
|
<indexterm zone="postmaster-shutdown">
|
|
<primary>shutdown</>
|
|
</indexterm>
|
|
|
|
<para>
|
|
There are several ways to shut down the database server. You control
|
|
the type of shutdown by sending different signals to the
|
|
<command>postmaster</command> process.
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><systemitem>SIGTERM</systemitem><indexterm><primary>SIGTERM</></></term>
|
|
<listitem>
|
|
<para>
|
|
After receiving <systemitem>SIGTERM</systemitem>, the server
|
|
disallows new connections, but lets existing sessions end their
|
|
work normally. It shuts down only after all of the sessions
|
|
terminate normally. This is the <firstterm>Smart
|
|
Shutdown</firstterm>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem>SIGINT</systemitem><indexterm><primary>SIGINT</></></term>
|
|
<listitem>
|
|
<para>
|
|
The server disallows new connections and sends all existing
|
|
server processes <systemitem>SIGTERM</systemitem>, which will cause them
|
|
to abort their current transactions and exit promptly. It then
|
|
waits for the server processes to exit and finally shuts down. This is the
|
|
<firstterm>Fast Shutdown</firstterm>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem>SIGQUIT</systemitem><indexterm><primary>SIGQUIT</></></term>
|
|
<listitem>
|
|
<para>
|
|
This is the <firstterm>Immediate Shutdown</firstterm>, which
|
|
will cause the <command>postmaster</command> process to send a
|
|
<systemitem>SIGQUIT</systemitem> to all child processes and exit
|
|
immediately, without properly shutting itself down. The child processes
|
|
likewise exit immediately upon receiving
|
|
<systemitem>SIGQUIT</systemitem>. This will lead to recovery (by
|
|
replaying the WAL log) upon next start-up. This is recommended
|
|
only in emergencies.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
|
|
<para>
|
|
The <xref linkend="app-pg-ctl"> program provides a convenient
|
|
interface for sending these signals to shut down the server.
|
|
</para>
|
|
|
|
<para>
|
|
Alternatively, you can send the signal directly using <command>kill</>.
|
|
The <acronym>PID</> of the <command>postmaster</command> process can be
|
|
found using the <command>ps</command> program, or from the file
|
|
<filename>postmaster.pid</filename> in the data directory. For
|
|
example, to do a fast shutdown:
|
|
<screen>
|
|
$ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<important>
|
|
<para>
|
|
It is best not to use <systemitem>SIGKILL</systemitem> to shut down
|
|
the server. Doing so will prevent the server from releasing
|
|
shared memory and semaphores, which may then have to be done
|
|
manually before a new server can be started. Furthermore,
|
|
<systemitem>SIGKILL</systemitem> kills the <command>postmaster</command>
|
|
process without letting it relay the signal to its subprocesses,
|
|
so it will be necessary to kill the individual subprocesses by hand as
|
|
well.
|
|
</para>
|
|
</important>
|
|
</sect1>
|
|
|
|
<sect1 id="encryption-options">
|
|
<title>Encryption Options</title>
|
|
|
|
<indexterm zone="encryption-options">
|
|
<primary>encryption</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</productname> offers encryption at several
|
|
levels, and provides flexibility in protecting data from disclosure
|
|
due to database server theft, unscrupulous administrators, and
|
|
insecure networks. Encryption might also be required to secure
|
|
sensitive data such as medical records or financial transactions.
|
|
</para>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term>Password Storage Encryption</term>
|
|
<listitem>
|
|
|
|
<para>
|
|
By default, database user passwords are stored as MD5 hashes, so
|
|
the administrator cannot determine the actual password assigned
|
|
to the user. If MD5 encryption is used for client authentication,
|
|
the unencrypted password is never even temporarily present on the
|
|
server because the client MD5 encrypts it before being sent
|
|
across the network.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encryption For Specific Columns</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
The <filename>/contrib</> function library
|
|
<function>pgcrypto</function> allows certain fields to be stored
|
|
encrypted. This is useful if only some of the data is sensitive.
|
|
The client supplies the decryption key and the data is decrypted
|
|
on the server and then sent to the client.
|
|
</para>
|
|
|
|
<para>
|
|
The decrypted data and the decryption key are present on the
|
|
server for a brief time while it is being decrypted and
|
|
communicated between the client and server. This presents a brief
|
|
moment where the data and keys can be intercepted by someone with
|
|
complete access to the database server, such as the system
|
|
administrator.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Data Partition Encryption</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
On Linux, encryption can be layered on top of a file system mount
|
|
using a <quote>loopback device</quote>. This allows an entire
|
|
file system partition be encrypted on disk, and decrypted by the
|
|
operating system. On FreeBSD, the equivalent facility is called
|
|
GEOM Based Disk Encryption, or <acronym>gbde</acronym>.
|
|
</para>
|
|
|
|
<para>
|
|
This mechanism prevents unencrypted data from being read from the
|
|
drives if the drives or the entire computer is stolen. This does
|
|
not protect against attacks while the file system is mounted,
|
|
because when mounted, the operating system provides an unencrypted
|
|
view of the data. However, to mount the file system, you need some
|
|
way for the encryption key to be passed to the operating system,
|
|
and sometimes the key is stored somewhere on the host that mounts
|
|
the disk.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encrypting Passwords Across A Network</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
The <literal>MD5</> authentication method double-encrypts the
|
|
password on the client before sending it to the server. It first
|
|
MD5 encrypts it based on the user name, and then encrypts it
|
|
based on a random salt sent by the server when the database
|
|
connection was made. It is this double-encrypted value that is
|
|
sent over the network to the server. Double-encryption not only
|
|
prevents the password from being discovered, it also prevents
|
|
another connection from using the same encrypted password to
|
|
connect to the database server at a later time.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encrypting Data Across A Network</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
SSL connections encrypt all data sent across the network: the
|
|
password, the queries, and the data returned. The
|
|
<filename>pg_hba.conf</> file allows administrators to specify
|
|
which hosts can use non-encrypted connections (<literal>host</>)
|
|
and which require SSL-encrypted connections
|
|
(<literal>hostssl</>). Also, clients can specify that they
|
|
connect to servers only via SSL. <application>Stunnel</> or
|
|
<application>SSH</> can also be used to encrypt transmissions.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>SSL Host Authentication</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
It is possible for both the client and server to provide SSL keys
|
|
or certificates to each other. It takes some extra configuration
|
|
on each side, but this provides stronger verification of identity
|
|
than the mere use of passwords. It prevents a computer from
|
|
pretending to be the server just long enough to read the password
|
|
send by the client. It also helps prevent "man in the middle"
|
|
attacks where a computer between the client and server pretends to
|
|
be the server and reads and passes all data between the client and
|
|
server.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Client-Side Encryption</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
If the system administrator cannot be trusted, it is necessary
|
|
for the client to encrypt the data; this way, unencrypted data
|
|
never appears on the database server. Data is encrypted on the
|
|
client before being sent to the server, and database results have
|
|
to be decrypted on the client before being used.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="ssl-tcp">
|
|
<title>Secure TCP/IP Connections with SSL</title>
|
|
|
|
<indexterm zone="ssl-tcp">
|
|
<primary>SSL</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> has native support for using
|
|
<acronym>SSL</> connections to encrypt client/server communications
|
|
for increased security. This requires that
|
|
<productname>OpenSSL</productname> is installed on both client and
|
|
server systems and that support in <productname>PostgreSQL</> is
|
|
enabled at build time (see <xref linkend="installation">).
|
|
</para>
|
|
|
|
<para>
|
|
With <acronym>SSL</> support compiled in, the
|
|
<productname>PostgreSQL</> server can be started with
|
|
<acronym>SSL</> enabled by setting the parameter
|
|
<xref linkend="guc-ssl"> to <literal>on</> in
|
|
<filename>postgresql.conf</>. When
|
|
starting in <acronym>SSL</> mode, the server will look for the
|
|
files <filename>server.key</> and <filename>server.crt</> in the
|
|
data directory, which must contain the server private key
|
|
and certificate, respectively. These files must be set up correctly
|
|
before an <acronym>SSL</>-enabled server can start. If the private key is
|
|
protected with a passphrase, the server will prompt for the
|
|
passphrase and will not start until it has been entered.
|
|
</para>
|
|
|
|
<para>
|
|
The server will listen for both standard and <acronym>SSL</>
|
|
connections on the same TCP port, and will negotiate with any
|
|
connecting client on whether to use <acronym>SSL</>. By default,
|
|
this is at the client's option; see <xref
|
|
linkend="auth-pg-hba-conf"> about how to set up the server to
|
|
require use of <acronym>SSL</> for some or all connections.
|
|
</para>
|
|
|
|
<para>
|
|
For details on how to create your server private key and certificate,
|
|
refer to the <productname>OpenSSL</> documentation. A
|
|
self-signed certificate can be used for testing, but a
|
|
certificate signed by a certificate authority (<acronym>CA</>)
|
|
(either one of the global
|
|
<acronym>CAs</> or a local one) should be used in production so the
|
|
client can verify the server's identity. To create a quick
|
|
self-signed certificate, use the following
|
|
<productname>OpenSSL</productname> command:
|
|
<programlisting>
|
|
openssl req -new -text -out server.req
|
|
</programlisting>
|
|
Fill out the information that <command>openssl</> asks for. Make sure
|
|
that you enter the local host name as <quote>Common Name</>; the challenge
|
|
password can be left blank. The program will generate a key that is
|
|
passphrase protected; it will not accept a passphrase that is less
|
|
than four characters long. To remove the passphrase (as you must if
|
|
you want automatic start-up of the server), run the commands
|
|
<programlisting>
|
|
openssl rsa -in privkey.pem -out server.key
|
|
rm privkey.pem
|
|
</programlisting>
|
|
Enter the old passphrase to unlock the existing key. Now do
|
|
<programlisting>
|
|
openssl req -x509 -in server.req -text -key server.key -out server.crt
|
|
chmod og-rwx server.key
|
|
</programlisting>
|
|
to turn the certificate into a self-signed certificate and to copy the
|
|
key and certificate to where the server will look for them.
|
|
</para>
|
|
|
|
<para>
|
|
If verification of client certificates is required, place the
|
|
certificates of the <acronym>CA</acronym>(s) you wish to check for in
|
|
the file <filename>root.crt</filename> in the data directory. When
|
|
present, a client certificate will be requested from the client
|
|
during SSL connection startup, and it must have been signed by one of the
|
|
certificates present in <filename>root.crt</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
When the <filename>root.crt</filename> file is not present, client
|
|
certificates will not be requested or checked. In this mode, SSL
|
|
provides communication security but not authentication.
|
|
</para>
|
|
|
|
<para>
|
|
The files <filename>server.key</>, <filename>server.crt</>,
|
|
and <filename>root.crt</filename> are only examined during server
|
|
start; so you must restart the server to make changes in them take
|
|
effect.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="ssh-tunnels">
|
|
<title>Secure TCP/IP Connections with <application>SSH</application> Tunnels</title>
|
|
|
|
<indexterm zone="ssh-tunnels">
|
|
<primary>ssh</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
One can use <application>SSH</application> to encrypt the network
|
|
connection between clients and a
|
|
<productname>PostgreSQL</productname> server. Done properly, this
|
|
provides an adequately secure network connection, even for non-SSL-capable
|
|
clients.
|
|
</para>
|
|
|
|
<para>
|
|
First make sure that an <application>SSH</application> server is
|
|
running properly on the same machine as the
|
|
<productname>PostgreSQL</productname> server and that you can log in using
|
|
<command>ssh</command> as some user. Then you can establish a secure
|
|
tunnel with a command like this from the client machine:
|
|
<programlisting>
|
|
ssh -L 3333:foo.com:5432 joe@foo.com
|
|
</programlisting>
|
|
The first number in the <option>-L</option> argument, 3333, is the
|
|
port number of your end of the tunnel; it can be chosen freely. The
|
|
second number, 5432, is the remote end of the tunnel: the port
|
|
number your server is using. The name or IP address between
|
|
the port numbers is the host with the database server you are going
|
|
to connect to. In order to connect to the database server using
|
|
this tunnel, you connect to port 3333 on the local machine:
|
|
<programlisting>
|
|
psql -h localhost -p 3333 postgres
|
|
</programlisting>
|
|
To the database server it will then look as though you are really
|
|
user <literal>joe@foo.com</literal> and it will use whatever
|
|
authentication procedure was configured for connections from this
|
|
user and host. Note that the server will not think the connection is
|
|
SSL-encrypted, since in fact it is not encrypted between the
|
|
<application>SSH</application> server and the
|
|
<productname>PostgreSQL</productname> server. This should not pose any
|
|
extra security risk as long as they are on the same machine.
|
|
</para>
|
|
<para>
|
|
In order for the
|
|
tunnel setup to succeed you must be allowed to connect via
|
|
<command>ssh</command> as <literal>joe@foo.com</literal>, just
|
|
as if you had attempted to use <command>ssh</command> to set up a
|
|
terminal session.
|
|
</para>
|
|
|
|
<tip>
|
|
<para>
|
|
Several other applications exist that can provide secure tunnels using
|
|
a procedure similar in concept to the one just described.
|
|
</para>
|
|
</tip>
|
|
|
|
</sect1>
|
|
|
|
</chapter>
|
|
|
|
<!-- Keep this comment at the end of the file
|
|
Local variables:
|
|
mode:sgml
|
|
sgml-omittag:nil
|
|
sgml-shorttag:t
|
|
sgml-minimize-attributes:nil
|
|
sgml-always-quote-attributes:t
|
|
sgml-indent-step:1
|
|
sgml-indent-data:t
|
|
sgml-parent-document:nil
|
|
sgml-default-dtd-file:"./reference.ced"
|
|
sgml-exposed-tags:nil
|
|
sgml-local-catalogs:("/usr/lib/sgml/catalog")
|
|
sgml-local-ecat-files:nil
|
|
End:
|
|
-->
|