diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc index afce6c24d..7fbeb71bd 100644 --- a/docs/usage/create.rst.inc +++ b/docs/usage/create.rst.inc @@ -39,7 +39,7 @@ borg create +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--stdin-user USER`` | set user USER in archive for stdin data (default: 'root') | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--stdin-group GROUP`` | set group GROUP in archive for stdin data (default: 'root') | + | | ``--stdin-group GROUP`` | set group GROUP in archive for stdin data (default: 'wheel') | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--stdin-mode M`` | set mode to M in archive for stdin data (default: 0660) | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -53,9 +53,9 @@ borg create +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.bford.info/cachedir/spec.html) | +-------------------------------------------------------+---------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -128,7 +128,7 @@ borg create --no-files-cache do not load/update the file metadata cache used to detect unchanged files --stdin-name NAME use NAME in archive for stdin data (default: "stdin") --stdin-user USER set user USER in archive for stdin data (default: 'root') - --stdin-group GROUP set group GROUP in archive for stdin data (default: 'root') + --stdin-group GROUP set group GROUP in archive for stdin data (default: 'wheel') --stdin-mode M set mode to M in archive for stdin data (default: 0660) @@ -138,8 +138,8 @@ borg create Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.bford.info/cachedir/spec.html) --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive @@ -241,16 +241,15 @@ how much your repository will grow. Please note that the "All archives" stats re the state after creation. Also, the ``--stats`` and ``--dry-run`` options are mutually exclusive because the data is not actually compressed and deduplicated during a dry run. -See the output of the "borg help patterns" command for more help on exclude patterns. +For more help on include/exclude patterns, see the :ref:`borg_patterns` command output. -See the output of the "borg help placeholders" command for more help on placeholders. +For more help on placeholders, see the :ref:`borg_placeholders` command output. .. man NOTES The ``--exclude`` patterns are not like tar. In tar ``--exclude`` .bundler/gems will exclude foo/.bundler/gems. In borg it will not, you need to use ``--exclude`` -'\*/.bundler/gems' to get the same effect. See ``borg help patterns`` for -more information. +'\*/.bundler/gems' to get the same effect. In addition to using ``--exclude`` patterns, it is possible to use ``--exclude-if-present`` to specify the name of a filesystem object (e.g. a file diff --git a/docs/usage/diff.rst.inc b/docs/usage/diff.rst.inc index c04887443..7ad3b210f 100644 --- a/docs/usage/diff.rst.inc +++ b/docs/usage/diff.rst.inc @@ -12,39 +12,39 @@ borg diff .. class:: borg-options-table - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | **positional arguments** | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``REPO::ARCHIVE1`` | repository location and ARCHIVE1 name | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | **optional arguments** | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--numeric-owner`` | only consider numeric user and group identifiers | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--same-chunker-params`` | Override check of chunker parameters. | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--sort`` | Sort the output lines by file path. | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--json-lines`` | Format output as JSON Lines. | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | .. class:: borg-common-opt-ref | - | | - | :ref:`common_options` | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | **Exclusion options** | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | - +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``REPO::ARCHIVE1`` | repository location and ARCHIVE1 name | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | **optional arguments** | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--numeric-owner`` | only consider numeric user and group identifiers | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--same-chunker-params`` | Override check of chunker parameters. | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--sort`` | Sort the output lines by file path. | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--json-lines`` | Format output as JSON Lines. | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | **Exclusion options** | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+ .. raw:: html @@ -77,8 +77,8 @@ borg diff Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line Description @@ -99,4 +99,4 @@ If you did not create the archives with different chunker params, pass ``--same-chunker-params``. Note that the chunker params changed from Borg 0.xx to 1.0. -See the output of the "borg help patterns" command for more help on exclude patterns. \ No newline at end of file +For more help on include/exclude patterns, see the :ref:`borg_patterns` command output. \ No newline at end of file diff --git a/docs/usage/export-tar.rst.inc b/docs/usage/export-tar.rst.inc index 9fa16e40b..4dd4c93a5 100644 --- a/docs/usage/export-tar.rst.inc +++ b/docs/usage/export-tar.rst.inc @@ -37,9 +37,9 @@ borg export-tar +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ @@ -73,8 +73,8 @@ borg export-tar Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line --strip-components NUMBER Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. @@ -110,7 +110,7 @@ By default the entire archive is extracted but a subset of files and directories can be selected by passing a list of ``PATHs`` as arguments. The file selection can further be restricted by using the ``--exclude`` option. -See the output of the "borg help patterns" command for more help on exclude patterns. +For more help on include/exclude patterns, see the :ref:`borg_patterns` command output. ``--progress`` can be slower than no progress display, since it makes one additional pass over the archive metadata. \ No newline at end of file diff --git a/docs/usage/extract.rst.inc b/docs/usage/extract.rst.inc index 141f68d08..082028e2d 100644 --- a/docs/usage/extract.rst.inc +++ b/docs/usage/extract.rst.inc @@ -47,9 +47,9 @@ borg extract +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ @@ -87,8 +87,8 @@ borg extract Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line --strip-components NUMBER Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. @@ -100,7 +100,7 @@ archive is extracted but a subset of files and directories can be selected by passing a list of ``PATHs`` as arguments. The file selection can further be restricted by using the ``--exclude`` option. -See the output of the "borg help patterns" command for more help on exclude patterns. +For more help on include/exclude patterns, see the :ref:`borg_patterns` command output. By using ``--dry-run``, you can do all extraction steps except actually writing the output data: reading metadata and data chunks from the repo, checking the hash/hmac, @@ -112,4 +112,8 @@ pass over the archive metadata. .. note:: Currently, extract always writes into the current working directory ("."), - so make sure you ``cd`` to the right place before calling ``borg extract``. \ No newline at end of file + so make sure you ``cd`` to the right place before calling ``borg extract``. + + When parent directories are not extracted (because of using file/directory selection + or any other reason), borg can not restore parent directories' metadata, e.g. owner, + group, permission, etc. \ No newline at end of file diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc index 6dea4cbea..f8fd0a69d 100644 --- a/docs/usage/help.rst.inc +++ b/docs/usage/help.rst.inc @@ -24,10 +24,14 @@ patterns when matching archive content in commands like ``extract`` or ``mount``. Starting with Borg 1.2 this behaviour will be changed to accept both absolute and relative paths. +A directory exclusion pattern can end either with or without a slash ('/'). +If it ends with a slash, such as `some/path/`, the directory will be +included but not its content. If it does not end with a slash, such as +`some/path`, both the directory and content will be excluded. + File patterns support these styles: fnmatch, shell, regular expressions, path prefixes and path full-matches. By default, fnmatch is used for -``--exclude`` patterns and shell-style is used for the experimental -``--pattern`` option. +``--exclude`` patterns and shell-style is used for the ``--pattern`` option. If followed by a colon (':') the first two characters of a pattern are used as a style selector. Explicit style selection is necessary when a @@ -140,60 +144,85 @@ Examples:: EOF $ borg create --exclude-from exclude.txt backup / -.. container:: experimental +A more general and easier to use way to define filename matching patterns exists +with the ``--pattern`` and ``--patterns-from`` options. Using these, you may +specify the backup roots (starting points) and patterns for inclusion/exclusion. +A root path starts with the prefix `R`, followed by a path (a plain path, not a +file pattern). An include rule starts with the prefix +, an exclude rule starts +with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern. - A more general and easier to use way to define filename matching patterns exists - with the experimental ``--pattern`` and ``--patterns-from`` options. Using these, you - may specify the backup roots (starting points) and patterns for inclusion/exclusion. - A root path starts with the prefix `R`, followed by a path (a plain path, not a - file pattern). An include rule starts with the prefix +, an exclude rule starts - with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern. +.. note:: - .. note:: + Via ``--pattern`` or ``--patterns-from`` you can define BOTH inclusion and exclusion + of files using pattern prefixes ``+`` and ``-``. With ``--exclude`` and + ``--exclude-from`` ONLY excludes are defined. - Via ``--pattern`` or ``--patterns-from`` you can define BOTH inclusion and exclusion - of files using pattern prefixes ``+`` and ``-``. With ``--exclude`` and - ``--exclude-from`` ONLY excludes are defined. +Inclusion patterns are useful to include paths that are contained in an excluded +path. The first matching pattern is used so if an include pattern matches before +an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches +a directory, it won't recurse into it and won't discover any potential matches for +include rules below that directory. - Inclusion patterns are useful to include paths that are contained in an excluded - path. The first matching pattern is used so if an include pattern matches before - an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches - a directory, it won't recurse into it and won't discover any potential matches for - include rules below that directory. +.. note:: - Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is - shell style (`sh:`), so those patterns behave similar to rsync include/exclude - patterns. The pattern style can be set via the `P` prefix. + It's possible that a sub-directory/file is matched while parent directories are not. + In that case, parent directories are not backed up thus their user, group, permission, + etc. can not be restored. - Patterns (``--pattern``) and excludes (``--exclude``) from the command line are - considered first (in the order of appearance). Then patterns from ``--patterns-from`` - are added. Exclusion patterns from ``--exclude-from`` files are appended last. +Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is +shell style (`sh:`), so those patterns behave similar to rsync include/exclude +patterns. The pattern style can be set via the `P` prefix. - Examples:: +Patterns (``--pattern``) and excludes (``--exclude``) from the command line are +considered first (in the order of appearance). Then patterns from ``--patterns-from`` +are added. Exclusion patterns from ``--exclude-from`` files are appended last. - # backup pics, but not the ones from 2018, except the good ones: - # note: using = is essential to avoid cmdline argument parsing issues. - borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics +Examples:: - # use a file with patterns: - borg create --patterns-from patterns.lst repo::arch + # backup pics, but not the ones from 2018, except the good ones: + # note: using = is essential to avoid cmdline argument parsing issues. + borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics - The patterns.lst file could look like that:: + # use a file with patterns: + borg create --patterns-from patterns.lst repo::arch - # "sh:" pattern style is the default, so the following line is not needed: - P sh - R / - # can be rebuild - - /home/*/.cache - # they're downloads for a reason - - /home/*/Downloads - # susan is a nice person - # include susans home - + /home/susan - # don't backup the other home directories - - /home/* - # don't even look in /proc - ! /proc +The patterns.lst file could look like that:: + + # "sh:" pattern style is the default, so the following line is not needed: + P sh + R / + # can be rebuild + - /home/*/.cache + # they're downloads for a reason + - /home/*/Downloads + # susan is a nice person + # include susans home + + /home/susan + # also back up this exact file + + pf:/home/bobby/specialfile.txt + # don't backup the other home directories + - /home/* + # don't even look in /proc + ! /proc + +You can specify recursion roots either on the command line or in a patternfile:: + + # these two commands do the same thing + borg create --exclude /home/bobby/junk repo::arch /home/bobby /home/susan + borg create --patterns-from patternfile.lst repo::arch + +The patternfile:: + + # note that excludes use fm: by default and patternfiles use sh: by default. + # therefore, we need to specify fm: to have the same exact behavior. + P fm + R /home/bobby + R /home/susan + + - /home/bobby/junk + +This allows you to share the same patterns between multiple repositories +without needing to specify them on the command line. .. _borg_placeholders: diff --git a/docs/usage/init.rst.inc b/docs/usage/init.rst.inc index a3e1e0d73..0e9a7d423 100644 --- a/docs/usage/init.rst.inc +++ b/docs/usage/init.rst.inc @@ -62,31 +62,41 @@ Description This command initializes an empty repository. A repository is a filesystem directory containing the deduplicated data from zero or more archives. -Encryption can be enabled at repository init time. It cannot be changed later. +Encryption mode TLDR +++++++++++++++++++++ -It is not recommended to work without encryption. Repository encryption protects -you e.g. against the case that an attacker has access to your backup repository. +The encryption mode can only be configured when creating a new repository - +you can neither configure it on a per-archive basis nor change the +encryption mode of an existing repository. -Borg relies on randomly generated key material and uses that for chunking, id -generation, encryption and authentication. The key material is encrypted using -the passphrase you give before it is stored on-disk. +Use ``repokey``:: -You need to be careful with the key / the passphrase: + borg init --encryption repokey /path/to/repo -If you want "passphrase-only" security, use one of the repokey modes. The -key will be stored inside the repository (in its "config" file). In above -mentioned attack scenario, the attacker will have the key (but not the -passphrase). +Or ``repokey-blake2`` depending on which is faster on your client machines (see below):: -If you want "passphrase and having-the-key" security, use one of the keyfile -modes. The key will be stored in your home directory (in .config/borg/keys). -In the attack scenario, the attacker who has just access to your repo won't -have the key (and also not the passphrase). + borg init --encryption repokey-blake2 /path/to/repo -Make a backup copy of the key file (keyfile mode) or repo config file -(repokey mode) and keep it at a safe place, so you still have the key in -case it gets corrupted or lost. Also keep the passphrase at a safe place. -The backup that is encrypted with that key won't help you with that, of course. +Borg will: + +1. Ask you to come up with a passphrase. +2. Create a borg key (which contains 3 random secrets. See :ref:`key_files`). +3. Encrypt the key with your passphrase. +4. Store the encrypted borg key inside the repository directory (in the repo config). + This is why it is essential to use a secure passphrase. +5. Encrypt and sign your backups to prevent anyone from reading or forging them unless they + have the key and know the passphrase. Make sure to keep a backup of + your key **outside** the repository - do not lock yourself out by + "leaving your keys inside your car" (see :ref:`borg_key_export`). + For remote backups the encryption is done locally - the remote machine + never sees your passphrase, your unencrypted key or your unencrypted files. + Chunking and id generation are also based on your key to improve + your privacy. +6. Use the key when extracting files to decrypt them and to verify that the contents of + the backups have not been accidentally or maliciously altered. + +Picking a passphrase +++++++++++++++++++++ Make sure you use a good passphrase. Not too short, not too simple. The real encryption / decryption key is encrypted with / locked by your passphrase. @@ -106,14 +116,22 @@ a different keyboard layout. You can change your passphrase for existing repos at any time, it won't affect the encryption/decryption key or other secrets. -Encryption modes -++++++++++++++++ +More encryption modes ++++++++++++++++++++++ -You can choose from the encryption modes seen in the table below on a per-repo -basis. The mode determines encryption algorithm, hash/MAC algorithm and also the -key storage location. +Only use ``--encryption none`` if you are OK with anyone who has access to +your repository being able to read your backups and tamper with their +contents without you noticing. -Example: `borg init --encryption repokey ...` +If you want "passphrase and having-the-key" security, use ``--encryption keyfile``. +The key will be stored in your home directory (in ``~/.config/borg/keys``). + +If you do **not** want to encrypt the contents of your backups, but still +want to detect malicious tampering use ``--encryption authenticated``. + +If ``BLAKE2b`` is faster than ``SHA-256`` on your hardware, use ``--encryption authenticated-blake2``, +``--encryption repokey-blake2`` or ``--encryption keyfile-blake2``. Note: for remote backups +the hashing is done on your local machine. .. nanorst: inline-fill diff --git a/docs/usage/key_export.rst.inc b/docs/usage/key_export.rst.inc index a295138a1..e91b2dbee 100644 --- a/docs/usage/key_export.rst.inc +++ b/docs/usage/key_export.rst.inc @@ -76,4 +76,15 @@ data backup. For repositories using the repokey encryption the key is saved in the repository in the config file. A backup is thus not strictly needed, but guards against the repository becoming inaccessible if the file -is damaged for some reason. \ No newline at end of file +is damaged for some reason. + +Examples:: + + borg key export /path/to/repo > encrypted-key-backup + borg key export --paper /path/to/repo > encrypted-key-backup.txt + borg key export --qr-html /path/to/repo > encrypted-key-backup.html + # Or pass the output file as an argument instead of redirecting stdout: + borg key export /path/to/repo encrypted-key-backup + borg key export --paper /path/to/repo encrypted-key-backup.txt + borg key export --qr-html /path/to/repo encrypted-key-backup.html + diff --git a/docs/usage/list.rst.inc b/docs/usage/list.rst.inc index 02195533b..54736bae0 100644 --- a/docs/usage/list.rst.inc +++ b/docs/usage/list.rst.inc @@ -23,7 +23,7 @@ borg list +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--short`` | only print file/directory names, nothing else | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}") | + | | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file or archive listing (default for files: "{mode} {user:6} {group:6} {size:8} {mtime} {path}{extra}{NL}"; for archives: "{archive:<36} {time} [{id}]{NL}") | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--json`` | Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available. | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -51,9 +51,9 @@ borg list +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. raw:: html @@ -74,7 +74,7 @@ borg list optional arguments --short only print file/directory names, nothing else - --format FORMAT, --list-format FORMAT specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}") + --format FORMAT, --list-format FORMAT specify format for file or archive listing (default for files: "{mode} {user:6} {group:6} {size:8} {mtime} {path}{extra}{NL}"; for archives: "{archive:<36} {time} [{id}]{NL}") --json Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available. --json-lines Only valid for listing archive contents. Format output as JSON Lines. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "bpath" key is therefore not available. @@ -93,8 +93,8 @@ borg list Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line Description @@ -102,21 +102,50 @@ Description This command lists the contents of a repository or an archive. -See the "borg help patterns" command for more help on exclude patterns. +For more help on include/exclude patterns, see the :ref:`borg_patterns` command output. .. man NOTES -The following keys are available for ``--format``: +The FORMAT specifier syntax ++++++++++++++++++++++++++++ + +The ``--format`` option uses python's `format string syntax +`_. + +Examples: +:: + + $ borg list --format '{archive}{NL}' /path/to/repo + ArchiveFoo + ArchiveBar + ... + + # {VAR:NUMBER} - pad to NUMBER columns. + # Strings are left-aligned, numbers are right-aligned. + # Note: time columns except ``isomtime``, ``isoctime`` and ``isoatime`` cannot be padded. + $ borg list --format '{archive:36} {time} [{id}]{NL}' /path/to/repo + ArchiveFoo Thu, 2021-12-09 10:22:28 [0b8e9a312bef3f2f6e2d0fc110c196827786c15eba0188738e81697a7fa3b274] + $ borg list --format '{mode} {user:6} {group:6} {size:8} {mtime} {path}{extra}{NL}' /path/to/repo::ArchiveFoo + -rw-rw-r-- user user 1024 Thu, 2021-12-09 10:22:17 file-foo + ... + + # {VAR:NUMBER} - pad to NUMBER columns right-aligned. + $ borg list --format '{mode} {user:>6} {group:>6} {size:<8} {mtime} {path}{extra}{NL}' /path/to/repo::ArchiveFoo + -rw-rw-r-- user user 1024 Thu, 2021-12-09 10:22:17 file-foo + ... + +The following keys are always available: - NEWLINE: OS dependent line separator - NL: alias of NEWLINE -- NUL: NUL character for creating print0 / xargs -0 like output, see barchive/bpath +- NUL: NUL character for creating print0 / xargs -0 like output, see barchive and bpath keys below - SPACE - TAB - CR - LF -Keys for listing repository archives: +Keys available only when listing archives in a repository: - archive: archive name interpreted as text (might be missing non-text characters, see barchive) - name: alias of "archive" @@ -128,12 +157,13 @@ Keys for listing repository archives: - start: time (start) of creation of the archive - time: alias of "start" - end: time (end) of creation of the archive +- command_line: command line which was used to create the archive - hostname: hostname of host on which this archive was created - username: username of user who created this archive -Keys for listing archive files: +Keys available only when listing files in an archive: - type - mode @@ -173,8 +203,6 @@ Keys for listing archive files: - sha3_384 - sha3_512 - sha512 -- shake_128 -- shake_256 - archiveid - archivename diff --git a/docs/usage/mount.rst.inc b/docs/usage/mount.rst.inc index f813f5583..d1334ad8f 100644 --- a/docs/usage/mount.rst.inc +++ b/docs/usage/mount.rst.inc @@ -49,9 +49,9 @@ borg mount +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -93,8 +93,8 @@ borg mount Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line --strip-components NUMBER Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. @@ -115,6 +115,23 @@ To allow a regular user to use fstab entries, add the ``user`` option: For FUSE configuration and mount options, see the mount.fuse(8) manual page. +Borg's default behavior is to use the archived user and group names of each +file and map them to the system's respective user and group ids. +Alternatively, using ``numeric-ids`` will instead use the archived user and +group ids without any mapping. + +The ``uid`` and ``gid`` mount options (implemented by Borg) can be used to +override the user and group ids of all files (i.e., ``borg mount -o +uid=1000,gid=1000``). + +The man page references ``user_id`` and ``group_id`` mount options +(implemented by fuse) which specify the user and group id of the mount owner +(aka, the user who does the mounting). It is set automatically by libfuse (or +the filesystem if libfuse is not used). However, you should not specify these +manually. Unlike the ``uid`` and ``gid`` mount options which affect all files, +``user_id`` and ``group_id`` affect the user and group id of the mounted +(base) directory. + Additional mount options supported by borg: - versions: when used with a repository mount, this gives a merged, versioned diff --git a/docs/usage/prune.rst.inc b/docs/usage/prune.rst.inc index dc42e18a3..a1312da6d 100644 --- a/docs/usage/prune.rst.inc +++ b/docs/usage/prune.rst.inc @@ -100,6 +100,9 @@ Description The prune command prunes a repository by deleting all archives not matching any of the specified retention options. This command is normally used by automated backup scripts wanting to keep a certain number of historic backups. +This retention policy is commonly referred to as +`GFS `_ +(Grandfather-father-son) backup rotation scheme. Also, prune automatically removes checkpoint archives (incomplete archives left behind by interrupted backup runs) except if the checkpoint is the latest diff --git a/docs/usage/recreate.rst.inc b/docs/usage/recreate.rst.inc index 05dab0e2a..9e345d26a 100644 --- a/docs/usage/recreate.rst.inc +++ b/docs/usage/recreate.rst.inc @@ -39,9 +39,9 @@ borg recreate +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + | | ``--pattern PATTERN`` | include/exclude paths matching PATTERN | +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + | | ``--patterns-from PATTERNFILE`` | read include/exclude patterns from PATTERNFILE, one per line | +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.bford.info/cachedir/spec.html) | +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -95,8 +95,8 @@ borg recreate Exclusion options -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line - --pattern PATTERN experimental: include/exclude paths matching PATTERN - --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --pattern PATTERN include/exclude paths matching PATTERN + --patterns-from PATTERNFILE read include/exclude patterns from PATTERNFILE, one per line --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.bford.info/cachedir/spec.html) --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive