From 851e85b42671d08b367cdcbf5ce44a114cc76367 Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Sun, 5 Nov 2017 23:09:14 +0100 Subject: [PATCH] build_usage --- docs/usage/create.rst.inc | 8 +++-- docs/usage/delete.rst.inc | 7 +++- docs/usage/diff.rst.inc | 71 ++++++++++++++++--------------------- docs/usage/help.rst.inc | 6 ++-- docs/usage/list.rst.inc | 9 ----- docs/usage/prune.rst.inc | 7 +++- docs/usage/recreate.rst.inc | 13 ++++++- docs/usage/upgrade.rst.inc | 14 ++++---- 8 files changed, 71 insertions(+), 64 deletions(-) diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc index 1ef6c23b5..1b6d22167 100644 --- a/docs/usage/create.rst.inc +++ b/docs/usage/create.rst.inc @@ -212,6 +212,11 @@ The ``--progress`` option shows (from left to right) Original, Compressed and De (O, C and D, respectively), then the Number of files (N) processed so far, followed by the currently processed path. +When using ``--stats``, you will get some statistics about how much data was +added - the "This Archive" deduplicated size there is most interesting as that is +how much your repository will grow. +Please note that the "All archives" stats refer to the state after creation. + See the output of the "borg help patterns" command for more help on exclude patterns. See the output of the "borg help placeholders" command for more help on placeholders. @@ -231,9 +236,6 @@ only include the objects specified by ``--exclude-if-present`` in your backup, and not include any other contents of the containing folder, this can be enabled through using the ``--keep-exclude-tags`` option. -Borg respects the nodump flag. Files flagged nodump will be marked as excluded (x) -in ``--list`` output. - Item flags ++++++++++ diff --git a/docs/usage/delete.rst.inc b/docs/usage/delete.rst.inc index 93a3ec3cb..5951c71a2 100644 --- a/docs/usage/delete.rst.inc +++ b/docs/usage/delete.rst.inc @@ -85,4 +85,9 @@ Description This command deletes an archive from the repository or the complete repository. Disk space is reclaimed accordingly. If you delete the complete repository, the -local cache for it (if any) is also deleted. \ No newline at end of file +local cache for it (if any) is also deleted. + +When using ``--stats``, you will get some statistics about how much data was +deleted - the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after deletion. \ No newline at end of file diff --git a/docs/usage/diff.rst.inc b/docs/usage/diff.rst.inc index ef5b831a6..5074a0993 100644 --- a/docs/usage/diff.rst.inc +++ b/docs/usage/diff.rst.inc @@ -12,43 +12,37 @@ borg diff .. class:: borg-options-table - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **positional arguments** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``REPO_ARCHIVE1`` | repository location and ARCHIVE1 name | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **optional arguments** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--numeric-owner`` | only consider numeric user and group identifiers | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--same-chunker-params`` | Override check of chunker parameters. | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--sort`` | Sort the output lines by file path. | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | .. class:: borg-common-opt-ref | - | | - | :ref:`common_options` | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **Exclusion options** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-if-present NAME`` | exclude directories that are tagged by containing a filesystem object with the given NAME | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``REPO_ARCHIVE1`` | repository location and ARCHIVE1 name | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **optional arguments** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--numeric-owner`` | only consider numeric user and group identifiers | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--same-chunker-params`` | Override check of chunker parameters. | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--sort`` | Sort the output lines by file path. | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **Exclusion options** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ .. raw:: html @@ -82,9 +76,6 @@ borg diff --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --pattern PATTERN experimental: include/exclude paths matching PATTERN --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line - --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) - --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME - --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive Description diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc index 152e01b54..c5fb6a71e 100644 --- a/docs/usage/help.rst.inc +++ b/docs/usage/help.rst.inc @@ -124,10 +124,12 @@ Examples:: may specify the backup roots (starting points) and patterns for inclusion/exclusion. A root path starts with the prefix `R`, followed by a path (a plain path, not a file pattern). An include rule starts with the prefix +, an exclude rule starts - with the prefix -, both followed by a pattern. + with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern. Inclusion patterns are useful to include paths that are contained in an excluded path. The first matching pattern is used so if an include pattern matches before - an exclude pattern, the file is backed up. + an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches + a directory, it won't recurse into it and won't discover any potential matches for + include rules below that directory. Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is shell style (`sh:`), so those patterns behave similar to rsync include/exclude diff --git a/docs/usage/list.rst.inc b/docs/usage/list.rst.inc index e6b898884..6cf097665 100644 --- a/docs/usage/list.rst.inc +++ b/docs/usage/list.rst.inc @@ -55,12 +55,6 @@ borg list +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-if-present NAME`` | exclude directories that are tagged by containing a filesystem object with the given NAME | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. raw:: html @@ -101,9 +95,6 @@ borg list --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --pattern PATTERN experimental: include/exclude paths matching PATTERN --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line - --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) - --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME - --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive Description diff --git a/docs/usage/prune.rst.inc b/docs/usage/prune.rst.inc index f98fbab84..81c2d32f9 100644 --- a/docs/usage/prune.rst.inc +++ b/docs/usage/prune.rst.inc @@ -134,4 +134,9 @@ negative number of archives to keep means that there is no limit. The ``--keep-last N`` option is doing the same as ``--keep-secondly N`` (and it will keep the last N archives under the assumption that you do not create more than one -backup archive in the same second). \ No newline at end of file +backup archive in the same second). + +When using ``--stats``, you will get some statistics about how much data was +deleted - the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after pruning. \ No newline at end of file diff --git a/docs/usage/recreate.rst.inc b/docs/usage/recreate.rst.inc index ef258e694..b3071ab3d 100644 --- a/docs/usage/recreate.rst.inc +++ b/docs/usage/recreate.rst.inc @@ -150,4 +150,15 @@ With ``--target`` the original archive is not replaced, instead a new archive is When rechunking space usage can be substantial, expect at least the entire deduplicated size of the archives using the previous chunker params. When recompressing expect approx. (throughput / checkpoint-interval) in space usage, -assuming all chunks are recompressed. \ No newline at end of file +assuming all chunks are recompressed. + +If you recently ran borg check --repair and it had to fix lost chunks with all-zero +replacement chunks, please first run another backup for the same data and re-run +borg check --repair afterwards to heal any archives that had lost chunks which are +still generated from the input data. + +Important: running borg recreate to re-chunk will remove the chunks_healthy +metadata of all items with replacement chunks, so healing will not be possible +any more after re-chunking (it is also unlikely it would ever work: due to the +change of chunking parameters, the missing chunk likely will never be seen again +even if you still have the data that produced it). \ No newline at end of file diff --git a/docs/usage/upgrade.rst.inc b/docs/usage/upgrade.rst.inc index eafa4362a..47a240dd0 100644 --- a/docs/usage/upgrade.rst.inc +++ b/docs/usage/upgrade.rst.inc @@ -130,13 +130,13 @@ make sure the cache files are also removed: borg delete borg -Unless ``--inplace`` is specified, the upgrade process first -creates a backup copy of the repository, in -REPOSITORY.before-upgrade-DATETIME, using hardlinks. This takes -longer than in place upgrades, but is much safer and gives -progress information (as opposed to ``cp -al``). Once you are -satisfied with the conversion, you can safely destroy the -backup copy. +Unless ``--inplace`` is specified, the upgrade process first creates a backup +copy of the repository, in REPOSITORY.before-upgrade-DATETIME, using hardlinks. +This requires that the repository and its parent directory reside on same +filesystem so the hardlink copy can work. +This takes longer than in place upgrades, but is much safer and gives +progress information (as opposed to ``cp -al``). Once you are satisfied +with the conversion, you can safely destroy the backup copy. WARNING: Running the upgrade in place will make the current copy unusable with older version, with no way of going back