diff --git a/docs/deployment/image-backup.rst b/docs/deployment/image-backup.rst index cee40faaa..2224607a5 100644 --- a/docs/deployment/image-backup.rst +++ b/docs/deployment/image-backup.rst @@ -9,8 +9,8 @@ technique makes sure only the modified parts of the file are stored. Borg also h optional simple sparse file support for extract. It is of utmost importancy to pin down the disk you want to backup. -You need to use the SERIAL for that. -Use: +You need to use the SERIAL for that. +Use: .. code-block:: bash @@ -26,10 +26,10 @@ Use: echo "Disk Identifier: $DISK_ID" # Use the following line to perform a borg backup for the full disk: - # borg create --read-special {now} "$DISK_ID" + # borg create --read-special disk-backup "$DISK_ID" # Use the following to perform a borg backup for all partitions of the disk - # borg create --read-special {now} "${PARTITIONS[@]}" + # borg create --read-special partitions-backup "${PARTITIONS[@]}" # Example output: # Partitions of /dev/nvme1n1: @@ -37,8 +37,8 @@ Use: # /dev/nvme1n1p2 # /dev/nvme1n1p3 # Disk Identifier: /dev/nvme1n1 - # borg create --read-special {now} /dev/nvme1n1 - # borg create --read-special {now} /dev/nvme1n1p1 /dev/nvme1n1p2 /dev/nvme1n1p3 + # borg create --read-special disk-backup /dev/nvme1n1 + # borg create --read-special partitions-backup /dev/nvme1n1p1 /dev/nvme1n1p2 /dev/nvme1n1p3 diff --git a/docs/faq.rst b/docs/faq.rst index 00a73b19e..20c1c145d 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -318,16 +318,16 @@ How do I configure different prune policies for different directories? Say you want to prune ``/var/log`` faster than the rest of ``/``. How do we implement that? The answer is to back up to different -archive *names* and then implement different prune policies for -different prefixes. For example, you could have a script that does:: +archive *series* and then implement different prune policies for the +different series. For example, you could have a script that does:: - borg create --exclude var/log main-$(date +%Y-%m-%d) / - borg create logs-$(date +%Y-%m-%d) /var/log + borg create --exclude var/log main / + borg create logs /var/log Then you would have two different prune calls with different policies:: - borg prune --verbose --list -d 30 -a 'sh:main-*' - borg prune --verbose --list -d 7 -a 'sh:logs-*' + borg prune --verbose --list -d 30 main + borg prune --verbose --list -d 7 logs This will keep 7 days of logs and 30 days of everything else. diff --git a/docs/quickstart.rst b/docs/quickstart.rst index 88f867c22..918f059cb 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -167,7 +167,7 @@ backed up and that the ``prune`` command keeps and deletes the correct backups. --exclude 'home/*/.cache/*' \ --exclude 'var/tmp/*' \ \ - '{hostname}-{now}' \ + '{hostname}' \ /etc \ /home \ /root \ @@ -178,16 +178,16 @@ backed up and that the ``prune`` command keeps and deletes the correct backups. info "Pruning repository" # Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly - # archives of THIS machine. The '{hostname}-*' globbing is very important to - # limit prune's operation to this machine's archives and not apply to - # other machines' archives also: + # archives of THIS machine. The '{hostname}' matching is very important to + # limit prune's operation to archives with exactly that name and not apply + # to archives with other names also: - borg prune \ - --list \ - --match-archives 'sh:{hostname}-*' \ - --show-rc \ - --keep-daily 7 \ - --keep-weekly 4 \ + borg prune \ + '{hostname}' \ + --list \ + --show-rc \ + --keep-daily 7 \ + --keep-weekly 4 \ --keep-monthly 6 prune_exit=$? @@ -196,7 +196,7 @@ backed up and that the ``prune`` command keeps and deletes the correct backups. info "Compacting repository" - borg compact + borg compact -v compact_exit=$? @@ -501,11 +501,11 @@ Example with **borg mount**: # open a new, separate terminal (this terminal will be blocked until umount) - # now we find out the archive names we have in the repo: + # now we find out the archive ID of the archive we want to mount: borg repo-list - # mount one archive from a borg repo: - borg mount -a myserver-system-2019-08-11 /mnt/borg + # mount one archive giving its archive ID prefix: + borg mount -a aid:d34db33f /mnt/borg # alternatively, mount all archives from a borg repo (slower): borg mount /mnt/borg @@ -527,17 +527,17 @@ Example with **borg extract**: mkdir borg_restore cd borg_restore - # now we find out the archive names we have in the repo: + # now we find out the archive ID of the archive we want to extract: borg repo-list - # we could find out the archive contents, esp. the path layout: - borg list myserver-system-2019-08-11 + # find out how the paths stored in the the archive look like: + borg list aid:d34db33f # we extract only some specific path (note: no leading / !): - borg extract myserver-system-2019-08-11 path/to/extract + borg extract aid:d34db33f path/to/extract # alternatively, we could fully extract the archive: - borg extract myserver-system-2019-08-11 + borg extract aid:d34db33f # now move the files to the correct place... diff --git a/docs/quickstart_example.rst.inc b/docs/quickstart_example.rst.inc index 03e698967..15c40a50e 100644 --- a/docs/quickstart_example.rst.inc +++ b/docs/quickstart_example.rst.inc @@ -3,13 +3,13 @@ $ borg -r /path/to/repo repo-create --encryption=repokey-aes-ocb 2. Back up the ``~/src`` and ``~/Documents`` directories into an archive called - *Monday*:: + *docs*:: - $ borg -r /path/to/repo create Monday ~/src ~/Documents + $ borg -r /path/to/repo create docs ~/src ~/Documents -3. The next day create a new archive called *Tuesday*:: +3. The next day create a new archive using the same archive name:: - $ borg -r /path/to/repo create --stats Tuesday ~/src ~/Documents + $ borg -r /path/to/repo create --stats docs ~/src ~/Documents This backup will be a lot quicker and a lot smaller since only new, never before seen data is stored. The ``--stats`` option causes Borg to @@ -17,7 +17,7 @@ size (the amount of unique data not shared with other archives):: Repository: /path/to/repo - Archive name: Tuesday + Archive name: docs Archive fingerprint: bcd1b53f9b4991b7afc2b339f851b7ffe3c6d030688936fe4552eccc1877718d Time (start): Sat, 2022-06-25 20:21:43 Time (end): Sat, 2022-06-25 20:21:43 @@ -30,32 +30,30 @@ 4. List all archives in the repository:: $ borg -r /path/to/repo repo-list - Monday Sat, 2022-06-25 20:21:14 [b80e24d2...b179f298] - Tuesday Sat, 2022-06-25 20:21:43 [bcd1b53f...1877718d] + docs Sat, 2022-06-25 20:21:14 [b80e24d2...b179f298] + docs Sat, 2022-06-25 20:21:43 [bcd1b53f...1877718d] -5. List the contents of the *Monday* archive:: +5. List the contents of the first archive:: - $ borg -r /path/to/repo list Monday + $ borg -r /path/to/repo list aid:b80e24d2 drwxr-xr-x user group 0 Mon, 2016-02-15 18:22:30 home/user/Documents -rw-r--r-- user group 7961 Mon, 2016-02-15 18:22:30 home/user/Documents/Important.doc ... -6. Restore the *Monday* archive by extracting the files relative to the current directory:: +6. Restore the first archive by extracting the files relative to the current directory:: - $ borg -r /path/to/repo extract Monday + $ borg -r /path/to/repo extract aid:b80e24d2 -7. Delete the *Monday* archive (please note that this does **not** free repo disk space):: +7. Delete the first archive (please note that this does **not** free repo disk space):: - $ borg -r /path/to/repo delete -a Monday + $ borg -r /path/to/repo delete aid:b80e24d2 - Please note the ``-a`` option here (short for ``--match-archives``) which enables you - to give a pattern to delete multiple archives, like ``-a 'sh:oldcrap-*'``. - You can also combine this with ``--first``, ``--last`` and ``--sort-by``. - Be careful, always first use with ``--dry-run`` and ``--list``! + Be careful if you use an archive NAME (and not an archive ID), that might match multiple archives! + Always first use with ``--dry-run`` and ``--list``! 8. Recover disk space by compacting the segment files in the repo:: - $ borg -r /path/to/repo compact + $ borg -r /path/to/repo compact -v .. Note:: Borg is quiet by default (it defaults to WARNING log level). diff --git a/docs/usage/create.rst b/docs/usage/create.rst index 0bec42ff2..df8235159 100644 --- a/docs/usage/create.rst +++ b/docs/usage/create.rst @@ -23,18 +23,18 @@ Examples # /home//.thumbnails is excluded, not /home/*/*/.thumbnails etc.) $ borg create my-files /home --exclude 'sh:home/*/.thumbnails' - # Backup the root filesystem into an archive named "root-YYYY-MM-DD" + # Backup the root filesystem into an archive named "root-archive" # use zlib compression (good, but slow) - default is lz4 (fast, low compression ratio) - $ borg create -C zlib,6 --one-file-system root-{now:%Y-%m-%d} / + $ borg create -C zlib,6 --one-file-system root-archive / - # Backup into an archive name like FQDN-root-TIMESTAMP - $ borg create '{fqdn}-root-{now}' / + # Backup into an archive name like FQDN-root + $ borg create '{fqdn}-root' / # Backup a remote host locally ("pull" style) using sshfs $ mkdir sshfs-mount $ sshfs root@example.com:/ sshfs-mount $ cd sshfs-mount - $ borg create example.com-root-{now:%Y-%m-%d} . + $ borg create example.com-root . $ cd .. $ fusermount -u sshfs-mount @@ -63,17 +63,13 @@ Examples # Only compress compressible data with lzma,N (N = 0..9) $ borg create --compression auto,lzma,N arch ~ - # Use short hostname, user name and current time in archive name - $ borg create '{hostname}-{user}-{now}' ~ - # Similar, use the same datetime format that is default as of borg 1.1 - $ borg create '{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S}' ~ - # As above, but add nanoseconds - $ borg create '{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f}' ~ + # Use short hostname and user name as archive name + $ borg create '{hostname}-{user}' ~ # Backing up relative paths by moving into the correct directory first $ cd /home/user/Documents # The root directory of the archive will be "projectA" - $ borg create 'daily-projectA-{now:%Y-%m-%d}' projectA + $ borg create 'daily-projectA' projectA # Use external command to determine files to archive # Use --paths-from-stdin with find to back up only files less than 1MB in size diff --git a/docs/usage/delete.rst b/docs/usage/delete.rst index 92b9c7d2f..8d35fe3a3 100644 --- a/docs/usage/delete.rst +++ b/docs/usage/delete.rst @@ -4,11 +4,14 @@ Examples ~~~~~~~~ :: - # delete a single backup archive: - $ borg delete -a Monday + # delete all backup archives named "kenny-files": + $ borg delete -a kenny-files # actually free disk space: $ borg compact + # delete a specific backup archive using its unique archive ID prefix + $ borg delete aid:d34db33f + # delete all archives whose names begin with the machine's hostname followed by "-" $ borg delete -a 'sh:{hostname}-*' diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc index 4833db2dc..f96396b61 100644 --- a/docs/usage/help.rst.inc +++ b/docs/usage/help.rst.inc @@ -342,9 +342,8 @@ If literal curly braces need to be used, double them for escaping:: Examples:: - borg create /path/to/repo::{hostname}-{user}-{utcnow} ... - borg create /path/to/repo::{hostname}-{now:%Y-%m-%d_%H:%M:%S%z} ... - borg prune -a 'sh:{hostname}-*' ... + borg create '{hostname}-{user}' ... + borg prune '{hostname}-{user}' ... .. note:: systemd uses a difficult, non-standard syntax for command lines in unit files (refer to diff --git a/docs/usage/info.rst b/docs/usage/info.rst index b248aaeeb..22f1d97d2 100644 --- a/docs/usage/info.rst +++ b/docs/usage/info.rst @@ -4,8 +4,8 @@ Examples ~~~~~~~~ :: - $ borg info Tuesday2022-06-25T20:51:39 - Archive name: Tuesday2022-06-25T20:51:39 + $ borg info aid:f7dea078 + Archive name: source-backup Archive fingerprint: f7dea0788dfc026cc2be1c0f5b94beb4e4084eb3402fc40c38d8719b1bf2d943 Comment: Hostname: mba2020 @@ -13,7 +13,7 @@ Examples Time (start): Sat, 2022-06-25 20:51:40 Time (end): Sat, 2022-06-25 20:51:40 Duration: 0.03 seconds - Command line: /Users/tw/w/borg-env/bin/borg -r path/to/repo create --stats 'Tuesday{now}' src --progress + Command line: /usr/bin/borg -r path/to/repo create source-backup src Utilization of maximum supported archive size: 0% Number of files: 244 Original size: 13.80 MB diff --git a/docs/usage/prune.rst b/docs/usage/prune.rst index 7fd648e83..478498263 100644 --- a/docs/usage/prune.rst +++ b/docs/usage/prune.rst @@ -7,24 +7,34 @@ Be careful, prune is a potentially dangerous command, it will remove backup archives. The default of prune is to apply to **all archives in the repository** unless -you restrict its operation to a subset of the archives using ``-a`` / ``--match-archives``. +you restrict its operation to a subset of the archives. + +The recommended way to name archives (with ``borg create``) is to use the +identical archive name within a series of archives. Then you can simply give +that name to prune also, so it operates just on that series of archives. + +Alternatively, you can use ``-a`` / ``--match-archives`` to do a match on the +archive names to select some of them. When using ``-a``, be careful to choose a good pattern - e.g. do not use a prefix "foo" if you do not also want to match "foobar". It is strongly recommended to always run ``prune -v --list --dry-run ...`` first so you will see what it would do without it actually doing anything. +Don't forget to run ``borg compact -v`` after prune to actually free disk space. + :: # Keep 7 end of day and 4 additional end of week archives. # Do a dry-run without actually deleting anything. $ borg prune -v --list --dry-run --keep-daily=7 --keep-weekly=4 - # Same as above but only apply to archive names starting with the hostname + # Similar as above but only apply to the archive series named '{hostname}': + $ borg prune -v --list --keep-daily=7 --keep-weekly=4 '{hostname}' + + # Similar as above but apply to archive names starting with the hostname # of the machine followed by a "-" character: $ borg prune -v --list --keep-daily=7 --keep-weekly=4 -a 'sh:{hostname}-*' - # actually free disk space: - $ borg compact # Keep 7 end of day, 4 additional end of week archives, # and an end of month archive for every month: diff --git a/docs/usage/tar.rst b/docs/usage/tar.rst index 05070c50d..ee5c0a346 100644 --- a/docs/usage/tar.rst +++ b/docs/usage/tar.rst @@ -34,9 +34,9 @@ Outputs a script that copies all archives from repo1 to repo2: :: - for A T in `borg list --format='{archive} {time:%Y-%m-%dT%H:%M:%S}{NL}'` + for N I T in `borg list --format='{archive} {id} {time:%Y-%m-%dT%H:%M:%S}{NL}'` do - echo "borg -r repo1 export-tar --tar-format=BORG $A - | borg -r repo2 import-tar --timestamp=$T $A -" + echo "borg -r repo1 export-tar --tar-format=BORG aid:$I - | borg -r repo2 import-tar --timestamp=$T $N -" done Kept: diff --git a/src/borg/archiver/create_cmd.py b/src/borg/archiver/create_cmd.py index 166e302be..fe8ba0580 100644 --- a/src/borg/archiver/create_cmd.py +++ b/src/borg/archiver/create_cmd.py @@ -575,7 +575,9 @@ class CreateMixIn: The archive will consume almost no disk space for files or parts of files that have already been stored in other archives. - The archive name needs to be unique. + The archive name does NOT need to be unique, you can and should use the same + name for a series of archives. The unique archive identifier is its ID (hash) + and you can abbreviate the ID as long as it is unique. In the archive name, you may use the following placeholders: {now}, {utcnow}, {fqdn}, {hostname}, {user} and some others. diff --git a/src/borg/archiver/prune_cmd.py b/src/borg/archiver/prune_cmd.py index fc2a13c4f..e68824ad2 100644 --- a/src/borg/archiver/prune_cmd.py +++ b/src/borg/archiver/prune_cmd.py @@ -163,17 +163,18 @@ class PruneMixIn: `GFS `_ (Grandfather-father-son) backup rotation scheme. - If you use --match-archives (-a), then only archives that match the pattern are - considered for deletion and only those archives count towards the totals - specified by the rules. + The recommended way to use prune is to give the archive series name to it via the + NAME argument (assuming you have the same name for all archives in a series). + Alternatively, you can also use --match-archives (-a), then only archives that + match the pattern are considered for deletion and only those archives count + towards the totals specified by the rules. Otherwise, *all* archives in the repository are candidates for deletion! There is no automatic distinction between archives representing different contents. These need to be distinguished by specifying matching globs. - If you have multiple sequences of archives with different data sets (e.g. + If you have multiple series of archives with different data sets (e.g. from different machines) in one shared repository, use one prune call per - data set that matches only the respective archives using the --match-archives - (-a) option. + series. The ``--keep-within`` option takes an argument of the form "", where char is "H", "d", "w", "m", "y". For example, ``--keep-within 2d`` means