From 821baf05282d072339fdd58534cecd5a76769d9c Mon Sep 17 00:00:00 2001 From: 8bit Date: Tue, 3 Oct 2017 13:21:28 -0400 Subject: [PATCH 1/3] Minor grammar correction and added consistency with subsistitution of 'Borg' --- docs/quickstart.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/quickstart.rst b/docs/quickstart.rst index 597929eac..3030b62f8 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -21,14 +21,14 @@ a good amount of free space on the filesystem that has your backup repository (and also on ~/.cache). A few GB should suffice for most hard-drive sized repositories. See also :ref:`cache-memory-usage`. -Borg doesn't use space reserved for root on repository disks (even when run as root), +|project_name| doesn't use space reserved for root on repository disks (even when run as root), on file systems which do not support this mechanism (e.g. XFS) we recommend to -reserve some space in Borg itself just to be safe by adjusting the +reserve some space in |project_name| itself just to be safe by adjusting the ``additional_free_space`` setting in the ``[repository]`` section of a repositories ``config`` file. A good starting point is ``2G``. If |project_name| runs out of disk space, it tries to free as much space as it -can while aborting the current operation safely, which allows to free more space +can while aborting the current operation safely, which allows it to free more space by deleting/pruning archives. This mechanism is not bullet-proof in some circumstances [1]_. @@ -153,14 +153,14 @@ backed up and that the ``prune`` command is keeping and deleting the correct bac Pitfalls with shell variables and environment variables ------------------------------------------------------- -This applies to all environment variables you want borg to see, not just +This applies to all environment variables you want |project_name| to see, not just ``BORG_PASSPHRASE``. The short explanation is: always ``export`` your variable, and use single quotes if you're unsure of the details of your shell's expansion behavior. E.g.:: export BORG_PASSPHRASE='complicated & long' -This is because ``export`` exposes variables to subprocesses, which borg may be +This is because ``export`` exposes variables to subprocesses, which |project_name| may be one of. More on ``export`` can be found in the "ENVIRONMENT" section of the bash(1) man page. From 8db0c770c32992ee8fa93c7c98661ee2a7bb8c84 Mon Sep 17 00:00:00 2001 From: 8bit Date: Tue, 3 Oct 2017 16:32:58 -0400 Subject: [PATCH 2/3] The user frees space, not borg --- docs/quickstart.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/quickstart.rst b/docs/quickstart.rst index 3030b62f8..a7a1ba0fb 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -28,7 +28,7 @@ reserve some space in |project_name| itself just to be safe by adjusting the ``config`` file. A good starting point is ``2G``. If |project_name| runs out of disk space, it tries to free as much space as it -can while aborting the current operation safely, which allows it to free more space +can while aborting the current operation safely, which allows the user to free more space by deleting/pruning archives. This mechanism is not bullet-proof in some circumstances [1]_. From 8d830d069f1c168e5402d21f95e44c7614663e3e Mon Sep 17 00:00:00 2001 From: 8bit Date: Tue, 17 Oct 2017 11:50:55 -0500 Subject: [PATCH 3/3] Removed all |project_name | instances, replaced with Borg --- docs/development.rst | 4 +-- docs/faq.rst | 48 +++++++++++++++--------------- docs/global.rst.inc | 1 - docs/installation.rst | 18 +++++------ docs/internals/data-structures.rst | 16 +++++----- docs/quickstart.rst | 24 +++++++-------- docs/support.rst | 2 +- 7 files changed, 56 insertions(+), 57 deletions(-) diff --git a/docs/development.rst b/docs/development.rst index 05719df8f..9e06cce8a 100644 --- a/docs/development.rst +++ b/docs/development.rst @@ -5,9 +5,9 @@ Development =========== -This chapter will get you started with |project_name| development. +This chapter will get you started with Borg development. -|project_name| is written in Python (with a little bit of Cython and C for +Borg is written in Python (with a little bit of Cython and C for the performance critical parts). Contributions diff --git a/docs/faq.rst b/docs/faq.rst index e23f45a30..b6015420b 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -12,7 +12,7 @@ Can I backup VM disk images? ---------------------------- Yes, the `deduplication`_ technique used by -|project_name| makes sure only the modified parts of the file are stored. +Borg makes sure only the modified parts of the file are stored. Also, we have optional simple sparse file support for extract. If you use non-snapshotting backup tools like Borg to back up virtual machines, @@ -51,16 +51,16 @@ to start tackling them. Can I backup from multiple servers into a single repository? ------------------------------------------------------------ -Yes, but in order for the deduplication used by |project_name| to work, it +Yes, but in order for the deduplication used by Borg to work, it needs to keep a local cache containing checksums of all file chunks already stored in the repository. This cache is stored in -``~/.cache/borg/``. If |project_name| detects that a repository has been +``~/.cache/borg/``. If Borg detects that a repository has been modified since the local cache was updated it will need to rebuild the cache. This rebuild can be quite time consuming. So, yes it's possible. But it will be most efficient if a single repository is only modified from one place. Also keep in mind that -|project_name| will keep an exclusive lock on the repository while creating +Borg will keep an exclusive lock on the repository while creating or deleting archives, which may make *simultaneous* backups fail. Can I copy or synchronize my repo to another location? @@ -116,7 +116,7 @@ Are there other known limitations? If a backup stops mid-way, does the already-backed-up data stay there? ---------------------------------------------------------------------- -Yes, |project_name| supports resuming backups. +Yes, Borg supports resuming backups. During a backup a special checkpoint archive named ``.checkpoint`` is saved every checkpoint interval (the default value for this is 30 @@ -134,7 +134,7 @@ just invoke ``borg create`` as you always do. You may use the same archive name as in previous attempt or a different one (e.g. if you always include the current datetime), it does not matter. -|project_name| always does full single-pass backups, so it will start again +Borg always does full single-pass backups, so it will start again from the beginning - but it will be much faster, because some of the data was already stored into the repo (and is still referenced by the checkpoint archive), so it does not need to get transmitted and stored again. @@ -167,8 +167,8 @@ all the part files and manually concatenate them together. For more details, see :ref:`checkpoints_parts`. -Can |project_name| add redundancy to the backup data to deal with hardware malfunction? ---------------------------------------------------------------------------------------- +Can Borg add redundancy to the backup data to deal with hardware malfunction? +----------------------------------------------------------------------------- No, it can't. While that at first sounds like a good idea to defend against some defect HDD sectors or SSD flash blocks, dealing with this in a @@ -180,8 +180,8 @@ storage or just make backups to different locations / different hardware. See also :issue:`225`. -Can |project_name| verify data integrity of a backup archive? -------------------------------------------------------------- +Can Borg verify data integrity of a backup archive? +--------------------------------------------------- Yes, if you want to detect accidental data damage (like bit rot), use the ``check`` operation. It will notice corruption using CRCs and hashes. @@ -551,7 +551,7 @@ If you run into that, try this: I am seeing 'A' (added) status for an unchanged file!? ------------------------------------------------------ -The files cache is used to determine whether |project_name| already +The files cache is used to determine whether Borg already "knows" / has backed up a file and if so, to skip the file from chunking. It does intentionally *not* contain files that have a modification time (mtime) same as the newest mtime in the created archive. @@ -563,7 +563,7 @@ This is expected: it is to avoid data loss with files that are backed up from a snapshot and that are immediately changed after the snapshot (but within mtime granularity time, so the mtime would not change). Without the code that removes these files from the files cache, the change that happened right after -the snapshot would not be contained in the next backup as |project_name| would +the snapshot would not be contained in the next backup as Borg would think the file is unchanged. This does not affect deduplication, the file will be chunked, but as the chunks @@ -586,13 +586,13 @@ already used. It always chunks all my files, even unchanged ones! --------------------------------------------------- -|project_name| maintains a files cache where it remembers the mtime, size and -inode of files. When |project_name| does a new backup and starts processing a +Borg maintains a files cache where it remembers the mtime, size and +inode of files. When Borg does a new backup and starts processing a file, it first looks whether the file has changed (compared to the values stored in the files cache). If the values are the same, the file is assumed unchanged and thus its contents won't get chunked (again). -|project_name| can't keep an infinite history of files of course, thus entries +Borg can't keep an infinite history of files of course, thus entries in the files cache have a "maximum time to live" which is set via the environment variable BORG_FILES_CACHE_TTL (and defaults to 20). Every time you do a backup (on the same machine, using the same user), the @@ -609,11 +609,11 @@ it would be much faster. Another possible reason is that files don't always have the same path, for example if you mount a filesystem without stable mount points for each backup or if you are running the backup from a filesystem snapshot whose name is not stable. If the directory where you mount a filesystem is different every time, -|project_name| assume they are different files. +Borg assume they are different files. -Is there a way to limit bandwidth with |project_name|? ------------------------------------------------------- +Is there a way to limit bandwidth with Borg? +-------------------------------------------- To limit upload (i.e. :ref:`borg_create`) bandwidth, use the ``--remote-ratelimit`` option. @@ -634,7 +634,7 @@ Add BORG_RSH environment variable to use pipeviewer wrapper script with ssh. :: export BORG_RSH='/usr/local/bin/pv-wrapper ssh' -Now |project_name| will be bandwidth limited. Nice thing about pv is that you can change rate-limit on the fly: :: +Now Borg will be bandwidth limited. Nice thing about pv is that you can change rate-limit on the fly: :: pv -R $(pidof pv) -L 102400 @@ -644,7 +644,7 @@ Now |project_name| will be bandwidth limited. Nice thing about pv is that you ca I am having troubles with some network/FUSE/special filesystem, why? -------------------------------------------------------------------- -|project_name| is doing nothing special in the filesystem, it only uses very +Borg is doing nothing special in the filesystem, it only uses very common and compatible operations (even the locking is just "mkdir"). So, if you are encountering issues like slowness, corruption or malfunction @@ -652,13 +652,13 @@ when using a specific filesystem, please try if you can reproduce the issues with a local (non-network) and proven filesystem (like ext4 on Linux). If you can't reproduce the issue then, you maybe have found an issue within -the filesystem code you used (not with |project_name|). For this case, it is +the filesystem code you used (not with Borg). For this case, it is recommended that you talk to the developers / support of the network fs and maybe open an issue in their issue tracker. Do not file an issue in the -|project_name| issue tracker. +Borg issue tracker. If you can reproduce the issue with the proven filesystem, please file an -issue in the |project_name| issue tracker about that. +issue in the Borg issue tracker about that. Why does running 'borg check --repair' warn about data loss? @@ -670,7 +670,7 @@ instances, such as malfunctioning storage hardware, additional repo corruption may occur. If you can't afford to lose the repo, it's strongly recommended that you perform repair on a copy of the repo. -In other words, the warning is there to emphasize that |project_name|: +In other words, the warning is there to emphasize that Borg: - Will perform automated routines that modify your backup repository - Might not actually fix the problem you are experiencing - Might, in very rare cases, further corrupt your repository diff --git a/docs/global.rst.inc b/docs/global.rst.inc index b6b98ab71..14a06bdf5 100644 --- a/docs/global.rst.inc +++ b/docs/global.rst.inc @@ -1,5 +1,4 @@ .. highlight:: bash -.. |project_name| replace:: Borg .. |package_dirname| replace:: borgbackup-|version| .. |package_filename| replace:: |package_dirname|.tar.gz .. |package_url| replace:: https://pypi.python.org/packages/source/b/borgbackup/|package_filename| diff --git a/docs/installation.rst b/docs/installation.rst index 683560fc7..bf519f75d 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -5,7 +5,7 @@ Installation ============ -There are different ways to install |project_name|: +There are different ways to install Borg: - :ref:`distribution-package` - easy and fast if a package is available from your distribution. @@ -29,11 +29,11 @@ Some distributions might offer a ready-to-use ``borgbackup`` package which can be installed with the package manager. .. important:: Those packages may not be up to date with the latest - |project_name| releases. Before submitting a bug + Borg releases. Before submitting a bug report, check the package version and compare that to our latest release then review :doc:`changes` to see if the bug has been fixed. Report bugs to the package - maintainer rather than directly to |project_name| if the + maintainer rather than directly to Borg if the package is out of date in the distribution. .. keep this list in alphabetical order @@ -87,7 +87,7 @@ Standalone Binary .. note:: Releases are signed with an OpenPGP key, see :ref:`security-contact` for more instructions. -|project_name| binaries (generated with `pyinstaller`_) are available +Borg binaries (generated with `pyinstaller`_) are available on the releases_ page for the following platforms: * **Linux**: glibc >= 2.13 (ok for most supported Linux releases). @@ -107,9 +107,9 @@ alias for ``borg mount``:: ln -s /usr/local/bin/borg /usr/local/bin/borgfs -Note that the binary uses /tmp to unpack |project_name| with all dependencies. +Note that the binary uses /tmp to unpack Borg with all dependencies. It will fail if /tmp has not enough free space or is mounted with the ``noexec`` option. -You can change the temporary directory by setting the ``TEMP`` environment variable before running |project_name|. +You can change the temporary directory by setting the ``TEMP`` environment variable before running Borg. If a new version is released, you will have to manually download it and replace the old version using the same steps as shown above. @@ -133,7 +133,7 @@ From Source Dependencies ~~~~~~~~~~~~ -To install |project_name| from a source package (including pip), you have to install the +To install Borg from a source package (including pip), you have to install the following dependencies first: * `Python 3`_ >= 3.5.0, plus development headers. Even though Python 3 is not @@ -285,7 +285,7 @@ You can then install ``pip`` and ``virtualenv``:: Using pip ~~~~~~~~~ -Virtualenv_ can be used to build and install |project_name| without affecting +Virtualenv_ can be used to build and install Borg without affecting the system Python or requiring root access. Using a virtual environment is optional, but recommended except for the most simple use cases. @@ -305,7 +305,7 @@ This will use ``pip`` to install the latest release from PyPi:: # or alternatively (if you want FUSE support): pip install borgbackup[fuse] -To upgrade |project_name| to a new version later, run the following after +To upgrade Borg to a new version later, run the following after activating your virtual environment:: pip install -U borgbackup # or ... borgbackup[fuse] diff --git a/docs/internals/data-structures.rst b/docs/internals/data-structures.rst index 7c6d46b68..d4e441154 100644 --- a/docs/internals/data-structures.rst +++ b/docs/internals/data-structures.rst @@ -28,7 +28,7 @@ the concept of archives or items. Each repository has the following file structure: README - simple text file telling that this is a |project_name| repository + simple text file telling that this is a Borg repository config repository configuration @@ -578,7 +578,7 @@ A chunk is stored as an object as well, of course. Chunks ~~~~~~ -The |project_name| chunker uses a rolling hash computed by the Buzhash_ algorithm. +The Borg chunker uses a rolling hash computed by the Buzhash_ algorithm. It triggers (chunks) when the last HASH_MASK_BITS bits of the hash are zero, producing chunks of 2^HASH_MASK_BITS Bytes on average. @@ -686,7 +686,7 @@ i.e. the reference count is pinned to MAX_VALUE. Indexes / Caches memory usage ----------------------------- -Here is the estimated memory usage of |project_name| - it's complicated:: +Here is the estimated memory usage of Borg - it's complicated:: chunk_count ~= total_file_size / 2 ^ HASH_MASK_BITS @@ -822,7 +822,7 @@ Each chunk consists of ``TYPE(1)`` + ``MAC(32)`` + ``NONCE(8)`` + ``CIPHERTEXT`` In AES-CTR mode you can think of the IV as the start value for the counter. The counter itself is incremented by one after each 16 byte block. The IV/counter is not required to be random but it must NEVER be reused. -So to accomplish this |project_name| initializes the encryption counter to be +So to accomplish this Borg initializes the encryption counter to be higher than any previously used counter value before encrypting new data. To reduce payload size, only 8 bytes of the 16 bytes nonce is saved in the @@ -845,7 +845,7 @@ Key files .. seealso:: The :ref:`key_encryption` section for an in-depth review of the key encryption. -When initialized with the ``init -e keyfile`` command, |project_name| +When initialized with the ``init -e keyfile`` command, Borg needs an associated file in ``$HOME/.config/borg/keys`` to read and write the repository. The format is based on msgpack_, base64 encoding and PBKDF2_ SHA256 hashing, which is then encoded again in a msgpack_. @@ -909,7 +909,7 @@ representation of the repository id. Compression ----------- -|project_name| supports the following compression methods: +Borg supports the following compression methods: - none (no compression, pass through data 1:1) - lz4 (low compression, but super fast) @@ -945,7 +945,7 @@ See ``borg create --help`` about how to specify the compression level and its de Lock files ---------- -|project_name| uses locks to get (exclusive or shared) access to the cache and +Borg uses locks to get (exclusive or shared) access to the cache and the repository. The locking system is based on creating a directory `lock.exclusive` (for @@ -963,7 +963,7 @@ The cache lock is usually in `~/.cache/borg/REPOID/lock.*`. The repository lock is in `repository/lock.*`. In case you run into troubles with the locks, you can use the ``borg break-lock`` -command after you first have made sure that no |project_name| process is +command after you first have made sure that no Borg process is running on any machine that accesses this resource. Be very careful, the cache or repository might get damaged if multiple processes use it at the same time. diff --git a/docs/quickstart.rst b/docs/quickstart.rst index a7a1ba0fb..4ea265b11 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -5,7 +5,7 @@ Quick Start =========== -This chapter will get you started with |project_name| and covers +This chapter will get you started with Borg and covers various use cases. A step by step example @@ -21,19 +21,19 @@ a good amount of free space on the filesystem that has your backup repository (and also on ~/.cache). A few GB should suffice for most hard-drive sized repositories. See also :ref:`cache-memory-usage`. -|project_name| doesn't use space reserved for root on repository disks (even when run as root), +Borg doesn't use space reserved for root on repository disks (even when run as root), on file systems which do not support this mechanism (e.g. XFS) we recommend to -reserve some space in |project_name| itself just to be safe by adjusting the +reserve some space in Borg itself just to be safe by adjusting the ``additional_free_space`` setting in the ``[repository]`` section of a repositories ``config`` file. A good starting point is ``2G``. -If |project_name| runs out of disk space, it tries to free as much space as it +If Borg runs out of disk space, it tries to free as much space as it can while aborting the current operation safely, which allows the user to free more space by deleting/pruning archives. This mechanism is not bullet-proof in some circumstances [1]_. If you *really* run out of disk space, it can be hard or impossible to free space, -because |project_name| needs free space to operate - even to delete backup +because Borg needs free space to operate - even to delete backup archives. You can use some monitoring process or just include the free space information @@ -153,14 +153,14 @@ backed up and that the ``prune`` command is keeping and deleting the correct bac Pitfalls with shell variables and environment variables ------------------------------------------------------- -This applies to all environment variables you want |project_name| to see, not just +This applies to all environment variables you want Borg to see, not just ``BORG_PASSPHRASE``. The short explanation is: always ``export`` your variable, and use single quotes if you're unsure of the details of your shell's expansion behavior. E.g.:: export BORG_PASSPHRASE='complicated & long' -This is because ``export`` exposes variables to subprocesses, which |project_name| may be +This is because ``export`` exposes variables to subprocesses, which Borg may be one of. More on ``export`` can be found in the "ENVIRONMENT" section of the bash(1) man page. @@ -220,7 +220,7 @@ means that an attacker who manages to compromise the host containing an encrypted archive will not be able to access any of the data, even while the backup is being made. -|project_name| supports different methods to store the AES and HMAC keys. +Borg supports different methods to store the AES and HMAC keys. ``repokey`` mode The key is stored inside the repository (in its "config" file). @@ -265,8 +265,8 @@ For automated backups the passphrase can be specified using the Remote repositories ------------------- -|project_name| can initialize and access repositories on remote hosts if the -host is accessible using SSH. This is fastest and easiest when |project_name| +Borg can initialize and access repositories on remote hosts if the +host is accessible using SSH. This is fastest and easiest when Borg is installed on the remote host, in which case the following syntax is used:: $ borg init user@hostname:/path/to/repo @@ -275,12 +275,12 @@ Note: please see the usage chapter for a full documentation of repo URLs. Remote operations over SSH can be automated with SSH keys. You can restrict the use of the SSH keypair by prepending a forced command to the SSH public key in -the remote server's `authorized_keys` file. This example will start |project_name| +the remote server's `authorized_keys` file. This example will start Borg in server mode and limit it to a specific filesystem path:: command="borg serve --restrict-to-path /path/to/repo",restrict ssh-rsa AAAAB3[...] -If it is not possible to install |project_name| on the remote host, +If it is not possible to install Borg on the remote host, it is still possible to use the remote host to store a repository by mounting the remote filesystem, for example, using sshfs:: diff --git a/docs/support.rst b/docs/support.rst index 8bd3fcbed..746fa6562 100644 --- a/docs/support.rst +++ b/docs/support.rst @@ -63,7 +63,7 @@ specific feature suggestion, you can post a new bounty or back an existing one (they always refer to an issue in our `issue tracker`_). As a developer, you can become a Bounty Hunter and win bounties (earn money) by -contributing to |project_name|, a free and open source software project. +contributing to Borg, a free and open source software project. We might also use BountySource to fund raise for some bigger goals.