mirror of
https://github.com/borgbackup/borg.git
synced 2026-06-11 09:59:19 -04:00
Merge pull request #7907 from ThomasWaldmann/update-docs-master
Update docs (master)
This commit is contained in:
TW
GitHub
commit
d9fde26243
2 changed files with 388 additions and 2 deletions
|
|
@ -33,14 +33,17 @@ Below, if we speak of borg 1.2.6, we mean a borg version >= 1.2.6 **or** a
|
|||
borg version that has the relevant security patches for this vulnerability applied
|
||||
(could be also an older version in that case).
|
||||
|
||||
Steps you must take to upgrade a repository:
|
||||
Steps you must take to upgrade a repository (this applies to all kinds of repos
|
||||
no matter what encryption mode they use, including "none"):
|
||||
|
||||
1. Upgrade all clients using this repository to borg 1.2.6.
|
||||
Note: it is not required to upgrade a server, except if the server-side borg
|
||||
is also used as a client (and not just for "borg serve").
|
||||
|
||||
Do **not** run ``borg check`` with borg > 1.2.4 before completing the upgrade steps.
|
||||
Do **not** run ``borg check`` with borg 1.2.6 before completing the upgrade steps:
|
||||
|
||||
- ``borg check`` would complain about archives without a valid archive TAM.
|
||||
- ``borg check --repair`` would remove such archives!
|
||||
2. Run ``BORG_WORKAROUNDS=ignore_invalid_archive_tam borg info --debug <repo> 2>&1 | grep TAM | grep -i manifest``.
|
||||
|
||||
a) If you get "TAM-verified manifest", continue with 3.
|
||||
|
|
@ -285,6 +288,91 @@ The best check that everything is ok is to run a dry-run extraction::
|
|||
|
||||
borg extract -v --dry-run REPO::ARCHIVE
|
||||
|
||||
|
||||
.. _upgradenotes:
|
||||
|
||||
Upgrade Notes
|
||||
=============
|
||||
|
||||
borg 1.1.x to 1.2.x
|
||||
-------------------
|
||||
|
||||
Some things can be recommended for the upgrade process from borg 1.1.x
|
||||
(please also read the important compatibility notes below):
|
||||
|
||||
- first upgrade to a recent 1.1.x release - especially if you run some older
|
||||
1.1.* or even 1.0.* borg release.
|
||||
- using that, run at least one `borg create` (your normal backup), `prune`
|
||||
and especially a `check` to see everything is in a good state.
|
||||
- check the output of `borg check` - if there is anything special, consider
|
||||
a `borg check --repair` followed by another `borg check`.
|
||||
- if everything is fine so far (borg check reports no issues), you can consider
|
||||
upgrading to 1.2.x. if not, please first fix any already existing issue.
|
||||
- if you want to play safer, first **create a backup of your borg repository**.
|
||||
- upgrade to latest borg 1.2.x release (you could use the fat binary from
|
||||
github releases page)
|
||||
- borg 1.2.6 has a security fix for the pre-1.2.5 archives spoofing vulnerability
|
||||
(CVE-2023-36811), see details and necessary upgrade procedure described above.
|
||||
- run `borg compact --cleanup-commits` to clean up a ton of 17 bytes long files
|
||||
in your repo caused by a borg 1.1 bug
|
||||
- run `borg check` again (now with borg 1.2.x) and check if there is anything
|
||||
special.
|
||||
- run `borg info` (with borg 1.2.x) to build the local pre12-meta cache (can
|
||||
take significant time, but after that it will be fast) - for more details
|
||||
see below.
|
||||
- check the compatibility notes (see below) and adapt your scripts, if needed.
|
||||
- if you run into any issues, please check the github issue tracker before
|
||||
posting new issues there or elsewhere.
|
||||
|
||||
If you follow this procedure, you can help avoiding that we get a lot of
|
||||
"borg 1.2" issue reports that are not really 1.2 issues, but existed before
|
||||
and maybe just were not noticed.
|
||||
|
||||
Compatibility notes:
|
||||
|
||||
- matching of path patterns has been aligned with borg storing relative paths.
|
||||
Borg archives file paths without leading slashes. Previously, include/exclude
|
||||
patterns could contain leading slashes. You should check your patterns and
|
||||
remove leading slashes.
|
||||
- dropped support / testing for older Pythons, minimum requirement is 3.8.
|
||||
In case your OS does not provide Python >= 3.8, consider using our binary,
|
||||
which does not need an external Python interpreter. Or continue using
|
||||
borg 1.1.x, which is still supported.
|
||||
- freeing repository space only happens when "borg compact" is invoked.
|
||||
- mount: the default for --numeric-ids is False now (same as borg extract)
|
||||
- borg create --noatime is deprecated. Not storing atime is the default behaviour
|
||||
now (use --atime if you want to store the atime).
|
||||
- --prefix is deprecated, use -a / --glob-archives, see #6806
|
||||
- list: corrected mix-up of "isomtime" and "mtime" formats.
|
||||
Previously, "isomtime" was the default but produced a verbose human format,
|
||||
while "mtime" produced a ISO-8601-like format.
|
||||
The behaviours have been swapped (so "mtime" is human, "isomtime" is ISO-like),
|
||||
and the default is now "mtime".
|
||||
"isomtime" is now a real ISO-8601 format ("T" between date and time, not a space).
|
||||
- create/recreate --list: file status for all files used to get announced *AFTER*
|
||||
the file (with borg < 1.2). Now, file status is announced *BEFORE* the file
|
||||
contents are processed. If the file status changes later (e.g. due to an error
|
||||
or a content change), the updated/final file status will be printed again.
|
||||
- removed deprecated-since-long stuff (deprecated since):
|
||||
|
||||
- command "borg change-passphrase" (2017-02), use "borg key ..."
|
||||
- option "--keep-tag-files" (2017-01), use "--keep-exclude-tags"
|
||||
- option "--list-format" (2017-10), use "--format"
|
||||
- option "--ignore-inode" (2017-09), use "--files-cache" w/o "inode"
|
||||
- option "--no-files-cache" (2017-09), use "--files-cache=disabled"
|
||||
- removed BORG_HOSTNAME_IS_UNIQUE env var.
|
||||
to use borg you must implement one of these 2 scenarios:
|
||||
|
||||
- 1) the combination of FQDN and result of uuid.getnode() must be unique
|
||||
and stable (this should be the case for almost everybody, except when
|
||||
having duplicate FQDN *and* MAC address or all-zero MAC address)
|
||||
- 2) if you are aware that 1) is not the case for you, you must set
|
||||
BORG_HOST_ID env var to something unique.
|
||||
- exit with 128 + signal number, #5161.
|
||||
if you have scripts expecting rc == 2 for a signal exit, you need to update
|
||||
them to check for >= 128.
|
||||
|
||||
|
||||
.. _changelog_1x:
|
||||
|
||||
Change Log 1.x
|
||||
|
|
@ -336,6 +424,301 @@ Other changes:
|
|||
- remove support for OpenSSL < 1.1.1
|
||||
|
||||
|
||||
Version 1.2.6 (2023-08-31)
|
||||
--------------------------
|
||||
|
||||
Fixes:
|
||||
|
||||
- The upgrade procedure docs as published with borg 1.2.5 did not work, if the
|
||||
repository had archives resulting from a borg rename or borg recreate operation.
|
||||
|
||||
The updated docs now use BORG_WORKAROUNDS=ignore_invalid_archive_tam at some
|
||||
places to avoid that issue, #7791.
|
||||
|
||||
See: fix pre-1.2.5 archives spoofing vulnerability (CVE-2023-36811),
|
||||
details and necessary upgrade procedure described above.
|
||||
|
||||
Other changes:
|
||||
|
||||
- updated 1.2.5 changelog entry: 1.2.5 already has the fix for rename/recreate.
|
||||
- remove cython restrictions. recommended is to build with cython 0.29.latest,
|
||||
because borg 1.2.x uses this since years and it is very stable.
|
||||
you can also try to build with cython 3.0.x, there is a good chance that it works.
|
||||
as a 3rd option, we also bundle the `*.c` files cython outputs in the release
|
||||
pypi package, so you can also just use these and not need cython at all.
|
||||
|
||||
|
||||
Version 1.2.5 (2023-08-30)
|
||||
--------------------------
|
||||
|
||||
Fixes:
|
||||
|
||||
- Security: fix pre-1.2.5 archives spoofing vulnerability (CVE-2023-36811),
|
||||
see details and necessary upgrade procedure described above.
|
||||
- rename/recreate: correctly update resulting archive's TAM, see #7791
|
||||
- create: do not try to read parent dir of recursion root, #7746
|
||||
- extract: fix false warning about pattern never matching, #4110
|
||||
- diff: remove surrogates before output, #7535
|
||||
- compact: clear empty directories at end of compact process, #6823
|
||||
- create --files-cache=size: fix crash, #7658
|
||||
- keyfiles: improve key sanity check, #7561
|
||||
- only warn about "invalid" chunker params, #7590
|
||||
- ProgressIndicatorPercent: fix space computation for wide chars, #3027
|
||||
- improve argparse validator error messages
|
||||
|
||||
New features:
|
||||
|
||||
- mount: make up volname if not given (macOS), #7690.
|
||||
macFUSE supports a volname mount option to give what finder displays on the
|
||||
desktop / in the directory view. if the user did not specify it, we make
|
||||
something up, because otherwise it would be "macFUSE Volume 0 (Python)" and
|
||||
hide the mountpoint directory name.
|
||||
- BORG_WORKAROUNDS=authenticated_no_key to extract from authenticated repos
|
||||
without key, #7700
|
||||
|
||||
Other changes:
|
||||
|
||||
- add `utcnow()` helper function to avoid deprecated `datetime.utcnow()`
|
||||
- stay on latest Cython 0.29 (0.29.36) for borg 1.2.x (do not use Cython 3.0 yet)
|
||||
- docs:
|
||||
|
||||
- move upgrade notes to own section, see #7546
|
||||
- mount -olocal: how to show mount in finder's sidebar, #5321
|
||||
- list: fix --pattern examples, #7611
|
||||
- improve patterns help
|
||||
- incl./excl. options, path-from-stdin exclusiveness
|
||||
- obfuscation docs: markup fix, note about MAX_DATA_SIZE
|
||||
- --one-file-system: add macOS apfs notes, #4876
|
||||
- improve --one-file-system help string, #5618
|
||||
- rewrite borg check docs
|
||||
- improve the docs for --keep-within, #7687
|
||||
- fix borg init command in environment.rst.inc
|
||||
- 1.1.x upgrade notes: more precise borg upgrade instructions, #3396
|
||||
|
||||
- tests:
|
||||
|
||||
- fix repo reopen
|
||||
- avoid long ids in pytest output
|
||||
- check buzhash chunksize distribution, see #7586
|
||||
|
||||
|
||||
Version 1.2.4 (2023-03-24)
|
||||
--------------------------
|
||||
|
||||
New features:
|
||||
|
||||
- import-tar: add --ignore-zeros to process concatenated tars, #7432.
|
||||
- debug id-hash: computes file/chunk content id-hash, #7406
|
||||
- diff: --content-only does not show mode/ctime/mtime changes, #7248
|
||||
- diff: JSON strings in diff output are now sorted alphabetically
|
||||
|
||||
Bug fixes:
|
||||
|
||||
- xattrs: fix namespace processing on FreeBSD, #6997
|
||||
- diff: fix path related bug seen when addressing deferred items.
|
||||
- debug get-obj/put-obj: always give chunkid as cli param, see #7290
|
||||
(this is an incompatible change, see also borg debug id-hash)
|
||||
- extract: fix mtime when ResourceFork xattr is set (macOS specific), #7234
|
||||
- recreate: without --chunker-params, do not re-chunk, #7337
|
||||
- recreate: when --target is given, do not detect "nothing to do".
|
||||
use case: borg recreate -a src --target dst can be used to make a copy
|
||||
of an archive inside the same repository, #7254.
|
||||
- set .hardlink_master for ALL hardlinkable items, #7175
|
||||
- locking: fix host, pid, tid order.
|
||||
tid (thread id) must be parsed as hex from lock file name.
|
||||
- update development.lock.txt, including a setuptools security fix, #7227
|
||||
|
||||
Other changes:
|
||||
|
||||
- requirements: allow msgpack 1.0.5 also
|
||||
- upgrade Cython to 0.29.33
|
||||
- hashindex minor fixes, refactor, tweaks, tests
|
||||
- use os.replace not os.rename
|
||||
- remove BORG_LIBB2_PREFIX (not used any more)
|
||||
- docs:
|
||||
|
||||
- BORG_KEY_FILE: clarify docs, #7444
|
||||
- update FAQ about locale/unicode issues, #6999
|
||||
- improve mount options rendering, #7359
|
||||
- make timestamps in manual pages reproducible
|
||||
- installation: update Fedora in distribution list, #7357
|
||||
- tests:
|
||||
|
||||
- fix test_size_on_disk_accurate for large st_blksize, #7250
|
||||
- add same_ts_ns function and use it for relaxed timestamp comparisons
|
||||
- "auto" compressor tests: don't assume a specific size,
|
||||
do not assume zlib is better than lz4, #7363
|
||||
- add test for extracted directory mtime
|
||||
- vagrant:
|
||||
|
||||
- upgrade local freebsd 12.1 box -> generic/freebsd13 box (13.1)
|
||||
- use pythons > 3.8 which work on freebsd 13.1
|
||||
- pyenv: also install python 3.11.1 for testing
|
||||
- pyenv: use python 3.10.1, 3.10.0 build is broken on freebsd
|
||||
|
||||
|
||||
Version 1.2.3 (2022-12-24)
|
||||
--------------------------
|
||||
|
||||
Fixes:
|
||||
|
||||
- create: fix --list --dry-run output for directories, #7209
|
||||
- diff/recreate: normalize chunker params before comparing them, #7079
|
||||
- check: fix uninitialised variable if repo is completely empty, #7034
|
||||
- xattrs: improve error handling, #6988
|
||||
- fix args.paths related argparsing, #6994
|
||||
- archive.save(): always use metadata from stats (e.g. nfiles, size, ...), #7072
|
||||
- tar_filter: recognize .tar.zst as zstd, #7093
|
||||
- get_chunker: fix missing sparse=False argument, #7056
|
||||
- file_integrity.py: make sure file_fd is always closed on exit
|
||||
- repository: cleanup(): close segment before unlinking
|
||||
- repository: use os.replace instead of os.rename
|
||||
|
||||
Other changes:
|
||||
|
||||
- remove python < 3.7 compatibility code
|
||||
- do not use version_tuple placeholder in setuptools_scm template
|
||||
- CI: fix tox4 passenv issue, #7199
|
||||
- vagrant: update to python 3.9.16, use the openbsd 7.1 box
|
||||
- misc. test suite and docs fixes / improvements
|
||||
- remove deprecated --prefix from docs, #7109
|
||||
- Windows: use MSYS2 for Github CI, remove Appveyor CI
|
||||
|
||||
|
||||
Version 1.2.2 (2022-08-20)
|
||||
--------------------------
|
||||
|
||||
New features:
|
||||
|
||||
- prune/delete --checkpoint-interval=1800 and ctrl-c/SIGINT support, #6284
|
||||
|
||||
Fixes:
|
||||
|
||||
- SaveFile: use a custom mkstemp with mode support, #6933, #6400, #6786.
|
||||
This fixes umask/mode/ACL issues (and also "chmod not supported" exceptions
|
||||
seen in 1.2.1) of files updated using SaveFile, e.g. the repo config.
|
||||
- hashindex_compact: fix eval order (check idx before use), #5899
|
||||
- create --paths-from-(stdin|command): normalize paths, #6778
|
||||
- secure_erase: avoid collateral damage, #6768.
|
||||
If a hardlink copy of a repo was made and a new repo config shall be saved,
|
||||
do NOT fill in random garbage before deleting the previous repo config,
|
||||
because that would damage the hardlink copy.
|
||||
- list: fix {flags:<WIDTH>} formatting, #6081
|
||||
- check: try harder to create the key, #5719
|
||||
- misc commands: ctrl-c must not kill other subprocesses, #6912
|
||||
|
||||
- borg create with a remote repo via ssh
|
||||
- borg create --content-from-command
|
||||
- borg create --paths-from-command
|
||||
- (de)compression filter process of import-tar / export-tar
|
||||
|
||||
Other changes:
|
||||
|
||||
- deprecate --prefix, use -a / --glob-archives, see #6806
|
||||
- make setuptools happy ("package would be ignored"), #6874
|
||||
- fix pyproject.toml to create a fixed _version.py file, compatible with both
|
||||
old and new setuptools_scm version, #6875
|
||||
- automate asciinema screencasts
|
||||
- CI: test on macOS 12 without fuse / fuse tests
|
||||
(too troublesome on github CI due to kernel extensions needed by macFUSE)
|
||||
- tests: fix test_obfuscate byte accounting
|
||||
- repository: add debug logging for issue #6687
|
||||
- _chunker.c: fix warnings on macOS
|
||||
- requirements.lock.txt: use the latest cython 0.29.32
|
||||
- docs:
|
||||
|
||||
- add info on man page installation, #6894
|
||||
- update archive_progress json description about "finished", #6570
|
||||
- json progress_percent: some values are optional, #4074
|
||||
- FAQ: full quota / full disk, #5960
|
||||
- correct shell syntax for installation using git
|
||||
|
||||
|
||||
Version 1.2.1 (2022-06-06)
|
||||
--------------------------
|
||||
|
||||
Fixes:
|
||||
|
||||
- create: skip with warning if opening the parent dir of recursion root fails, #6374
|
||||
- create: fix crash. metadata stream can produce all-zero chunks, #6587
|
||||
- fix crash when computing stats, escape % chars in archive name, #6500
|
||||
- fix transaction rollback: use files cache filename as found in txn.active/, #6353
|
||||
- import-tar: kill filter process in case of borg exceptions, #6401 #6681
|
||||
- import-tar: fix mtime type bug
|
||||
- ensure_dir: respect umask for created directory modes, #6400
|
||||
- SaveFile: respect umask for final file mode, #6400
|
||||
- check archive: improve error handling for corrupt archive metadata block, make
|
||||
robust_iterator more robust, #4777
|
||||
- pre12-meta cache: do not use the cache if want_unique is True, #6612
|
||||
- fix scp-style repo url parsing for ip v6 address, #6526
|
||||
- mount -o versions: give clear error msg instead of crashing.
|
||||
it does not make sense to request versions view if you only look at 1 archive,
|
||||
but the code shall not crash in that case as it did, but give a clear error msg.
|
||||
- show_progress: add finished=true/false to archive_progress json, #6570
|
||||
- delete/prune: fix --iec mode output (decimal vs. binary units), #6606
|
||||
- info: fix authenticated mode repo to show "Encrypted: No", #6462
|
||||
- diff: support presence change for blkdev, chrdev and fifo items, #6615
|
||||
|
||||
New features:
|
||||
|
||||
- delete: add repository id and location to prompt, #6453
|
||||
- borg debug dump-repo-objs --ghost: new --segment=S --offset=O options
|
||||
|
||||
Other changes:
|
||||
|
||||
- support python 3.11
|
||||
- allow msgpack 1.0.4, #6716
|
||||
- load_key: no key is same as empty key, #6441
|
||||
- give a more helpful error msg for unsupported key formats, #6561
|
||||
- better error msg for defect or unsupported repo configs, #6566
|
||||
- docs:
|
||||
|
||||
- document borg 1.2 pattern matching behavior change, #6407
|
||||
Make clear that absolute paths always go into the matcher as if they are
|
||||
relative (without leading slash). Adapt all examples accordingly.
|
||||
- authentication primitives: improved security and performance infos
|
||||
- mention BORG_FILES_CACHE_SUFFIX as alternative to BORG_FILES_CACHE_TTL, #5602
|
||||
- FAQ: add a hint about --debug-topic=files_cache
|
||||
- improve borg check --max-duration description
|
||||
- fix values of TAG bytes, #6515
|
||||
- borg compact --cleanup-commits also runs a normal compaction, #6324
|
||||
- virtualization speed tips
|
||||
- recommend umask for passphrase file perms
|
||||
- borg 1.2 is security supported
|
||||
- update link to ubuntu packages, #6485
|
||||
- use --numeric-ids in pull mode docs
|
||||
- remove blake2 docs, blake2 code not bundled any more, #6371
|
||||
- clarify on-disk order and size of segment file log entry fields, #6357
|
||||
- docs building: do not transform --/--- to unicode dashes
|
||||
- tests:
|
||||
|
||||
- check that borg does not require pytest for normal usage, fixes #6563
|
||||
- fix OpenBSD symlink mode test failure, #2055
|
||||
- vagrant:
|
||||
|
||||
- darwin64: remove fakeroot, #6314
|
||||
- update development.lock.txt
|
||||
- use pyinstaller 4.10 and python 3.9.13 for binary build
|
||||
- upgrade VMCPUS and xdistn from 4 to 16, maybe this speeds up the tests
|
||||
- crypto:
|
||||
|
||||
- use hmac.compare_digest instead of ==, #6470
|
||||
- hmac_sha256: replace own cython wrapper code by hmac.digest python stdlib (since py38)
|
||||
- hmac and blake2b minor optimizations and cleanups
|
||||
- removed some unused crypto related code, #6472
|
||||
- avoid losing the key (potential use-after-free). this never could happen in
|
||||
1.2 due to the way we use the code. The issue was discovered in master after
|
||||
other changes, so we also "fixed" it here before it bites us.
|
||||
- setup / build:
|
||||
|
||||
- add pyproject.toml, fix sys.path, #6466
|
||||
- setuptools_scm: also require it via pyproject.toml
|
||||
- allow extra compiler flags for every extension build
|
||||
- fix misc. C / Cython compiler warnings, deprecation warnings
|
||||
- fix zstd.h include for bundled zstd, #6369
|
||||
- source using python 3.8 features: ``pyupgrade --py38-plus ./**/*.py``
|
||||
|
||||
|
||||
Version 1.2.0 (2022-02-22 22:02:22 :-)
|
||||
--------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -86,6 +86,9 @@ run into this by yourself by restoring an older copy of your repository.
|
|||
"attack": maybe an attacker has replaced your repo by an older copy, trying to
|
||||
trick you into AES counter reuse, trying to break your repo encryption.
|
||||
|
||||
Borg users have also reported that fs issues (like hw issues / I/O errors causing
|
||||
the fs to become read-only) can cause this warning, see :issue:`7853`.
|
||||
|
||||
If you decide to ignore this and accept unsafe operation for this repository,
|
||||
you could delete the manifest-timestamp and the local cache:
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue