Some features like append-only repositories rely on a server-side component
that enforces them (because that shall only be controllable server-side,
not client-side).
So, that can only work, if such a server-side component exists, which is the
case for borg 1.x ssh: repositories (but not for borg 1.x non-ssh: repositories).
For borg2, we currently have:
- fs repos
- sftp: repos
- rclone: repos (enabling many different cloud providers)
- s3/b3: repos
- ssh: repos using client/server rpc code similar as in borg 1.x
So, only for the last method we have a borg server-side process that could enforce some features, but not for any of the other repo types.
For append-only the current idea is that this should not be done within borg,
but solved by a missing repo object delete permission enforced by the storage.
borg create could then use credentials that miss permission to delete,
while borg compact would use credentials that include permission to delete.
Some features like repository quotas rely on a server-side component
that enforces them (because that shall only be controllable server-side,
not client-side).
So, that can only work, if such a server-side component exists, which is the
case for borg 1.x ssh: repositories (but not for borg 1.x non-ssh: repositories).
For borg2, we currently have:
- fs repos
- sftp: repos
- rclone: repos (enabling many different cloud providers)
- s3/b3: repos
- ssh: repos using client/server rpc code similar as in borg 1.x
So, only for the last method we have a borg server-side process that could enforce some features, but not for any of the other repo types.
For quotas the current idea is that this should not be done within borg,
but enforced by a storage specific quota implementation (like fs quota,
or quota of the cloud storage provider). borg could offer information
about overall repo space used, but would not enforce quotas within borg.
Well, it's not totally removed, some code in Item, Archive and
borg transfer --from-borg1 needs to stay in place, so that we
can pick the CORRECT chunks list that is in .chunks_healthy
for all-zero-replacement-chunk-patched items when transferring
archives from borg1 to borg2 repos.
transfer: do not transfer replacement chunks, deal with missing chunks in other_repo
FUSE fs read: IOError or all-zero result
fixes#8641
In the example, setting SYSTEMD_WANTS instead of appending may prevent
other autostart services attached by earlier udev rules from launching.
This commit changes = to += to fix this behavior.
fixes#8639
The priority of 40 for the udev rules as stated in to documentation
applies the rule too early on some systems, which prevents the rule from
matching. This commit changes the priority to 80.
/Users/tw/w/borg/docs/internals/data-structures.rst:971:
WARNING: Lexing literal_block
'
[cache]
version = 1
repository = 3c4...e59
manifest = 10e...21c
timestamp = 2017-06-01T21:31:39.699514
key_type = 2
previous_location = /path/to/repo
[integrity]
manifest = 10e...21c
files = {"algorithm": "XXH64", "digests": {"HashHeader": "eab...39e3", "final": "e2a...b24"}}
'
as "ini" resulted in an error at token: '}'.
Retrying in relaxed mode. [misc.highlighting_failure]
Note: this part of the docs didn't change for a long time, so I guess
the sudden warning comes from a change in sphinx' lexers.
Main problem is that rc != 0 will abort our CI pipeline.
see #8318
so long as it can be assumed that the user has configured a POSIX
compliant login shell, using a simple command [1] looks cleaner, as
no ``export`` or ``;`` are used.
[1] Section "2.9.1 Simple Commands" in volume "Shell & Utilities" of POSIX.1-2024
the python package pkgconfig does not need to be "preinstalled"
anymore, because our pyproject.toml cares for that. otoh, the cli tool
pkg-config must be preinstalled so that libs and headers can be found
automagically.
Also be a bit more clear about the FUSE stuff.
`setup.py` hardcoded crypto library paths for OpenBSD, causing build
issue when OpenBSD drops specific OpenSSL version. Solution is to make
paths configurable.
Addresses #8553.
- Mention zstd as the best general choice when not using lz4
(as often acknowledged by public benchmarks)
- Mention 'auto' more prominently as a good heuristic to improve
speed while retaining good compression
- Link to compression options
borg 1.x encouraged users to put everything into the archive name:
- name of the dataset
- timestamp (usually used to make the archive name unique)
- maybe also hostname (when backing up to same repo from multiple hosts)
- maybe also username (when backing up to same repo from multiple users)
borg2 now discourages users from putting the timestamp into the name,
because we rather want same name within a series of archives - thus,
the field width for the name can be narrower.
the ID of the archive is now the only unique identifier, thus it is
moved to the leftmost place.
256bits (64 hex digits) was a bit much and as borg can also deal with
abbreviated IDs, we only show 32bits (8 hex digits) by default.
the ID is followed by the timestamp (also quite "interesting", because
it usually differs for different archives).
then following are: archive name, user name, host name - these might be
always the same if there is only one series of archives in a repo.
use 2 blanks separating the fields for better readability.
- changes to locally stored files cache:
- store as files.<H(archive_name)>
- user can manually control suffix via env var
- if local files cache is not found, build from previous archive.
- enable rebuilding the files cache via loading the previous
archive's metadata from the repo (better than starting with
empty files cache and needing to read/chunk/hash all files).
previous archive == same archive name, latest timestamp in repo.
- remove AdHocCache (not needed any more, slow)
- remove BORG_CACHE_IMPL, we only have one
- remove cache lock (this was blocking parallel backups to same
repo from same machine/user).
Cache entries now have ctime AND mtime.
Note: TTL and age still needed for discarding removed files.
But due to the separate files caches per series, the TTL
was lowered to 2 (from 20).
in borg 1.x, we used to put a timestamp into the archive name to make
it unique, because borg1 required that.
borg2 does not require unique archive names, but it encourages you
to even use an identical archive name within the same SERIES of archives.
that makes matching (e.g. for prune, but also at other places) much
simpler and borg KNOWS which archives belong to the same series.
borg1 needed this due to its transactional / rollback behaviour:
if there was uncommitted stuff in the repo, next repo opening automatically
rolled back to last commit. thus we needed checkpoint archives to reference
chunks and commit the repo.
borg2 does not do that anymore, unused chunks are only removed when the
user invokes borg compact.
thus, if a borg create gets interrupted, the user can just run borg create
again and it will find some chunks are already in the repo, making progress
even if borg create gets frequently interrupted.
Note: this is the default cache implementation in borg 1.x,
it worked well, but there were some issues:
- if the local chunks cache got out of sync with the repository,
it needed an expensive rebuild from the infos in all archives.
- to optimize that, a local chunks.archive.d cache was used to
speed that up, but at the price of quite significant space needs.
AdhocCacheWithFiles replaced this with a non-persistent chunks cache,
requesting all chunkids from the repository to initialize a simplified
non-persistent chunks index, that does not do real refcounting and also
initially does not have size information for pre-existing chunks.
We want to move away from precise refcounting, LocalCache needs to die.
Also: support a "cli" env var value, that does not determine
the implementation from the env var, but rather from cli options (similar to as it was before adding BORG_CACHE_IMPL).
- macOS: run on macos-14 (on Apple Silicon!)
- macOS: use OpenSSL 3.0 from brew
- macOS: run with Python 3.11
- pip install -e .: add -v
- use up-to-date github actions
- remove libb2 references - since borg 1.2, we use blake2 indirectly via python stdlib
borg init calls this. If there is a PermissionError, it is
usually fs permission issue at path or its parent directory.
Don't give a traceback, but rather an error msg and a specific exit code.
- implement updating exit code based on severity, including modern codes
- extend print_warning with kwargs wc (warning code) and wt (warning type)
- update a global warnings_list with warning_info elements
- create a class hierarchy below BorgWarning class similar to Error class
- diff: change harmless warnings about speed to rc == 0
- delete --force --force: change harmless warnings to rc == 0
Also:
- have BackupRaceConditionError as a more precise subclass of BackupError
Hint for Cygwin users to make sure they use a virtual environment.
Not using a virtual environment will be likely troublesome if there is already a Python installed on Windows.
last coala release (0.11.0) is now over 6y old.
when using pip install coala, a ton of stuff gets installed (expected)
and a part of that downgrades some stuff we use to outdated, incompatible
versions.
when trying to run coala with python 3.11, it just crashes because the
last release was made for py35/py36 (as seen in their setup.py).
a lot of PRs and tickets pile up at the coala project on github,
but noone is maintaining it.
Move the explanation below the general explanation of the `--keep-*` option
behavior rephrase the last sentence to make it clear that it works like the
other options that were explained in the previous paragraph.
Resolves#7687
- pattern needs to start with + - !
- first match wins
- the default is to list everything, thus a 2nd pattern
is needed to exclude everything not matched by 1st pattern.
at some places, the docs were not updated yet.
for borg 1.x, -a (aka --glob-archives) expected
sh: style glob patterns ONLY (but one must not
give sh: explicitly).
for borg 2, -a (aka --match-archives) defaults
to id: style (identical match), so one must give
sh: if one wants shell-style globbing.
