Borg used to read the manifest's key-type byte and then look for the key in
exactly one place (keyfile or repokey) depending on the key class that byte
selected. As a result every crypto suite was duplicated into a keyfile class
and a repokey class that differed only in TYPE, NAME, ARG_NAME and STORAGE.
Now key *location* is independent of the type byte: detection tries keyfiles
first and repokeys afterwards until a passphrase unlocks a key. The type byte
still selects the crypto suite (id hash, MAC, cipher) to instantiate. Where a
key is stored (keyfile vs repokey) is therefore a per-key property
(self.storage), not a separate class, so a repository may even hold a mix of
keyfile- and repo-stored borg keys.
With storage decoupled from class identity, the keyfile/repokey class pairs
collapse into one class per crypto suite:
- modern AEAD: AESOCBKey, CHPOKey, Blake3AESOCBKey, Blake3CHPOKey
- legacy borg 1.x (read-only): AESCTRKey, Blake2AESCTRKey
There is now exactly one type byte per modern crypto suite (the old separate
repokey type bytes 0x11/0x21/0x31/0x41 were removed; borg2 is beta and only
needs to read repos it created). identify_key() matches on TYPES_ACCEPTABLE.
CLI: --encryption selects only the crypto suite (aes-ocb, chacha20-poly1305,
blake3-aes-ocb, blake3-chacha20-poly1305, authenticated*, none); the storage
location is chosen with the new --key-location=repokey|keyfile (default
repokey). The old combined modes (repokey-aes-ocb etc.) were removed.
borg key import also gained --key-location. borg key change-location no longer
swaps key classes or rewrites the manifest; it just re-saves the unlocked key
at the new location.
Keyfile removal (key remove, change-location) now overwrites the keyfile with
random data via secure_erase() before unlinking, consistent with save().
borg 1.x legacy read compatibility is preserved (the legacy class merge is a
behavior-preserving rename; the legacy type bytes incl. PASSPHRASE stay in
TYPES_ACCEPTABLE).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The unix-socket transport (socket:// repositories, the --socket option and
"borg serve" over a socket) was never part of a stable borg 2 release and the
old RPC protocol it relied on is gone, so the remaining code was dead:
- legacy remote: drop the unreachable proto == "socket" connection branch and
the now-unused self.sock handling, "import socket" and get_socket_filename
import (LegacyRemoteRepository is only built for proto == "ssh")
- helpers: remove get_socket_filename() and its export
- parseformat: drop "socket" from local_path_re - socket:// is now treated like
any other unknown scheme (a local path) rather than being special-cased
- tests: drop test_socket and the self.sock check in the legacy reopen helper
- docs: drop the stale --socket entry from the manually maintained
common-options.rst.inc (the auto-generated usage/man docs are left untouched
here and will be rebuilt in a separate commit)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
For sftp/http(s)/s3/b2/rclone repositories, borg only detects the scheme now
and hands the raw URL to borgstore, which parses and validates it - removing
the duplicate parsing borg used to do. Precise field extraction (user/host/
port/path) is kept only for the protocols borg itself reads: file, rest and
legacy ssh.
- drop http_re, s3_re, rclone_re and the sftp arm of the old ssh_or_sftp_re
- add a single scheme-detection pass-through against BORGSTORE_SCHEMES; reject
unknown schemes (e.g. socket://) as before
- canonical_path() returns the processed URL for the delegated protocols, with
embedded credentials stripped so secrets never reach the security state file
or logs
- source local_path_re's scheme exclusions from BORGSTORE_SCHEMES
- create: use proto == "file" instead of "not location.host" for the local
repo-dir inode skip
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
If `borg compact` was interrupted after deleting repository objects but
before writing the updated chunk index, the still-existing cache/chunks.*
kept claiming the deleted objects were present. A later `borg create` would
trust that stale index, skip re-uploading the affected chunks and silently
create an archive with dangling object references that extracts to zero bytes.
Invalidate all cached chunk indexes via delete_chunkindex_cache() before the
first object is deleted, so an interruption is conservative: the next client
rebuilds the index from actual repository contents and re-uploads any deleted
data. The post-deletion save_chunk_index() still writes a fresh, valid index.
Add a regression test covering both compact paths (default and --stats) that
interrupts compaction right before save_chunk_index() and verifies no cached
chunk index survives and a later create+extract reproduces the original bytes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FilesCacheMixin initialized _newest_cmtime to 0, but _write_files_cache()
only treats None as "no file was chunked this run" (falling back to a
max_time_ns cutoff that keeps all current entries).
When a backup reuses all files from the cache without chunking anything,
_newest_cmtime stayed at 0, so the race-protection cutoff became the unix
epoch and every current (age == 0) entry was discarded. The next backup
then had to re-read, chunk and hash all files again.
Initialize _newest_cmtime to None to match the documented contract in
_write_files_cache(), and make the comparisons in _build_files_cache()
None-safe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make `borg serve` able to be the server-side component of a rest:// repository,
selected with a new --rest option. Plain `borg serve` (no option) keeps serving
legacy borg-1.x repos and stays command-line compatible with borg 1.x.
- serve_cmd.py: add --rest and --backend. With --rest, serve the given
--backend FILE:<path> on stdio via borgstore.server.rest.serve(); honor
--restrict-to-path/--restrict-to-repository (validated against the FILE path)
and --permissions (mapped via borg_permissions). Without --rest, run the legacy
RepositoryServer as before.
- repository.py: for rest:// locations, build the borgstore REST backend with a
command that runs `borg serve --rest --backend FILE:<path>` (locally via
sys.executable, or over ssh reusing borgstore's ssh_cmd / BORG_REMOTE_PATH),
instead of borgstore's hardcoded `borgstore-server-rest`. So a remote only needs
borg installed. Extracted the permissions string->dict mapping into the reusable
borg_permissions().
- tests: unit tests for the rest serve command builder. The existing
remote_archiver (rest:///) suite now runs against `borg serve --rest`.
- docs: changelog + quickstart updated.
Legacy serve and the legacy ssh client are unchanged (client still spawns plain
`borg serve`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The modern client/server transport (RemoteRepository served by `borg serve`
over an msgpack RPC protocol) is now redundant for current (borg 2) repos:
its functionality is replaced by rest:// (which can tunnel over ssh to a
remote borgstore REST server).
Remove the modern RemoteRepository (both ssh:// and socket://) entirely.
Legacy v1 (borg 1.x) repos remain reachable over ssh:// via the separate
LegacyRemoteRepository client, and `borg serve` / RepositoryServer is kept,
trimmed to the legacy-only path, so a remote borg2 can still serve a v1 repo
for `borg transfer --from-borg1`.
Details:
- remote.py: delete RemoteRepository, SleepingBandwidthLimiter and the `api`
decorator; trim RepositoryServer to legacy-only (drop modern _rpc_methods,
socket serving, non-legacy open() branch); keep cache_if_remote /
RepositoryCache / RepositoryNoCache (used by all repos).
- get_repository(): non-legacy ssh:// now raises a clear "use rest://" error;
socket:// route and the global --socket option removed.
- parseformat: drop the socket:// scheme (now an invalid location).
- borg serve: keep the command (serves legacy v1 ssh only); update epilog.
- borg version: drop modern remote query; keep legacy ssh path.
- update isinstance/import sites (cache, archive, fuse/hlfuse, analyze/compact,
archiver __init__ -> LegacyRemoteRepository.RPCError).
- tests/docs updated; obsolete socket serve test removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The move to platformdirs and its current usage _does_ honor XDG_*
variables on macOS if they are set. Tests were set up to assume this to
be untrue and the docs matched that.
This commit adds tests asserting that XDG_* variables are used when they
are present on macOS, with default locations still in ~/Library.
Remove the handwritten bash and zsh shell completion scripts now that
auto-generated completions via borg completion bash/zsh (powered by
shtab, #9172) are tested and working. Fish completions are kept as
shtab does not yet support fish.
Replace string-matching tests with focused behavior tests: script size
sanity, shell syntax validation (bash -n / zsh -n), and tests that
invoke the custom preamble functions in bash (sortby key dedup,
filescachemode mutual exclusivity, archive name and aid: prefix
completion against a real repository).
