PathNotAllowed lived in borg.legacy.remote, but borg serve --rest
(non-legacy) now also raises it via check_rest_restrictions, which made
non-legacy code import from the legacy package just for an exception.
It is a generic "repository path not allowed" error, so move it next to
the other cross-cutting Error subclasses in helpers/errors.py and
re-export it from helpers. Pure relocation; exit code stays 83.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A rest:// repository is now served by "borg serve --rest" spawned over ssh
rather than borgstore's "borgstore-server-rest".
CI: chmod o+x $HOME so the rest test's ssh user (sftpuser) can run borg
The rest repo test starts "borg serve --rest" over ssh as sftpuser, which runs
the borg under test from the tox venv under the runner $HOME.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make `borg serve` able to be the server-side component of a rest:// repository,
selected with a new --rest option. Plain `borg serve` (no option) keeps serving
legacy borg-1.x repos and stays command-line compatible with borg 1.x.
- serve_cmd.py: add --rest and --backend. With --rest, serve the given
--backend FILE:<path> on stdio via borgstore.server.rest.serve(); honor
--restrict-to-path/--restrict-to-repository (validated against the FILE path)
and --permissions (mapped via borg_permissions). Without --rest, run the legacy
RepositoryServer as before.
- repository.py: for rest:// locations, build the borgstore REST backend with a
command that runs `borg serve --rest --backend FILE:<path>` (locally via
sys.executable, or over ssh reusing borgstore's ssh_cmd / BORG_REMOTE_PATH),
instead of borgstore's hardcoded `borgstore-server-rest`. So a remote only needs
borg installed. Extracted the permissions string->dict mapping into the reusable
borg_permissions().
- tests: unit tests for the rest serve command builder. The existing
remote_archiver (rest:///) suite now runs against `borg serve --rest`.
- docs: changelog + quickstart updated.
Legacy serve and the legacy ssh client are unchanged (client still spawns plain
`borg serve`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The existing --from-borg1 transfer tests only use a local v1 repo, so they
exercise LegacyRepository but never the ssh path (LegacyRemoteRepository +
borg serve / RepositoryServer) that this branch preserves.
Add test_transfer_from_borg1_ssh: extract the repo12.tar.gz borg 1.2 repo and
transfer from it via --other-repo=ssh://__testsuite__/<abspath> --from-borg1.
The __testsuite__ host makes the legacy client spawn a local "borg serve"
(no real ssh), driving the full client -> serve -> LegacyRepository chain, then
asserts all archives transferred. Local/non-win32 only, like the sibling tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
borg.remote no longer fit its name: it held the legacy-only borg serve server
plus generic repository cache wrappers used by current repos. Split by purpose
and remove the module:
- Move RepositoryServer into borg.legacy.remote (it only serves legacy v1 ssh
repositories). It reuses the exception classes (PathNotAllowed,
InvalidRPCMethod, UnexpectedRPCDataFormatFromClient) and BORG_VERSION / MSGID
constants already defined there; open() uses the module-level LegacyRepository.
serve_cmd.py now imports RepositoryServer from ..legacy.remote.
- Move RepositoryNoCache and cache_if_remote into borg.repository (they wrap a
Repository and are used by Archive.check and mount of current repos).
archive.py and mount_cmds.py import them from ..repository now.
- Move the cache_if_remote tests into repository_test.py; delete remote_test.py.
- Delete src/borg/remote.py; fix the stale BUFSIZE comment in constants.py.
Pure relocation, no behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
borg.legacy.remote.cache_if_remote (and the RepositoryCache / RepositoryNoCache
classes it returns) are dead code: nothing imports or calls them. Every
cache_if_remote consumer (archive check, mount, tests) uses the non-legacy
borg.remote version, and legacy repos never reach it (Archive.check rejects
legacy repos). The trio was copied wholesale during the borg.legacy split (#9556).
Delete RepositoryNoCache, RepositoryCache and cache_if_remote, plus the imports
that only they used (shutil, struct, tempfile, xxhash.xxh64, compress.Compressor,
helpers.safe_unlink). LegacyRemoteRepository and the rest of the module are
unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After removing the modern RemoteRepository, cache_if_remote always returned
RepositoryNoCache in production (the only RepositoryCache path was the removed
isinstance(RemoteRepository) check; force_cache=True was used only by a test).
Delete the vestigial RepositoryCache class and simplify cache_if_remote: drop
the pack/unpack/force_cache parameters and the LZ4/xxh64 cache-file machinery,
keep building the decrypted_cache -> transform closure, and always return
RepositoryNoCache. Remove the imports that only RepositoryCache used.
Replace the RepositoryCache tests with a focused test of the surviving
cache_if_remote path (plain passthrough and decrypted (csize, plaintext) tuples).
The legacy copy in borg/legacy/remote.py is intentionally left untouched (its
RepositoryCache is still used for LegacyRemoteRepository).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The modern client/server transport (RemoteRepository served by `borg serve`
over an msgpack RPC protocol) is now redundant for current (borg 2) repos:
its functionality is replaced by rest:// (which can tunnel over ssh to a
remote borgstore REST server).
Remove the modern RemoteRepository (both ssh:// and socket://) entirely.
Legacy v1 (borg 1.x) repos remain reachable over ssh:// via the separate
LegacyRemoteRepository client, and `borg serve` / RepositoryServer is kept,
trimmed to the legacy-only path, so a remote borg2 can still serve a v1 repo
for `borg transfer --from-borg1`.
Details:
- remote.py: delete RemoteRepository, SleepingBandwidthLimiter and the `api`
decorator; trim RepositoryServer to legacy-only (drop modern _rpc_methods,
socket serving, non-legacy open() branch); keep cache_if_remote /
RepositoryCache / RepositoryNoCache (used by all repos).
- get_repository(): non-legacy ssh:// now raises a clear "use rest://" error;
socket:// route and the global --socket option removed.
- parseformat: drop the socket:// scheme (now an invalid location).
- borg serve: keep the command (serves legacy v1 ssh only); update epilog.
- borg version: drop modern remote query; keep legacy ssh path.
- update isinstance/import sites (cache, archive, fuse/hlfuse, analyze/compact,
archiver __init__ -> LegacyRemoteRepository.RPCError).
- tests/docs updated; obsolete socket serve test removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BLAKE3 is generally faster and provides a more modern construction for
keyed hashing (using its internal keyed mode instead of the construction
used for BLAKE2b).
Key types changed:
- authenticated-blake2 -> authenticated-blake3
- {keyfile,repokey}-blake2-aes-ocb -> {keyfile,repokey}-blake3-aes-ocb
- {keyfile,repokey}-blake2-chacha20-poly1305 -> {keyfile,repokey}-blake3-chacha20-poly1305
This also fixes the slightly unusual way how we used blake2b,
it is only supported for importing borg 1.x repos.
New repos either use HMAC-SHA256 or BLAKE3.
working with r1beta5 (from 2024) is just too much pain.
the system packages only have python 3.10.
if one install python 3.11 from HaikuPorts, it has no ssl support.
if one also installs openssl3 from HaikuPorts, creating a venv fails...
also: rust toolchains issues, thread-local storage ("TLS") issues, as seen in #9463.
thus: no haiku CI until they release next beta and cross-platform-actions have it.
RepoObj.extract_crypted_data() / parse_meta() / parse() unpacked the
fixed-size object header without first checking the input length, and
guarded the meta/data sizes only with assert. A too-short object (e.g. a
truncated or malicious repository object) therefore raised an uncaught
struct.error, and a header claiming more meta/data than present raised
AssertionError.
Callers handle repository corruption by catching IntegrityError, so these
unintended exception types escaped that handling and aborted the operation
with a traceback instead of a clean "corrupted object" report.
Validate the input length before unpacking the header and turn the size
consistency checks into IntegrityError.
Add regression tests for too-short objects and inconsistent meta/data sizes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add three new fields to ChunkIndexEntry and update all call sites:
- pack_id (32 bytes): identifies the pack file containing the chunk
- obj_offset (uint32): byte offset of the chunk within the pack
- obj_size (uint32): stored (compressed) size of the chunk on disk
At N=1 (one chunk per pack), chunk_id == pack_id, obj_offset == 0,
and obj_size == pack_size. All sites use chunk_id as the ChunkIndex
key and extract pack_id as a separate variable with an N=1 comment.
compact_cmd.py: use obj_size (stored size) in repository_size sum.
cache.py: preserve pack fields when serializing the chunk index cache.
repository.py: populate pack_id/obj_size from borgstore object info.
archive.py: extract pack_id on its own line, obj_size=0 for now.
hashindex.pyx: update add(), namedtuple, format string, and docstring.
hashindex.pyi: add new fields to ChunkIndexEntry and CIE type alias.
testsuite/hashindex_test.py: update all ChunkIndexEntry constructions.
