Commit graph

10243 commits

Author SHA1 Message Date
Hugo Wallenburg
47d88682c1
Fixes pytest warning for non-list in parametrize 2026-06-20 18:09:40 +02:00
Hugo Wallenburg
01aff2b7e2
Sets base_timestamp for prune only once 2026-06-20 18:09:39 +02:00
Daniel Rudolf
1073da9eb1
Refactor the interval-based pruning example
Co-Authored-By: Hugo Wallenburg <hugo@hugow.no>
2026-06-20 18:09:39 +02:00
Hugo Wallenburg
d9b35f2eeb
Simplifies retention granularity ordering check 2026-06-20 18:09:39 +02:00
Hugo Wallenburg
304a2fa5b9
Removes --keep-last and --keep-within, superseded by --keep 2026-06-20 18:09:38 +02:00
Hugo Wallenburg
d9f9f1dd45
Updates prune parser epilog to match new int/interval behavior
Along with reordering and improvements.
2026-06-20 18:09:38 +02:00
Hugo Wallenburg
4382b3a8f2
Updates usage docs for prune and new int/interval handling 2026-06-20 18:09:38 +02:00
Hugo Wallenburg
7948625442
Fixes prune error tests with binary borg 2026-06-20 18:09:37 +02:00
Hugo Wallenburg
f7a848ed6f
Rewords original prune example with more precise terminology 2026-06-20 18:09:37 +02:00
Hugo Wallenburg
318c53fcf9
Adds full-scale example using pruning intervals
Also rewrites some of the older example to match terminology and wording
from the newer example.
2026-06-20 18:09:37 +02:00
Hugo Wallenburg
369fd466f1
Fixes timestamp parsing error on Python 3.10 2026-06-20 18:09:36 +02:00
Hugo Wallenburg
72e72dc431
Adds 'prune --since', base timestamp to prune from
A flag to explicitly "save" all archives older than a certain timestamp,
and to function as a base timestamp from which to base interval
timedelta calculations. Allows for precise time interval manipulation
for superusers and as a bonus simplifies time-based testing and
alleviates the need for an external dependency to freeze time in test.

Includes some refactoring of do_prune for logical flow & naming that
came up while iterating on these changes.
2026-06-20 18:09:36 +02:00
Hugo Wallenburg
eca719aeec
Assert output format without dry run in basic pruning test 2026-06-20 18:09:36 +02:00
Hugo Wallenburg
4bf60336b2
Adds optional interval support for all prune retention flags
Support is added for setting prune retention with either an int (keep n
archives) or an interval (keep within). This works much like
--keep-within currently does, but extends support to all retention
filters.

Additionally adds a generic --keep flag to take over (or live alongside)
both --keep-last and --keep-within. --keep-last is no longer an alias of
--keep-secondly, now keeps archives made on the same second.

Comparisons against archive timestamp are made to use local timezone
instead of UTC. Should be equal result in practice, but allows for
easier testing with frozen local time.
2026-06-20 18:09:36 +02:00
Hugo Wallenburg
5f07e76ef7
Adds int_or_interval format parser
Accepts either int or interval, first tries parsing int then tries
parsing as interval if that fails. Returns a timedelta for easy date
math later. Now allows intervals of length 0 as a 0-length timedelta is
perfectly fine to work with.
2026-06-20 18:09:35 +02:00
TW
5eb194cdc6
Merge pull request #9795 from mr-raj12/packreader-class
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
repository: add PackReader to walk pack object headers
2026-06-20 17:26:21 +02:00
Mrityunjay Raj
ccf8084cc4 repository: address review, build PackReader key from pack_id and detect in-memory packs in iter_headers 2026-06-20 19:32:57 +05:30
Mrityunjay Raj
e0d781d50c repository: add PackReader to walk pack object headers, refs #8476
Move pack-level header iteration out of RepoObj into a PackReader class and use it to build the chunk index.
2026-06-20 17:13:17 +05:30
TW
5d80b95a12
Merge pull request #9794 from mr-raj12/pack-files-step9-check-repo-readonly-storehash
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
repository: read-only check by hashing pack/index objects
2026-06-19 21:30:05 +02:00
Mrityunjay Raj
27d693c222 repository: read-only check by hashing pack/index objects, refs #8476 #8572
Implement the read-only repository check by hashing pack and index
objects and comparing against the stored hashes, without writing to the
repository.

Report check progress separately for the index and for the packs, each
ending at 100%.
2026-06-19 23:30:33 +05:30
TW
cd91cfdab8
Merge pull request #9793 from ThomasWaldmann/support-msgpack121-master
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
support msgpack 1.2.1
2026-06-18 23:42:00 +02:00
Thomas Waldmann
890da42276
support msgpack 1.2.1
Port of borgbackup/borg#9790 to master.

Note: the PR's second commit (Archive.delete: don't reuse msgpack Unpacker
after an unpacking failure) does not apply here - on master Archive.delete
no longer unpacks item metadata (it just removes the archive and lets
"borg compact" reclaim space), so the reuse-after-failure code path does
not exist. The streaming RobustUnpacker already creates a fresh Unpacker
on resync.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 22:08:48 +02:00
TW
1dce2eba20
Merge pull request #9789 from mr-raj12/pack-files-step8-sha256-pack-id
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
repository: always name packs by sha256(pack)
2026-06-18 17:52:47 +02:00
Mrityunjay Raj
0ece418991 cleanup: drop leftover pack_id==chunk_id references after sha256 naming, refs #8572
Remove the dead BORG_TESTONLY_SHA256_PACK_ID tox env and CI job, fix the packs.rst pack-id docs, and reword comments/tests to describe sha256 pack naming instead of the removed shortcut.
2026-06-18 19:33:50 +05:30
Mrityunjay Raj
fd17753561 repository: always name packs by sha256(pack), refs #8572
Drop the single-chunk shortcut that reused the chunk_id as the pack_id.
A pack is now always named by sha256 of its bytes, even when it holds a
single chunk, so no code can depend on pack_id == chunk_id.
2026-06-18 18:50:46 +05:30
TW
c63435ebc2
Merge pull request #9787 from ThomasWaldmann/cleanup-common-options
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI / sha256 pack-id (informational) (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
remote: mark legacy cli options for ssh-rpc remoting
2026-06-18 00:44:23 +02:00
Thomas Waldmann
509c90df3b
remote: mark legacy cli options for ssh-rpc remoting 2026-06-18 00:42:21 +02:00
TW
7df216206f
Merge pull request #9783 from mr-raj12/repository-check-delete-sha256-pack-ids
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI / sha256 pack-id (informational) (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
repository: make check and delete work at N=1 with sha256 pack ids
2026-06-17 17:41:12 +02:00
Mrityunjay Raj
bec9446a45 repository, repoobj, cache: make check/delete/compact work with sha256 pack_ids, refs #8572 2026-06-17 19:16:53 +05:30
TW
47764e2f70
Merge pull request #9785 from ThomasWaldmann/docs-fixes-master
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI / sha256 pack-id (informational) (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
docs fixes (master)
2026-06-16 15:56:02 +02:00
Thomas Waldmann
912b2c50c8
docs: update structure diagram 2026-06-16 15:51:00 +02:00
Thomas Waldmann
48308f1969
misc docs updates 2026-06-16 15:21:57 +02:00
Thomas Waldmann
a7990c3702
update general docs 2026-06-16 14:46:57 +02:00
Thomas Waldmann
ce19629b34
docs: update repo-create examples for split --encryption/--id-hash
The repo-create.rst examples were not updated after --encryption was
split into --encryption + --id-hash. Use the real mode name (aes256-ocb)
and show --id-hash as an orthogonal option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 14:28:03 +02:00
TW
2a82eb0b94
Merge pull request #9784 from ThomasWaldmann/build-usage-man-master
build usage / man
2026-06-16 14:20:17 +02:00
Thomas Waldmann
c4347a9dce
build usage / man 2026-06-16 14:17:43 +02:00
TW
8f6d55cafb
Merge pull request #9782 from ThomasWaldmann/remove-repository-wait-async
repository: drop "wait" arg, make put/delete synchronous
2026-06-16 14:08:41 +02:00
TW
972ccabf29
Merge pull request #9781 from borgbackup/dependabot/pip/requirements.d/pip-dependencies-39deabeb19
build(deps-dev): bump the pip-dependencies group in /requirements.d with 2 updates
2026-06-16 11:53:36 +02:00
Thomas Waldmann
06b10043ec
repository: drop the dead "wait" arg, make put/delete always synchronous
The new borgstore-based Repository inherited put(..., wait=) / delete(..., wait=)
and async_response() from the legacy RemoteRepository's pipelined RPC protocol.
In the new architecture these are dead: borgstore's API is strictly synchronous,
so put/delete ignored wait and ran synchronously, and async_response() was an
empty stub always returning None.

Remove wait and async_response() so the synchronous behavior is explicit, and
clean up every caller that still threaded wait=False / drained async_response()
(cache.add_chunk, archive.py, transfer_cmd.py, and the archive_test mock).

The legacy repository/remote keep their real wait/async implementation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 11:38:35 +02:00
dependabot[bot]
a13672fbd7
build(deps-dev): bump the pip-dependencies group
Bumps the pip-dependencies group in /requirements.d with 2 updates: [tox](https://github.com/tox-dev/tox) and [coverage](https://github.com/coveragepy/coveragepy).


Updates `tox` from 4.53.1 to 4.54.0
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/tox/compare/4.53.1...4.54.0)

Updates `coverage` from 7.13.5 to 7.14.0
- [Release notes](https://github.com/coveragepy/coveragepy/releases)
- [Changelog](https://github.com/coveragepy/coveragepy/blob/main/CHANGES.rst)
- [Commits](https://github.com/coveragepy/coveragepy/compare/7.13.5...7.14.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-version: 4.54.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: pip-dependencies
- dependency-name: coverage
  dependency-version: 7.14.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: pip-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-16 05:43:04 +00:00
TW
b483c9ee11
Merge pull request #9779 from ThomasWaldmann/split-encryption-id-hash-9168
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI / sha256 pack-id (informational) (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
repo-create: split --encryption into --encryption + --id-hash (#9168)
2026-06-15 20:19:12 +02:00
TW
ebfe0a1855
Merge pull request #9778 from mr-raj12/chunks-index-storage
index: move chunk index to index/ namespace, drop hash seed
2026-06-15 20:17:41 +02:00
Thomas Waldmann
2eef6c8035
crypto: drop the NAME attribute; repo-info builds the suite string from ENC_NAME/IDHASH_NAME
NAME was only read by "borg repo-info" to show the crypto suite. Remove it from
all (current and legacy) key classes and let repo-info assemble the display from
the two real dimensions instead: "<mode>, <ENC_NAME>, <IDHASH_NAME>", e.g.
"Encrypted: Yes (repokey, aes256-ocb, sha256)" or
"Encrypted: No (repokey, authenticated, blake3)".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 19:38:32 +02:00
Mrityunjay Raj
eb30123614 index: move chunk index to index/ namespace, drop hash seed
Name objects by pure sha256 so borgstore can verify them. closes #9758
2026-06-15 23:04:23 +05:30
Thomas Waldmann
98b428b146
repo-create: gate key/passphrase warning on ENC_NAME, not the NAME display string
The "you need KEY AND PASSPHRASE" warning was gated on key.NAME != "plaintext",
a brittle dependency on a human-readable display string. Use the cipher dimension
instead: only plaintext has ENC_NAME == "none"; every key-bearing suite (including
the authenticated ones, which should warn) differs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 19:32:13 +02:00
Thomas Waldmann
ee2aee0a31
repo-info/list --json: emit encryption.keyfile based on key storage, not NAME
The keyfile detection used `key.NAME.startswith("key file")`, but since the
keyfile/repokey unification no key class has such a NAME, so encryption.keyfile
was never emitted (even for keyfile repos). Decide it from the key storage
(KeyBlobStorage.KEYFILE), matching the text repo-info output. Add a test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 19:31:35 +02:00
Thomas Waldmann
2f59abe205
crypto: drop dead ARG_NAME; give legacy key classes ENC_NAME/IDHASH_NAME
ARG_NAME had no readers left after the JSON output switched to ENC_NAME/IDHASH_NAME,
so remove it from all (current and legacy) key classes.

Give the legacy (borg 1.x, read-only) key classes the two dimensions so that
repo-info/list --json reports a meaningful crypto suite for legacy repos instead
of null/null:
- AESCTRKey:            encryption=aes256-ctr, id_hash=sha256
- Blake2AESCTRKey:      encryption=aes256-ctr, id_hash=blake2
- Blake2AuthenticatedKey: encryption=authenticated, id_hash=blake2

IDHASH_NAME="blake2" is set on the ID_BLAKE2b_256 mix-in (parallel to the
ID_HMAC_SHA_256 / ID_BLAKE3_256 mix-ins). These legacy values never become CLI
choices: encryption_argument_names()/id_hash_argument_names() only iterate
AVAILABLE_KEY_TYPES, not LEGACY_KEY_TYPES.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 18:39:41 +02:00
Thomas Waldmann
479a77b512
repo-info/list --json: report encryption + id_hash separately, #9168
Stop using key.ARG_NAME (the combined crypto-suite name) for the JSON output.
The "encryption" object now mirrors the split CLI options: the "mode" field is
replaced by "encryption" (cipher / AE algorithm, key.ENC_NAME) and "id_hash"
(id hash function, key.IDHASH_NAME).

This is a breaking change to the documented JSON API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 18:31:27 +02:00
Thomas Waldmann
29a760e2b3
repo-create: split --encryption into --encryption + --id-hash, #9168
The combined --encryption value packed two orthogonal dimensions (cipher / AE
algorithm and id hash function) into a single string, causing a combinatorial
explosion of mode names. Key location was already split out into --key-location.

Now:
- --encryption selects only the cipher / AE algorithm:
  none, authenticated, aes256-ocb, chacha20-poly1305
- --id-hash selects the id hash function: sha256 (default) or blake3
- --key-location (unchanged) selects key storage: repokey (default) or keyfile

The old combined names were removed (clean break): select a BLAKE3 suite via
--encryption ... --id-hash blake3 instead of blake3-*. aes-ocb was renamed to
aes256-ocb (key NAME shown by repo-info and ARG_NAME in JSON updated to match).
"none" has no key, so it only supports the sha256 id hash.

No on-disk format, key-type byte, or crypto behavior changes: the existing key
classes form a clean cross-product of {cipher} x {id-hash}, selected via the new
ENC_NAME / IDHASH_NAME class attributes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 17:44:38 +02:00
TW
dbc38ab66c
Merge pull request #9740 from borgbackup/dependabot/github_actions/actions-484570b1b1
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI / sha256 pack-id (informational) (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
build(deps): bump codecov/codecov-action from 6 to 7 in the actions group
2026-06-15 01:54:28 +02:00