Commit graph

10285 commits

Author SHA1 Message Date
Mrityunjay Raj
2dcb50f30e repository: assert compact_pack complete coverage reaches pack end 2026-06-28 03:46:35 +05:30
Mrityunjay Raj
2a4671c1ab tests: extract corrupt_chunk_on_disk helper into repository_test.py 2026-06-28 01:43:01 +05:30
Mrityunjay Raj
f295b3e77a repository: simplify delete/compact_pack docstrings and comments
Rename delete's persist kwarg to update_index; corrupt pack bytes on disk
in test_verify_data instead of via delete/put.
2026-06-28 00:07:38 +05:30
Mrityunjay Raj
4fe7e0ea16 repository: fix Repository.delete edge cases (#9815)
Tolerate a stale earlier object in the pack via a compact_pack complete flag, skip the per-delete index write in check --repair via a delete persist flag, and make verify_data report the files whose chunks it removes. Add a regression test for the stale-earlier-object case.
2026-06-27 10:54:57 +05:30
TW
f3933988c5
Merge pull request #9813 from mr-raj12/pack-files-cleanup
Some checks failed
Lint / lint (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
CI / asan_ubsan (push) Has been cancelled
CI / native_tests (push) Has been cancelled
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Has been cancelled
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Has been cancelled
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Has been cancelled
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Has been cancelled
CI / windows_tests (push) Has been cancelled
pack-files: never drop a whole pack to delete a single object
2026-06-26 20:58:01 +02:00
Mrityunjay Raj
106dfbadae tests: drop the delete_chunk wrapper, call Repository.delete directly
Inline repository.delete at the check/extract/mount call sites. Drop the borgstore HEAD note from compact_pack now that it is fixed upstream.
2026-06-26 23:24:57 +05:30
Mrityunjay Raj
e74d7ef9a0 repository: make Repository.delete a real single-object delete
Rewrite the object's pack via compact_pack and persist a full chunk index. Wire debug delete-obj and the delete_chunk test helper to it, and drop the store.info HEAD coverage check from compact_pack.
2026-06-26 09:51:57 +05:30
Mrityunjay Raj
5747a19cf8 tests: delete_chunk drops a single object via compact_pack, not the whole pack
Rewrite the damage helper to rewrite the pack without the target chunk (neighbours survive) and persist the index, then revive three check tests that were skipped because the old whole-pack drop left an inconsistent index.
2026-06-26 04:05:32 +05:30
Mrityunjay Raj
82c589f3da debug delete-obj: make single-object delete a no-op instead of dropping the whole pack
Add explicit flush() in two repository tests so they no longer rely on the put count filling whole packs before close()/get().
2026-06-26 03:40:51 +05:30
Mrityunjay Raj
9fff040d58 comments: drop N=1/N>1 pack-files scaffolding, describe the general case
These comments were phrased around development milestones (N=1, N>1, pack_id == chunk_id); with multi-object packs in place that framing is stale. Reword for the general case. Comment-only, no behavior change.
2026-06-26 01:46:32 +05:30
Mrityunjay Raj
62f2e8b05f
repository: store more than one object per pack (#9808)
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
* repository: store more than one object per pack

Raise PackWriter max_count to 2 and flush buffered packs before the archives pointer is written; also flush after debug put-obj and in the affected tests.

* archive: flush buffered packs in Archives.create() so no caller writes the pointer over buffered objects
2026-06-25 21:33:09 +02:00
TW
e57e22d174
Merge pull request #9801 from mr-raj12/pack-files-compact
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
compact: free space per pack with --threshold
2026-06-24 18:04:01 +02:00
Mrityunjay Raj
4a28995f04 compact: --stats reports sizes from the cached index, drop the redundant full scan and fix the epilog 2026-06-24 20:15:46 +05:30
Mrityunjay Raj
0586cc7b7b compact: report progress after each unit so it reaches 100% 2026-06-24 18:14:05 +05:30
Mrityunjay Raj
78b8bbab50 compact: pack-level compaction with --threshold and --dry-run
Rewrite a pack only when its unused bytes reach --threshold percent; --dry-run reports the space compact would free without changing the repository (#9379).
2026-06-24 11:10:42 +05:30
Mrityunjay Raj
58cd86c3d4 compact: free space per pack, honor --threshold
Group objects by their pack and act per pack: drop fully-unused packs, and
rewrite mixed packs whose unused bytes reach --threshold (default 40%) by
copying the survivors forward with compact_pack. Two index scans keep the
memory use bounded. refs #8572 #8514
2026-06-24 11:10:42 +05:30
TW
13fa6237ee
Merge pull request #9804 from ThomasWaldmann/interals-docs-update
Some checks failed
Lint / lint (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
CI / asan_ubsan (push) Has been cancelled
CI / native_tests (push) Has been cancelled
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Has been cancelled
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Has been cancelled
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Has been cancelled
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Has been cancelled
CI / windows_tests (push) Has been cancelled
docs: update / fix "internals" section
2026-06-23 11:36:08 +02:00
Thomas Waldmann
66d010c2ac
docs: update / fix "internals" section 2026-06-23 11:19:38 +02:00
TW
dd67bf371f
Merge pull request #9800 from ThomasWaldmann/requirements-fixes-master
Requirements fixes (master)
2026-06-23 10:45:37 +02:00
Thomas Waldmann
5237f9a002
setuptools-scm: tag.strict = false 2026-06-23 09:46:25 +02:00
Thomas Waldmann
84281c3670
requirements.d/development.*: remove requirements that are in pyproject.toml build-system 2026-06-23 09:46:22 +02:00
Thomas Waldmann
4fdf38ebcf
setuptools-scm: be consistent with dash and toml requirement 2026-06-23 09:46:20 +02:00
Thomas Waldmann
aa9a4c1416
blake3: exclude 1.0.9 release
1.0.9 does not build on windows/msys2/mingw.
2026-06-23 09:46:14 +02:00
TW
cf6d70fd61
Merge pull request #9803 from borgbackup/dependabot/github_actions/actions-d9db5a10fc
build(deps): bump the actions group with 2 updates
2026-06-23 09:44:04 +02:00
TW
cb6b4a32d5
Merge pull request #9802 from borgbackup/dependabot/pip/requirements.d/pip-dependencies-7be846ec3e
build(deps-dev): bump coverage from 7.14.0 to 7.14.3 in /requirements.d in the pip-dependencies group
2026-06-23 09:43:31 +02:00
dependabot[bot]
ea6ef5ffbf
build(deps): bump the actions group with 2 updates
Bumps the actions group with 2 updates: [actions/checkout](https://github.com/actions/checkout) and [cross-platform-actions/action](https://github.com/cross-platform-actions/action).


Updates `actions/checkout` from 6 to 7
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

Updates `cross-platform-actions/action` from 1.2.0 to 1.3.0
- [Release notes](https://github.com/cross-platform-actions/action/releases)
- [Changelog](https://github.com/cross-platform-actions/action/blob/master/changelog.md)
- [Commits](https://github.com/cross-platform-actions/action/compare/v1.2.0...v1.3.0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: cross-platform-actions/action
  dependency-version: 1.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-23 05:43:34 +00:00
dependabot[bot]
7e66e1e1e1
build(deps-dev): bump coverage
Bumps the pip-dependencies group in /requirements.d with 1 update: [coverage](https://github.com/coveragepy/coveragepy).


Updates `coverage` from 7.14.0 to 7.14.3
- [Release notes](https://github.com/coveragepy/coveragepy/releases)
- [Changelog](https://github.com/coveragepy/coveragepy/blob/main/CHANGES.rst)
- [Commits](https://github.com/coveragepy/coveragepy/compare/7.14.0...7.14.3)

---
updated-dependencies:
- dependency-name: coverage
  dependency-version: 7.14.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: pip-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-23 05:42:48 +00:00
TW
840de26662
Merge pull request #9798 from mr-raj12/pack-files-remove-objects-helper
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
repository: add replace_pack helper
2026-06-22 17:10:57 +02:00
Mrityunjay Raj
bf36090497 repository: compact_pack takes sets, unifies keep/drop loop, renames kept
keep_ids/drop_ids are now keyword-only sets; one loop over their union
replaces two; survivors renamed to kept; intersection asserted empty.
2026-06-22 19:05:17 +05:30
Mrityunjay Raj
a56518e41f repository: compact_pack takes keep and drop ids, owns the index update
Rename replace_pack to compact_pack; it now repoints kept and deletes dropped
index entries, asserting keep + drop tile the whole pack. refs #8572 #8514
2026-06-22 17:34:11 +05:30
Mrityunjay Raj
55dc65c1c4 repository: clarify replace_pack docs and cover the keep-all case
Document the old_pack_id return and the caller's index-durability duty, reword
the delete-old-last rationale, and test the reproduced-pack skip and survivor sort.
2026-06-22 05:15:53 +05:30
Mrityunjay Raj
e2c7a2c8e0 repository: add replace_pack to keep a subset of a pack's objects, refs #8572 #8514
Copies the kept objects into a new pack via store.defrag, repoints their chunk
index entries, and deletes the old pack; an empty keep set drops the pack whole.
2026-06-22 04:41:13 +05:30
TW
17c3d67624
Merge pull request #8775 from Goddesen/prune-timely-by-interval
Some checks failed
Lint / lint (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
CI / asan_ubsan (push) Has been cancelled
CI / native_tests (push) Has been cancelled
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Has been cancelled
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Has been cancelled
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Has been cancelled
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Has been cancelled
CI / windows_tests (push) Has been cancelled
2026-06-21 11:59:17 +02:00
Hugo Wallenburg
ab2d78b84a
Tests keep-all with shorter archive span
33333 days in the past is before 1970 and may not be representable on
some platforms. 3333 is just ~9 years.
2026-06-21 10:00:00 +02:00
TW
c2b325b76f
Merge pull request #9796 from mr-raj12/fix-flaky-windows-corruption-offset
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
testsuite: corrupt archive metadata mid-object to fix flaky windows check test
2026-06-20 22:40:35 +02:00
Mrityunjay Raj
abd3f6a4fb testsuite: corrupt archive metadata mid-object to fix flaky windows check test
The last byte can land on store-level framing instead of the authenticated payload, making test_manifest_rebuild_corrupted_chunk fail intermittently on Windows.
2026-06-20 22:16:07 +05:30
Hugo Wallenburg
c0a69611a8
Renames --since to --from 2026-06-20 18:36:59 +02:00
Hugo Wallenburg
fe0d2d117d
Adds a specific keep=all test 2026-06-20 18:35:34 +02:00
Hugo Wallenburg
89c6a2cccc
Improves interval example
Makes specific notes on week numbers, clears some ambiguity in the
yearly candidate wording, and more.

The matching test was previously missing assertions on the weekly
archives, now fixed along with better explanations.
2026-06-20 18:09:41 +02:00
Hugo Wallenburg
378f7c27cc
Moves keep-all test
Moves it out of the nicely organized increasing retention rule tests.
2026-06-20 18:09:40 +02:00
Hugo Wallenburg
790871ee59
Comments on zfill
While the exact stated reason doesn't currently apply in the code, I
believe that any future work and debugging is aided by this ordering
guarantee.
2026-06-20 18:09:40 +02:00
Hugo Wallenburg
79388320c2
Makes archive interval timestamp check inclusive 2026-06-20 18:09:40 +02:00
Hugo Wallenburg
47d88682c1
Fixes pytest warning for non-list in parametrize 2026-06-20 18:09:40 +02:00
Hugo Wallenburg
01aff2b7e2
Sets base_timestamp for prune only once 2026-06-20 18:09:39 +02:00
Daniel Rudolf
1073da9eb1
Refactor the interval-based pruning example
Co-Authored-By: Hugo Wallenburg <hugo@hugow.no>
2026-06-20 18:09:39 +02:00
Hugo Wallenburg
d9b35f2eeb
Simplifies retention granularity ordering check 2026-06-20 18:09:39 +02:00
Hugo Wallenburg
304a2fa5b9
Removes --keep-last and --keep-within, superseded by --keep 2026-06-20 18:09:38 +02:00
Hugo Wallenburg
d9f9f1dd45
Updates prune parser epilog to match new int/interval behavior
Along with reordering and improvements.
2026-06-20 18:09:38 +02:00
Hugo Wallenburg
4382b3a8f2
Updates usage docs for prune and new int/interval handling 2026-06-20 18:09:38 +02:00
Hugo Wallenburg
7948625442
Fixes prune error tests with binary borg 2026-06-20 18:09:37 +02:00