10387 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Mrityunjay Raj	028def7c31	Merge 0d2a6daa47 into d2d8ea7ec7	2026-07-07 18:12:49 +00:00
Mrityunjay Raj	0d2a6daa47	compact: address review: re-check object spans against pack file size ... compact_pack raises IntegrityError if an object ends past EOF. On a damaged repo, skip nuking soft-deleted archives and compact only packs without unindexed bytes instead of refusing entirely. Add tests, fix docs wording.	2026-07-07 23:42:27 +05:30
TW	d2d8ea7ec7	Merge pull request #9876 from mr-raj12/create-log-archive-name-9865 ... create: log the archive name, not just the repository	2026-07-07 11:30:11 +02:00
TW	072fe773ce	Merge pull request #9875 from borgbackup/dependabot/pip/requirements.d/pip-dependencies-7dc801e80b ... build(deps-dev): bump the pip-dependencies group in /requirements.d with 2 updates	2026-07-07 11:28:46 +02:00
TW	a968a3d081	Merge pull request #9874 from ThomasWaldmann/retry-no-sleep-on-last-try ... create file retries: do not sleep before giving up on the last try	2026-07-07 11:28:10 +02:00
Mrityunjay Raj	afb332264b	compact: handle pack objects no chunk index entry covers, #9868 ... Tolerate unindexed/superseded bytes instead of asserting a fully tiled pack; raise IntegrityError only on real overlap. Size packs from the store file so those bytes and orphan packs are reclaimed; skip compaction while chunks are missing.	2026-07-07 14:09:52 +05:30
Mrityunjay Raj	94c9f0ef90	create: log the archive name, not just the repository (fixes #9865)	2026-07-07 12:26:06 +05:30
dependabot[bot]	72b1523ba2	build(deps-dev): bump the pip-dependencies group ... Bumps the pip-dependencies group in /requirements.d with 2 updates: [virtualenv](https://github.com/pypa/virtualenv) and [tox](https://github.com/tox-dev/tox). Updates `virtualenv` from 21.4.1 to 21.4.3 - [Release notes](https://github.com/pypa/virtualenv/releases) - [Changelog](https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst) - [Commits](https://github.com/pypa/virtualenv/compare/21.4.1...21.4.3) Updates `tox` from 4.55.0 to 4.55.1 - [Release notes](https://github.com/tox-dev/tox/releases) - [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst) - [Commits](https://github.com/tox-dev/tox/compare/4.55.0...4.55.1) --- updated-dependencies: - dependency-name: virtualenv dependency-version: 21.4.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: pip-dependencies - dependency-name: tox dependency-version: 4.55.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: pip-dependencies ... Signed-off-by: dependabot[bot] <support@github.com>	2026-07-07 05:43:03 +00:00
Thomas Waldmann	9bd269090b	create file retries: do not sleep before giving up on the last try ... When all retries for a file were used up, the error is re-raised anyway, so sleeping before that is pointless - on the last try, the sleep was up to ~31.6s (retry 9) per permanently failing file, e.g. on a disk that gives I/O errors for many files. Check last_try first and re-raise immediately; only sleep (and log the retry warning) when another attempt will actually follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-07 06:00:58 +02:00
TW	9a920e017c	Merge pull request #9866 from mr-raj12/create-stats-bytes-9405 ... create --stats: populate bytes read/sent from store stats	2026-07-07 04:49:43 +02:00
TW	3539fccc64	Merge pull request #9871 from ThomasWaldmann/fix-lock-acquire-timeout ... locking: do not leave a lock behind when exclusive acquire times out	2026-07-07 04:47:19 +02:00
Thomas Waldmann	88f470ea8e	locking: do not leave a lock behind when exclusive acquire times out ... When Lock.acquire() for an exclusive lock had created its lock and then timed out waiting for non-exclusive locks to go away, it raised LockTimeout without deleting the lock it had just created. That zombie exclusive lock then blocked all other clients until it expired as stale (up to 30 minutes) or until a client on the same host noticed the owning process was dead. Clients on other hosts were locked out for the full staleness period. Delete our lock before giving up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-07 00:15:09 +02:00
Mrityunjay Raj	54dede3e5c	create: report full borgstore stats instead of two byte counters ... Replace the rx_bytes/tx_bytes summary with the complete store.stats dict (call counts, timings, throughput, cache hit ratio) in --stats and --json output.	2026-07-06 23:37:57 +05:30
TW	d3a9799563	Merge pull request #9864 from mr-raj12/pack-not-found-exception ... repository: raise PackNotFound when get_many hits a missing pack	2026-07-06 19:38:15 +02:00
Mrityunjay Raj	9a593118ca	create: populate --stats bytes read/sent from store stats ... rx_bytes/tx_bytes only exist on the legacy RemoteRepository, so they always read 0 for borgstore repositories. Sample repository.store.stats after the cache is closed, rename the labels remote->repository, and add them to --json. fixes #9405	2026-07-06 12:44:09 +05:30
Mrityunjay Raj	cbc94ae837	repository: raise PackNotFound when get_many hits a missing pack	2026-07-06 08:34:59 +05:30
TW	87bd65c054	Merge pull request #9863 from ThomasWaldmann/compact-use-size-better-master ... compact: use size better	2026-07-05 22:04:39 +02:00
Thomas Waldmann	230f6ef400	compact: only output statistics when --stats is given	2026-07-05 21:11:35 +02:00
Thomas Waldmann	2b2ceaeb2c	compact: show deduplication and compression factors, fixes #9856	2026-07-05 20:14:51 +02:00
TW	10df5a898b	Merge pull request #9862 from PhrozenByte/drop-symlink-mount-warning-master ... Drop borgfs runtime warning about symlinks and improve corresponding docs [master]	2026-07-05 19:45:47 +02:00
Daniel Rudolf	88e186e053	Improve docs note about symlink behaviour	2026-07-05 18:11:12 +02:00
Daniel Rudolf	a56eb9e1db	Drop non-actionable borgfs warning about symlinks	2026-07-05 18:09:28 +02:00
TW	fc8c8e0fe9	Merge pull request #9811 from ThomasWaldmann/env-hostname-username-master ... Replace create --hostname/--username with BORG_HOSTNAME/BORG_USERNAME env vars, fixes #9651	2026-07-05 14:56:52 +02:00
TW	08f91d4d62	Merge pull request #9860 from mvanhorn/docs/fix-contruct-typo ... docs: fix typo in serve comment (contruct -> construct)	2026-07-05 14:55:21 +02:00
Charmi Kadi	cef78c99aa	json: support BORG_JSON_INDENT env var for JSON output formatting	2026-07-05 14:45:59 +02:00
Matt Van Horn	f59258e305	docs: fix typo in serve comment (contruct -> construct)	2026-07-05 04:26:48 -07:00
TW	caf1fe55fe	Merge pull request #9836 from mr-raj12/pack-files-get-many-grouping ... repository: cache whole packs in get_many, replacing adjacent-read grouping	2026-07-05 08:24:49 +02:00
Mrityunjay Raj	cc098ecd5f	repository: use try/except/else in get_many, fix misleading comment	2026-07-05 06:41:24 +05:30
TW	1926368ff9	Merge pull request #9857 from borgbackup/ai-findings-autofix/src-borg-testsuite-archiver-extract_cmd_test.py ... extract tests: add missing comma between argument strings	2026-07-05 00:29:57 +02:00
TW	779b57e322	Merge pull request #9858 from borgbackup/ai-findings-autofix/src-borg-testsuite-helpers-parseformat_test.py ... fix typo: zettabytes	2026-07-05 00:29:02 +02:00
Mrityunjay Raj	2e2ef6f47a	repository: peek the pack cache in get(), switch to peekable helpers LRUCache	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	ec62e5884c	repository: use functools.lru_cache for the get_many pack cache	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	f6712364df	repository: cache whole packs in get_many, replacing adjacent-read grouping ... Keep up to PACK_READER_CACHE_SIZE whole packs in an LRU so repeated and out-of-order ids read from memory instead of refetching from the store.	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	d076788dc5	repository_test: cover get_many pack-grouping (one load per pack, order, missing ids)	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	5d557e1bb6	repository: group adjacent same-pack reads in get_many into one store.load	2026-07-04 22:14:55 +05:30
TW	aba4e2dcf4	Apply suggested fix to src/borg/testsuite/helpers/parseformat_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:39:43 +02:00
TW	811175f0d1	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:14 +02:00
TW	cf0968ac82	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:14 +02:00
TW	6529346861	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:13 +02:00
TW	14f96bc0b8	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:13 +02:00
TW	196625f7c4	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:12 +02:00
TW	985cbc4178	Merge pull request #9853 from ThomasWaldmann/fix-create-enospc-atomicity ... create: do not wrap repository writes in backup_io("read") (silent data loss on ENOSPC)	2026-07-03 20:35:37 +02:00
Thomas Waldmann	5aaa2ddc4b	create: don't wrap repository writes in backup_io("read") ... process_file() ran process_file_chunks() inside `with backup_io("read")`. That block was meant to guard reading the *source* file, but the source reads are already guarded individually by backup_io_iter(). The outer wrapper additionally caught add_chunk()'s *repository* writes, so a critical repository IO failure -- e.g. the repo running out of space during a pack flush -- was wrapped into a per-file BackupOSError. Borg then only warned, skipped the file, and continued, and create_inner() still committed the archive via archive.save(). The result: `borg create` on an out-of-space repo printed a normal success summary ("Error files: 0"), exited 0, and committed an archive that references chunks which were never durably stored. `borg check` afterwards reports "Missing file chunk detected" and `borg compact` reports "Repository has N missing objects!" -- silent, unrestorable-backup data loss. Drop the outer backup_io("read") wrapper. Source reads stay per-file warnings (backup_io_iter is unchanged); repository OSErrors are now left unwrapped and therefore critical, aborting create before archive.save() runs, exactly as the BackupOSError docstring prescribes. Reproduced on a space-limited macOS ramdisk (source > free space): before, create exited 0 with a corrupt archive; after, create fails and commits nothing, and the repository stays consistent across a create/delete/compact churn matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 19:48:35 +02:00
TW	98bf11363d	Merge pull request #9852 from mr-raj12/pack-defaults-compact-index-fix ... size-bound 50 MB packs by default, fix stale chunk index after compact pack rewrites	2026-07-03 14:47:54 +02:00
Mrityunjay Raj	a8c2c7b1ec	PackWriter: move pack size default to constants.py, keep it caller-set	2026-07-03 17:37:02 +05:30
TW	64d89c8d1b	Merge pull request #9851 from ThomasWaldmann/cache-close-resource-fixes ... Release cache resources properly in Cache.close()	2026-07-03 14:04:18 +02:00
TW	f829eef49a	Merge pull request #9848 from mr-raj12/docs-pack-files-figures ... docs: illustrate pack file format and sync packs.rst to multi-object packs	2026-07-03 14:03:35 +02:00
TW	e779e55278	Merge pull request #9847 from mr-raj12/fs-dir-subdirs ... helpers/fs: let get_*_dir helpers take extra path components	2026-07-03 13:57:45 +02:00
Mrityunjay Raj	55535a21eb	docs: address review, add excalidraw sources for pack figures ... Fix outdated wording in pack figures and captions, document the diagram workflow in development.rst, and check in editable sources.	2026-07-03 17:22:36 +05:30
Mrityunjay Raj	1930c0cead	compact: fix chunk index going stale when packs are rewritten, fixes #9850 ... compact_pack() updated repository.chunks, but compact computes keep/drop from and persists its own index, which kept pointing at the deleted packs. Also: tolerate deleting already-gone packs, start rebuilt index entries as unused.	2026-07-03 16:52:32 +05:30