10371 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
TW	87bd65c054	Merge pull request #9863 from ThomasWaldmann/compact-use-size-better-master ... compact: use size better	2026-07-05 22:04:39 +02:00
Thomas Waldmann	230f6ef400	compact: only output statistics when --stats is given	2026-07-05 21:11:35 +02:00
Thomas Waldmann	2b2ceaeb2c	compact: show deduplication and compression factors, fixes #9856	2026-07-05 20:14:51 +02:00
TW	10df5a898b	Merge pull request #9862 from PhrozenByte/drop-symlink-mount-warning-master ... Drop borgfs runtime warning about symlinks and improve corresponding docs [master]	2026-07-05 19:45:47 +02:00
Daniel Rudolf	88e186e053	Improve docs note about symlink behaviour	2026-07-05 18:11:12 +02:00
Daniel Rudolf	a56eb9e1db	Drop non-actionable borgfs warning about symlinks	2026-07-05 18:09:28 +02:00
TW	fc8c8e0fe9	Merge pull request #9811 from ThomasWaldmann/env-hostname-username-master ... Replace create --hostname/--username with BORG_HOSTNAME/BORG_USERNAME env vars, fixes #9651	2026-07-05 14:56:52 +02:00
TW	08f91d4d62	Merge pull request #9860 from mvanhorn/docs/fix-contruct-typo ... docs: fix typo in serve comment (contruct -> construct)	2026-07-05 14:55:21 +02:00
Charmi Kadi	cef78c99aa	json: support BORG_JSON_INDENT env var for JSON output formatting	2026-07-05 14:45:59 +02:00
Matt Van Horn	f59258e305	docs: fix typo in serve comment (contruct -> construct)	2026-07-05 04:26:48 -07:00
TW	caf1fe55fe	Merge pull request #9836 from mr-raj12/pack-files-get-many-grouping ... repository: cache whole packs in get_many, replacing adjacent-read grouping	2026-07-05 08:24:49 +02:00
Mrityunjay Raj	cc098ecd5f	repository: use try/except/else in get_many, fix misleading comment	2026-07-05 06:41:24 +05:30
TW	1926368ff9	Merge pull request #9857 from borgbackup/ai-findings-autofix/src-borg-testsuite-archiver-extract_cmd_test.py ... extract tests: add missing comma between argument strings	2026-07-05 00:29:57 +02:00
TW	779b57e322	Merge pull request #9858 from borgbackup/ai-findings-autofix/src-borg-testsuite-helpers-parseformat_test.py ... fix typo: zettabytes	2026-07-05 00:29:02 +02:00
Mrityunjay Raj	2e2ef6f47a	repository: peek the pack cache in get(), switch to peekable helpers LRUCache	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	ec62e5884c	repository: use functools.lru_cache for the get_many pack cache	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	f6712364df	repository: cache whole packs in get_many, replacing adjacent-read grouping ... Keep up to PACK_READER_CACHE_SIZE whole packs in an LRU so repeated and out-of-order ids read from memory instead of refetching from the store.	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	d076788dc5	repository_test: cover get_many pack-grouping (one load per pack, order, missing ids)	2026-07-04 22:14:55 +05:30
Mrityunjay Raj	5d557e1bb6	repository: group adjacent same-pack reads in get_many into one store.load	2026-07-04 22:14:55 +05:30
TW	aba4e2dcf4	Apply suggested fix to src/borg/testsuite/helpers/parseformat_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:39:43 +02:00
TW	811175f0d1	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:14 +02:00
TW	cf0968ac82	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:14 +02:00
TW	6529346861	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:13 +02:00
TW	14f96bc0b8	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:13 +02:00
TW	196625f7c4	Apply suggested fix to src/borg/testsuite/archiver/extract_cmd_test.py from Copilot Autofix ... Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-07-04 12:37:12 +02:00
TW	985cbc4178	Merge pull request #9853 from ThomasWaldmann/fix-create-enospc-atomicity ... create: do not wrap repository writes in backup_io("read") (silent data loss on ENOSPC)	2026-07-03 20:35:37 +02:00
Thomas Waldmann	5aaa2ddc4b	create: don't wrap repository writes in backup_io("read") ... process_file() ran process_file_chunks() inside `with backup_io("read")`. That block was meant to guard reading the *source* file, but the source reads are already guarded individually by backup_io_iter(). The outer wrapper additionally caught add_chunk()'s *repository* writes, so a critical repository IO failure -- e.g. the repo running out of space during a pack flush -- was wrapped into a per-file BackupOSError. Borg then only warned, skipped the file, and continued, and create_inner() still committed the archive via archive.save(). The result: `borg create` on an out-of-space repo printed a normal success summary ("Error files: 0"), exited 0, and committed an archive that references chunks which were never durably stored. `borg check` afterwards reports "Missing file chunk detected" and `borg compact` reports "Repository has N missing objects!" -- silent, unrestorable-backup data loss. Drop the outer backup_io("read") wrapper. Source reads stay per-file warnings (backup_io_iter is unchanged); repository OSErrors are now left unwrapped and therefore critical, aborting create before archive.save() runs, exactly as the BackupOSError docstring prescribes. Reproduced on a space-limited macOS ramdisk (source > free space): before, create exited 0 with a corrupt archive; after, create fails and commits nothing, and the repository stays consistent across a create/delete/compact churn matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 19:48:35 +02:00
TW	98bf11363d	Merge pull request #9852 from mr-raj12/pack-defaults-compact-index-fix ... size-bound 50 MB packs by default, fix stale chunk index after compact pack rewrites	2026-07-03 14:47:54 +02:00
Mrityunjay Raj	a8c2c7b1ec	PackWriter: move pack size default to constants.py, keep it caller-set	2026-07-03 17:37:02 +05:30
TW	64d89c8d1b	Merge pull request #9851 from ThomasWaldmann/cache-close-resource-fixes ... Release cache resources properly in Cache.close()	2026-07-03 14:04:18 +02:00
TW	f829eef49a	Merge pull request #9848 from mr-raj12/docs-pack-files-figures ... docs: illustrate pack file format and sync packs.rst to multi-object packs	2026-07-03 14:03:35 +02:00
TW	e779e55278	Merge pull request #9847 from mr-raj12/fs-dir-subdirs ... helpers/fs: let get_*_dir helpers take extra path components	2026-07-03 13:57:45 +02:00
Mrityunjay Raj	55535a21eb	docs: address review, add excalidraw sources for pack figures ... Fix outdated wording in pack figures and captions, document the diagram workflow in development.rst, and check in editable sources.	2026-07-03 17:22:36 +05:30
Mrityunjay Raj	1930c0cead	compact: fix chunk index going stale when packs are rewritten, fixes #9850 ... compact_pack() updated repository.chunks, but compact computes keep/drop from and persists its own index, which kept pointing at the deleted packs. Also: tolerate deleting already-gone packs, start rebuilt index entries as unused.	2026-07-03 16:52:32 +05:30
Mrityunjay Raj	dd1bb247a4	PackWriter: size-bound packs of 50 MB by default, fixes #9843 ... max_count=3 was a testing value. The default is now max_count=None, max_size=50_000_000; the testsuite sets BORG_PACK_MAX_COUNT=3 instead, so small test data still produces multiple multi-object packs.	2026-07-03 16:52:07 +05:30
Mrityunjay Raj	09a1627860	repository/compact: use get_cache_dir subdirs arg, pathlib for store cache dir	2026-07-03 15:55:27 +05:30
Thomas Waldmann	a200591bb4	Release files cache dict in Cache.close() ... close() persists the files cache but never drops its reference to self._files, unlike self._chunks which is cleared right after being saved. For a backup of many files this dict can hold hundreds of MB and stays alive for as long as the closed cache object does. Set self._files = None after writing it, mirroring the _chunks handling. Accessing .files after close() was never supported anyway (close() also tears down cache_config, which _read_files_cache needs), so this only frees memory earlier. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-03 11:59:36 +02:00
Thomas Waldmann	9f5c6c5b73	Invalidate repository chunk index after clearing it in Cache.close() ... AdHocWithFilesCache._chunks is the very same ChunkIndex object the Repository holds (bound via the .chunks property). On close() we persist it and clear the table in place (_maybe_write_chunks_cache(clear=True)), then set self._chunks = None. But that only drops the *cache's* reference: the Repository still points at the now-empty table and is_chunk_index_loaded stays True. A later access to repository.chunks in the same process (e.g. delete_object doing self.chunks.get(id)) would then see a valid-looking but empty index and conclude every chunk is gone, instead of rebuilding from the repository. delete_chunkindex_from_repo() already guards its analogous case with invalidate_chunk_index(); do the same here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-03 11:59:07 +02:00
TW	3d56f5a7c4	Merge pull request #9844 from ThomasWaldmann/update-changes-master ... update CHANGES	2026-07-03 09:44:21 +02:00
Thomas Waldmann	b220fba833	update CHANGES	2026-07-03 09:43:41 +02:00
Mrityunjay Raj	8a24775180	docs: illustrate pack file format and sync packs.rst to multi-object packs, refs #8572	2026-07-03 10:40:36 +05:30
Mrityunjay Raj	db89931645	helpers/fs: let get_*_dir helpers take extra path components ... get_config/cache/data/runtime_dir now accept *subdirs and ensure_dir the full path. get_keys_dir and get_security_dir use this instead of hand-joining "keys"/"security", and the store cache in repository.py uses get_cache_dir("storecache") instead of its own makedirs.	2026-07-03 08:01:11 +05:30
TW	15d55d48e3	Merge pull request #9845 from ThomasWaldmann/rest-perf-test ... borg2 performance harness for rest:// repositories	2026-07-02 21:33:28 +02:00
Thomas Waldmann	a0cd1b3441	borg2 performance harness for rest:// repositories	2026-07-02 21:26:37 +02:00
TW	13d6d54a13	Merge pull request #9842 from wszqkzqk/allocation-granularity ... testsuite: relax sparse file SEEK_DATA check	2026-07-02 21:11:23 +02:00
Zhou Qiankang	3969aeac0b	testsuite: relax sparse file SEEK_DATA check ... On filesystems with large allocation units like tmpfs with hugepages, SEEK_DATA can report an offset before the actual first non-zero byte. The test already verifies file contents and checks st_blocks to confirm the file is stored sparsely, so only require that the first data region is not past the expected content. Signed-off-by: Zhou Qiankang <wszqkzqk@qq.com>	2026-07-03 00:42:43 +08:00
TW	a14029755c	Merge pull request #9838 from mr-raj12/pack-files-pack-cache ... repository: env-gated writethrough cache for the packs/ namespace	2026-07-02 17:46:34 +02:00
Mrityunjay Raj	bccbfde960	CI: run one linux job per event with BORG_STORE_CACHE enabled	2026-07-02 20:29:28 +05:30
Mrityunjay Raj	7cc2701cc0	repository: rename BORG_PACK_CACHE to BORG_STORE_CACHE and document the shared pack cache	2026-07-02 17:32:33 +05:30
TW	c944ace472	Merge pull request #9824 from ThomasWaldmann/fastcdc-chunker ... Add fastcdc chunker (keyed Gear hash)	2026-07-02 09:30:18 +02:00