Commit graph

9229 commits

Author SHA1 Message Date
Thomas Waldmann
aea3d6aefe
pathlib refactor compact_cmd 2025-06-11 09:01:48 +02:00
Thomas Waldmann
4866d49e77
pathlib refactor shell_completions_test 2025-06-11 09:01:47 +02:00
Thomas Waldmann
afb47cd940
pathlib refactor - remove unused imports 2025-06-11 09:01:45 +02:00
Thomas Waldmann
4dbec57716
pathlib refactor logger 2025-06-11 09:01:44 +02:00
Thomas Waldmann
c48918edc9
pathlib refactor file_integrity 2025-06-11 09:01:43 +02:00
Thomas Waldmann
baecf9cccb
pathlib refactor cache 2025-06-11 09:01:42 +02:00
Thomas Waldmann
3c8cfd0e29
pathlib refactor fslocking 2025-06-11 09:01:38 +02:00
Thomas Waldmann
4ff49a6e91
pathlib refactor platform.base 2025-06-11 08:42:42 +02:00
Thomas Waldmann
a0f73d7a61
pathlib refactor fs 2025-06-11 08:42:40 +02:00
TW
9a65d5245d
Merge pull request #8903 from ThomasWaldmann/buzhash64
buzhash64 chunker
2025-06-11 08:31:27 +02:00
Thomas Waldmann
d23704e112
buzhash64: docs 2025-06-10 23:42:22 +02:00
Thomas Waldmann
b9646f236e
buzhash64: init table using a 256bit key derived from ID key
That way we can feed lots of entropy into the table creation.

The bh64_key is derived from the id_key (NOT the crypt_key), thus
it will create the same key for related repositories (even if they
use different encryption/authentication keys). Due to that, it will
also create the same buzhash64 table, will cut chunks at the same
points and deduplication will work amongst the related repositories.
2025-06-10 22:44:54 +02:00
Thomas Waldmann
544b3f41a9
get_chunker: give it the key instead of the seed
the buzhash seed only has 32bits, but we rather
want 64bits for buzhash64.

just take them from crypt_key for now.
2025-06-10 22:44:52 +02:00
Thomas Waldmann
6f55cba0ce
ChunkerParams: add support for buzhash64 2025-06-10 22:44:51 +02:00
Thomas Waldmann
dc2dab1535
buzhash64: integrate into borg benchmark command 2025-06-10 22:44:50 +02:00
Thomas Waldmann
63ff136dfe
buzhash64: integrate into build 2025-06-10 22:44:48 +02:00
Thomas Waldmann
6a6622f9d8
buzhash64: adapt buzhash and tests for 64bit 2025-06-10 22:44:47 +02:00
Thomas Waldmann
cbe6ba719d
chunkers: prepare for buzhash64
Added some *64*.* files that are just 1:1 copies of their
32bit counterparts, so that the changes for the 64bit
adaption will later be better visible.
2025-06-10 22:44:43 +02:00
TW
6487a9875c
Merge pull request #8912 from ThomasWaldmann/update-changes
update CHANGES
2025-06-10 11:22:43 +02:00
Thomas Waldmann
b116971217
update CHANGES 2025-06-10 01:09:25 +02:00
TW
31a1ac18c7
Merge pull request #8899 from ThomasWaldmann/fish-completions
fish: fix archive name completion
2025-06-09 19:21:11 +02:00
TW
82a525c5be
Merge pull request #8908 from ThomasWaldmann/3818simplifyexport-master2
small fixes for docs update in 3818simplifyexport-master
2025-06-09 14:35:43 +02:00
Thomas Waldmann
7741b503ad
reformat text 2025-06-09 14:19:14 +02:00
Thomas Waldmann
c850f508f8
fix grammar 2025-06-09 14:19:12 +02:00
TW
f3cfc72b9d
Merge pull request #8331 from qyanu-pull-requests/3818simplifyexport-master
docs: borg-serve: simplify example of env in authorized_keys
2025-06-09 14:17:46 +02:00
TW
9dc18e3707
Merge pull request #8906 from ThomasWaldmann/permissions-docs
docs: add docs for serve --permissions / BORG_REPO_PERMISSIONS
2025-06-08 14:01:58 +02:00
Thomas Waldmann
81bacd04c5
docs: add docs for serve --permissions / BORG_REPO_PERMISSIONS 2025-06-08 13:49:09 +02:00
TW
3cf8d7cf2f
Merge pull request #8905 from ThomasWaldmann/derive-key
key: add derive_key to derive new keys from existing key material
2025-06-06 21:37:29 +02:00
TW
f569668474
Merge pull request #8904 from ThomasWaldmann/allow-msgpack-111
Allow msgpack 1.1.1(rc1)
2025-06-06 20:56:49 +02:00
Thomas Waldmann
e86bae79c2
key: add derive_key to derive new keys from existing key material.
Just a slight refactor of existing code to make
it more useful for other key-generation purposes.
2025-06-06 20:55:39 +02:00
Thomas Waldmann
467d0604da
msgpack version check: ignore "rc" or other version elements
Only compare the main version number, e.g. 1.1.1 (first 3 elements
of the version tuple).

Without this change, it would not accept 1.1.1rc1 because that is
not "<= (1, 1, 1)" in that simplistic version comparison.
2025-06-06 18:38:10 +02:00
Thomas Waldmann
862f19aab9
msgpack: allow 1.1.1
1.1.1rc1 looked good in testing, so hopefully 1.1.1 will also be ok.
2025-06-06 18:35:25 +02:00
TW
fb527051cb
Merge pull request #8902 from ThomasWaldmann/separate-chunkers
Separate chunkers
2025-06-05 11:46:55 +02:00
Thomas Waldmann
25ec20410c
chunkers: add fixed chunker tests to selftest 2025-06-05 09:49:12 +02:00
Thomas Waldmann
3c459f0ae7
chunkers: rename test modules 2025-06-05 09:46:19 +02:00
Thomas Waldmann
0754138a89
Split remaining chunker/reader tests into separate modules 2025-06-05 01:26:06 +02:00
Thomas Waldmann
ae93e67254
Rename chunker_slow_test.py to buzhash_pytest_test.py 2025-06-05 00:54:11 +02:00
Thomas Waldmann
cd6360a894
Split chunker_test into fixed_test and buzhash_test modules
Separated `chunker_test` into two dedicated test modules: `fixed_test` (for `ChunkerFixed`) and `buzhash_test` (for `Chunker`). Updated imports and adjusted references accordingly.
2025-06-05 00:49:50 +02:00
Thomas Waldmann
e7c9db9506
tests: move chunker tests to own package 2025-06-05 00:41:31 +02:00
Thomas Waldmann
90fe56e20a
make mypy happy 2025-06-05 00:22:26 +02:00
Thomas Waldmann
4f6605b115
chunker_test: fix import issue
name collision: module buzhash vs function buzhash.
2025-06-05 00:00:25 +02:00
Thomas Waldmann
f44b1742f5
Extract ChunkerFixed into a dedicated module under chunkers.
Moved the `ChunkerFixed` implementation from `chunker` to a new `fixed` module for better modularity. Updated imports and type hints.

Removed now empty chunkers.chunker module.
2025-06-04 23:48:50 +02:00
Thomas Waldmann
3f4d7ef910
Extract ChunkerFailing into a dedicated module under chunkers
Moved the `ChunkerFailing` implementation from `chunker` to a new `failing` module for better modularity. Updated imports and type hints. Adjusted related definitions in `chunker.pyi` accordingly.
2025-06-04 23:07:05 +02:00
Thomas Waldmann
31494f200d
Extract buzhash logic into a dedicated module under chunkers
Moved `buzhash` implementation from `chunker` to a new `buzhash` module for better separation of concerns. Updated imports, adjusted `setup.py` and build configuration accordingly. Removed deprecated `Chunker` definitions from `chunker.pyi`.
2025-06-04 22:57:03 +02:00
Thomas Waldmann
322e2018ec
Move get_chunker to __init__.py and update Chunker signature
Relocated `get_chunker` function from `chunker` module to `chunkers.__init__.py` for improved organization. Updated `Chunker` class signature to include a `sparse` parameter with a default value. Adjusted imports and type hints accordingly.
2025-06-04 22:46:06 +02:00
Thomas Waldmann
a78c310b72
Split reader functionality into a separate module under chunkers package
Extracted the `reader` logic from `chunker` into a dedicated `reader` module to improve modularity and maintainability. Updated imports, references, and build configurations accordingly.
2025-06-04 22:40:39 +02:00
Thomas Waldmann
bc86bd4bb0
Move chunker module to chunkers package
Refactor by relocating the `chunker` module under a new `chunkers` package, adjusting imports and file references accordingly.
2025-06-04 22:12:45 +02:00
TW
ae80be7f43
Merge pull request #8901 from ThomasWaldmann/chunker-refactor3
Small Chunker refactor
2025-06-04 21:55:39 +02:00
Thomas Waldmann
f58d26671d
ChunkerFixed: do not assert on short header read
ChunkerFixed can be configured to support files with a specific header size.

But we do not want to get an AssertionError if we encounter a 0-byte file
or a file that is shorter than the header size.
2025-06-04 13:40:16 +02:00
Thomas Waldmann
66f10712b4
chunker: use safe_fadvise
also: refactor safe_advise a bit, use has_posix_fadvise.
2025-06-04 13:28:32 +02:00