Commit graph

9215 commits

Author SHA1 Message Date
Thomas Waldmann
dc2dab1535
buzhash64: integrate into borg benchmark command 2025-06-10 22:44:50 +02:00
Thomas Waldmann
63ff136dfe
buzhash64: integrate into build 2025-06-10 22:44:48 +02:00
Thomas Waldmann
6a6622f9d8
buzhash64: adapt buzhash and tests for 64bit 2025-06-10 22:44:47 +02:00
Thomas Waldmann
cbe6ba719d
chunkers: prepare for buzhash64
Added some *64*.* files that are just 1:1 copies of their
32bit counterparts, so that the changes for the 64bit
adaption will later be better visible.
2025-06-10 22:44:43 +02:00
TW
6487a9875c
Merge pull request #8912 from ThomasWaldmann/update-changes
update CHANGES
2025-06-10 11:22:43 +02:00
Thomas Waldmann
b116971217
update CHANGES 2025-06-10 01:09:25 +02:00
TW
31a1ac18c7
Merge pull request #8899 from ThomasWaldmann/fish-completions
fish: fix archive name completion
2025-06-09 19:21:11 +02:00
TW
82a525c5be
Merge pull request #8908 from ThomasWaldmann/3818simplifyexport-master2
small fixes for docs update in 3818simplifyexport-master
2025-06-09 14:35:43 +02:00
Thomas Waldmann
7741b503ad
reformat text 2025-06-09 14:19:14 +02:00
Thomas Waldmann
c850f508f8
fix grammar 2025-06-09 14:19:12 +02:00
TW
f3cfc72b9d
Merge pull request #8331 from qyanu-pull-requests/3818simplifyexport-master
docs: borg-serve: simplify example of env in authorized_keys
2025-06-09 14:17:46 +02:00
TW
9dc18e3707
Merge pull request #8906 from ThomasWaldmann/permissions-docs
docs: add docs for serve --permissions / BORG_REPO_PERMISSIONS
2025-06-08 14:01:58 +02:00
Thomas Waldmann
81bacd04c5
docs: add docs for serve --permissions / BORG_REPO_PERMISSIONS 2025-06-08 13:49:09 +02:00
TW
3cf8d7cf2f
Merge pull request #8905 from ThomasWaldmann/derive-key
key: add derive_key to derive new keys from existing key material
2025-06-06 21:37:29 +02:00
TW
f569668474
Merge pull request #8904 from ThomasWaldmann/allow-msgpack-111
Allow msgpack 1.1.1(rc1)
2025-06-06 20:56:49 +02:00
Thomas Waldmann
e86bae79c2
key: add derive_key to derive new keys from existing key material.
Just a slight refactor of existing code to make
it more useful for other key-generation purposes.
2025-06-06 20:55:39 +02:00
Thomas Waldmann
467d0604da
msgpack version check: ignore "rc" or other version elements
Only compare the main version number, e.g. 1.1.1 (first 3 elements
of the version tuple).

Without this change, it would not accept 1.1.1rc1 because that is
not "<= (1, 1, 1)" in that simplistic version comparison.
2025-06-06 18:38:10 +02:00
Thomas Waldmann
862f19aab9
msgpack: allow 1.1.1
1.1.1rc1 looked good in testing, so hopefully 1.1.1 will also be ok.
2025-06-06 18:35:25 +02:00
TW
fb527051cb
Merge pull request #8902 from ThomasWaldmann/separate-chunkers
Separate chunkers
2025-06-05 11:46:55 +02:00
Thomas Waldmann
25ec20410c
chunkers: add fixed chunker tests to selftest 2025-06-05 09:49:12 +02:00
Thomas Waldmann
3c459f0ae7
chunkers: rename test modules 2025-06-05 09:46:19 +02:00
Thomas Waldmann
0754138a89
Split remaining chunker/reader tests into separate modules 2025-06-05 01:26:06 +02:00
Thomas Waldmann
ae93e67254
Rename chunker_slow_test.py to buzhash_pytest_test.py 2025-06-05 00:54:11 +02:00
Thomas Waldmann
cd6360a894
Split chunker_test into fixed_test and buzhash_test modules
Separated `chunker_test` into two dedicated test modules: `fixed_test` (for `ChunkerFixed`) and `buzhash_test` (for `Chunker`). Updated imports and adjusted references accordingly.
2025-06-05 00:49:50 +02:00
Thomas Waldmann
e7c9db9506
tests: move chunker tests to own package 2025-06-05 00:41:31 +02:00
Thomas Waldmann
90fe56e20a
make mypy happy 2025-06-05 00:22:26 +02:00
Thomas Waldmann
4f6605b115
chunker_test: fix import issue
name collision: module buzhash vs function buzhash.
2025-06-05 00:00:25 +02:00
Thomas Waldmann
f44b1742f5
Extract ChunkerFixed into a dedicated module under chunkers.
Moved the `ChunkerFixed` implementation from `chunker` to a new `fixed` module for better modularity. Updated imports and type hints.

Removed now empty chunkers.chunker module.
2025-06-04 23:48:50 +02:00
Thomas Waldmann
3f4d7ef910
Extract ChunkerFailing into a dedicated module under chunkers
Moved the `ChunkerFailing` implementation from `chunker` to a new `failing` module for better modularity. Updated imports and type hints. Adjusted related definitions in `chunker.pyi` accordingly.
2025-06-04 23:07:05 +02:00
Thomas Waldmann
31494f200d
Extract buzhash logic into a dedicated module under chunkers
Moved `buzhash` implementation from `chunker` to a new `buzhash` module for better separation of concerns. Updated imports, adjusted `setup.py` and build configuration accordingly. Removed deprecated `Chunker` definitions from `chunker.pyi`.
2025-06-04 22:57:03 +02:00
Thomas Waldmann
322e2018ec
Move get_chunker to __init__.py and update Chunker signature
Relocated `get_chunker` function from `chunker` module to `chunkers.__init__.py` for improved organization. Updated `Chunker` class signature to include a `sparse` parameter with a default value. Adjusted imports and type hints accordingly.
2025-06-04 22:46:06 +02:00
Thomas Waldmann
a78c310b72
Split reader functionality into a separate module under chunkers package
Extracted the `reader` logic from `chunker` into a dedicated `reader` module to improve modularity and maintainability. Updated imports, references, and build configurations accordingly.
2025-06-04 22:40:39 +02:00
Thomas Waldmann
bc86bd4bb0
Move chunker module to chunkers package
Refactor by relocating the `chunker` module under a new `chunkers` package, adjusting imports and file references accordingly.
2025-06-04 22:12:45 +02:00
TW
ae80be7f43
Merge pull request #8901 from ThomasWaldmann/chunker-refactor3
Small Chunker refactor
2025-06-04 21:55:39 +02:00
Thomas Waldmann
f58d26671d
ChunkerFixed: do not assert on short header read
ChunkerFixed can be configured to support files with a specific header size.

But we do not want to get an AssertionError if we encounter a 0-byte file
or a file that is shorter than the header size.
2025-06-04 13:40:16 +02:00
Thomas Waldmann
66f10712b4
chunker: use safe_fadvise
also: refactor safe_advise a bit, use has_posix_fadvise.
2025-06-04 13:28:32 +02:00
TW
2295c83c7e
Merge pull request #8900 from ThomasWaldmann/gitignore-update
.gitignore: update entries
2025-06-04 11:51:13 +02:00
Thomas Waldmann
b26f222ca4
.gitignore: update entries 2025-06-04 11:40:55 +02:00
TW
6e2b550a0e
Merge pull request #8897 from ThomasWaldmann/update-changes
update CHANGES
2025-06-04 01:01:46 +02:00
Thomas Waldmann
a1ba5c3c81
fish: fix archive name completion 2025-06-04 01:01:08 +02:00
Thomas Waldmann
5ca22e9c10
update CHANGES 2025-06-03 20:26:10 +02:00
TW
7ec02dce46
Merge pull request #8895 from ThomasWaldmann/exclude-based-on-xattrs
Exclude based on xattrs or NODUMP flag
2025-06-03 19:54:00 +02:00
Thomas Waldmann
2129d81fdb
create: auto-exclude items based on xattrs or NODUMP, see #4972
no options yet, just hardcoded macOS and Linux xattrs.
removed the --exclude-nodump option, it is also done automagically now.

also: create: call stat_ext_attrs early

this reads bsdflags, xattrs and ACLs from the
filesystem, except if the user chose to disable that.

notable:
- borg always reads these, even for unchanged files
- if we read them early, borg can now behave differently
  based e.g. on a xattr value (and e.g. exclude the file)
2025-06-03 12:56:30 +02:00
TW
1c07087e00
Merge pull request #8883 from ThomasWaldmann/chunker-refactor2
Chunker refactor
2025-06-03 12:51:30 +02:00
TW
0fa548beac
Merge pull request #8894 from ThomasWaldmann/bandit
use bandit
2025-06-03 09:07:35 +02:00
Thomas Waldmann
3b94130b1c
suppress some bandit warnings 2025-06-01 18:00:37 +02:00
Thomas Waldmann
338f9b1db9
add bandit, a security-oriented static analysis tool 2025-06-01 18:00:35 +02:00
TW
dda9c445e1
Merge pull request #8893 from ThomasWaldmann/borg-serve-permissions
serve: add --permissions option as an alternative to BORG_REPO_PERMISSIONS env var
2025-05-30 13:08:29 +02:00
Thomas Waldmann
bdc36d451f
serve: add --permissions option as an alternative to BORG_REPO_PERMISSIONS env var 2025-05-30 11:42:56 +02:00
TW
8c235d760b
Merge pull request #8891 from ThomasWaldmann/refactor-chunker-slow-master
test_chunkpoints_unchanged: do not use blake2b_256
2025-05-29 19:36:24 +02:00