Commit graph

9833 commits

Author SHA1 Message Date
TW
f12f464099
Merge pull request #9481 from ThomasWaldmann/rel200b21
Some checks failed
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
Lint / lint (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
Release 2.0.0b21
2026-03-15 19:50:30 +01:00
Thomas Waldmann
2d6d900231
update CHANGES 2026-03-15 17:03:12 +01:00
Thomas Waldmann
5eb2dd5328
canary CI job: use bash shell
Even on Windows.

Also:
- run daily (until all issues have been found / fixed)
- fix trailing blanks.
2026-03-15 17:03:10 +01:00
Thomas Waldmann
2322d24e91
add .DS_Store to .gitignore 2026-03-15 14:37:51 +01:00
Thomas Waldmann
f12f67a76d
build_usage build_man 2026-03-15 14:37:50 +01:00
TW
1706e0d36b
Merge pull request #9480 from ThomasWaldmann/add-rest-store-support
add REST store support
2026-03-15 14:29:49 +01:00
Thomas Waldmann
a2ea1e9883
add rest store support 2026-03-15 13:19:39 +01:00
TW
93dfcf4565
Merge pull request #9475 from ThomasWaldmann/update-changes-master
Some checks failed
Lint / lint (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
CI / asan_ubsan (push) Has been cancelled
CI / native_tests (push) Has been cancelled
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Has been cancelled
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Has been cancelled
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Has been cancelled
CI / vm_tests (OpenBSD, false, openbsd, 7.8) (push) Has been cancelled
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Has been cancelled
CI / windows_tests (push) Has been cancelled
update CHANGES
2026-03-13 15:30:48 +01:00
TW
0d784d00d2
Merge pull request #9474 from ThomasWaldmann/use-jsonargparse-release
jsonargparse: pull from pypi, fixes #9460
2026-03-13 15:29:54 +01:00
Thomas Waldmann
d0c5a04a2a
update CHANGES 2026-03-13 15:29:33 +01:00
Thomas Waldmann
2f99890487
jsonargparse: pull from pypi, fixes #9460 2026-03-13 13:12:14 +01:00
TW
a32d9724f0
Merge pull request #9472 from ThomasWaldmann/ci-openbsd78
adapt for OpenBSD 7.8 (CI and setup.py, master)
2026-03-13 11:36:45 +01:00
TW
ff9af9d4d5
Merge pull request #9471 from ThomasWaldmann/ci-timeouts-master
Some checks are pending
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.7) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI timeouts (master)
2026-03-13 00:47:50 +01:00
Thomas Waldmann
b35a76dd3c
adapt for OpenBSD 7.8 (CI and setup.py) 2026-03-13 00:39:13 +01:00
Thomas Waldmann
8ca88165ac
CI: increase timeout, see #9470
Especially OpenBSD with Coverage is rather slow.
2026-03-12 23:59:27 +01:00
TW
20078f5ab8
Merge pull request #9469 from ThomasWaldmann/ci-locked-and-canary
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.7) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CI: use locked requirements, add canary job, fixes #9361
2026-03-12 16:56:36 +01:00
Thomas Waldmann
0f1e3f90fe
CI: use locked requirements, add canary job, fixes #9361 2026-03-12 15:01:00 +01:00
TW
463aafde65
Merge pull request #9468 from ThomasWaldmann/tag-validator
Some checks failed
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.7) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Has been cancelled
enhance tag_validator, fixes #9466
2026-03-12 01:17:07 +01:00
Thomas Waldmann
65de69943e
enhance tag_validator, fixes #9466 2026-03-11 23:11:38 +01:00
TW
7f7e6061fb
Merge pull request #9464 from ThomasWaldmann/create-set-metadata
borg create: add --hostname/--username/--tags
2026-03-11 21:04:45 +01:00
Thomas Waldmann
5f4e921b9a
create: add --tags, fixes #9401
- add `--tags TAG [TAG ...]` option to `borg create` to tag newly created archives.
- validate the tags exactly like `borg tag` does, including checking that any special tags starting with `@` are known `SPECIAL_TAGS`.
- add `test_create_tags` and `test_create_invalid_tags` to ensure proper behavior.
2026-03-11 19:28:56 +01:00
Thomas Waldmann
90f84e7219
create: add --hostname/--username, fixes #9402
- Added `--hostname` and `--username` command-line options to `borg create`
- Updated Archive to capture and store these explicit values, falling back to system defaults
- Added `test_explicit_hostname_and_username` to verify the functionality
2026-03-11 18:21:38 +01:00
TW
a03d1780a5
Merge pull request #9462 from ThomasWaldmann/trailing-sparse-windows-master
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.7) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
fix spurious sparse test fail on win32, fixes #7616
2026-03-11 13:57:56 +01:00
TW
1c75f505d6
Merge pull request #9461 from ThomasWaldmann/fix-with-lock-test-master
fix race condition in test_with_lock, fixes #8810
2026-03-11 13:23:36 +01:00
Thomas Waldmann
871c89bea0
Fix intermittent test_with_lock failure (race condition), fixes #8810.
test_with_lock previously relied on a hardcoded timeout
(`time.sleep(4)`) to ensure the first background command acquired
the repository lock before the second command tried to get it. On
extremely slow CI runners, this was sometimes too short, allowing
the second command to falsely acquire the lock.

This commit replaces the arbitrary sleep with true synchronization:
- The first command now blocks indefinitely using `sys.stdin.read()`.
- The test deterministically waits for lock acquisition by reading
  `p1.stdout.readline()` which guarantees the lock is held.
- After the second command correctly fails, the first command is
  smoothly unblocked and terminated by passing `input=""` to
  `p1.communicate()`.
2026-03-11 11:40:42 +01:00
Thomas Waldmann
6d85812e12
fix spurious sparse test fail on win32, fixes #7616
Python's `os.truncate()` on Windows relies on `SetEndOfFile()`, which does
not initialize the extended disk space with zeroes. This means that
trailing sparse holes simply leave uninitialized garbage data at the end
of the file.

During sparse file extraction, when the very last chunk is a sparse hole,
the VDL (Valid Data Length) is not properly advanced by `os.truncate()`.
As a result, reading from the end of the file fetches random disk garbage
instead of zeroes, causing spurious test failures at boundaries (like
2MB or 8MB) depending on what was in the uninitialized disk sectors.

Fix this by tracking trailing holes and manually writing a single `b"\0"`
byte at the end of the file before truncating on Windows. Writing explicit
data forces NTFS to officially advance the VDL and securely zero-fill the
preceding hole space.

Re-enable `test_sparse_file` on Windows.
2026-03-11 10:58:54 +01:00
Thomas Waldmann
c98e93bc0b
fix race condition in test_with_lock, fixes #8810 2026-03-11 00:13:53 +01:00
TW
3d437d8589
Merge pull request #9413 from ThomasWaldmann/jsonargparse2
Some checks are pending
Lint / lint (push) Waiting to run
CI / lint (push) Waiting to run
CI / security (push) Waiting to run
CI / asan_ubsan (push) Blocked by required conditions
CI / native_tests (push) Blocked by required conditions
CI / vm_tests (Haiku, false, haiku, r1beta5) (push) Blocked by required conditions
CI / vm_tests (NetBSD, false, netbsd, 10.1) (push) Blocked by required conditions
CI / vm_tests (OmniOS, false, omnios, r151056) (push) Blocked by required conditions
CI / vm_tests (OpenBSD, false, openbsd, 7.7) (push) Blocked by required conditions
CI / vm_tests (borg-freebsd-14-x86_64-gh, FreeBSD, true, freebsd, 14.3) (push) Blocked by required conditions
CI / windows_tests (push) Blocked by required conditions
CodeQL / Analyze (push) Waiting to run
use jsonargparse
2026-03-10 23:44:48 +01:00
Thomas Waldmann
4f2f2255c3
create --paths-from-shell-command, fixes #5968
This adds the `--paths-from-shell-command` option to the `create` command, enabling the use of shell-specific features like pipes and redirection when specifying input paths. Includes related test coverage.
2026-03-10 20:16:26 +01:00
Thomas Waldmann
bcce0c13f7
pyproject.toml: pull jsonargparse from main branch zip 2026-03-10 20:16:03 +01:00
Thomas Waldmann
e15547a685
import ActionSubCommands from jsonargparse 2026-03-10 20:15:59 +01:00
Thomas Waldmann
4a6b4130ef
completion: use new get_completion_script method 2026-03-10 18:36:35 +01:00
Thomas Waldmann
f2bdb689c5
improve configuration precedence docs 2026-03-10 18:36:34 +01:00
Thomas Waldmann
cf6fd0769d
use jsonargparse's PositiveInt 2026-03-10 18:36:32 +01:00
Thomas Waldmann
678df16bad
add support for auto-generated environment variables (jsonargparse) 2026-03-10 18:36:31 +01:00
Thomas Waldmann
63a45c6c21
add support for yaml config files, default config 2026-03-10 18:36:30 +01:00
Thomas Waldmann
2f778b974b
argparsing: add ArgumentParser subclass with borg's usual defaults
Borg's ArgumentParser (in borg.helpers.argparsing) now subclasses
jsonargparse's ArgumentParser and pre-sets two defaults that every
borg parser uses:

  formatter_class = RawDescriptionHelpFormatter
  add_help        = False
2026-03-10 18:36:28 +01:00
Thomas Waldmann
975212bbec
simplify flatten_namespace, add docstring for argparsing 2026-03-10 18:36:27 +01:00
Thomas Waldmann
fac47c6785
archiver: replace CommonOptions suffix hack with jsonargparse-native approach
The old code worked around argparse's flat namespace by appending
_maincommand / _midcommand / _subcommand suffixes to every common
option's dest (e.g. log_level_subcommand), then resolving them with
CommonOptions.resolve() after parsing.  This polluted config key names
and env var names (BORG_LOG_LEVEL_SUBCOMMAND instead of BORG_LOG_LEVEL).

jsonargparse nests subcommand arguments automatically, so the workaround
is no longer needed.  Each parser level now registers common options with
their clean dest name.  flatten_namespace() is updated to a two-pass
depth-first walk so the most-specific (innermost) value wins naturally:

  borg --info create --debug  →  log_level = "debug"  (subcommand wins)
  borg --info create          →  log_level = "info"   (top-level fills gap)

For append-action options (--debug-topic) values from all levels are
merged (outer + inner) to preserve the accumulation behaviour.
2026-03-10 18:36:26 +01:00
Thomas Waldmann
71212d2d7f
add octal_int validator 2026-03-10 18:36:24 +01:00
Thomas Waldmann
ee58cb4e44
reorg imports 2026-03-10 18:36:23 +01:00
Thomas Waldmann
e91f4efab1
make build_usage / build_man: do not show options with SUPPRESS 2026-03-10 18:36:22 +01:00
Thomas Waldmann
83b8935abf
move compress.CompressionSpec to helpers.parseformat.CompressionSpec
other validators / specs are also there and it is easier to maintain
as Python code. the compress module is Cython code.
2026-03-10 18:36:20 +01:00
Thomas Waldmann
7250fd947e
helpers.jap_helper -> helpers.argparsing 2026-03-10 18:36:19 +01:00
Thomas Waldmann
be2b3363e2
reorg imports 2026-03-10 18:36:18 +01:00
Thomas Waldmann
a708397612
fix argparsing test 2026-03-10 18:36:16 +01:00
Thomas Waldmann
223a834a3d
adapt borg completion for jsonargparse 2026-03-10 18:36:12 +01:00
Thomas Waldmann
393b2cbc11
import from jsonargparse: SUPPRESS, REMAINDER, NameSpace 2026-03-10 18:36:11 +01:00
Thomas Waldmann
e39d06ee98
blacken src 2026-03-10 18:36:09 +01:00
Thomas Waldmann
27561bd9cc
SortBySpec: avoid triggering bandit security checker
Guess it was triggered due to naming the variable "token",
maybe "sort_key" is less problematic.
2026-03-10 18:36:08 +01:00