ci.yml already has timeout-minutes on every job, but these three
workflows had no timeout configured. Without an explicit timeout,
GitHub Actions defaults to 6 hours, wasting CI minutes if a job
gets stuck.
Added timeouts consistent with ci.yml:
- codeql-analysis.yml: 20 min (builds from source + analysis)
- backport.yml: 5 min (simple checkout + PR creation)
- black.yaml: 5 min (matches ci.yml ruff lint job)
Fixes#9298
if an already existing fs directory has the correct (as archived) mtime,
we have already extracted it in a previous borg extract run and we do not
need and should not call restore_attrs for it again.
if the directory exists, but does not have the correct mtime, restore_attrs
will be called and its attributes will be extracted (and mtime set to
correct value).
Enable JUnit XML generation for `native_tests` and `windows_tests` to allow Codecov to process test analytics.
Upload the generated `test-results.xml` using `codecov/codecov-action`.
The upload step uses `if: !cancelled()` to ensure results are uploaded even if tests fail (to analyze failures), but skipped if the workflow is explicitly cancelled.
Originally brought up in PR #8752 by @katia-sentry, but missed the !cancelled() check.
Also: upgrade to codecov-action@v5.
This allows users to compare file content efficiently without reading the
full file data, by exposing a hash of the chunk IDs and the relevant
conditions for valid comparisons, like chunker params, chunker seed/key,
id key, key type, etc.
This is based on PR #5167 by @hrehfeld, code + discussion, with some changes:
- the conditions hash now includes more relevant input params
- returning a single value that is composed of 2 parts
- tests (including new buzhash64)
Example output (different files in same archive):
1e88bfb02d0a5320-a539587200c33b857f9827d01fcb7dabacf30501c83929e7308668d43f4a6302 file1
1e88bfb02d0a5320-9ed78a4c14d0506d9ae75d914cca90db64655ddea22647dd1c89f19e2fc080ae file2
The fingerprint has 2 parts:
First part: same hash, indicates same chunking / chunk id generation params,
meaning that the second part is valid to be compared.
Second part: different hash, because file content is different.
same hash here would mean same content.
A pre-existing directory might be a btrfs subvolume that was created by
the user ahead of time when restoring several nested subvolumes from a
single archive.
If the archive item to be extracted is a directory and there is already
a directory at the destination path, do not remove (and recreate) it,
but just use it.
That way, btrfs subvolumes (which look like directories) are not deleted.
Fix originally contributed by @intelfx in #7866, but needed more work,
so I thought more about the implications and added a test.
Note:
In the past, we first removed (empty) directories, then created a fresh
one, then called restore_attrs for that. That produced correct metadata,
but only for the case of an EMPTY exisiting directory. If the existing
directory was not empty, the simply os.rmdir we tried did not work
anyway and did not remove the existing directory.
Usually we extract to an empty base directory, thus encountering this
edge case is mostly limited to continuing a previous extraction.
In that case, calling restore_attrs again on a directory that already has
existing attrs should be harmless, because they are identical.
This implementation should be good enough for our usecase (paths) and has no external dependencies.
There is also a wcwidth library which might be even better, but would add another dependency.
This commit implements a comprehensive approach to Windows path compatibility
by standardizing on forward slashes (/) for all internal path representations
while maintaining cross-platform archive compatibility.
Core Strategy:
- All internal paths now use forward slashes as separators on all platforms
- Boundary normalization: backslashes converted to forward slashes at entry
points on Windows (filesystem paths only, not user patterns)
- Literal backslashes from POSIX archives replaced with % on Windows extraction
Key Changes:
Path Handling (helpers/fs.py):
- Added slashify(): converts backslashes to forward slashes on Windows
- Added percentify(): replaces backslashes with % for POSIX-to-Windows extraction
- Updated make_path_safe() to check for Windows-style .. patterns
- Changed get_strip_prefix() to use posixpath.normpath instead of os.path.normpath
- Updated remove_dotdot_prefixes() to use forward slashes consistently
Pattern Matching (patterns.py):
- Replaced os.path with posixpath throughout for consistent separator handling
- Updated PathFullPattern, PathPrefixPattern, FnmatchPattern, ShellPattern
- All pattern matching now uses / as separator regardless of platform
- Removed platform-specific os.sep usage
Archive Operations (archive.py, item.pyx):
- Applied slashify() to paths during archive creation on Windows
- Added percentify/slashify encoding/decoding for symlink targets
- Ensures archived paths always use forward slashes
Command Line (archiver/create_cmd.py, extract_cmd.py):
- Replaced os.path.join/normpath with posixpath equivalents
- Added slashify() for stdin-provided paths on Windows
- Updated strip_components to use / separator
- Changed PathSpec to FilesystemPathSpec for proper path handling
Repository (repository.py, legacyrepository.py):
- Replaced custom _local_abspath_to_file_url() with Path.as_uri()
Documentation (archiver/help_cmd.py):
- Clarified that all archived paths use forward slashes
- Added note about Windows absolute paths in archives (e.g., C/Windows/System32)
- Documented backslash-to-percent replacement for POSIX archives on Windows
Impact:
- Windows users can now create and extract archives with consistent path handling
- Cross-platform archives remain compatible
- Pattern matching works identically on all platforms
Seen this on the macOS arm64 runner:
ImportError: dlopen(/Users/runner/work/borg/borg/.tox/py311-none/lib/python3.11/site-packages/_argon2_cffi_bindings/_ffi.abi3.so, 0x0002): tried: '/Users/runner/work/borg/borg/.tox/py311-none/lib/python3.11/site-packages/_argon2_cffi_bindings/_ffi.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/borg/borg/.tox/py311-none/lib/python3.11/site-packages/_argon2_cffi_bindings/_ffi.abi3.so' (no such file), '/Users/runner/work/borg/borg/.tox/py311-none/lib/python3.11/site-packages/_argon2_cffi_bindings/_ffi.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))
Consolidate key backup documentation into `borg key export` and reference
it from Quickstart and FAQ to avoid duplication and inconsistency.
Clarify that while `repokey` or `authenticated` mode stores the key in the
repo, a separate backup is still recommended to protect against repository
corruption or data loss.
