When running as a Pyinstaller-made binary, sys.executable points to the
borg binary itself. Invoking it with "-m borg" resulted in an incorrect
command line (e.g., "borg -m borg ..."), which confused the argument
parser in the subprocess.
This change checks sys.frozen to determine the correct invocation:
- If frozen: [sys.executable, ...args]
- If not frozen: [sys.executable, "-m", "borg", ...args]
The check_python() function verified that the Python runtime supported
'follow_symlinks' for os.stat, os.utime, and os.chown. This check is no
longer necessary because:
1. Borg now requires Python >= 3.10.
2. On POSIX systems (Linux, macOS, *BSD, Haiku, OmniOS), support for these
operations relies on the *at syscalls (fstatat, etc.), which have been
implemented in standard libc for well over a decade (e.g., FreeBSD 8.0+,
NetBSD 6.0+, Solaris 11+).
3. On Windows (MSYS2/MinGW), Python has supported follow_symlinks for
os.stat since Python 3.2. The removed check specifically inspected only
os.stat on Windows, avoiding the problematic os.utime/os.chown checks.
Any platform capable of running Python 3.10 will inherently support these
standard file operations.
if an already existing fs directory has the correct (as archived) mtime,
we have already extracted it in a previous borg extract run and we do not
need and should not call restore_attrs for it again.
if the directory exists, but does not have the correct mtime, restore_attrs
will be called and its attributes will be extracted (and mtime set to
correct value).
This allows users to compare file content efficiently without reading the
full file data, by exposing a hash of the chunk IDs and the relevant
conditions for valid comparisons, like chunker params, chunker seed/key,
id key, key type, etc.
This is based on PR #5167 by @hrehfeld, code + discussion, with some changes:
- the conditions hash now includes more relevant input params
- returning a single value that is composed of 2 parts
- tests (including new buzhash64)
Example output (different files in same archive):
1e88bfb02d0a5320-a539587200c33b857f9827d01fcb7dabacf30501c83929e7308668d43f4a6302 file1
1e88bfb02d0a5320-9ed78a4c14d0506d9ae75d914cca90db64655ddea22647dd1c89f19e2fc080ae file2
The fingerprint has 2 parts:
First part: same hash, indicates same chunking / chunk id generation params,
meaning that the second part is valid to be compared.
Second part: different hash, because file content is different.
same hash here would mean same content.
A pre-existing directory might be a btrfs subvolume that was created by
the user ahead of time when restoring several nested subvolumes from a
single archive.
If the archive item to be extracted is a directory and there is already
a directory at the destination path, do not remove (and recreate) it,
but just use it.
That way, btrfs subvolumes (which look like directories) are not deleted.
Fix originally contributed by @intelfx in #7866, but needed more work,
so I thought more about the implications and added a test.
Note:
In the past, we first removed (empty) directories, then created a fresh
one, then called restore_attrs for that. That produced correct metadata,
but only for the case of an EMPTY exisiting directory. If the existing
directory was not empty, the simply os.rmdir we tried did not work
anyway and did not remove the existing directory.
Usually we extract to an empty base directory, thus encountering this
edge case is mostly limited to continuing a previous extraction.
In that case, calling restore_attrs again on a directory that already has
existing attrs should be harmless, because they are identical.
This implementation should be good enough for our usecase (paths) and has no external dependencies.
There is also a wcwidth library which might be even better, but would add another dependency.
This commit implements a comprehensive approach to Windows path compatibility
by standardizing on forward slashes (/) for all internal path representations
while maintaining cross-platform archive compatibility.
Core Strategy:
- All internal paths now use forward slashes as separators on all platforms
- Boundary normalization: backslashes converted to forward slashes at entry
points on Windows (filesystem paths only, not user patterns)
- Literal backslashes from POSIX archives replaced with % on Windows extraction
Key Changes:
Path Handling (helpers/fs.py):
- Added slashify(): converts backslashes to forward slashes on Windows
- Added percentify(): replaces backslashes with % for POSIX-to-Windows extraction
- Updated make_path_safe() to check for Windows-style .. patterns
- Changed get_strip_prefix() to use posixpath.normpath instead of os.path.normpath
- Updated remove_dotdot_prefixes() to use forward slashes consistently
Pattern Matching (patterns.py):
- Replaced os.path with posixpath throughout for consistent separator handling
- Updated PathFullPattern, PathPrefixPattern, FnmatchPattern, ShellPattern
- All pattern matching now uses / as separator regardless of platform
- Removed platform-specific os.sep usage
Archive Operations (archive.py, item.pyx):
- Applied slashify() to paths during archive creation on Windows
- Added percentify/slashify encoding/decoding for symlink targets
- Ensures archived paths always use forward slashes
Command Line (archiver/create_cmd.py, extract_cmd.py):
- Replaced os.path.join/normpath with posixpath equivalents
- Added slashify() for stdin-provided paths on Windows
- Updated strip_components to use / separator
- Changed PathSpec to FilesystemPathSpec for proper path handling
Repository (repository.py, legacyrepository.py):
- Replaced custom _local_abspath_to_file_url() with Path.as_uri()
Documentation (archiver/help_cmd.py):
- Clarified that all archived paths use forward slashes
- Added note about Windows absolute paths in archives (e.g., C/Windows/System32)
- Documented backslash-to-percent replacement for POSIX archives on Windows
Impact:
- Windows users can now create and extract archives with consistent path handling
- Cross-platform archives remain compatible
- Pattern matching works identically on all platforms
Consolidate key backup documentation into `borg key export` and reference
it from Quickstart and FAQ to avoid duplication and inconsistency.
Clarify that while `repokey` or `authenticated` mode stores the key in the
repo, a separate backup is still recommended to protect against repository
corruption or data loss.
fuse2 was a bit misleading. it meant "our 2nd fuse implementation",
but could be misunderstood to refer to fuse v2.
hlfuse.py now means highlevel fuse, as opposed to the lowlevel fuse in fuse.py.
Updated mount_cmds_test.py to work with both llfuse/pyfuse3 and mfusepy
by checking for either implementation in skip conditions.
mfusepy: 2 test fails due to hardlink implementation differences
