Include all changes from dependabot PR #9603 plus fix the broken
virtualenv pin: tox 4.52.1 requires virtualenv>=21.1, but PR #9603
kept virtualenv==20.39.1 (20.x). Bumped virtualenv to 21.3.2.
Fixes: https://github.com/borgbackup/borg/pull/9603
When using the slashdot hack (e.g. `borg create ARCHIVE rootfs/./`),
the source directory's metadata was being excluded instead of archived as
the archive root. This happened because `create_helper` treated the
slashdot target directory the same as its parent directories (which should
be stripped), rather than recognizing it as the root of the archive.
Added a new condition in `create_helper` to detect when the current path
matches the strip prefix target exactly (`path + "/" == strip_prefix`) and
archive it as `"."` (the archive root) instead of excluding it.
The @pytest.mark.skipif(not fs_supports_sparse(), ...) decorator on
test_chunkify_sparse was commented out and is not needed because the
zeros.startswith(result) fix in FileReader.read() detects zero-filled
slices as CH_ALLOC regardless of sparse FS support, and ChunkerFixed
with sparse=True gracefully falls back when SEEK_HOLE/SEEK_DATA is
not available.
The test_sparsemap tests were failing on Linux CI because SEEK_HOLE/
SEEK_DATA naturally coalesces adjacent ranges of the same type (data
or hole), but the tests compared against the raw per-block sparse maps
which list each block separately.
Add a coalesce_sparse_map() helper that merges adjacent ranges with
the same is_data flag, and compare sparsemap() output against the
coalesced expected map instead of the raw per-block map.
When FileReader.read() sliced a large CH_DATA block (read at 1MB
granularity) into smaller block_size chunks (e.g. 4096 bytes), zero-filled
slices were returned as CH_DATA with zero bytes instead of CH_ALLOC.
Add a zeros.startswith(result) check before returning a CH_DATA chunk,
converting all-zero slices to CH_ALLOC. This ensures sparse-aware
consumers correctly identify allocated-but-zero regions regardless of
whether the file was read with sparse=True or sparse=False.
- Update fixed_test.py expectations for non-sparse chunking.
- Enable `sparse=True` in interaction_test.py and reader_test.py where zero detection is required.
- Catch `ValueError` in _build_fmap to support `BytesIO` seeking.
When a generator for get_many() or call_many() is destroyed early (for example, if a BackupError occurs during extraction and aborts fetching preloaded chunks), a GeneratorExit is raised inside call_many().
Previously, call_many() lacked a try/finally block, so it failed to mark the abandoned msgids in self.ignore_responses. When the remote server eventually sent the data, it was indefinitely cached in self.responses and self.chunkid_to_msgids, causing a memory leak.
This fix wraps the request loop in try/finally to guarantee that all pending waiting_for message IDs, as well as any unrequested preloaded chunk IDs in calls, are properly added to ignore_responses.
For example, this memory leak could be triggered when extracting files:
- by permission errors or other OSErrors with the extracted file
- if the archived file had all-zero replacement chunks or inconsistent size
The previous code performed allocations and buffer acquisitions before the
`try` block. If a later allocation or buffer acquisition failed, execution did
not enter the `finally` block, so resources acquired earlier in the setup path
could leak.
Move allocation and buffer acquisition into the guarded block, initialize raw
output pointers to `NULL`, and only call `PyMem_Free` or `PyBuffer_Release`
for resources that were actually acquired.
The previous code performed allocations and buffer acquisitions before the
`try` block. If a later allocation or buffer acquisition failed, execution did
not enter the `finally` block, so resources acquired earlier in the setup path
could leak.
Move allocation and buffer acquisition into the guarded block, initialize raw
output pointers to `NULL`, and only call `PyMem_Free` or `PyBuffer_Release`
for resources that were actually acquired.
