- better check return value of fd.read(n) and reject if it returns more bytes than requested.
- avoid giving len<=0 to posix_fadvise(), which could drop the rest of the file from cache.
- buzhash: check for len == 0 edge case
- correctly Py_DECREF in cases of errors
- check for malloc/calloc failures
The previous code performed allocations and buffer acquisitions before the
`try` block. If a later allocation or buffer acquisition failed, execution did
not enter the `finally` block, so resources acquired earlier in the setup path
could leak.
Move allocation and buffer acquisition into the guarded block, initialize raw
output pointers to `NULL`, and only call `PyMem_Free` or `PyBuffer_Release`
for resources that were actually acquired.
- avoid buckets_length integer overflow on 32bit systems via huge num_buckets
- always initialize index-> min_empty and num_empty
- correctly free memory when header validation fails.
this is a minor issue, because borg will terminate in that case anyway.
- make it possible to lookup in compacted hashtables
- deal safely with empty index: we must use num_buckets = 1 to avoid division
by zero and sanity check in hashindex_read.
- reinitialize upper/lower limit and min_empty after compact
- fix size_idx / fit_size / grow_size / shrink_size (mind array bounds)
- deal with growing when already at max capacity
- hashindex_resize: replace num_entries assertion, rather return error
- BaseIndex.clear: always stay in valid state
Do not free the old index before we successfully have allocated a new one.
This is a minor issue as the Exception raised would terminate borg anyway.
When a generator for get_many() or call_many() is destroyed early (for example, if a BackupError occurs during extraction and aborts fetching preloaded chunks), a GeneratorExit is raised inside call_many().
Previously, call_many() lacked a try/finally block, so it failed to mark the abandoned msgids in self.ignore_responses. When the remote server eventually sent the data, it was indefinitely cached in self.responses and self.chunkid_to_msgids, causing a memory leak.
This fix wraps the request loop in try/finally to guarantee that all pending waiting_for message IDs, as well as any unrequested preloaded chunk IDs in calls, are properly added to ignore_responses.
For example, this memory leak could be triggered when extracting files:
- by permission errors or other OSErrors with the extracted file
- if the archived file had all-zero replacement chunks or inconsistent size
fix: resolve KeyError and memory leaks in LRUCache
- __setitem__: assign value before popping from _lru to avoid KeyError when exceeding capacity.
- clear(): clear the _lru list also to prevent stale keys causing KeyErrors during future evictions.
That way, right below the docs version number that is currently
being displayed, it is easier to find for users.
Also: hide the default readthedocs-flyout (bottom right)
Co-authored-by: Junie <junie@jetbrains.com>
If the repo index is not uptodate, any borg operation might trigger replay_segments and thus even operations that are usually quick can take rather long, leaving the user wondering about what's going on.
While there is already a progress indicator in replay_segments, it is logged at INFO level and thus not visible if borg runs at the default WARNING level.
Consolidate key backup documentation into `borg key export` and reference
it from Quickstart and FAQ to avoid duplication and inconsistency.
Clarify that while `repokey` or `authenticated` mode stores the key in the
repo, a separate backup is still recommended to protect against repository
corruption or data loss.
Python's `os.truncate()` on Windows relies on `SetEndOfFile()`, which does
not initialize the extended disk space with zeroes. This means that
trailing sparse holes simply leave uninitialized garbage data at the end
of the file.
During sparse file extraction, when the very last chunk is a sparse hole,
the VDL (Valid Data Length) is not properly advanced by `os.truncate()`.
As a result, reading from the end of the file fetches random disk garbage
instead of zeroes, causing spurious test failures at boundaries (like
2MB or 8MB) depending on what was in the uninitialized disk sectors.
Fix this by tracking trailing holes and manually writing a single `b"\0"`
byte at the end of the file before truncating on Windows. Writing explicit
data forces NTFS to officially advance the VDL and securely zero-fill the
preceding hole space.
Re-enable `test_sparse_file` on Windows.
Fixes#9448.
borg mount forks into a background daemon, so coverage was missing the process that actually handles the FUSE requests, leaving fuse.py and hlfuse.py at 0%.
Enable coverage patches:
patch = ["subprocess", "_exit"]
This lets coverage follow spawned subprocesses and still record data for paths that terminate via os._exit().
(cherry picked from commit 6df608ab6b)
