When using the slashdot hack (e.g. `borg create ARCHIVE rootfs/./`),
the source directory's metadata was being excluded instead of archived as
the archive root. This happened because `create_helper` treated the
slashdot target directory the same as its parent directories (which should
be stripped), rather than recognizing it as the root of the archive.
Added a new condition in `create_helper` to detect when the current path
matches the strip prefix target exactly (`path + "/" == strip_prefix`) and
archive it as `"."` (the archive root) instead of excluding it.
On macOS, os.fsync() is an OS-level flush (kernel page cache -> drive
write buffer). It does not issue a HW-level flush, so a power loss can
still discard data the drive has buffered. F_FULLFSYNC additionally
flushes the drive's own write buffer to persistent storage, giving true
durability guarantees.
Add fdatasync() and sync_dir() to darwin.pyx using F_FULLFSYNC with
an os.fsync() fallback for network filesystems that do not support it.
Import them in platform/__init__.py so they override the base
implementations on macOS.
Backport of #9385 to 1.4-maint.
Signed-off-by: Mrityunjay Raj <mr.raj.earth@gmail.com>
- better check return value of fd.read(n) and reject if it returns more bytes than requested.
- avoid giving len<=0 to posix_fadvise(), which could drop the rest of the file from cache.
- buzhash: check for len == 0 edge case
- correctly Py_DECREF in cases of errors
- check for malloc/calloc failures
The previous code performed allocations and buffer acquisitions before the
`try` block. If a later allocation or buffer acquisition failed, execution did
not enter the `finally` block, so resources acquired earlier in the setup path
could leak.
Move allocation and buffer acquisition into the guarded block, initialize raw
output pointers to `NULL`, and only call `PyMem_Free` or `PyBuffer_Release`
for resources that were actually acquired.
- avoid buckets_length integer overflow on 32bit systems via huge num_buckets
- always initialize index-> min_empty and num_empty
- correctly free memory when header validation fails.
this is a minor issue, because borg will terminate in that case anyway.
- make it possible to lookup in compacted hashtables
- deal safely with empty index: we must use num_buckets = 1 to avoid division
by zero and sanity check in hashindex_read.
- reinitialize upper/lower limit and min_empty after compact
- fix size_idx / fit_size / grow_size / shrink_size (mind array bounds)
- deal with growing when already at max capacity
- hashindex_resize: replace num_entries assertion, rather return error
- BaseIndex.clear: always stay in valid state
Do not free the old index before we successfully have allocated a new one.
This is a minor issue as the Exception raised would terminate borg anyway.
When a generator for get_many() or call_many() is destroyed early (for example, if a BackupError occurs during extraction and aborts fetching preloaded chunks), a GeneratorExit is raised inside call_many().
Previously, call_many() lacked a try/finally block, so it failed to mark the abandoned msgids in self.ignore_responses. When the remote server eventually sent the data, it was indefinitely cached in self.responses and self.chunkid_to_msgids, causing a memory leak.
This fix wraps the request loop in try/finally to guarantee that all pending waiting_for message IDs, as well as any unrequested preloaded chunk IDs in calls, are properly added to ignore_responses.
For example, this memory leak could be triggered when extracting files:
- by permission errors or other OSErrors with the extracted file
- if the archived file had all-zero replacement chunks or inconsistent size
fix: resolve KeyError and memory leaks in LRUCache
- __setitem__: assign value before popping from _lru to avoid KeyError when exceeding capacity.
- clear(): clear the _lru list also to prevent stale keys causing KeyErrors during future evictions.
Add `--quick-stats` to `borg create` and `borg import-tar`.
This is a lightweight alternative to `--stats`: it prints the existing
per-archive statistics, but skips repository-wide cache statistics such as
"All archives" and the chunk index summary. Those repository-wide stats can be
slow because they may require walking archive metadata.
Keep existing `--stats` and `--json` behavior unchanged.
With `--json --quick-stats`, emit archive stats but omit the top-level `cache`
object so JSON output does not trigger cache stats calculation.
Reject `--stats --quick-stats` as ambiguous. Ignore `--quick-stats` with
`--dry-run`, matching the existing `--stats` dry-run behavior.
That way, right below the docs version number that is currently
being displayed, it is easier to find for users.
Also: hide the default readthedocs-flyout (bottom right)
Co-authored-by: Junie <junie@jetbrains.com>
