This feature allows multiple repositories to share deduplication-relevant secrets (id_key and chunk_seed) while maintaining secure, independent encryption keys.
`borg key export-related-secrets <REPO> <SPATH>` to export the secrets to a JSON file.
`borg init --import-related-secrets <SPATH> <REPO>` to initialize a new repository using the secrets from the JSON file.
Both repositories must use the same chunk id algorithm (both HMAC-SHA256 or both BLAKE2b).
If you create related repositories with borg 1.4.x, you can later transfer their archives
to one or multiple related new borg2 repositories without breaking deduplication.
But please note that we might remove BLAKE2b support for new borg2 repos, see #8867,
so this might only work for HMAC-SHA256 in the end.
When using the slashdot hack (e.g. `borg create ARCHIVE rootfs/./`),
the source directory's metadata was being excluded instead of archived as
the archive root. This happened because `create_helper` treated the
slashdot target directory the same as its parent directories (which should
be stripped), rather than recognizing it as the root of the archive.
Added a new condition in `create_helper` to detect when the current path
matches the strip prefix target exactly (`path + "/" == strip_prefix`) and
archive it as `"."` (the archive root) instead of excluding it.
On macOS, os.fsync() is an OS-level flush (kernel page cache -> drive
write buffer). It does not issue a HW-level flush, so a power loss can
still discard data the drive has buffered. F_FULLFSYNC additionally
flushes the drive's own write buffer to persistent storage, giving true
durability guarantees.
Add fdatasync() and sync_dir() to darwin.pyx using F_FULLFSYNC with
an os.fsync() fallback for network filesystems that do not support it.
Import them in platform/__init__.py so they override the base
implementations on macOS.
Backport of #9385 to 1.4-maint.
Signed-off-by: Mrityunjay Raj <mr.raj.earth@gmail.com>
- better check return value of fd.read(n) and reject if it returns more bytes than requested.
- avoid giving len<=0 to posix_fadvise(), which could drop the rest of the file from cache.
- buzhash: check for len == 0 edge case
- correctly Py_DECREF in cases of errors
- check for malloc/calloc failures
The previous code performed allocations and buffer acquisitions before the
`try` block. If a later allocation or buffer acquisition failed, execution did
not enter the `finally` block, so resources acquired earlier in the setup path
could leak.
Move allocation and buffer acquisition into the guarded block, initialize raw
output pointers to `NULL`, and only call `PyMem_Free` or `PyBuffer_Release`
for resources that were actually acquired.
- avoid buckets_length integer overflow on 32bit systems via huge num_buckets
- always initialize index-> min_empty and num_empty
- correctly free memory when header validation fails.
this is a minor issue, because borg will terminate in that case anyway.
- make it possible to lookup in compacted hashtables
- deal safely with empty index: we must use num_buckets = 1 to avoid division
by zero and sanity check in hashindex_read.
- reinitialize upper/lower limit and min_empty after compact
- fix size_idx / fit_size / grow_size / shrink_size (mind array bounds)
- deal with growing when already at max capacity
- hashindex_resize: replace num_entries assertion, rather return error
- BaseIndex.clear: always stay in valid state
Do not free the old index before we successfully have allocated a new one.
This is a minor issue as the Exception raised would terminate borg anyway.
When a generator for get_many() or call_many() is destroyed early (for example, if a BackupError occurs during extraction and aborts fetching preloaded chunks), a GeneratorExit is raised inside call_many().
Previously, call_many() lacked a try/finally block, so it failed to mark the abandoned msgids in self.ignore_responses. When the remote server eventually sent the data, it was indefinitely cached in self.responses and self.chunkid_to_msgids, causing a memory leak.
This fix wraps the request loop in try/finally to guarantee that all pending waiting_for message IDs, as well as any unrequested preloaded chunk IDs in calls, are properly added to ignore_responses.
For example, this memory leak could be triggered when extracting files:
- by permission errors or other OSErrors with the extracted file
- if the archived file had all-zero replacement chunks or inconsistent size
fix: resolve KeyError and memory leaks in LRUCache
- __setitem__: assign value before popping from _lru to avoid KeyError when exceeding capacity.
- clear(): clear the _lru list also to prevent stale keys causing KeyErrors during future evictions.
Add `--quick-stats` to `borg create` and `borg import-tar`.
This is a lightweight alternative to `--stats`: it prints the existing
per-archive statistics, but skips repository-wide cache statistics such as
"All archives" and the chunk index summary. Those repository-wide stats can be
slow because they may require walking archive metadata.
Keep existing `--stats` and `--json` behavior unchanged.
With `--json --quick-stats`, emit archive stats but omit the top-level `cache`
object so JSON output does not trigger cache stats calculation.
Reject `--stats --quick-stats` as ambiguous. Ignore `--quick-stats` with
`--dry-run`, matching the existing `--stats` dry-run behavior.
