mirror of
https://github.com/borgbackup/borg.git
synced 2026-03-21 10:00:06 -04:00
8834f6fdbd
For small remainders of files (last chunk), we do not need to buzhash if it
is already clear that there is not enough left (we want at least min_size big
chunks).
Small files are handled by same code - as they only give 1 chunk, that is
the last chunk (see above).
See "Cases" considerations below.
For big files, we do not need to buzhash the first min_size bytes of a chunk -
we do not want to cut there anyway, so we can start buzhashing at offset
min_size.
Cases (before this change)
--------------------------
- A) remaining <= window_size
- would do 2 chunker_fill calls (both line 253) and trigger eof with the 2nd call
- no buzhashing
- result is 1 <remaining> length chunk
- B) window_size < remaining <= min_size:
- the chunker would do 1 chunker_fill call (line 253) that would read the entire remaining file (but not trigger eof yet)
- would compute all possible remaining - window_size + 1 buzhashes, but without a chance for a cut,
because there is also the n < min_size condition
- would do another chunker_fill call (line 282), but not get more data, so loop ends
- result is 1 <remaining> length chunk
- C) file > min_size:
- normal chunking
Cases (after this change)
-------------------------
- A) similar to above A), but up to remaining < min_size + window_size + 1,
so it does not buzhash if there is no chance for a cut.
- B) see C) above
|
||
|---|---|---|
| .. | ||
| testsuite | ||
| __init__.py | ||
| __main__.py | ||
| _chunker.c | ||
| _hashindex.c | ||
| archive.py | ||
| archiver.py | ||
| cache.py | ||
| chunker.pyx | ||
| compress.pyx | ||
| constants.py | ||
| crypto.pyx | ||
| fuse.py | ||
| hash_sizes.py | ||
| hashindex.pyx | ||
| helpers.py | ||
| item.py | ||
| key.py | ||
| locking.py | ||
| logger.py | ||
| lrucache.py | ||
| platform.py | ||
| platform_base.py | ||
| platform_darwin.pyx | ||
| platform_freebsd.pyx | ||
| platform_linux.pyx | ||
| platform_posix.pyx | ||
| remote.py | ||
| repository.py | ||
| selftest.py | ||
| shellpattern.py | ||
| upgrader.py | ||
| xattr.py | ||