{now} and {utcnow} should point to the same exact momemt, but they don't
because .now() and .utcnow() create two different objects at different
times. Although the difference will be on the order of microseconds on all
but the slowest machines, this bug still tickles my inner pedant...
reading the files cache can take considerable amount of time (a user
reported 1h 42min for a 700MB files cache for a repo with 8M files and
15TB total), so we must init the checkpoint timer after that or borg
will create the checkpoint too early.
creating a checkpoint means (among other stuff) saving the files cache,
which will also take a lot of time in such a case, one time too much.
doing this in a clean way required some refactoring:
- cache_mode is now given to Cache initializer and stored in instance
- the files cache is loaded early in _do_open (if needed)
- size inconsistencies
- file has all-zero replacement chunks
introduced new BackupError exception. when raised while extracting
files, gets handled via emitting a warning, setting rc=1 and
proceeding to next file.
if interactive passphrase query fails and the env vars are not set,
show a clear error message about this.
users often do 'BORG_PASSPHRASE=secret', forgetting the 'export'.
or they use sudo (and not sudo -E).
in both cases, the env vars won't be available for the borg process.
C code and the repo index use uint32 type for segment file offsets,
so when opening a repo and the config max_segment_size is too big,
fail early.
Also disallow setting a too big value via "borg config".
Just checking for None in patterns.py (and removing the set_defaults
calls in archiver.py) did not work as other arg parsing code relies on
having an empty list (in .patterns, .paths) to append to.
Thus I stayed away from pattern.py and rather fixed the defaults for
.paths and .patterns in all parsers.
the problem was that the upper layer code did not have enough information
about the file, whether it is known or not - and thus, could not decide
correctly whether status should be M)odified or A)dded.
now, file_known_and_unchanged method returns an additional "known"
boolean to fix this.
also: add comment about files cache loading in cache_mode='r'
now deals with:
- corrupted files cache (truncated or modified not by borg)
- inaccessible/unreadable files cache
- missing files cache
The latter fix is not sufficient, the cache transaction processing
would still stumble over expected, but missing files in the cache.
(cherry picked from commit 423ec4ba1e)
- move stuff to platform.base (should be platform independent according
to the docs).
- bump platform API version
- parseformat: import fqdn from platform instead of recomputing it
This is not yet fixing #3471, just a preparation for it.
