all archives, all items are read to build a unified view.
files are represented by a same-name directory with the versions of the file.
A filename suffix computed by adler32(chunkids) is used to disambiguate the versions.
also: refactor code a little, create methods for leaves, inner nodes.
Reusing the nonce totally breaks AES-CTR confidentiality.
This code uses a reservation of nonce space and stores the next nonce available for
a future reservation on the client and in the repository.
Local storage is needed to protect against evil repositories that try to gain access to
encrypted data by not saving nonce reservations and aborting the connection or otherwise
forcing a rollback.
Storage in the repository is needed to protect against another client writing to the
repository after a transaction was aborted and thus not seeing the last used nonce from
the manifest.
With a real counter mode cipher protection for the multiple client case with an actively
evil repository is not possible. But this still protects against cases where the attacker can
not arbitrarily change the repository but can read everything stored and abort connections
or crash the server.
Fixes#22
ie. it means that if all the recompressed chunks were already in the repo
no data would be written, so there would be no active txn, so failure
ensues.
n.b. we only need to store them for items that we wouldn't extract.
this also fixes an intersting edge case in extracting hard links
with --strip-components
The filter function passed to iter_items (with preload=True) may never return True for items that
are not really extracted later because that would leak prefetched items.
For restoring hard linked files the item containing the actual chunks might not be matched
or implicitly removed from the restore by strip_components. For this reason the chunk list or all
items that can potentially be used as hardlink target needs to be stored.
To achive both requirements at the same time the filter function needs to store the needed information
for the hardlinks while not returning True just because it could be a hardlink target.
Known problems: When using progress indication the calculated extracted_size now can be smaller
than the actual extracted size in presence of hard links (master is not restored) instead of
bigger (potential master not used in restore).
archiver: split traceback and msg, have separate log level for traceback, log LockTimeout at debug level,
for "Error" exceptions: always log the traceback, either at ERROR or DEBUG level.
remote: if we have an "Error" typed exception instance, we can use its traceback flag and .get_message()
as we do locally.
