if there are too many deleted buckets (tombstones), hashtable performance goes down the drain.
in the worst case of 0 empty buckets and lots of tombstones, this results in full table scans for
new / unknown keys.
thus we make sure we always have a good amount of empty buckets.
* trigger bug in --verify-data, see #2221
* raise decompression errors as DecompressionError, fixes#2221
this is a subclass of IntegrityError, so borg check --verify-data works correctly if
the decompressor stumbles over corrupted data before the plaintext gets verified
(in a unencrypted repository, otherwise the MAC check would fail first).
* fixup: fix exception docstring, add placeholder, change wording
Obviously this means that --log-json with remote repos requires 1.1
on the remote end, but if you don't have that, then random "Remote:"
lines would break stderr anyway.
the chunk accounting code tried to reflect repo space usage via the st_blocks of the files.
so, a specific chunk that was shared between multiple files [inodes] was only accounted for one specific file.
thus, the overall "du" of everything in the fuse mounted repo was maybe correctly reflecting the repo space usage,
but the decision which file has the chunk (the space) was kind of arbitrary and not really useful.
otoh, a simple fuse getattr() was rather expensive due to this as it needed to iterate over the chunks list
to compute the st_blocks value. also it needed quite some memory for the accounting.
thus, st_blocks is now just ceil(size / blocksize).
also: fixed bug that st_blocks was a floating point value previously.
also: preparing for further optimization of size computation (see next cs)
if an item has a chunk list, pre-compute the total size and store it into "size" metadata entry.
this speeds up access to item size (e.g. for regular files) and could also be used to verify the validity of the chunks list.
note about hardlinks: size is only stored for hardlink masters (only they have an own chunk list)
